Telegraf is a server-based agent for **collecting and sending metrics for further processing**. It's a piece of software that you can install anywhere in your infrastructure and it will read metrics from specified sources – typically application logs, events, or data outputs. This tutorial demonstrates how to send your metrics to Coralogix via Telegraf. ## Prerequisites - Telegraf [installed](https://docs.influxdata.com/telegraf/v1.21/introduction/installation/) - Active Coralogix [account](https://signup.coralogix.com/#/) with [metric bucket](https://coralogix.com/docs/user-guides/data-flow/s3-archive/connect-s3-archive/index.md) and a working [Grafana dashboard](https://coralogix.com/docs/user-guides/visualizations/hosted-grafana-view/index.md) for metrics ## Configuration If you are using Telegraf version `v1.23.4` or above, you can use the Coralogix dialect for a more straightforward configuration. In order to send your data to Coralogix, you are required to declare the following variables in your configuration: - **`service_address`**: Coralogix [OpenTelemetry endpoint](https://coralogix.com/docs/integrations/coralogix-endpoints/index.md) associated with your Coralogix domain - **`private_key`**: Access your Coralogix [Send-Your-Data API key](https://coralogix.com/docs/user-guides/account-management/api-keys/send-your-data-api-key/index.md). Your key is recorded in the override file as a secret in order to ensure that this sensitive information remains protected and unexposed. - **`application`** & **`subsystem`**: Customize and organize your data in your Coralogix dashboard using [application](https://coralogix.com/docs/user-guides/account-management/account-settings/application-and-subsystem-names/index.md) and [subsystem](https://coralogix.com/docs/user-guides/account-management/account-settings/application-and-subsystem-names/index.md) names. Application name is available as a `cx_application_name` label for all metrics for this Telegraf instance. Subsystem name is available as a `cx_subsystem_name` label for all metrics for this Telegraf instance. The following example shows how to configure Telegraf using the Coralogix dialect: ```text [[outputs.opentelemetry]] service_address = "ingress.[[DOMAIN_VALUE]]:443" compression = "gzip" [outputs.opentelemetry.coralogix] private_key = "" application = "" subsystem = "" ``` For older versions of Telegraf (`v1.23.3` or earlier), you should use the following configuration to send telemetry data using the OpenTelemetry output plugin: ```text [[outputs.opentelemetry]] service_address = "ingress.[[DOMAIN_VALUE]]:443" insecure_skip_verify = true compression = "gzip" [outputs.opentelemetry.headers] Authorization = "Bearer " ApplicationName = "" ApiName = "" ``` ## Monitoring Node In this example, we set up a Telegraf agent to monitor Node, which runs a website. It's a development environment, so we set ApplicationName to "development" and subsystem to "website". Additionally, we provide more metadata in the `\\\\[global\\\_tags\\\\]` section, such as `dc="us-east-1"` and `rack="1a"`. For input plugins, we use: - cpu - disk - diskio - kernel - mem - processes - swap - system These plugins give us a good overview of the Node's metrics. The following example shows the complete Telegraf configuration: ```text # Telegraf Configuration [global_tags] # will tag all metrics with dc=us-east-1 and rack = "1a" dc = "us-east-1" rack = "1a" # Configuration for telegraf agent [agent] interval = "10s" round_interval = true metric_batch_size = 1000 metric_buffer_limit = 10000 flush_interval = "10s" flush_jitter = "0s" precision = "" # Send OpenTelemetry metrics over gRPC [[outputs.opentelemetry]] service_address = "ingress.[[DOMAIN_VALUE]]:443" insecure_skip_verify = true compression = "gzip" [outputs.opentelemetry.headers] Authorization = "Bearer " ApplicationName = "development" ApiName = "website" # Input plugins # Read metrics about cpu usage [[inputs.cpu]] percpu = true totalcpu = true collect_cpu_time = false report_active = false # Read metrics about disk usage by mount point [[inputs.disk]] ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"] [[inputs.diskio]] [[inputs.kernel]] [[inputs.mem]] [[inputs.processes]] [[inputs.swap]] [[inputs.system]] ``` In Coralogix, you can query all the data using PromQL via Grafana. The following PromQL query shows us all the metrics exported by this Telegraf instance: ```text {cx_application_name="development",cx_subsystem_name="website"} ``` Results in: ```text cpu_usage_guest{cx_application_name="development", cx_subsystem_name="website", cpu="cpu-total", cx_tenant_id="32680", dc="us-east-1", host="website-host", hostname="$HOSTNAME", rack="1a"} .... diskio_io_time_total{cx_application_name="development", cx_subsystem_name="website", cx_tenant_id="32680", dc="us-east-1", host="website-host, hostname="$HOSTNAME", name="loop0", rack="1a"} ``` Note: application name, subsystem, and global tags are available as labels on the exported metrics. Next, you can start building dashboards and configuring different input plugins. ## Troubleshooting If you are having issues, you can enable debug logs via the Telegraf configuration: ```text [agent] debug = true ... ``` Telegraf will then start logging a detailed overview of what is happening. For example: ```text 2022-10-19T11:12:34Z D! [outputs.opentelemetry] Wrote batch of 43 metrics in 87.062989ms 2022-10-19T11:12:34Z D! [outputs.opentelemetry] Buffer fullness: 0 / 10000 metrics 2022-10-19T11:13:20Z D! [inputs.disk] [SystemPS] -> using mountpoint "/run/user/1000/doc"... 2022-10-19T11:13:20Z D! [inputs.disk] [SystemPS] => dropped by disk usage ("/run/user/1000/doc"): operation not permitted ``` ## Additional Resources [Telegraf Docs by InfluxDB](https://docs.influxdata.com/telegraf/v1.21/introduction/installation/) ## Support **Need help?** Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up. Feel free to reach out to us **via our in-app chat** or by sending us an email at [support@coralogix.com](mailto:support@coralogix.com).