This tutorial demonstrates how to set up OpenTelemetry Collector Contrib on a Linux EC2 instance. There are some optional sections in the example configuration that can be enabled as needed.

Setup

Installation

STEP 1 If you don't already have one, create an EC2 instance.

STEP 2 Download and install OpenTelemetry.

On this page, find and copy the otelcol-contrib download link for your OS and CPU architecture in the list of Assets.

For example:

For Debian based Linux on AMD64 CPU architecture:

otelcol-contrib_0.115.1_linux_amd64.deb

Link:

https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.115.1/otelcol-contrib_0.115.1_linux_amd64.deb

Download and install the desired package onto the EC2 host:

Example using the CLI:

wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.115.1/otelcol-contrib_0.115.1_linux_amd64.deb
sudo dpkg -i otelcol-contrib_0.115.1_linux_amd64.deb

STEP 3 Configure

Review and replace the default /etc/otelcol-contrib/config.yaml with this example configuration. Adjust as necessary:

receivers:
  # To enable traces, uncomment this OTLP receiver and traces pipeline sections.
  # otlp:
  #   protocols:
  #     grpc:
  #       endpoint: localhost:4317
  #     http:
  #       endpoint: localhost:4318
  filelog:
    force_flush_period: 0
    include:
    - /var/log/syslog
    - /var/log/more/examples/*/*.log
    include_file_name: true
    include_file_path: true
    retry_on_failure:
      enabled: true
    start_at: beginning
    # Uncomment for file tracking, you will need to enable the file_storage "extension" too.
    # storage: file_storage
  hostmetrics:
    root_path: /
    collection_interval: 10s
    scrapers:
      cpu: { metrics: { system.cpu.utilization: { enabled: true } } }
      filesystem:
        exclude_fs_types: { fs_types: [autofs, binfmt_misc, bpf, cgroup2, configfs, debugfs, devpts, devtmpfs, fusectl, hugetlbfs, iso9660, mqueue, nsfs, overlay, proc, procfs, pstore, rpc_pipefs, securityfs, selinuxfs, squashfs, sysfs, tracefs], match_type: strict }
        exclude_mount_points: { match_type: regexp, mount_points: [/dev/*, /proc/*, /sys/*, /run/k3s/containerd/*, /run/containerd/runc/*, /var/lib/docker/*, /var/lib/kubelet/*, /snap/*] }
      memory: { metrics: { system.memory.utilization: { enabled: true } } }
      network:
      process:
        metrics:
          process.cpu.utilization: { enabled: true }
          process.memory.utilization: { enabled: true }
          process.threads: { enabled: true }
        mute_process_exe_error: true
        mute_process_user_error: true

processors:
  batch:
    send_batch_size: 1024
    send_batch_max_size: 2048
    timeout: "1s"
  resourcedetection:
    detectors: [system, env, ec2]
    override: false
    timeout: 2s
  resource/metadata:
    attributes:
      - action: upsert
        key: cx.otel_integration.name
        value: coralogix-integration-OTEL-EC2-service

exporters:
  coralogix:
    domain: "coralogix.com"
    private_key: "cxtp_ExampleSndYourDataAPIKey"
    application_name: "DefaultApplicationName"
    subsystem_name: "DefaultSubsystemName"
    # These configuration values can be used to dynamically set the application_name and subsystem_name
    # application_name_attributes:
    # - host.id
    # subsystem_name_attributes: 
    # - host.name
    timeout: 30s

# To use file_storage, you need to create or select a folder to which otelcol-contrib can write.
# extensions:
#   file_storage:
#     directory: /var/log/otelcol

service:
  # Uncomment extensions if you intend to do file tracking of filelog receiver
  # extensions:
  # - file_storage
  pipelines:
    logs:
      receivers: [ filelog ]
      processors: [ resourcedetection, batch ]
      exporters: [ coralogix ]
    metrics:
      receivers: [ hostmetrics ]
      processors: [ resourcedetection, batch ]
      exporters: [ coralogix ]
    # To enable traces, uncomment this traces pipeline and the OTLP receiver sections.
    # traces:
    #   receivers: [ otlp ]
    #   processors: [ resourcedetection, batch ]
    #   exporters: [ coralogix ]

Notes:

• Coralogix recommends the default otel-integration chart settings for batch processors in all collectors. Learn more here.

• Learn more about receivers and exporters, the filelog receiver, and the hostmetrics receiver.

• Due to Linux permissions, you'll likely get errors in the otelcol-contrib service logs. There are options in the hostmetrics reciever process config to mute these errors. We've enabled the mute_process_exe_error and mute_process_user_error in our example configuration.

STEP 4 Reload service

To consume the new configuration, you'll need to restart the service. Depending on which package you installed, the process will be different.

Example:

Systems with systemctl:

sudo systemctl restart otelcol-contrib

Check service logs to ensure proper function.

Optional Configurations

file_storage extension

The file_storage extension is used to provide the OTEL Collector space on the local filesystem to store checkpoints or other files. As Linux permissions are user based, and OTEL Collector service runs as otelcol-contrib, you'll need to designate an approprite folder with appropriate write permissions for that user.

Once you determine an appropriate location, uncomment the extensions: section and adjust the directory: accordingly.

Additionally, you'l need to uncomment the extension under service: and the storage: line of the filelog receiver.

Dynamic application and subsystem names

The Coralogix Exporter supports dynamic application and subsystem names through the use of application_name_attributes and subsystem_name_attributes configuration options. You can use any resource attribute from your enriched payloads as the keys. The first key that is non-null will be used. If all options are null, it will fall back to the value set in application_name and subsystem_name.

Trace pipeline

Local deployment of OTEL can be used to process Traces generated by applications hosted on the host at localhost:4317 and localhost:4318 when you enable the traces pipeline and otlp receiver.

To do so, uncomment the following sections: - traces: under service > pipelines - otlp: under receivers

Collect EC2 Tags

To add EC2 tags, add a role to your instance.

Create a New Policy

STEP 1. Navigate to IAM > Policies. Click Create policy.

STEP 2. Select the following options.

STEP 3. Enter a policy name. Click Save.

Create a New Role

STEP 1. Navigate to IAM > Roles. Click Create role.

STEP 2. Select the following options.

STEP 3. Click Next.

STEP 4. Click on the checkbox next to the rule you just created.

STEP 5. Enter a rule name. Click Create rule.

Add IAM Role to Your Instance

STEP 1. Open your instance from the EC2 instance screen.

STEP 2. Add the IAM role to the empty IAM.

STEP 3. Select the IAM role. Click Update IAM role.

STEP 4. Allow tags in the metadata instance.

STEP 5. Update the resourcedetection processor at the processors.

resourcedetection:
    # ecs & docker detectors not required when using ecslogresourcedetection for logs
    detectors: [env, ec2, system]
    ec2:
      tags:
        - ^TagName$
    timeout: 2s
    override: false

Notes:

• It is mandatory to specify the tags parameter, as it is not added by default.

• Set any Regex to limit the number of tags or use ^.*$ to get all tags.

Validation

Start Otel to validate your setup, using this example from Ubuntu Linux.

/usr/bin/otelcol-contrib --config=file:/etc/otelcol-contrib/config.yaml

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to contact us via our in-app chat or by emailing [email protected].

Previous Windows Event Logs & OpenTelemetry

Next OpenTelemetry using Docker

Theme

Light

light

dark