AWS ECS-EC2 OpenTelemetry instrumentation

Open in ChatGPTOpen in Claude

This tutorial demonstrates how to deploy OpenTelemetry to ECS to facilitate the collection of logs, metrics, traces, and profiles, and send them to Coralogix.

Image

This implementation utilizes the wrapper image coralogixrepo/coralogix-otel-collector, based on the official OpenTelemetry contrib image.

Notes:

• The wrapper image is used to dynamically apply the OpenTelemetry configuration at runtime from an environment variable.

• The image configuration utilizes the otlp receiver for both HTTP (on 4318) and GRPC (on 4317). Data can be sent using either endpoint.

Prerequisites

• Coralogix account

• Coralogix Send Your Data - API Key

• AWS credentials configured

• aws-cli

• ecs-cli (optional, used in cluster creation and verification examples)

• Existing ECS EC2 cluster name

• Coralogix ECS-EC2 CloudFormation template

If you do not already have an ECS cluster, create one first.

Configuration

OpenTelemetry

The OpenTelemetry configuration for the agent is stored in a Base64-encoded environment variable and is applied at runtime. This allows you to dynamically pass any configuration values you choose as a parameter to CloudFormation.

To send logs, metrics, and traces to Coralogix, use this CloudFormation template as a reference configuration. The template includes a default OpenTelemetry (OTel) configuration, which you can override by specifying a value for the CustomConfig parameter.

Notes:

• The Coralogix Exporter enables the use of enrichments such as dynamic application or subsystem name, defined using application_name_attributes and subsystem_name_attributes, respectively.

ECS Cluster

Deploy a new ECS cluster. If you already have an existing ECS Cluster, skip this step.

STEP 1. Deploy a new ECS cluster:

ecs-cli up --region <region> --keypair <your-key-pair> --cluster <cluster-name> --size <no. of instances> --capability-iam 

Note

The --keypair flag and STEP 2 are not mandatory. However, if no Key Pair is supplied, you will be unable to connect to any of the EC2 instances in the cluster via SSH.

STEP 2. Create a key pair using the command below:

aws ec2 create-key-pair --key-name MyKeyPair --query 'KeyMaterial' --output text > MyKeyPair.pem

STEP 3. Control default values.

The ecs-cli up command will leverage CloudFormation to create an ECS Cluster. Default values will be used to create and configure a VPC and Subnets. These values can also be controlled using:

ecs-cli up --help

Deploy with CLI

Without profiling feature

1. Create collector configuration file:
   • coralogix-otel-config.yaml
2. Upload the file to S3.
3. Deploy CloudFormation.

Use this command when deploying only the base collector:

aws cloudformation deploy \
  --region <aws-region> \
  --template-file template.yaml \
  --stack-name <installation-name> \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides \
    S3ConfigBucket=<config-s3-bucket> \
    S3ConfigKey=<config-s3-object-key> \
    ClusterName=<ecs-cluster-name> \
    CDOTImageVersion=<collector-image-version> \
    CoralogixApiKey=<send-your-data-api-key> \
    CoralogixRegion=<coralogix-region>

ProfilingEnabled defaults to false when not provided.

With profiling

1. Create both configuration files:
   • coralogix-otel-config.yaml
   • coralogix-profiling-config.yaml
2. Upload both files to S3.
3. Deploy CloudFormation with profiling enabled.

Use this command when deploying both the base collector and the profiling one:

aws cloudformation deploy \
  --region <aws-region> \
  --template-file template.yaml \
  --stack-name <installation-name> \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides \
    S3ConfigBucket=<config-s3-bucket> \
    S3ConfigKey=coralogix-otel-config.yaml \
    ProfilingS3ConfigBucket=<profiling-s3-bucket> \
    ProfilingS3ConfigKey=coralogix-profiling-config.yaml \
    ProfilingEnabled=true \
    ClusterName=<ecs-cluster-name> \
    CDOTImageVersion=<collector-image-version> \
    CoralogixApiKey=<send-your-data-api-key> \
    CoralogixRegion=<coralogix-region>

The description for each parameter and its valid values can be found in the CloudFormation template.

Deploy from the Coralogix Integration UI

When deploying with the Coralogix UI (Integrations -> AWS ECS-EC2 -> Deploy), onboarding uses a single CloudFormation stack.

Observability features

• Enable APM & Tracing to configure tracing options.
• Enable Profiling to add the profiling collector deployment flow.

Review YAML file

When profiling is enabled, the Review YAML file step shows two files:

• coralogix-otel-config.yaml (collector configuration)
• coralogix-profiling-config.yaml (profiling configuration)

When profiling is disabled, only coralogix-otel-config.yaml is shown.

CloudFormation tab

The Cloud formation button opens AWS quick-create for the current template revision. Profiling parameters are added only when profiling is enabled.

Deployment code in UI

The Deployment code section provides a CLI command with --template-file template.yaml.

• With profiling enabled, it includes profiling parameters (ProfilingS3ConfigBucket, ProfilingS3ConfigKey, ProfilingEnabled).
• With profiling disabled, profiling parameters are omitted.

Verify deployment

Verify services on your ECS cluster:

ecs-cli ps --region <region> -c <cluster-name>