AWS ECS-EC2 using OpenTelemetry

This tutorial demonstrates how to deploy OpenTelemetry to AWS ECS-EC2 to facilitate the collection of logs, metrics, and traces.

Telemetry is sent to Coralogix via the Coralogix Exporter, which allows for the use of enrichments such as dynamic application or subsystem name, defined using application_name_attributes and subsystem_name_attributes, respectively. Find out more here.

Prerequisites

• Coralogix account
• AWS account with AWS credentials configured
• ecs-cli
• aws-cli
• Coralogix Otel ECS agent CloudFormation template

Image

This implementation utilises the Coralogix Opentelemetry Collector image
coralogixrepo/coralogix-otel-collector. This image serves as an enhanced version of the official OpenTelemetry Contrib distribution, featuring custom components and extended configuration loading options, including environment variable support.

Note the latest tag on this image is not updated, you must explicitly select a version when pulling. Tags can be found here.

The image configuration utilizes the otlp receiver for both HTTP (on 4318) and GRPC (on 4317). Data can be sent using either endpoint.

OpenTelemetry Configuration

The OpenTelemetry configuration for the agent is stored in a Base64 encoded environment variable and applied at runtime. This allows you to dynamically pass any configuration values you choose as a parameter to CloudFormation.

The following configuration files work directly with the coralogixrepo/otel-coralogix-ecs-wrapper docker image for ECS:

• logs
• metrics & traces

Create other configurations by combining logs, metrics and/or traces.

Deploy a New ECS Cluster

If you already have an existing ECS cluster, skip this step.

Deploy a new cluster:

ecs-cli up --region <region> --keypair <your-key-pair> --cluster <cluster-name> --size <no. of instances> --capability-iam 

Notes:

• The --keypair flag is not mandatory. However, if not supplied, you will not be able to connect to any of the instances in the cluster via SSH. Create a key pair using the command below:

aws ec2 create-key-pair --key-name MyKeyPair --query 'KeyMaterial' --output text > MyKeyPair.pem

• The ecs-cli up command will leverage CloudFormation to create an ECS cluster.
• Default values will be used to create and configure a VPC and Subnets. These and other values can be controlled from ecs-cli using the following command:

ecs-cli up --help

ECS Task Definition & Service

STEP 1. Once an ECS cluster has been deployed, deploy a task definition to be used by ECS to create an ECS service to run OpenTelemetry.

• An AWS ECS task definition serves as template defining a container configuration.
• An ECS service is a configuration item that defines and orchestrates how a task definition should be run.

STEP 2. Deploy this CloudFormation template, with necessary parameters provided:

• ClusterName: Name of an existing ECS Cluster
• CoralogixRegion: Coralogix Region associated with your Coralogix account
• ApplicationName & SubsystemName: Application and subsystem names as they will appear in your Coralogix dashboard
• PrivateKey: Your Coralogix Send Your Data – API Key

STEP 3. Once the template is deployed successfully, verify that the container is running:

ecs-cli ps --region <region> -c <cluster name>

Additional Resources

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].