Skip to main content

Coralogix OpenTelemetry Agent for ECS-EC2. CloudFormation template.

This CloudFormation template deploys an ECS Service and Task Definition for running the OpenTelemetry Collector agent on an ECS cluster. This deployment is able to collect Logs, Metrics and Traces. The template will deploy a daemonset which runs an instance of OpenTelemetry Collector on each node in a cluster.

CloudFormation template to launch the Coralogix Distribution for OpenTelemetry ("CDOT") into an existing ECS cluster. This CDOT deployment is able to collect Logs, Metrics and Traces. CDOT is deployed in the OTEL Agent deployment pattern, as an ECS Daemon Service type, which runs an instance of the OpenTelemetry Collector agent on each node in a cluster.

Template

  • template.yaml - Runs CDOT directly with an S3 configuration or in supervised mode with embedded startup configurations and optional S3 overrides. Collector configs are typically generated by the Coralogix UI or obtained from the integration chart.

Container image

This solution uses the coralogixrepo/coralogix-otel-collector image which is a custom distribution of OpenTelemetry containing custom components developed by Coralogix. The image is available on Docker Hub. See the ECS components reference for details.

The OTEL Collector/agent/daemon image used is the Coralogix Distribution for OpenTelemetry Docker Hub image. It is deployed as a Daemon ECS Task, i.e. one OTEL Collector agent container on each EC2 instance (i.e. ECS container instance) across the cluster.

CDOT extends upon the main OpenTelemetry Collector Contrib project, adding features specifically to enhance integration with AWS ECS, among other improvements.

The OTEL agent is deployed as a Daemon ECS Task and connected using host network mode. OTEL-instrumented application containers that need to send telemetry to the local OTEL agent can lookup the IP address of the CDOT container using a number of methods, making it easier for Application Tasks using awsvpc and bridge network modes to connect with the OTEL agent. OTEL-instrumented application containers should also consider which resource attributes to use as telemetry identifiers.

The CDOT OTEL agent also features enhancements specific to ECS integration. These improvements are proprietary to the Coralogix Distribution for OpenTelemetry.

Logs

The OTEL agent uses a filelog receiver to read the docker logs of all containers on the EC2 host. OTLP is also accepted. Coralogix provides the awsecscontainermetricsd receiver which enables metrics collection of all tasks on the same host. The coralogix exporter forwards telemetry to your configured Coralogix endpoint.

Logs are collected from all containers that log to /var/lib/docker/containers/*/*.log. The container requires privileges to mount the read-only host filesystem path /var/lib/docker/.

Container metrics

Container metrics are collected from all containers running on the ECS cluster. The metrics are collected using the awsecscontainermetricsd receiver. If you do not wish to collect container metrics, comment out or delete the metrics/containermetrics pipeline from the configuration.

OpenTelemetry Collector metrics

The default configuration exposes OpenTelemetry Collector metrics on port 8888 via the path /metrics. The metrics are collected using a prometheus scrape job. These are performance metrics for the OpenTelemetry Collector containers, including records received and processed and submission faults.

Traces

A GRPC(4317) and HTTP(4318) endpoint is exposed for sending traces to the local OTLP endpoint.

By default, traces are sampled at 10% using head sampling — a probabilistic sampler applied at the collection point, before any processing. When enabled, it creates a separate pipeline for sampled traces, reducing trace volume while keeping a representative sample.

The sampling configuration can be adjusted using the following parameters:

  • EnableHeadSampler: Enable/disable head sampling
  • SamplerMode: Choose between proportional, equalizing, or hash_seed sampling modes
  • SamplingPercentage: Set the desired sampling rate (0-100%)

Span metrics

When enabled, the spanmetrics connector generates metrics from traces, providing insights into trace performance and patterns. This feature creates additional metrics pipelines that convert span data into metrics for monitoring and alerting purposes.

Database traces

When enabled, database operation traces are processed separately with dedicated metrics generation. This feature provides specialized monitoring for database operations with optimized bucket configurations and filtering.

Requires

  • An existing ECS cluster
  • AWS CLI (if deploying via CLI)

Parameters

ParameterDescriptionDefault ValueRequired
S3ConfigBucketS3 bucket containing collector and optional Supervisor configurations. Required in collector mode and when either fallback config parameter is set.""collector mode / fallbacks
S3ConfigKeyS3 object key for the collector configuration. Required in collector mode.""collector mode
S3SupervisorConfigKeyOptional S3 object key for the Supervisor configuration.""
ClusterNameThe name of an existing ECS cluster
CDOTImageVersionThe Coralogix OpenTelemetry Collector Image version/tag to use. See available tags here
SupervisorEnabledRun the collectors through the Supervisor (true or false).false
SupervisedImageRepositoryRepository for the supervised CDOT image.cgx.jfrog.io/coralogix-docker-images/coralogix-otel-supervised-cdot
SupervisedImageVersionVersion/tag for the supervised CDOT image.v0.11.0
InitialFallbackConfigsInitial Supervisor fallback configuration URLs for the main collector agent, entered as a comma-separated list.""
ImageThe OpenTelemetry Collector Image to use. If specified, this value will override the CDOTImageVersion parameter and the Coralogix OTEL Collector image.none
MemoryThe amount of memory to allocate to the OpenTelemetry container.
Assigning too much memory can lead to the ECS Service not being deployed. Make sure that values are within the range of what is available on your ECS cluster
2048
CoralogixRegionThe region of your Coralogix AccountAllowed Values:
- EU1
- EU2
- AP1
- AP2
- AP3
- US1
- US2
- US3
CoralogixApiKeyThe Send-Your-Data API key for your Coralogix account.
ProfilingS3ConfigBucketS3 bucket for the profiling configuration. Required in collector mode; optional override in supervised mode.""
ProfilingS3ConfigKeyS3 object key for the profiling configuration. Required in collector mode; optional override in supervised mode.""
ProfilingInitialFallbackConfigsInitial Supervisor fallback configuration URLs for the profiling agent, entered as a comma-separated list.""
ProfilingEnabledEnable separate profiling collector service.false
ProfilingMemoryMemory (MiB) for the profiling task.512
TaskRoleArnOptional ARN of a custom task role. If not provided, a minimal role with S3 read permissions is auto-created.""
HealthCheckEnabledEnable ECS container health check for the OTEL agent container. Requires OTEL collector image version v0.4.2 or later.false
HealthCheckIntervalHealth check interval (seconds)30
HealthCheckTimeoutHealth check timeout (seconds)5
HealthCheckRetriesHealth check retries3
HealthCheckStartPeriodHealth check start period (seconds)10

Deploy the CloudFormation template

Collector mode requires the OpenTelemetry configuration to be stored in S3. The config is typically generated by the Coralogix UI when deploying from the platform, or can be obtained from the integration chart.

aws cloudformation deploy --template-file template.yaml --stack-name <stack_name> \
--region <region> \
--parameter-overrides \
S3ConfigBucket=<your-s3-bucket> \
S3ConfigKey=<path/to/config.yaml> \
ClusterName=<ecs cluster name> \
CDOTImageVersion=<image tag> \
CoralogixApiKey=<your-api-key> \
CoralogixRegion=<coralogix-region> \
SupervisorEnabled=false \
HealthCheckEnabled=true

For supervised mode, omit the S3 parameters to use the embedded Supervisor configuration and the localhost-only nop collector configuration:

aws cloudformation deploy --template-file template.yaml --stack-name <stack_name> \
--region <region> \
--capabilities CAPABILITY_NAMED_IAM \
--parameter-overrides \
ClusterName=<ecs_cluster_name> \
CDOTImageVersion=<image_tag> \
CoralogixApiKey=<send_your_data_api_key> \
CoralogixRegion=<coralogix_region> \
SupervisorEnabled=true

The embedded startup config uses nop receivers and exporters for traces, metrics, and logs. It only exposes the health check on localhost:13133, so it does not send data until the Supervisor receives a remote config.

To override the embedded files in supervised mode, set S3ConfigBucket with S3ConfigKey, S3SupervisorConfigKey, or both. A configured key is copied from S3; an empty key keeps the corresponding embedded config.

InitialFallbackConfigs and ProfilingInitialFallbackConfigs are each a list of strings. In CloudFormation parameter overrides, write multiple URLs as one comma-separated value with no spaces. For example:

s3://<BUCKET>.s3.<REGION>.amazonaws.com/<ACCOUNT_ID>/<GROUP_NAME>/<COLLECTOR_VERSION>/<REMOTE_CONFIG_NAME>/config.yaml,s3://<BUCKET>.s3.<REGION>.amazonaws.com/<ACCOUNT_ID>/<GROUP_NAME>/EMPTY_VERSION/<REMOTE_CONFIG_NAME>/config.yaml

Their defaults are empty, which produces no initial fallback configuration for that agent. These parameters apply only to the embedded Supervisor configuration; an S3-provided Supervisor configuration is used as-is.

  • InitialFallbackConfigs is injected into the main collector Supervisor.
  • ProfilingInitialFallbackConfigs is injected into the profiling Supervisor.

When either is set, S3ConfigBucket is required and every fallback URL must reference that bucket. The generated task role grants read access to that bucket. When TaskRoleArn is provided, that custom role must grant s3:GetObject access to the fallback objects.

Supervised CDOT v0.11.0 or later is required when using initial fallback configuration with ProfilingEnabled=true.

When ProfilingEnabled=true, the profiling collector follows SupervisorEnabled. When the flag is true, it uses the supervised CDOT image and starts with the embedded localhost-only nop config. Set both ProfilingS3ConfigBucket and ProfilingS3ConfigKey to override that startup config from S3. When the flag is false, those S3 parameters are required and the file is run directly by CDOT.

Note that these are just examples of how this could be deployed. You can also deploy this template using the AWS Console or any CloudFormation management tools.

OpenTelemetry configuration

In collector mode, the template loads the OpenTelemetry configuration from S3. Supervised mode starts with the embedded nop config unless an S3 collector key is provided. The template passes these environment variables to the collector:

  • CORALOGIX_DOMAIN – region-specific domain (from CoralogixRegion parameter)
  • CORALOGIX_PRIVATE_KEY – your API key (from CoralogixApiKey parameter)

Configuration source: The canonical config is maintained in the telemetry-shippers integration chart:

Alternatively, use the config generated by the Coralogix UI when deploying from the platform.

The configuration monitors container logs, listens for logs, metrics and traces on port 4317/4318, and collects container metrics using the awsecscontainermetricsd receiver.

Health check

Requires OTEL collector image version v0.4.2 or later. The default config will expose a health check on port 13133 on localhost via the path /. The health check is exposed using the health_check extension.

The healthy response should look like this:

{
"status": "Server available",
"upSince": "2023-10-25T15:37:32.003837622Z",
"uptime": "2m5.2610063s"
}

ECS container health check

You can customize the health check settings using the following parameters:

  • HealthCheckInterval (default: 30)
  • HealthCheckTimeout (default: 5)
  • HealthCheckRetries (default: 3)
  • HealthCheckStartPeriod (default: 10)

Example deployment with custom health check settings:

aws cloudformation deploy --template-file template.yaml --stack-name <stack_name> \
--region <region> \
--parameter-overrides \
S3ConfigBucket=<your-s3-bucket> \
S3ConfigKey=<path/to/config.yaml> \
ClusterName=<ecs cluster name> \
CDOTImageVersion=<image tag> \
CoralogixApiKey=<your-api-key> \
CoralogixRegion=<coralogix-region> \
HealthCheckEnabled=true \
HealthCheckInterval=60 \
HealthCheckTimeout=10 \
HealthCheckRetries=5 \
HealthCheckStartPeriod=20

Further info

See documentation: AWS ECS-EC2 using OpenTelemetry.

Last updated on