# OpenTelemetry ECS fargate ## Overview This tutorial demonstrates how to add the OTEL Collector as a sidecar agent to your ECS Task Definitions. We use the standard OpenTelemetry Collector Contrib distribution, leveraging the `envprovider` to generate the configuration from an AWS SSM Parameter Store. You can review an example CloudFormation template [here](https://github.com/coralogix/cloudformation-coralogix-aws/tree/master/aws-integrations/ecs-fargate). ## Using `envprovider` for configuration The `envprovider` is utilized to load the OpenTelemetry configuration via S3. This allows for more dynamic and convenient configuration adjustments compared to embedding a static configuration within the container image. The [config.yaml](https://github.com/coralogix/telemetry-shippers/blob/master/otel-ecs-fargate/config.yaml) ffile includes an example configuration that'll ensure proper ingestion of logs, metrics and traces by our backend. Upload this configuration to an S3 bucket accessible by your ecs services. ## Adding the OTEL Collector to ECS task definitions Once the configuration has been uploaded to S3, you'll need to add the OTEL container to your existing Task Definition(s). In the container definition, make sure to adjust the command, CORALOGIX_PRIVATE_KEY and CORALOGIX_DOMAIN. The s3: reference needs to be the Object URI of the S3 object, but replace the https: prefix with s3: ### Example container declaration Below is an example of how to declare the OTEL collector container within your Task Definition: ```json "containerDefinitions": [ { }, { "name": "otel-collector", "image": "otel/opentelemetry-collector-contrib", "cpu": 0, "portMappings": [ { "name": "otel-collector-4317-tcp", "containerPort": 4317, "hostPort": 4317, "protocol": "tcp", "appProtocol": "grpc" }, { "name": "otel-collector-4318-tcp", "containerPort": 4318, "hostPort": 4318, "protocol": "tcp", } ], "essential": false, "command": [ "--config", "s3://example-bucket.s3.us-east-1.amazonaws.com/example_config.yaml" ], "environment": [ { "name": "CORALOGIX_PRIVATE_KEY", "value": "" }, { "name": "CORALOGIX_DOMAIN", "value": "" } ], "mountPoints": [], "volumesFrom": [], "user": "0", "logConfiguration": { "logDriver": "awsfirelens", "options": { "Name": "otel-collector" } }, "systemControls": [], "firelensConfiguration": { "type": "fluentbit" } } ] ``` ### Configuring log forwarding Make sure you set all your existing containers' logConfiguration to the same. ```json "logConfiguration": { "logDriver": "awsfirelens", "options": { "Name": "otel-collector" } } ``` ### Alternative log drivers (CloudWatch) If you don't want specific container logs to be sent to Coralogix, you can configure the log driver for that container to CloudWatch or any other preferred destination. Here's an example for CloudWatch: ```json "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-create-group": "true", "awslogs-group": "", "awslogs-region": "", "awslogs-stream-prefix": "" } } ``` ## Configuration Details ### Resource Catalog The Coralogix Resource Catalog can be used to visualize and track your ECS containers. It collects container details and lets you observe performance metrics and review logs of the associated containers. Resource Catalog data collection is enabled by default. It is composed of the following components: - transform/entity-event processor - coralogix/resource_catalog exporter - logs/resource_catalog pipeline under service To disable the Resource Catalog, comment out or delete the `logs/resource_catalog` pipeline under the `service` section in the configuration file. ### Using Secrets Manager for your private key If you prefer to store your Coralogix private key in AWS Secrets Manager, remove the `"PRIVATE_KEY"` config from the `"environment"` section and instead add it to `"secrets"`, referencing the Secret's ARN. ```json "secrets": [ { "name": "SSM_CONFIG", "valueFrom": "CX_OTEL_ECS_Fargate_config.yaml" }, { "name": "PRIVATE_KEY", "valueFrom": "arn:aws:secretsmanager:region:aws_account_id:secret:secret_name-AbCdEf" } ], ``` Create the Secret as "Plaintext" with only the API key with no quotation marks. You will also need to add the `secretsmanager:GetSecretValue` permission to your ECS Task Execution Role. ## Permissions ### Granting permissions for parameter store access To allow your container to access the Systems Manager Parameter Store, you need to provide `ssm:GetParameters` action permission to the ECS Task Execution Role. Here’s an example of the required permissions: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ssm:GetParameters" ], "Resource": [ "arn:aws:ssm:region:aws_account_id:parameter/parameter_name" ] } ] } ``` ### Granting permissions for Secrets Manager Secret access To allow your container to access the Secrets Manager Secret, you need to provide `secretsmanager:GetSecretValue` action permission to the ECS Task Execution Role. Here’s an example of the required permissions: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "secretsmanager:GetSecretValue" ], "Resource": [ "arn:aws:secretsmanager:region:aws_account_id:secret:secret_name-AbCdEf" ] } ] } ``` ## Submitting metrics and traces After adding the OTEL container to your Task Definition, your applications can submit traces and metrics to the following endpoints: - **Traces**: `http://localhost:4318/v1/traces` - **Metrics**: `http://localhost:4318/v1/metrics` The OTEL collector will also automatically gather container metrics from all containers within the Task Definition. ______________________________________________________________________ This guide outlines the steps to integrate the OTEL collector into your ECS Fargate setup, helping you efficiently collect telemetry data from your applications. ## Support **Need help?** Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up. Feel free to reach out to us **via our in-app chat** or by sending us an email to [support@coralogix.com](mailto:support@coralogix.com).