This tutorial demonstrates how to deploy [OpenTelemetry](https://coralogix.com/docs/opentelemetry/getting-started/index.md) to ECS to facilitate the collection of logs, metrics, traces, and profiles, and send them to Coralogix. ## Image This implementation utilizes the wrapper image `coralogixrepo/coralogix-otel-collector`, based on the official OpenTelemetry contrib image. **Notes**: - The wrapper image is used to dynamically apply the OpenTelemetry configuration at runtime from an environment variable. - The image configuration utilizes the otlp receiver for both HTTP (on 4318) and GRPC (on 4317). Data can be sent using either endpoint. ## Prerequisites - Coralogix [account](https://signup.coralogix.com/#/) - Coralogix [Send Your Data - API Key](https://coralogix.com/docs/user-guides/account-management/api-keys/send-your-data-api-key/index.md) - AWS credentials [configured](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html) - [aws-cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) - [ecs-cli](https://github.com/aws/amazon-ecs-cli#installing) (optional, used in cluster creation and verification examples) - Existing ECS EC2 cluster name - Coralogix ECS-EC2 [CloudFormation template](https://github.com/coralogix/cloudformation-coralogix-aws/tree/master/opentelemetry/ecs-ec2) If you do not already have an ECS cluster, create one first. ## Configuration ### OpenTelemetry The OpenTelemetry configuration for the agent is stored in a Base64-encoded environment variable and is applied at runtime. This allows you to dynamically pass any configuration values you choose as a parameter to CloudFormation. To send logs, metrics, and traces to Coralogix, use this [CloudFormation template](https://github.com/coralogix/cloudformation-coralogix-aws/blob/1bc3e03f12476ba87e38b4243fd611c13943e84d/opentelemetry/ecs-ec2/template.yaml) as a reference configuration. The template includes a default OpenTelemetry (OTel) configuration, which you can override by specifying a value for the CustomConfig parameter. **Notes**: - The [Coralogix Exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/coralogixexporter) enables the use of enrichments such as dynamic `application` or `subsystem` name, defined using `application_name_attributes` and `subsystem_name_attributes`, respectively. ### ECS Cluster Deploy a new ECS cluster. If you already have an existing ECS Cluster, skip this step. **STEP 1**. Deploy a new ECS cluster: ```bash ecs-cli up --region --keypair --cluster --size --capability-iam ``` Note The `--keypair` flag and STEP 2 are not mandatory. However, if no Key Pair is supplied, you will be unable to connect to any of the EC2 instances in the cluster via SSH. **STEP 2**. Create a key pair using the command below: ```bash aws ec2 create-key-pair --key-name MyKeyPair --query 'KeyMaterial' --output text > MyKeyPair.pem ``` **STEP 3**. Control default values. The `ecs-cli up` command will leverage CloudFormation to create an ECS Cluster. Default values will be used to create and configure a VPC and Subnets. These values can also be controlled using: ```bash ecs-cli up --help ``` ## Deploy with CLI ### Without profiling feature 1. Create collector configuration file: - `coralogix-otel-config.yaml` 1. Upload the file to S3. 1. Deploy CloudFormation. Use this command when deploying only the base collector: ```bash aws cloudformation deploy \ --region \ --template-file template.yaml \ --stack-name \ --capabilities CAPABILITY_NAMED_IAM \ --parameter-overrides \ S3ConfigBucket= \ S3ConfigKey= \ ClusterName= \ CDOTImageVersion= \ CoralogixApiKey= \ CoralogixRegion= ``` `ProfilingEnabled` defaults to `false` when not provided. ### With profiling 1. Create both configuration files: - `coralogix-otel-config.yaml` - `coralogix-profiling-config.yaml` 1. Upload both files to S3. 1. Deploy CloudFormation with profiling enabled. Use this command when deploying both the base collector and the profiling one: ```bash aws cloudformation deploy \ --region \ --template-file template.yaml \ --stack-name \ --capabilities CAPABILITY_NAMED_IAM \ --parameter-overrides \ S3ConfigBucket= \ S3ConfigKey=coralogix-otel-config.yaml \ ProfilingS3ConfigBucket= \ ProfilingS3ConfigKey=coralogix-profiling-config.yaml \ ProfilingEnabled=true \ ClusterName= \ CDOTImageVersion= \ CoralogixApiKey= \ CoralogixRegion= ``` The description for each parameter and its valid values can be found in the CloudFormation template. ## Deploy from the Coralogix Integration UI When deploying with the Coralogix UI (`Integrations -> AWS ECS-EC2 -> Deploy`), onboarding uses a single CloudFormation stack. ### Observability features - Enable **APM & Tracing** to configure tracing options. - Enable **Profiling** to add the profiling collector deployment flow. ### Review YAML file When profiling is enabled, the **Review YAML file** step shows two files: - `coralogix-otel-config.yaml` (collector configuration) - `coralogix-profiling-config.yaml` (profiling configuration) When profiling is disabled, only `coralogix-otel-config.yaml` is shown. ### CloudFormation tab The **Cloud formation** button opens AWS quick-create for the current template revision. Profiling parameters are added only when profiling is enabled. ### Deployment code in UI The **Deployment code** section provides a CLI command with `--template-file template.yaml`. - With profiling enabled, it includes profiling parameters (`ProfilingS3ConfigBucket`, `ProfilingS3ConfigKey`, `ProfilingEnabled`). - With profiling disabled, profiling parameters are omitted. ## Verify deployment Verify services on your ECS cluster: ```bash ecs-cli ps --region -c ``` ## **Support** **Need help?** Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up. Feel free to reach out to us **via our in-app chat** or by sending us an email at **[support@coralogix.com](mailto:support@coralogix.com)**.