# AWS EKS Fargate Copy as Markdown[Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fcoralogix.com%2Fdocs%2Fintegrations%2Faws%2Faws-eks-fargate.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.)[Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fcoralogix.com%2Fdocs%2Fintegrations%2Faws%2Faws-eks-fargate.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.) Integrate Coralogix seamlessly with **Amazon EKS on AWS Fargate** to effortlessly collect, analyze, and visualize **logs, metrics, and traces** from your containerized applications, empowering you with comprehensive full-stack monitoring and insights. ## Overview[​](#overview "Direct link to Overview") The **Coralogix AWS EKS Fargate** integration has two independent parts: metrics and traces via OpenTelemetry, and logs via the AWS log\_router framework. Each integration can be deployed separately from the other. For the upstream source and manifest reference, see the [OTEL EKS Fargate](https://coralogix.com/docs/docs/external/telemetry-shippers/otel-eks-fargate) README. ## Metrics & Traces[​](#metrics--traces "Direct link to Metrics & Traces") This integration leverages the OpenTelemetry Collector (Contrib) to collect your Fargate workload pod and container metrics. It does so by querying the Kubelet stats API running on each node. As Fargate hosts are managed by AWS, node metrics are not available. ### Prerequisites[​](#prerequisites "Direct link to Prerequisites") * `cx-eks-fargate-otel` namespace declared in your EKS cluster Note To run workloads on Fargate, this namespace must be linked to an AWS Fargate profile. If no profile exists, you must create one. * A secret containing your Coralogix [Send-Your-Data API key](https://coralogix.com/docs/docs/user-guides/account-management/api-keys/send-your-data-api-key/.md) in the `cx-eks-fargate-otel` namespace ### Create a Secret[​](#create-a-secret "Direct link to Create a Secret") Note This integration uses `PRIVATE_KEY` as the environment variable name for the Send-Your-Data API key. Note that the [OpenTelemetry ECS Fargate](https://coralogix.com/docs/docs/integrations/aws/opentelemetry-ecs-fargate/.md) integration uses `CORALOGIX_PRIVATE_KEY` instead — take care not to mix them up. **STEP 1**. Export your API key to a local variable. ``` export PRIVATE_KEY= ``` **STEP 2.** Set your namespace variable. ``` export NAMESPACE=cx-eks-fargate-otel ``` **STEP 3**. Create the secret using kubectl. ``` kubectl create secret generic coralogix-keys -n $NAMESPACE --from-literal=PRIVATE_KEY=$PRIVATE_KEY ``` **STEP 4**. Validate the newly-created secret. ``` kubectl get secret coralogix-keys -o yaml -n $NAMESPACE ``` ### Create a ServiceAccount[​](#create-a-serviceaccount "Direct link to Create a ServiceAccount") In order for the OpenTelemetry Collector to get full access to the Kubernetes API, it will need to bind to a ServiceAccount. Create the service account by running the following bash script. Set the CLUSTER\_NAME and REGION accordingly, with the remainder unchanged. ``` ##!/bin/bash CLUSTER_NAME= REGION= SERVICE_ACCOUNT_NAMESPACE=cx-eks-fargate-otel SERVICE_ACCOUNT_NAME=cx-otel-collector SERVICE_ACCOUNT_IAM_ROLE=EKS-Fargate-cx-OTEL-ServiceAccount-Role SERVICE_ACCOUNT_IAM_POLICY=arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy eksctl utils associate-iam-oidc-provider \\ --cluster=$CLUSTER_NAME \\ --approve eksctl create iamserviceaccount \\ --cluster=$CLUSTER_NAME \\ --region=$REGION \\ --name=$SERVICE_ACCOUNT_NAME \\ --namespace=$SERVICE_ACCOUNT_NAMESPACE \\ --role-name=$SERVICE_ACCOUNT_IAM_ROLE \\ --attach-policy-arn=$SERVICE_ACCOUNT_IAM_POLICY \\ --approve ``` ### Configure and Deploy OTEL Collector Service[​](#configure-and-deploy-otel-collector-service "Direct link to Configure and Deploy OTEL Collector Service") The attached yaml manifest will deploy the following: * an OpenTelemetry Collector, * a clusterIP service for submission of application traces and metrics, and * cluster permissions required to query the Kubernetes API. **STEP 1**. Set the container environment variables detailed at the top of the [yaml file](https://github.com/coralogix/telemetry-shippers/blob/master/otel-eks-fargate/cx-eks-fargate-otel.yaml). **STEP 2**. Once you’ve adjusted the manifests appropriately, deploy using the kubectl apply command. ``` kubectl apply -f cx-eks-fargate-otel.yaml ``` **Notes**: * This manifest is all that is required to collect metrics from your EKS Fargate cluster and process application metrics and traces from gRPC sources. * The OTLP gRPC endpoint is: `http://cx-otel-collector-service.cx-eks-fargate-otel.svc.cluster.local:4317`. ### Configure and Deploy the Self Monitoring Pod[​](#configure-and-deploy-the-self-monitoring-pod "Direct link to Configure and Deploy the Self Monitoring Pod") Since Fargate workloads are unable to directly communicate with their host due to networking restrictions, we cannot monitor the OTEL collector pod’s performance directly. Instead, we have constructed a secondary manifest that’ll deploy a second OTEL collector to collect just these missing pod metrics. [This manifest](https://github.com/coralogix/telemetry-shippers/blob/master/otel-eks-fargate/cx-eks-fargate-otel-self-monitoring.yaml) also has some required environment variables that need to be set, which detailed at the top. Again, after setting the environment variables, deploy using kubectl apply: ``` kubectl apply -f cx-eks-fargate-otel-self-monitoring.yaml ``` ## Logs[​](#logs "Direct link to Logs") For EKS Fargate logs, leverage the AWS log\_router built into the Fargate Kubelet to route your application logs to the Coralogix platform through a Amazon Data Firehose. ### Prerequisites[​](#prerequisites-1 "Direct link to Prerequisites") * Amazon Data Firehose [configured](https://coralogix.com/docs/docs/integrations/aws/amazon-data-firehose/send-logs-using-amazon-data-firehose/.md) \[**Note**: By following our documentation, you will have configured Firehose to use with your log\_router. Though there are other exporters available for the log\_router (fluent-bit), it is restricted to ElasticSearch, Firehose and Cloudwatch. We **recommend** Amazon Data Firehose as it limits added cost while allowing direct submission to the Coralogix Platform.] * Proper permissions added to each of your Fargate profile pod execution roles * Deployment of the log\_router configuration ### Fargate Profile Permissions[​](#fargate-profile-permissions "Direct link to Fargate Profile Permissions") For the sidecar fluent-bit pod to submit messages to Amazon Data Firehose, it requires put and put batch permissions. This is accomplished by adding the following permissions to every Fargate profile that you wish to monitor for application logs. ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "firehose:PutRecord", "firehose:PutRecordBatch" ], "Resource": [ "" ] } ] } ``` ### log\_router Deployment[​](#log_router-deployment "Direct link to log_router Deployment") To deploy the log\_router, you’ll need a specific namespace with appropriate labels configured in your cluster. You won’t be deploying any workloads into this namespace, so you don’t need a Fargate Profile configured for it. Though the configuration will look like a standard fluent-bit configuration, only specific sections can be modified and only certain modules can be used. Below is our **recommended** Kubernetes manifest which will deploy the namespace and the fluent-bit ConfigMap. ``` --- kind: Namespace apiVersion: v1 metadata: name: aws-observability labels: aws-observability: enabled --- kind: ConfigMap apiVersion: v1 metadata: name: aws-logging namespace: aws-observability data: filters.conf: | [FILTER] Name parser Match * Key_name log Parser crio [FILTER] Name kubernetes Match kube.* Merge_Log On Buffer_Size 0 Kube_Meta_Cache_TTL 300s Keep_Log Off Merge_Log_Key log_obj K8S-Logging.Parser On K8S-Logging.Exclude On Annotations Off output.conf: | [OUTPUT] Name kinesis_firehose Match * region (no quotes) delivery_stream (no quotes) parsers.conf: | [PARSER] Name crio Format Regex Regex ^(?