# APM onboarding

This guide provides step-by-step instructions on configuring and using Coralogix Application Performance Monitoring (APM). Follow these steps to effectively onboard your services, install collectors, correlate resources, and leverage advanced features.

Ready to get started?

Start by [deploying the OpenTelemetry Collector](#deploy-the-opentelemetry-collector) and [instrumenting your services](#generate-metrics-and-spans).

## Overview

To begin monitoring your applications efficiently, follow these steps:

1. Deploy the OpenTelemetry Collector:
   - Install and configure the OpenTelemetry Collector to automatically collect spans and generate Span Metrics for comprehensive monitoring and performance tracking.
   - If needed, enable Span Metrics. By default, Span Metrics are enabled for efficient and scalable APM monitoring.
1. Generate metrics and spans. Use OpenTelemetry instrumentation or the [eBPF auto-instrumentation](https://coralogix.com/docs/opentelemetry/instrumentation-options/ebpf-auto-instrumentation/overview/index.md) to send traces and metrics from your services.
1. Optimize with head sampling for improved performance and data efficiency (10% by default).
1. Correlate logs and infrastructure resource data with your services.
1. If needed, configure serverless instrumentation. Learn how to send your serverless functions data to Coralogix.

## Deploy the OpenTelemetry Collector

### Deployment methods

Select from any of the following configuration options:

- [Docker](https://coralogix.com/docs/opentelemetry/configuration-options/opentelemetry-using-docker/index.md)

- [EC2](https://coralogix.com/docs/opentelemetry/configuration-options/install-opentelemetry-on-an-ec2-instance/index.md)

- [ECS-Fargate](https://coralogix.com/docs/integrations/aws/opentelemetry-ecs-fargate/index.md)

- [ECS-EC2](https://coralogix.com/docs/opentelemetry/configuration-options/aws-ecs-ec2-using-opentelemetry/index.md)

- Kubernetes Complete Observability:

  1. Download the [`values.yaml`](https://github.com/coralogix/telemetry-shippers/blob/master/otel-integration/k8s-helm/values.yaml) file (for users starting from scratch).

  1. Make sure all [K8s and Helm prerequisites](https://github.com/coralogix/telemetry-shippers/tree/master/otel-integration/k8s-helm#prerequisites) (minimum Kubernetes and Helm versions, secrets setup) are met.

  1. Make sure that the `spanMetrics` preset is enabled.

     ```yaml
      spanMetrics:
           enabled: true
     ```

  1. Use Helm installation commands detailed in the [installation guide](https://github.com/coralogix/telemetry-shippers/tree/master/otel-integration/k8s-helm#installation).

### Enable Span Metrics

By default, Span Metrics are enabled in the [latest Helm chart](https://github.com/coralogix/telemetry-shippers/blob/master/otel-integration/k8s-helm/Chart.yaml). However, you can add Span Metrics to your [configuration file](https://coralogix.com/docs/user-guides/apm/getting-started/span-metrics/#creating-span-metrics-using-your-own-opentelemetry) or enable it in your [values.yaml](https://coralogix.com/docs/user-guides/apm/getting-started/span-metrics/#creating-span-metrics-with-the-kubernetes-extension-for-otel). Refer to [Coralogix Span Metrics](https://coralogix.com/docs/user-guides/apm/getting-started/span-metrics/index.md) for details.

When Span Metrics are enabled, the following metrics are generated:

| Metric name (OTLP) | Prometheus exported name                                 | Metric type | Auto-generated labels (dimensions)                                         |
| ------------------ | -------------------------------------------------------- | ----------- | -------------------------------------------------------------------------- |
| `requests`         | `calls_total`                                            | Counter     | `service.name`, `span.name`, `span.kind`, `status.code`                    |
| `duration`         | `duration_ms_bucket, duration_ms_sum, duration_ms_count` | Histogram   | `service.name`, `span.name`, `span.kind`, `status.code`, `le` (for bucket) |
| `errors`           | `errors_total`                                           | Counter     | `service.name`, `span.name`, `span.kind`, `status.code`                    |

**Additional dimensions introduced by Coralogix**

When using the Coralogix `values.yaml`, the following dimensions are added to enable APM functionalities and enhance data analysis:

- Dimensions added by default as part of Coralogix `values.yaml`.

  ```yaml
  extraDimensions:
          - name: http.method
          - name: cgx.transaction
          - name: cgx.transaction.root
          - name: status_code
          - name: db.namespace
          - name: db.operation.name
          - name: db.collection.name
          - name: db.system
  ```

- Once you enable the Span Metrics preset, the `errorTracking` configuration will be enabled automatically. Learn more about [API error tracking](https://coralogix.com/docs/user-guides/apm/features/api-error-tracking/index.md).

- Once you enable the Span Metrics preset, the `serviceVersion` configuration will be enabled automatically. Learn more about [Grouping service metrics by version](https://coralogix.com/docs/user-guides/apm/features/group-by-service-version/index.md).

  - Make sure `service.version` attribute [is added](https://coralogix.com/docs/user-guides/apm/features/group-by-service-version/#adding-the-serviceversion-attribute-using-otelresourceattributes) to your service spans.

- Once you enable the Span Metrics preset, the `dbMetrics` configuration will be enabled automatically. It generates RED (Request, Errors, Duration) metrics for database spans. For example, `query db_calls_total` is used to view generated request metrics. This is needed to enable the [Database Monitoring](https://coralogix.com/docs/user-guides/apm/features/database-monitoring/index.md) feature inside Coralogix APM.

### Transform statements

OpenTelemetry sometimes introduces breaking changes that, if left unaddressed, can disrupt the experience for customers upgrading to newer versions. Coralogix APM and the OpenTelemetry Collector (deployed via the Coralogix Helm chart) support a `transform` processor to rename attributes before exporting them. Refer to [Aligning Coralogix and OTel Naming Conventions](https://coralogix.com/docs/user-guides/apm/getting-started/aligning-coralogix-and-otel-naming-conventions/index.md) for details.

### Optimize high cardinality in metrics and spans

High cardinality (over 300K), which occurs when metrics or spans contain labels with numerous unique values, such as user IDs, UUIDs, or session-specific data. This creates a large number of metric combinations, often exceeding practical limits. For example, using user-specific values in span names or labels can lead to exponentially growing cardinality, complicating metric analysis and visualization. In cases of high cardinality caused by overly unique span names, we recommend adjusting your instrumentation or using the `spanMetrics.spanNameReplacePattern` parameter to replace the problematic values with a generic placeholder. For example, if your span name corresponds to template `user-1234`, you can use the following pattern to replace the user ID with a generic placeholder. This will result in your spans having a generalized name `user-{id}`. Learn how to replace specific `span.name` with a generic one as detailed [here](https://github.com/coralogix/telemetry-shippers/tree/master/otel-integration/k8s-helm#about-span-metrics).

Use [`metricsExpiration`](https://github.com/coralogix/telemetry-shippers/blob/7ffd8bec5e63e92bd167e2e2435e05a0d5a3bb02/otel-integration/k8s-helm/values.yaml#L139) when you want to control how long unexported metrics are kept in memory.

## Generate metrics and spans

### OpenTelemetry instrumentation

Instrument your services to generate spans. Select your programming language:

- [Manual instrumentation](https://opentelemetry.io/docs/languages/).
- [Auto instrumentation](https://opentelemetry.io/docs/zero-code/).

### Quick enablement for Kubernetes using eBPF

Auto-instrument your applications using [eBPF with OBI](https://coralogix.com/docs/opentelemetry/instrumentation-options/ebpf-auto-instrumentation/overview/index.md).

## Optimize with head or tail sampling (recommended)

By default, our [latest Helm chart](https://github.com/coralogix/telemetry-shippers/tree/master/otel-integration/k8s-helm) includes head sampling of traces. Use one of these methods for upgrading to tail sampling:

- **Kubernetes**: [Tail sampling with OpenTelemetry using Coralogix Helm chart](https://coralogix.com/docs/opentelemetry/kubernetes-observability/tail-sampling-with-opentelemetry-using-kubernetes/index.md).
- **Docker Compose**: [Tail sampling with OpenTelemetry using Docker Compose](https://coralogix.com/docs/opentelemetry/tail-sampling/tail-sampling-with-opentelemetry-using-docker-compose/index.md).

## Log correlation

Investigate your service logs by enriching them with a [subsystem name](https://coralogix.com/docs/user-guides/account-management/account-settings/application-and-subsystem-names/index.md) that matches your service name. Alternatively, follow [these instructions](https://coralogix.com/docs/user-guides/apm/features/service-catalog/#logs) to set up your log correlation.

## Instrument serverless functions

Auto-instrument AWS Lambda by following [these instructions](https://coralogix.com/docs/opentelemetry/instrumentation-options/opentelemetry-lambda-auto-instrumentation/index.md).

## Next steps

Once your services are sending traces, learn how Coralogix turns them into long-term metrics in the [Span Metrics](https://coralogix.com/docs/user-guides/apm/getting-started/span-metrics/index.md) overview.

## Additional resources

|                      |                                                                                                                                       |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| GitHub documentation | [Coralogix OTel integration for K8s](https://github.com/coralogix/telemetry-shippers/blob/master/otel-integration/k8s-helm/README.md) |
| Tutorial             | [OpenTelemetry and Coralogix introduction](https://coralogix.com/academy/get-to-know-coralogix/otel-coralogix-intro/)                 |
| Tutorial             | [OpenTelemetry Lambda function integration](https://coralogix.com/academy/get-to-know-coralogix/otel-lambda-function-integration/)    |
| Tutorial             | [ECS to Coralogix using OpenTelemetry](https://coralogix.com/academy/get-to-know-coralogix/ecs-to-coralogix-using-opentelemetry/)     |

Introduction to APM