Copy as Markdown[Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fcoralogix.com%2Fdocs%2Fintegrations%2Fai-observability%2Fai-apps%2Flitellm.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.)[Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fcoralogix.com%2Fdocs%2Fintegrations%2Fai-observability%2Fai-apps%2Flitellm.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.) # LiteLLM [LiteLLM](https://github.com/BerriAI/litellm) is a proxy and Python SDK that routes calls to 100+ LLM providers behind one OpenAI-compatible API. It emits OpenTelemetry GenAI spans directly through a built-in `otel` callback, so you export straight to Coralogix over OTLP (OpenTelemetry Protocol) and every request flows into AI Center. LiteLLM needs no Coralogix-specific SDK, no separate instrumentation library, and no OpenTelemetry Collector. LiteLLM is a special case in the [Compatibility matrix](https://coralogix.com/docs/user-guides/ai/otel-integration/providers.md): most providers rely on a third-party instrumentation library, while LiteLLM carries native OpenTelemetry support. You opt in to the latest OpenTelemetry GenAI semantic conventions, and its spans carry the `gen_ai.*` attributes AI Center consumes. ## What you need[​](#what-you-need "Direct link to What you need") * A running [LiteLLM proxy](https://docs.litellm.ai/docs/proxy/quick_start) or an application that uses the LiteLLM Python SDK, version `1.86.0` or later. Earlier versions do not emit the latest OpenTelemetry GenAI semantic conventions that [Step 3](#step-3-opt-in-to-the-latest-genai-semantic-conventions) enables. * A Coralogix [Send-Your-Data API key](https://coralogix.com/docs/user-guides/account-management/api-keys/api-keys.md). AI Center processes **only trace data**, not logs, and retrieves it **exclusively from your [S3 archive](https://coralogix.com/docs/user-guides/data-flow/s3-archive/connect-s3-archive.md)**. AI Center ignores data stored in Frequent Search. Route LiteLLM traces to archive storage. ## Configure the integration[​](#configure-the-integration "Direct link to Configure the integration") This page assumes you already run a LiteLLM proxy. Select your Coralogix region using the domain selector at the top of this page — the endpoint in the configuration below updates to match the region you pick. ### Step 1: Enable the OpenTelemetry callback[​](#step-1-enable-the-opentelemetry-callback "Direct link to Step 1: Enable the OpenTelemetry callback") Add the `otel` callback to your proxy's `config.yaml`: ``` litellm_settings: callbacks: ["otel"] ``` If you use the LiteLLM Python SDK directly instead of the proxy, enable the callback in code: ``` import litellm litellm.callbacks = ["otel"] ``` ### Step 2: Send traces directly to Coralogix[​](#step-2-send-traces-directly-to-coralogix "Direct link to Step 2: Send traces directly to Coralogix") LiteLLM reads the standard OpenTelemetry export variables. Point the exporter at the Coralogix OTLP endpoint, pass your [Send-Your-Data API key](https://coralogix.com/docs/user-guides/account-management/api-keys/api-keys.md) as the authorization header, and label the traces with your Coralogix application and subsystem: ``` export OTEL_EXPORTER="otlp_grpc" export OTEL_ENDPOINT="https://ingress.eu2.coralogix.com:443" export OTEL_HEADERS="Authorization=Bearer%20" export OTEL_RESOURCE_ATTRIBUTES="cx.application.name=my-genai-app,cx.subsystem.name=my-service" ``` The space after `Bearer` is URL-encoded as `%20` because `OTEL_HEADERS` parses commas and equals signs as delimiters. `OTEL_EXPORTER` also accepts `otlp_http` — use it with the HTTP endpoint if your environment blocks gRPC. ### Step 3: Opt in to the latest GenAI semantic conventions[​](#step-3-opt-in-to-the-latest-genai-semantic-conventions "Direct link to Step 3: Opt in to the latest GenAI semantic conventions") Set the OpenTelemetry stability opt-in so LiteLLM emits spans that follow the latest GenAI semantic conventions instead of its legacy span format: ``` export OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_latest_experimental" ``` This step is mandatory. The opt-in makes LiteLLM: * Change the LLM-call span name, kind, and structure. * Add the `gen_ai.provider.name` attribute alongside `gen_ai.system`. * Populate additional request and cache-token attributes when present. * Consolidate the per-message events into a single `gen_ai.client.inference.operation.details` event. AI Center relies on these conventions — see [Span attribute inventory](https://coralogix.com/docs/user-guides/ai/otel-integration/span-attributes.md) for the full list. ### Step 4: Capture message content[​](#step-4-capture-message-content "Direct link to Step 4: Capture message content") To capture prompt and completion text on the spans, set the content-capture variable: ``` export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="SPAN_ONLY" ``` `SPAN_ONLY` writes prompts and responses as span attributes, which is what AI Center evaluations read. The variable also accepts `NO_CONTENT` (the default), `EVENT_ONLY`, and `SPAN_AND_EVENT`. With capture on, LiteLLM writes message content — the user prompts and the LLM responses — to your traces, and Coralogix stores it. `NO_CONTENT` keeps that content inside your environment; usage breakdowns by team and agent still work either way, because the identity and token-usage attributes carry no message content — see [Breaking down usage by teams](#breaking-down-usage-by-teams). Pick the value that matches your data-handling policy. Alternative: route through an OpenTelemetry Collector You do not need a collector, but you can run one if you prefer to keep Coralogix credentials out of the LiteLLM process or to batch and retry centrally. Point LiteLLM at the collector instead of Coralogix: ``` export OTEL_EXPORTER="otlp_grpc" export OTEL_ENDPOINT="http://:4317" ``` Configure the collector with the [Coralogix Exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/coralogixexporter). See [OpenTelemetry integration for AI Center](https://coralogix.com/docs/user-guides/ai/otel-integration.md#step-1-deploy-an-opentelemetry-collector-recommended) for a ready-to-use collector config. ## What AI Center receives[​](#what-ai-center-receives "Direct link to What AI Center receives") With the latest GenAI semantic conventions and content capture enabled, each LLM call produces a GenAI span carrying: * `gen_ai.provider.name` — the provider, for example `openai` * `gen_ai.operation.name` = `chat` * `gen_ai.request.model`, `gen_ai.response.model`, `gen_ai.response.id` * `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.usage.total_tokens` * `gen_ai.input.messages`, `gen_ai.output.messages` — the message content * `gen_ai.response.finish_reasons` AI Center detects the span through `gen_ai.provider.name` or `gen_ai.input.messages`. For the complete list of attributes AI Center consumes, see [Span attribute inventory](https://coralogix.com/docs/user-guides/ai/otel-integration/span-attributes.md). ## Validate the integration[​](#validate-the-integration "Direct link to Validate the integration") Confirm GenAI spans are arriving with a DataPrime query: ``` source spans | filter tags['gen_ai.provider.name']:string != null | select $m.traceID, tags['gen_ai.provider.name']:string, tags['gen_ai.request.model']:string, tags['gen_ai.usage.input_tokens']:string, tags['gen_ai.usage.output_tokens']:string | limit 10 ``` ## Breaking down usage by teams[​](#breaking-down-usage-by-teams "Direct link to Breaking down usage by teams") When you run LiteLLM as a proxy, it stamps request-identity attributes onto **every** span at the authentication boundary. Group spans by these attributes in Coralogix to break token usage, cost, and call volume down by team. This is part of LiteLLM's OpenTelemetry v2 tracing — see [LiteLLM request identity on every span](https://docs.litellm.ai/docs/observability/opentelemetry_v2#request-identity-on-every-span). What you need for this breakdown: * LiteLLM running as a proxy. Team identity comes from the proxy's authenticated virtual keys, so the SDK-only path does not carry it. * One or more teams created on the proxy, each with its own virtual key. * OpenTelemetry v2 tracing turned on with `LITELLM_OTEL_V2=true` in the proxy environment. It defaults to off, and only this path stamps the `litellm.*` identity attributes. It is a separate switch from the [Step 3](#step-3-opt-in-to-the-latest-genai-semantic-conventions) opt-in, which selects the GenAI convention shape rather than turning v2 on — set both. ### Identity attributes on every span[​](#identity-attributes-on-every-span "Direct link to Identity attributes on every span") LiteLLM writes these attributes automatically once requests authenticate with a team's virtual key: | Attribute | Value | | --------------------------------------- | ------------------------------- | | `litellm.team.id` | Team ID | | `litellm.team.alias` | Team display name | | `litellm.api_key.hash` | Hash of the calling virtual key | | `litellm.metadata.user_api_key_alias` | Virtual key name | | `litellm.metadata.user_api_key_org_id` | Organization ID | | `litellm.metadata.user_api_key_user_id` | Internal user ID | | `litellm.metadata.requester_ip_address` | Caller IP address | ### Distinguish coding agents[​](#distinguish-coding-agents "Direct link to Distinguish coding agents") When several coding agents — for example, Claude Code, Cursor, or Codex CLI — share one proxy, break usage down by both the team and the agent. Issue a separate virtual key per coding agent within each team, and name each key after the agent. LiteLLM promotes the key name to every span as `litellm.metadata.user_api_key_alias`, so: * `litellm.team.alias` identifies the team. * `litellm.metadata.user_api_key_alias` identifies the coding agent. For example, a Checkout team using Claude Code and Cursor gets two keys — one named `claude-code`, one named `cursor` — both scoped to the Checkout team. This needs no per-request wiring, since identity travels with the key. As an alternative, attach a custom `coding_agent` field to each request's metadata and promote it (see [Capture more identity](#capture-more-identity)). It then appears as `litellm.metadata.coding_agent`. ### Group usage in Coralogix[​](#group-usage-in-coralogix "Direct link to Group usage in Coralogix") Total tokens and call volume per team and coding agent: ``` source spans | filter tags['litellm.team.alias']:string != null | groupby tags['litellm.team.alias']:string as team, tags['litellm.metadata.user_api_key_alias']:string as coding_agent aggregate sum(tags['gen_ai.usage.total_tokens']:number) as total_tokens, count() as calls | sortby total_tokens desc ``` ### Capture more identity[​](#capture-more-identity "Direct link to Capture more identity") Two identity groups stay conservative by default. Turn them on when you need them: * **End-user breakdown** — `litellm.metadata.user_api_key_end_user_id` carries the end-user ID by default. For a dedicated top-level `litellm.end_user.id`, add it to the promoted keys. * **Custom request metadata and team metadata** — promote extra request-metadata sub-keys under `litellm.metadata.*`, and named sub-keys of a team's free-form metadata under `litellm.team.metadata`. A team's metadata list is empty by default, so none of it leaves the process until you name each sub-key. Set these lists in `config.yaml` under `callback_settings.otel`, or through the matching `LITELLM_OTEL_BAGGAGE_PROMOTED_KEYS`, `LITELLM_OTEL_BAGGAGE_METADATA_KEYS`, and `LITELLM_OTEL_BAGGAGE_TEAM_METADATA_KEYS` environment variables (comma-separated). Each list replaces its default, so include the defaults you want to keep: ``` callback_settings: otel: baggage_promoted_keys: - litellm.team.id - litellm.team.alias - litellm.team.metadata - litellm.api_key.hash - gen_ai.request.model - litellm.provider.model - litellm.end_user.id baggage_metadata_keys: - user_api_key_org_id - user_api_key_user_id - user_api_key_alias - user_api_key_end_user_id - requester_ip_address - coding_agent baggage_team_metadata_keys: - cost_center ``` ## Emit and collect metrics[​](#emit-and-collect-metrics "Direct link to Emit and collect metrics") Alongside traces, LiteLLM's OpenTelemetry v2 integration emits GenAI client metrics — histograms for call latency, token usage, and cost — over the same OTLP exporter. Coralogix ingests them as metrics you can chart and alert on. ### Turn on metrics[​](#turn-on-metrics "Direct link to Turn on metrics") Metrics stay off until you turn them on. Set this in the proxy environment: ``` export LITELLM_OTEL_INTEGRATION_ENABLE_METRICS="true" ``` On `litellm` earlier than `1.96.0`, the metrics ship only with OpenTelemetry v2, so also set `LITELLM_OTEL_V2=true`. The metric stream ships through the exporter you already configured for traces — `OTEL_EXPORTER`, `OTEL_ENDPOINT`, and `OTEL_HEADERS` — so the same Coralogix endpoint receives it with no extra configuration. gRPC export needs the `grpcio` package. ### Metrics emitted[​](#metrics-emitted "Direct link to Metrics emitted") Every metric is a histogram. These are OpenTelemetry instrument names, so they use dot notation; Coralogix stores them in Prometheus underscore notation, which is what you type in Metrics Explorer and PromQL. LiteLLM records: | Metric | In Coralogix | Unit | What it measures | | ------------------------------------- | ------------------------------------- | --------- | ------------------------------------------------------------------- | | `gen_ai.client.operation.duration` | `gen_ai_client_operation_duration` | `s` | Wall-clock time for the whole LLM call | | `gen_ai.client.token.usage` | `gen_ai_client_token_usage` | `{token}` | Tokens consumed, split into input and output by `gen_ai.token.type` | | `gen_ai.usage.cost` | `gen_ai_usage_cost` | `USD` | LiteLLM's computed cost for the call | | `gen_ai.server.time_to_first_token` | `gen_ai_server_time_to_first_token` | `s` | Time to the first streamed token, on streaming calls | | `gen_ai.server.time_per_output_token` | `gen_ai_server_time_per_output_token` | `s` | Average time per output token, on streaming calls | | `gen_ai.client.response.duration` | `gen_ai_client_response_duration` | `s` | Provider-side generation time | LiteLLM renamed the cost and streaming-latency instruments in `1.96.0`. On earlier versions they are `gen_ai.client.token.cost`, `gen_ai.client.response.time_to_first_token`, and `gen_ai.client.response.time_per_output_token`. Each sample carries the same identity attributes as the matching span — `gen_ai.operation.name`, `gen_ai.system`, `gen_ai.request.model`, `gen_ai.framework`, and selected `metadata.*` fields such as `metadata.user_api_key_team_id` — so you can group the histograms by model, provider, key, or team. `gen_ai.token.type` always stays on `gen_ai.client.token.usage`, which keeps the input and output split. ### Query metrics in Coralogix[​](#query-metrics-in-coralogix "Direct link to Query metrics in Coralogix") Coralogix ingests these over OTLP as metrics. Open Metrics Explorer or write PromQL to chart latency, token usage, and cost, grouped by model, provider, or team, using the underscore names from the table above. ### Control metric cardinality[​](#control-metric-cardinality "Direct link to Control metric cardinality") High-cardinality attributes multiply the number of metric series. Trim the attributes LiteLLM attaches with the filter under `callback_settings.otel.attributes` in `config.yaml`. List exactly what to keep with `include_list` (LiteLLM drops anything else), or remove specific attributes with `exclude_list`: ``` callback_settings: otel: attributes: include_list: - gen_ai.operation.name - gen_ai.system - gen_ai.request.model - gen_ai.framework - metadata.user_api_key_team_id - metadata.user_api_key_org_id ``` `gen_ai.token.type` is always kept, so the input and output token split survives any filter. For the full reference, see [LiteLLM OpenTelemetry v2 metrics](https://docs.litellm.ai/docs/observability/opentelemetry_v2#metrics). ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") Spans not appearing in AI Center AI Center filters for GenAI spans using `gen_ai.provider.name` or `gen_ai.input.messages`. LiteLLM emits `gen_ai.provider.name` only when you set `OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental` ([Step 3](#step-3-opt-in-to-the-latest-genai-semantic-conventions)). Confirm the proxy environment includes this variable and that traces reach your S3 archive. Missing message content Confirm the proxy environment sets `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY` ([Step 4](#step-4-capture-message-content)). Traces reach Coralogix but not AI Center AI Center reads trace data from your S3 archive, not from Frequent Search. Verify your traces route to archive storage. ## Next steps[​](#next-steps "Direct link to Next steps") Confirm which `gen_ai.*` attributes AI Center consumes in [Span attribute inventory](https://coralogix.com/docs/user-guides/ai/otel-integration/span-attributes.md). ## Related resources[​](#related-resources "Direct link to Related resources") * [OpenTelemetry integration for AI Center](https://coralogix.com/docs/user-guides/ai/otel-integration.md) * [Compatibility matrix](https://coralogix.com/docs/user-guides/ai/otel-integration/providers.md) * [Span attribute inventory](https://coralogix.com/docs/user-guides/ai/otel-integration/span-attributes.md) * [Code examples](https://coralogix.com/docs/user-guides/ai/otel-integration/code-examples.md) * [LiteLLM OpenTelemetry documentation](https://docs.litellm.ai/docs/observability/opentelemetry_integration)