Open in ChatGPTOpen in Claude

LiteLLM

LiteLLM is a proxy and Python SDK that routes calls to 100+ LLM providers behind one OpenAI-compatible API. It emits OpenTelemetry GenAI spans directly through a built-in otel callback, so you export straight to Coralogix over OTLP (OpenTelemetry Protocol) and every request flows into AI Center. LiteLLM needs no Coralogix-specific SDK, no separate instrumentation library, and no OpenTelemetry Collector.

LiteLLM is a special case in the Compatibility matrix: most providers rely on a third-party instrumentation library, while LiteLLM carries native OpenTelemetry support. You opt in to the latest OpenTelemetry GenAI semantic conventions, and its spans carry the gen_ai.* attributes AI Center consumes.

What you need

• A running LiteLLM proxy or an application that uses the LiteLLM Python SDK, version 1.86.0 or later. Earlier versions do not emit the latest OpenTelemetry GenAI semantic conventions that Step 3 enables.
• A Coralogix Send-Your-Data API key.

AI Center processes only trace data, not logs, and retrieves it exclusively from your S3 archive. AI Center ignores data stored in Frequent Search. Route LiteLLM traces to archive storage.

Configure the integration

This page assumes you already run a LiteLLM proxy. Select your Coralogix region using the domain selector at the top of this page — the endpoint in the configuration below updates to match the region you pick.

Step 1: Enable the OpenTelemetry callback

Add the otel callback to your proxy's config.yaml:

litellm_settings:
  callbacks: ["otel"]

If you use the LiteLLM Python SDK directly instead of the proxy, enable the callback in code:

import litellm

litellm.callbacks = ["otel"]

Step 2: Send traces directly to Coralogix

LiteLLM reads the standard OpenTelemetry export variables. Point the exporter at the Coralogix OTLP endpoint, pass your Send-Your-Data API key as the authorization header, and label the traces with your Coralogix application and subsystem:

export OTEL_EXPORTER="otlp_grpc"
export OTEL_ENDPOINT="https://ingress.eu2.coralogix.com:443"
export OTEL_HEADERS="Authorization=Bearer%20<your-api-key>"
export OTEL_RESOURCE_ATTRIBUTES="cx.application.name=my-genai-app,cx.subsystem.name=my-service"

The space after Bearer is URL-encoded as %20 because OTEL_HEADERS parses commas and equals signs as delimiters. OTEL_EXPORTER also accepts otlp_http — use it with the HTTP endpoint if your environment blocks gRPC.

Step 3: Opt in to the latest GenAI semantic conventions

Set the OpenTelemetry stability opt-in so LiteLLM emits spans that follow the latest GenAI semantic conventions instead of its legacy span format:

export OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_latest_experimental"

This step is mandatory. The opt-in makes LiteLLM:

• Change the LLM-call span name, kind, and structure.
• Add the gen_ai.provider.name attribute alongside gen_ai.system.
• Populate additional request and cache-token attributes when present.
• Consolidate the per-message events into a single gen_ai.client.inference.operation.details event.

AI Center relies on these conventions — see Span attribute inventory for the full list.

Step 4: Capture message content

To capture prompt and completion text on the spans, set the content-capture variable:

export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="SPAN_ONLY"

SPAN_ONLY writes prompts and responses as span attributes, which is what AI Center evaluations read. The variable also accepts NO_CONTENT (the default), EVENT_ONLY, and SPAN_AND_EVENT.

With capture on, LiteLLM writes message content — the user prompts and the LLM responses — to your traces, and Coralogix stores it. NO_CONTENT keeps that content inside your environment; usage breakdowns by team and agent still work either way, because the identity and token-usage attributes carry no message content — see Breaking down usage by teams. Pick the value that matches your data-handling policy.

What AI Center receives

With the latest GenAI semantic conventions and content capture enabled, each LLM call produces a GenAI span carrying:

• gen_ai.provider.name — the provider, for example openai
• gen_ai.operation.name = chat
• gen_ai.request.model, gen_ai.response.model, gen_ai.response.id
• gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.usage.total_tokens
• gen_ai.input.messages, gen_ai.output.messages — the message content
• gen_ai.response.finish_reasons

AI Center detects the span through gen_ai.provider.name or gen_ai.input.messages. For the complete list of attributes AI Center consumes, see Span attribute inventory.

Validate the integration

Confirm GenAI spans are arriving with a DataPrime query:

source spans
| filter tags['gen_ai.provider.name']:string != null
| select $m.traceID,
         tags['gen_ai.provider.name']:string,
         tags['gen_ai.request.model']:string,
         tags['gen_ai.usage.input_tokens']:string,
         tags['gen_ai.usage.output_tokens']:string
| limit 10

Breaking down usage by teams

When you run LiteLLM as a proxy, it stamps request-identity attributes onto every span at the authentication boundary. Group spans by these attributes in Coralogix to break token usage, cost, and call volume down by team. This is part of LiteLLM's OpenTelemetry v2 tracing — see LiteLLM request identity on every span.

What you need for this breakdown:

• LiteLLM running as a proxy. Team identity comes from the proxy's authenticated virtual keys, so the SDK-only path does not carry it.
• One or more teams created on the proxy, each with its own virtual key.
• OpenTelemetry v2 tracing turned on with LITELLM_OTEL_V2=true in the proxy environment. It defaults to off, and only this path stamps the litellm.* identity attributes. It is a separate switch from the Step 3 opt-in, which selects the GenAI convention shape rather than turning v2 on — set both.

Identity attributes on every span

LiteLLM writes these attributes automatically once requests authenticate with a team's virtual key:

Attribute	Value
litellm.team.id	Team ID
litellm.team.alias	Team display name
litellm.api_key.hash	Hash of the calling virtual key
litellm.metadata.user_api_key_alias	Virtual key name
litellm.metadata.user_api_key_org_id	Organization ID
litellm.metadata.user_api_key_user_id	Internal user ID
litellm.metadata.requester_ip_address	Caller IP address

Distinguish coding agents

When several coding agents — for example, Claude Code, Cursor, or Codex CLI — share one proxy, break usage down by both the team and the agent. Issue a separate virtual key per coding agent within each team, and name each key after the agent. LiteLLM promotes the key name to every span as litellm.metadata.user_api_key_alias, so:

• litellm.team.alias identifies the team.
• litellm.metadata.user_api_key_alias identifies the coding agent.

For example, a Checkout team using Claude Code and Cursor gets two keys — one named claude-code, one named cursor — both scoped to the Checkout team. This needs no per-request wiring, since identity travels with the key.

As an alternative, attach a custom coding_agent field to each request's metadata and promote it (see Capture more identity). It then appears as litellm.metadata.coding_agent.

Group usage in Coralogix

Total tokens and call volume per team and coding agent:

source spans
| filter tags['litellm.team.alias']:string != null
| groupby tags['litellm.team.alias']:string as team,
          tags['litellm.metadata.user_api_key_alias']:string as coding_agent
  aggregate sum(tags['gen_ai.usage.total_tokens']:number) as total_tokens,
            count() as calls
| sortby total_tokens desc

Capture more identity

Two identity groups stay conservative by default. Turn them on when you need them:

• End-user breakdown — litellm.metadata.user_api_key_end_user_id carries the end-user ID by default. For a dedicated top-level litellm.end_user.id, add it to the promoted keys.
• Custom request metadata and team metadata — promote extra request-metadata sub-keys under litellm.metadata.*, and named sub-keys of a team's free-form metadata under litellm.team.metadata. A team's metadata list is empty by default, so none of it leaves the process until you name each sub-key.

Set these lists in config.yaml under callback_settings.otel, or through the matching LITELLM_OTEL_BAGGAGE_PROMOTED_KEYS, LITELLM_OTEL_BAGGAGE_METADATA_KEYS, and LITELLM_OTEL_BAGGAGE_TEAM_METADATA_KEYS environment variables (comma-separated). Each list replaces its default, so include the defaults you want to keep:

callback_settings:
  otel:
    baggage_promoted_keys:
      - litellm.team.id
      - litellm.team.alias
      - litellm.team.metadata
      - litellm.api_key.hash
      - gen_ai.request.model
      - litellm.provider.model
      - litellm.end_user.id
    baggage_metadata_keys:
      - user_api_key_org_id
      - user_api_key_user_id
      - user_api_key_alias
      - user_api_key_end_user_id
      - requester_ip_address
      - coding_agent
    baggage_team_metadata_keys:
      - cost_center

Emit and collect metrics

Alongside traces, LiteLLM's OpenTelemetry v2 integration emits GenAI client metrics — histograms for call latency, token usage, and cost — over the same OTLP exporter. Coralogix ingests them as metrics you can chart and alert on.

Turn on metrics

Metrics stay off until you turn them on. Set this in the proxy environment:

export LITELLM_OTEL_INTEGRATION_ENABLE_METRICS="true"

On litellm earlier than 1.96.0, the metrics ship only with OpenTelemetry v2, so also set LITELLM_OTEL_V2=true. The metric stream ships through the exporter you already configured for traces — OTEL_EXPORTER, OTEL_ENDPOINT, and OTEL_HEADERS — so the same Coralogix endpoint receives it with no extra configuration. gRPC export needs the grpcio package.

Metrics emitted

Every metric is a histogram. These are OpenTelemetry instrument names, so they use dot notation; Coralogix stores them in Prometheus underscore notation, which is what you type in Metrics Explorer and PromQL. LiteLLM records:

Metric	In Coralogix	Unit	What it measures
gen_ai.client.operation.duration	gen_ai_client_operation_duration	s	Wall-clock time for the whole LLM call
gen_ai.client.token.usage	gen_ai_client_token_usage	{token}	Tokens consumed, split into input and output by gen_ai.token.type
gen_ai.usage.cost	gen_ai_usage_cost	USD	LiteLLM's computed cost for the call
gen_ai.server.time_to_first_token	gen_ai_server_time_to_first_token	s	Time to the first streamed token, on streaming calls
gen_ai.server.time_per_output_token	gen_ai_server_time_per_output_token	s	Average time per output token, on streaming calls
gen_ai.client.response.duration	gen_ai_client_response_duration	s	Provider-side generation time

LiteLLM renamed the cost and streaming-latency instruments in 1.96.0. On earlier versions they are gen_ai.client.token.cost, gen_ai.client.response.time_to_first_token, and gen_ai.client.response.time_per_output_token.

Each sample carries the same identity attributes as the matching span — gen_ai.operation.name, gen_ai.system, gen_ai.request.model, gen_ai.framework, and selected metadata.* fields such as metadata.user_api_key_team_id — so you can group the histograms by model, provider, key, or team. gen_ai.token.type always stays on gen_ai.client.token.usage, which keeps the input and output split.

Query metrics in Coralogix

Coralogix ingests these over OTLP as metrics. Open Metrics Explorer or write PromQL to chart latency, token usage, and cost, grouped by model, provider, or team, using the underscore names from the table above.

Control metric cardinality

High-cardinality attributes multiply the number of metric series. Trim the attributes LiteLLM attaches with the filter under callback_settings.otel.attributes in config.yaml. List exactly what to keep with include_list (LiteLLM drops anything else), or remove specific attributes with exclude_list:

callback_settings:
  otel:
    attributes:
      include_list:
        - gen_ai.operation.name
        - gen_ai.system
        - gen_ai.request.model
        - gen_ai.framework
        - metadata.user_api_key_team_id
        - metadata.user_api_key_org_id

gen_ai.token.type is always kept, so the input and output token split survives any filter. For the full reference, see LiteLLM OpenTelemetry v2 metrics.

Troubleshooting

Spans not appearing in AI Center

AI Center filters for GenAI spans using gen_ai.provider.name or gen_ai.input.messages. LiteLLM emits gen_ai.provider.name only when you set OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental (Step 3). Confirm the proxy environment includes this variable and that traces reach your S3 archive.

Missing message content

Confirm the proxy environment sets OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY (Step 4).

Traces reach Coralogix but not AI Center

AI Center reads trace data from your S3 archive, not from Frequent Search. Verify your traces route to archive storage.

Next steps

Confirm which gen_ai.* attributes AI Center consumes in Span attribute inventory.

Related resources

• OpenTelemetry integration for AI Center
• Compatibility matrix
• Span attribute inventory
• Code examples
• LiteLLM OpenTelemetry documentation