# LiteLLM

Coralogix's AI Observability integrations are designed to provide deep insight into applications leveraging large language models. Through a dedicated integration with the LiteLLM SDK, Coralogix delivers a unified view of calls across various LLM providers, enabling teams to track performance, costs, and errors in a single place. This helps teams standardize monitoring, compare model performance, and optimize the entire system for efficiency and accuracy.

## Overview

This library offers customized [OpenTelemetry instrumentation](https://github.com/open-telemetry/opentelemetry-python-contrib/) for [LiteLLM](https://github.com/BerriAI/litellm), optimized to support large language model (LLM) application development with streamlined integration, detailed production tracing, and effective debugging capabilities.

## Requirements

- Python 3.10–3.13.
- Coralogix [API keys](https://coralogix.com/docs/user-guides/account-management/api-keys/api-keys/index.md).

## Installation

Run the following command.

```bash
pip install "llm-tracekit-litellm"
```

## Authentication

Authentication data is passed during OTel Span Exporter definition:

1. Select the ingress.\[[DOMAIN_VALUE]\]:443 endpoint that corresponds to your Coralogix [domain](https://coralogix.com/docs/user-guides/account-management/account-settings/coralogix-domain/index.md) using the domain selector at the top of the page.
1. Use your [customized API key](https://coralogix.com/docs/user-guides/account-management/api-keys/api-keys/index.md) in the authorization request header.
1. Provide the [application and subsystem names](https://coralogix.com/docs/user-guides/account-management/account-settings/application-and-subsystem-names/index.md).

```python
from llm_tracekit.litellm import setup_export_to_coralogix

setup_export_to_coralogix(
    coralogix_token=<your_coralogix_token>,
    coralogix_endpoint="ingress.[[DOMAIN_VALUE]]:443",
    service_name="ai-service",
    application_name="ai-application",
    subsystem_name="ai-subsystem",
    capture_content=True,
)
```

Note

All of the authentication parameters can also be provided through environment variables (`CX_TOKEN`, `CX_ENDPOINT`, etc.).

## Usage

This section describes how to set up instrumentation for LiteLLM.

### Set up tracing

**Automatic**

Use the `setup_export_to_coralogix` function to set up tracing and export traces to Coralogix. See the code snippet in the [Authentication](#authentication) section.

**Manual**

Alternatively, you can set up tracing manually.

```python
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

tracer_provider = TracerProvider(
    resource=Resource.create({SERVICE_NAME: "ai-service"}),
)
exporter = OTLPSpanExporter()
span_processor = SimpleSpanProcessor(exporter)
tracer_provider.add_span_processor(span_processor)
trace.set_tracer_provider(tracer_provider)
```

### Instrument

To instrument all clients, call the `instrument` method.

```python
from llm_tracekit.litellm import LiteLLMInstrumentor

LiteLLMInstrumentor().instrument()
```

### Uninstrument

To uninstrument clients, call the `uninstrument` method.

```python
LiteLLMInstrumentor().uninstrument()
```

### Full example

```python
import litellm
from llm_tracekit.litellm import LiteLLMInstrumentor, setup_export_to_coralogix

# Optional: Configure sending spans to Coralogix
# Reads Coralogix connection details from the following environment variables:
# - CX_TOKEN
# - CX_ENDPOINT
setup_export_to_coralogix(
    service_name="ai-service",
    application_name="ai-application",
    subsystem_name="ai-subsystem",
    capture_content=True,
)

# Activate instrumentation
LiteLLMInstrumentor().instrument()

# LiteLLM usage example
response = litellm.completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a short poem on open telemetry."}],
    user="user@company.com",
)
```

### Enable message content capture

By default, message content — prompt contents, completions, function arguments, and return values — is not captured. To capture message content as span attributes:

- Pass `capture_content=True` when calling `setup_export_to_coralogix`.
- Set the environment variable `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` to `SPAN_AND_EVENT`.

Most Coralogix AI evaluations require message contents to function properly, so enabling message capture is strongly recommended.

## Semantic conventions

| Attribute                                                                            | Type   | Description                                                                               | Example                                                                     |
| ------------------------------------------------------------------------------------ | ------ | ----------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `gen_ai.prompt.<message_number>.role`                                                | string | Role of message author for user message `<message_number>`                                | `system`, `user`, `assistant`, `tool`                                       |
| `gen_ai.prompt.<message_number>.content`                                             | string | Contents of user message `<message_number>`                                               | `What's the weather in Paris?`                                              |
| `gen_ai.prompt.<message_number>.tool_calls.<tool_call_number>.id`                    | string | ID of tool call in user message `<message_number>`                                        | `call_O8NOz8VlxosSASEsOY7LDUcP`                                             |
| `gen_ai.prompt.<message_number>.tool_calls.<tool_call_number>.type`                  | string | Type of tool call in user message `<message_number>`                                      | `function`                                                                  |
| `gen_ai.prompt.<message_number>.tool_calls.<tool_call_number>.function.name`         | string | The name of the function used in tool call within user message `<message_number>`         | `get_current_weather`                                                       |
| `gen_ai.prompt.<message_number>.tool_calls.<tool_call_number>.function.arguments`    | string | Arguments passed to the function used in tool call within user message `<message_number>` | `{"location": "Seattle, WA"}`                                               |
| `gen_ai.prompt.<message_number>.tool_call_id`                                        | string | Tool call ID in user message `<message_number>`                                           | `call_mszuSIzqtI65i1wAUOE8w5H4`                                             |
| `gen_ai.completion.<choice_number>.role`                                             | string | Role of message author for choice `<choice_number>` in model response                     | `assistant`                                                                 |
| `gen_ai.completion.<choice_number>.finish_reason`                                    | string | Finish reason for choice `<choice_number>` in model response                              | `stop`, `tool_calls`, `error`                                               |
| `gen_ai.completion.<choice_number>.content`                                          | string | Contents of choice `<choice_number>` in model response                                    | `The weather in Paris is rainy and overcast, with temperatures around 57°F` |
| `gen_ai.completion.<choice_number>.tool_calls.<tool_call_number>.id`                 | string | ID of tool call in choice `<choice_number>`                                               | `call_O8NOz8VlxosSASEsOY7LDUcP`                                             |
| `gen_ai.completion.<choice_number>.tool_calls.<tool_call_number>.type`               | string | Type of tool call in choice `<choice_number>`                                             | `function`                                                                  |
| `gen_ai.completion.<choice_number>.tool_calls.<tool_call_number>.function.name`      | string | The name of the function used in tool call within choice `<choice_number>`                | `get_current_weather`                                                       |
| `gen_ai.completion.<choice_number>.tool_calls.<tool_call_number>.function.arguments` | string | Arguments passed to the function used in tool call within choice `<choice_number>`        | `{"location": "Seattle, WA"}`                                               |
| `gen_ai.request.tools.<tool_number>.type`                                            | string | Type of tool definition advertised to the model                                           | `function`                                                                  |
| `gen_ai.request.tools.<tool_number>.function.name`                                   | string | Name of the tool/function exposed to the model                                            | `get_current_weather`                                                       |
| `gen_ai.request.tools.<tool_number>.function.description`                            | string | Description of the tool/function                                                          | `Get the current weather in a given location`                               |
| `gen_ai.request.tools.<tool_number>.function.parameters`                             | string | JSON schema describing the tool/function parameters passed with the request               | `{"type": "object", "properties": {"city": {"type": "string"}}}`            |
| `gen_ai.request.user`                                                                | string | A unique identifier representing the end user (from the `user` parameter)                 | `user@company.com`                                                          |

## Next steps

Once your integration is set up, explore the [AI Center Overview](https://coralogix.com/docs/user-guides/ai/monitor/index.md) to monitor performance, costs, quality issues, and security across all your AI applications — and to set up [Guardrails](https://coralogix.com/docs/user-guides/ai/guardrails/index.md) for real-time policy enforcement.