eBPF application auto-instrumentation with OBI
Collecting application metrics and traces usually means adding language-specific agents or modifying application code. OBI (OpenTelemetry eBPF-based instrumentation) simplifies this process using eBPF for automatic application instrumentation. Without any code changes, OBI inspects executables and the OS networking stack to capture trace spans for web traffic and RED (Rate, Errors, Duration) metrics in Linux-based HTTP/S and gRPC services.
See our Getting started guide for setup instructions.
Key capabilities
- Zero-code, kernel-level instrumentation: Captures traces and RED metrics with no code changes, library installs, or application restarts.
- Broad language and protocol coverage: Works across Java, .NET, Go, Python, Ruby, Node.js, C, C++, and Rust, with initial Deno coverage, spanning HTTP/S, gRPC, gRPC-Web, JSON-RPC, plus major database, messaging, search, and AI/LLM protocols.
- Distributed tracing with context propagation: Automatically captures, propagates, and forwards trace spans across services.
- Encrypted-traffic visibility: Monitors TLS/SSL transactions without decrypting content. HTTPS instrumentation is supported for Java, Node.js, and generic Go protocols.
- AI/LLM observability: Captures payloads for OpenAI, Anthropic, Google Gemini (AI Studio and Vertex AI), AWS Bedrock, Qwen (DashScope), Ollama, OpenAI-compatible gateways, MCP over JSON-RPC, embedding providers (Voyage AI, Cohere, Jina AI), rerank providers, and vector databases (Pinecone, Qdrant, Milvus, Zilliz, Chroma, Weaviate). View the captured AI traffic in Coralogix AI Center.
- Kubernetes-first with auto-discovery and cloud enrichment: Deploys as a DaemonSet, automatically discovering services and enriching telemetry with Kubernetes, OpenShift, and AWS/Azure/GCP metadata.
OBI enables scalable, vendor-neutral observability without invasive changes.
How OBI works
OBI runs as a DaemonSet on each Kubernetes node, capturing every packet transmitted or received at the kernel level with no sampling and no code changes. The agent categorizes traffic by protocol (detected from raw data or via the originating library), maps service-to-service connections, and enriches each interaction with OpenTelemetry conventions. Your OpenTelemetry collector then processes the spans and metrics and forwards them to Coralogix.
This is built on eBPF (extended Berkeley Packet Filter), a Linux kernel technology that safely executes custom logic in-kernel for observability, networking, and security, without modifying applications. OBI uses the OpenTelemetry eBPF instrumentation agent, now part of the official CNCF OpenTelemetry project (upstream documentation).
Key concepts
- DaemonSet: One agent runs on every Kubernetes node, ensuring cluster-wide visibility and scaling naturally as the cluster grows. See Kubernetes DaemonSets.
- Automatic service discovery: Detects new pods, containers, and services in real-time without manual configuration.
- Service naming: Uses Kubernetes metadata (
deployment,daemonset, orstatefulsetnames) as the service name. - Network observability: Captures latency, packet drops, and errors across inter-service communication.
For background on the underlying technology, see the eBPF Foundation.
System requirements
- Architecture: AMD64, ARM64
- Kernel version: 5.8 and above (with BTF enabled), or RHEL 4.18 kernels build 348+
Supported protocols
OBI detects all listed protocols across all supported languages. All protocols support encrypted/secure variants except MQTT, AMQP, and Aerospike. Trace context is propagated automatically for HTTP and for gRPC/HTTP2 via HPACK header injection, including connections that were already open when OBI started. For other protocols, OBI emits spans but does not propagate context downstream.
- Web & RPC
- SQL
- NoSQL and cache
- Messaging
- Search
- AWS services
- AI/LLM
| Protocol | Versions | Methods | Notes |
|---|---|---|---|
| HTTP | 1.0/1.1/2.0 | All | - |
| gRPC | 1.0+ | All | Methods for long-lived connections opened before OBI starts can't be retrieved and are marked *. |
| JSON-RPC | 2.0 | All | - |
| GraphQL | All | All | Document export is off by default. |
| SunRPC / ONC RPC | All | All |
| Database | Versions | Methods | Notes |
|---|---|---|---|
| MySQL | All | All | Prepared statements created before OBI starts may be missed. |
| PostgreSQL | All | All | Prepared statements created before OBI starts may be missed. |
| MSSQL | All | All | Query text may be missing if statements were prepared before OBI starts. |
| Database | Versions | Methods | Notes |
|---|---|---|---|
| MongoDB | 5.0+ | insert, update, find, delete, findAndModify, aggregate, count, distinct, mapReduce | No support for compressed payloads. |
| Redis | All | All | For connections opened before OBI starts, the database number can't be inferred and db.namespace isn't added. |
| Couchbase | All | All | Bucket unknown if SELECT_BUCKET happened before OBI started; collection unknown if GET_COLLECTION_ID happened before OBI started. Full query text is captured in db.query.text. |
| SQL++ | All | All | - |
| Memcached | All | ASCII text subset, excluding quit and meta commands | Only the first key is recorded for multi-key retrieval; payload bytes aren't captured. |
| Aerospike | All | GET, EXISTS, PUT, TOUCH, OPERATE, DELETE, SCAN, QUERY, BATCH, UDF | Native client protocol on port 3000; not encrypted. Compressed (type-4) payloads aren't parsed, and only operation metadata (namespace, set, key) is captured, not record or bin values. db.query.text requires the client's sendKey write policy. Scan and query duration is measured to the first response frame. |
| Protocol | Versions | Methods | Notes |
|---|---|---|---|
| Kafka | All | produce, fetch | May fail to get the topic name for fetch requests in newer Kafka (Fetch API version ≥ 13). A multi-topic request emits one span per topic instead of one span per request. |
| MQTT | 3.1.1, 5.0 | publish, subscribe | Not encrypted. Only the first topic filter is used; payload is not captured. |
| NATS | All | publish, subscribe | - |
| AMQP | 1.0 | publish, process | Only transfer performatives generate spans. |
| Engine | Versions | Methods |
|---|---|---|
| Elasticsearch | 7.14+ | /_search, /_msearch, /_bulk, /_doc |
| Opensearch | 3.0.0+ | /_search, /_msearch, /_bulk, /_doc |
| Service | Versions | Methods |
|---|---|---|
| AWS S3 | All | CreateBucket, DeleteBucket, PutObject, DeleteObject, ListBuckets, ListObjects, GetObject |
| AWS SQS | All | All |
| Provider | Versions | Methods |
|---|---|---|
| OpenAI | All | All |
| Anthropic | All | All |
| Google Gemini (AI Studio) | All | All |
| Vertex AI (Gemini) | All | All |
| AWS Bedrock | All | All |
| Qwen (DashScope) | All | All |
Ollama (native /api/chat and /api/generate) | All | All |
OpenAI-compatible gateways (LiteLLM, vLLM, LocalAI, OpenRouter, Ollama /v1/) | All | All |
| MCP over JSON-RPC | All | All |
| Embedding providers (Voyage AI, Cohere, Jina AI) | All | All |
| Rerank providers | All | All |
| Vector databases (Pinecone, Qdrant, Milvus, Zilliz, Chroma, Weaviate) | All | All |
Tool-call arguments and results are extracted for OpenAI, Anthropic, Google Gemini, and Qwen, and SSE streaming responses are captured for OpenAI-compatible, Anthropic, and Qwen APIs.
Gemini SSE streaming responses are captured, and each tool call gets its own span.
Token usage is exported only when the provider reports it, for streaming responses, only when the captured stream carries the provider's usage event or final usage chunk. Provider error messages are not exported: to copy them verbatim into the span status.message, select gen_ai.response.error explicitly under attributes.select.traces.include (wildcards don't enable it). Review retention and downstream access first, because provider error text can carry prompts, completions, or credentials.
Additional instrumentation features
Beyond baseline protocol detection, OBI ships built-in enhancements that deepen telemetry quality for specific languages and environments: richer Go library coverage, automatic instrumentation of the Go Trace API, an embedded OpenTelemetry Java agent for JVM telemetry, and automatic cloud-provider metadata enrichment.
Go-specific instrumentations
For Go applications, the agent provides enhanced instrumentation for specific libraries:
| Library | Support |
|---|---|
| Standard Go net/http | ✅ |
| Gorilla Mux | ✅ |
| Gin | ✅ |
| gRPC-Go | ✅ |
| Go x/net/http2 | ✅ |
| Go-Redis v9 | ✅ |
| Sarama Kafka | ✅ |
| Kafka-Go | ✅ |
Go Trace API auto-instrumentation
Go applications that create spans through the global otel.Tracer API without registering a TracerProvider are instrumented automatically: OBI activates the OpenTelemetry Go Auto SDK inside the process and exports those application-authored spans alongside the spans it derives from network traffic. When the application already registers an SDK TracerProvider, OBI defers to it instead.
Activation requires all of the following:
| Requirement | Detail |
|---|---|
| Modules in the executable | go.opentelemetry.io/auto/sdk v1.1.0+, go.opentelemetry.io/otel v1.33.0+, and go.opentelemetry.io/otel/trace v1.33.0+, all canonical and unreplaced |
| Recognized module versions | Each OBI release recognizes the module versions validated when it was built. A later module version needs a newer OBI release. |
| Architecture | 64-bit Linux amd64 or arm64 executable |
| Permissions | Permission to use bpf_probe_write_user. On Linux 5.10 and later this means effective CAP_SYS_ADMIN and kernel lockdown mode [none]. |
When a requirement isn't met, OBI doesn't activate the Auto SDK. It may still construct a synthetic span from the observed Trace API calls, carrying the span name, parent relationship, status, and some primitive attributes, but not the instrumentation scope, events, or requested span kind. Each application-authored span must fit within a 16 KiB encoded payload; larger spans are dropped, and OBI doesn't report that with a warning or metric.
For the full activation contract, see the OBI support matrix.
Java agent integration
OBI embeds the OpenTelemetry Java agent directly in the OBI binary. The Java agent is enabled by default, no extra flag or download is required. To disable it, set java.enabled: false in your OBI config or OTEL_EBPF_JAVAAGENT_ENABLED=false in the environment.
This provides richer Java telemetry (for example, JVM metrics, detailed span attributes) without requiring a separate Java agent download or configuration.
Cloud metadata decoration
OBI automatically detects and enriches telemetry with cloud provider resource attributes when running in AWS, Azure, or GCP environments. This adds attributes such as cloud region, account ID, and instance metadata to your spans and metrics without additional configuration.
Benefits
Why teams adopt OBI over agent-based or SDK-based instrumentation.
Monitoring starts immediately after deployment, capturing kernel-level metrics and events without instrumenting or redeploying applications.
Deep, real-time visibility into system performance and application behavior, without impacting overall system efficiency.
Attaches to system calls and network events at the kernel level, so uninstrumented, proprietary, and legacy services are covered regardless of language or SDK support.
Runs sandboxed in the kernel with limited permissions, minimizing attack surface and requiring no application changes.
Built on the OpenTelemetry eBPF instrumentation agent, avoiding vendor lock-in and enabling community-driven development.
What's changed
OBI behavior changes that may affect existing dashboards, queries, and span filters are tracked in the OBI changelog, version by version. Review the relevant version there before upgrading.
Feed AI Center from OBI
Installing OBI on the host running your AI application is the simplest way to feed Coralogix AI Center. One install captures AI/LLM traffic from OpenAI, Anthropic, Google Gemini (AI Studio and Vertex AI), AWS Bedrock, Qwen (DashScope), Ollama, OpenAI-compatible gateways, MCP over JSON-RPC, embedding providers, rerank providers, and vector databases, alongside the other OBI capabilities (distributed tracing, database observability, encrypted-traffic visibility, cloud metadata decoration) on the same install. No code changes, no SDKs, no per-language agents.
APM feature matrix
The table below compares the features supported by Coralogix APM with full OpenTelemetry integration versus those of the eBPF-based APM.
| Feature (K8s environments only) | eBPF | Full OpenTelemetry |
|---|---|---|
| Service catalog | ✅ | ✅ |
| Database catalog | ✅ | ✅ |
| Service map (only for Events2Metrics as metrics data source) | ✅ | ✅ |
| Metrics contextualization | ✅ | ✅ |
| Service alerts | ✅ | ✅ |
| SLOs | ✅ | ✅ |
| Apdex | ✅ | ✅ |
| Span - log correlation | ✅ | ✅ |
| API error tracking | ✅ | ✅ |
| Service - log correlation | ✅ | ✅ |
| K8s resources correlation | ✅ | ✅ |
| Serverless monitoring | ❌ | ✅ |
| Transactions | ✅ | ✅ |
| Spans exploration | ✅ | ✅ |
| Traces exploration | ✅ | ✅ |
| Trace map | ✅ | ✅ |
| Sampling | ✅ | ✅ |
| Span metrics | ✅ | ✅ |