# Exposed metrics Copy as Markdown[Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fcoralogix.com%2Fdocs%2Fopentelemetry%2Finstrumentation-options%2Febpf-auto-instrumentation%2Fexported-metrics.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.)[Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fcoralogix.com%2Fdocs%2Fopentelemetry%2Finstrumentation-options%2Febpf-auto-instrumentation%2Fexported-metrics.md%20and%20help%20me%20with%20my%20question%20about%20this%20Coralogix%20documentation%20page.) The following tables describe the metrics exported by OBI in both OpenTelemetry (OTEL) and Prometheus formats, for use within Coralogix observability workflows. ## Application metrics[​](#application-metrics "Direct link to Application metrics") | Family | Name (OTEL) | Name (Prometheus) | Type | Unit | Description | | ------------------- | --------------------------------------- | --------------------------------------------- | ------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | Application | `http.client.request.duration` | `http_client_request_duration_seconds` | Histogram | seconds | Time taken for outbound HTTP calls from client applications | | Application | `http.client.request.body.size` | `http_client_request_body_size_bytes` | Histogram | bytes | Size of HTTP request payloads sent by clients | | Application | `http.client.response.body.size` | `http_client_response_body_size_bytes` | Histogram | bytes | Size of HTTP responses received by clients | | Application | `http.server.request.duration` | `http_server_request_duration_seconds` | Histogram | seconds | Time taken to process inbound HTTP requests on servers | | Application | `http.server.request.body.size` | `http_server_request_body_size_bytes` | Histogram | bytes | Size of incoming HTTP request bodies received by servers | | Application | `http.server.response.body.size` | `http_server_response_body_size_bytes` | Histogram | bytes | Size of outbound HTTP response bodies sent by servers | | Application | `rpc.client.duration` | `rpc_client_duration_seconds` | Histogram | seconds | Duration of outbound gRPC service calls | | Application | `rpc.server.duration` | `rpc_server_duration_seconds` | Histogram | seconds | Duration of inbound RPC service calls | | Application | `sql.client.duration` | `sql_client_duration_seconds` | Histogram | seconds | Time taken for SQL client operations (Experimental) | | Application | `redis.client.duration` | `redis_client_duration_seconds` | Histogram | seconds | Time taken for Redis client operations (Experimental) | | Application | `db.server.operation.duration` | `db_server_operation_duration_seconds` | Histogram | seconds | Server-side database operation duration for Redis, Memcached, and SQL. | | Application | `messaging.publish.duration` | `messaging_publish_duration` | Histogram | seconds | Time taken for message publication (e.g., Kafka) (Experimental) | | Application | `messaging.process.duration` | `messaging_process_duration` | Histogram | seconds | Time taken for message processing (e.g., Kafka) (Experimental) | | Application Process | `process.cpu.time` | `process_cpu_time_seconds_total` | Counter | seconds | Total CPU time consumed, broken down by state (system/user/wait) | | Application Process | `process.cpu.utilization` | `process_cpu_utilization_ratio` | Gauge | ratio | CPU time usage change over time, normalized by CPU count | | Application Process | `process.memory.usage` | `process_memory_usage_bytes` | UpDownCounter | bytes | Physical memory currently used | | Application Process | `process.memory.virtual` | `process_memory_virtual_bytes` | UpDownCounter | bytes | Virtual memory committed | | Application Process | `process.disk.io` | `process_disk_io_bytes_total` | Counter | bytes | Total disk I/O bytes transferred | | Application Process | `process.network.io` | `process_network_io_bytes_total` | Counter | bytes | Total network I/O bytes transferred | | Go runtime | `go.memory.limit` | `go_memory_limit_bytes` | UpDownCounter | bytes | Go runtime memory limit. | | Go runtime | `go.memory.gc.goal` | `go_memory_gc_goal_bytes` | UpDownCounter | bytes | Committed GC heap goal. Omitted when the runtime offset and the fallback symbol are both unavailable. | | Go runtime | `go.memory.gc.cycles` | `go_memory_gc_cycles_total` | Counter | count | Completed Go GC cycles. | | Go runtime | `go.memory.gc.pause.duration` | `go_memory_gc_pause_duration_seconds` | Histogram | seconds | Cumulative stop-the-world GC pause duration. Requires Go 1.22+. | | Go runtime | `go.memory.used` | `go_memory_used_bytes` | UpDownCounter | bytes | Memory in use, broken down by `go.memory.type` (`stack` or `other`). Requires Go 1.23+. | | Go runtime | `go.memory.allocated` | `go_memory_allocated_bytes_total` | Counter | bytes | Cumulative allocated heap bytes. Requires Go 1.23+. | | Go runtime | `go.memory.allocations` | `go_memory_allocations_total` | Counter | allocations | Cumulative heap allocation count. Requires Go 1.23+. | | Go runtime | `go.cpu.time` | `go_cpu_time_seconds_total` | Counter | seconds | Cumulative CPU time, broken down by `go.cpu.state` (and `go.cpu.detailed_state` where applicable). Requires Go 1.23+. | | Go runtime | `go.goroutine.count` | `go_goroutine_count` | UpDownCounter | goroutines | Current goroutine count. Omitted rather than reported partially when the runtime snapshot is incomplete. | | Go runtime | `go.processor.limit` | `go_processor_limit` | UpDownCounter | processors | `GOMAXPROCS` value. | | Go runtime | `go.config.gogc` | `go_config_gogc_percent` | UpDownCounter | percent | `GOGC` configuration value. | | Go runtime | `go.schedule.duration` | `go_schedule_duration_seconds` | Histogram | seconds | Cumulative runnable-to-running goroutine latency. Requires Go 1.20+. | | JVM | `jvm.memory.used` | `jvm_memory_used_bytes` | UpDownCounter | bytes | HotSpot JVM memory used, per memory pool. Sum by `jvm.memory.type` for heap and non-heap totals. | | JVM | `jvm.memory.committed` | `jvm_memory_committed_bytes` | UpDownCounter | bytes | HotSpot JVM memory committed. | | JVM | `jvm.memory.limit` | `jvm_memory_limit_bytes` | UpDownCounter | bytes | HotSpot JVM memory limit. | | JVM | `jvm.memory.used_after_last_gc` | `jvm_memory_used_after_last_gc_bytes` | UpDownCounter | bytes | HotSpot JVM memory used after the last GC. | | JVM | `jvm.class.loaded` | `jvm_class_loaded_total` | Counter | classes | Classes loaded since JVM start. | | JVM | `jvm.class.unloaded` | `jvm_class_unloaded_total` | Counter | classes | Classes unloaded since JVM start. | | JVM | `jvm.class.count` | `jvm_class_count` | UpDownCounter | classes | Classes currently loaded. | | JVM | `jvm.thread.count` | `jvm_thread_count` | UpDownCounter | threads | Current executing thread count. | | JVM | `jvm.cpu.time` | `jvm_cpu_time_seconds_total` | Counter | seconds | CPU time used by the JVM process. | | JVM | `jvm.cpu.count` | `jvm_cpu_count` | UpDownCounter | processors | Processors available to the JVM. | | JVM | `jvm.cpu.recent_utilization` | `jvm_cpu_recent_utilization_ratio` | Gauge | ratio | Recent CPU utilization of the JVM process. | | Node.js runtime | `nodejs.eventloop.utilization` | `nodejs_eventloop_utilization_ratio` | Gauge | ratio | Event-loop utilization. Requires Node.js 14.10+. | | Node.js runtime | `nodejs.eventloop.time` | `nodejs_eventloop_time_seconds_total` | Counter | seconds | Cumulative event-loop time, broken down by `nodejs.eventloop.state` (`idle` or `active`). Requires Node.js 14.10+. | | Node.js runtime | `nodejs.eventloop.delay.*` | `nodejs_eventloop_delay_*_seconds` | Gauge | seconds | Event-loop delay statistics. `min`, `max`, `mean`, `stddev`, `p50`, `p90`, `p99`. Requires Node.js 16.14+. | | Node.js runtime | `v8js.gc.duration` | `v8js_gc_duration_seconds` | Histogram | seconds | V8 garbage-collection pause duration. | | Node.js runtime | `v8js.memory.heap.used` | `v8js_memory_heap_used_bytes` | UpDownCounter | bytes | V8 heap memory used. | | Node.js runtime | `v8js.memory.heap.limit` | `v8js_memory_heap_limit_bytes` | UpDownCounter | bytes | V8 heap memory limit. | | Node.js runtime | `v8js.memory.heap.space.available_size` | `v8js_memory_heap_space_available_size_bytes` | UpDownCounter | bytes | Available V8 heap-space memory. | | Node.js runtime | `v8js.memory.heap.space.physical_size` | `v8js_memory_heap_space_physical_size_bytes` | UpDownCounter | bytes | Physical (committed) V8 heap-space memory. | | Python runtime | `cpython.gc.collections` | `cpython_gc_collections_total` | Counter | collections | CPython garbage-collection runs. | | Python runtime | `cpython.gc.collected_objects` | `cpython_gc_collected_objects_total` | Counter | objects | Objects reclaimed by the CPython garbage collector. | | Python runtime | `cpython.gc.uncollectable_objects` | `cpython_gc_uncollectable_objects_total` | Counter | objects | Uncollectable objects found by the CPython garbage collector. | | Network | `obi.network.flow.bytes` | `obi_network_flow_bytes` | Counter | bytes | Bytes transmitted between source and destination network endpoints | | Network | `obi.network.flow.packets` | `obi_network_flow_packets_total` | Counter | count | Packets transmitted between source and destination network endpoints. | | Network | `obi.network.inter.zone.bytes` | `obi_network_inter_zone_bytes` | Counter | bytes | Cross-zone network bytes within Kubernetes clusters (Experimental) | | Network | `obi.stat.tcp.io` | `obi_stat_tcp_io_bytes_total` | Counter | bytes | TCP bytes transferred. | | Network | `obi.stat.tcp.retransmits` | `obi_stat_tcp_retransmits_total` | Counter | count | TCP segment retransmissions. | | Network | `obi.stat.tcp.failed.connections` | `obi_stat_tcp_failed_connections_total` | Counter | connections | TCP failed connection count, labeled by `network.tcp.handshake.role` (client or server) and failure reason. The `reason` label is disabled by default. | ### Enabling runtime metrics[​](#enabling-runtime-metrics "Direct link to Enabling runtime metrics") Go, JVM, Node.js, and Python runtime metrics are all enabled through the shared `application_runtime` metrics feature; `jvm_runtime_metrics.sampling_interval` controls the JVM sampling rate. The Node.js event-loop and V8 metrics require the Node.js injector (`nodejs.enabled`, on by default). Each metric's introducing release is noted in the table above; see the [OBI changelog](https://coralogix.com/docs/opentelemetry/instrumentation-options/ebpf-auto-instrumentation/release-notes.md) for details. ### Span and service-graph metrics[​](#span-and-service-graph-metrics "Direct link to Span and service-graph metrics") OBI also supports exporting **Span metrics** and **Service Graph metrics**, which can be toggled via feature configuration options. The legacy `application_span` and `application_span_sizes` span-metric features are deprecated in favor of `application_span_otel`. The deprecated features still emit the Grafana-convention `traces_spanmetrics_*` metrics and keep working with a startup warning, while `application_span_otel` emits the OpenTelemetry-convention `traces_span_metrics_*` metrics. `application_span_sizes` has no direct replacement. The closest are `http.server.request.body.size` and `http.server.response.body.size`. Enabling a legacy feature together with `application_span_otel` is rejected, and the `*` (all) selector keeps only the OTel version. ### Metric naming[​](#metric-naming "Direct link to Metric naming") Prometheus metric names are derived from their OTLP definitions rather than declared separately. On the OTLP export path, `target_info`, `traces_target_info`, `traces_host_info`, `traces_span_metrics_calls_total`, and `traces_span_metrics_duration` are emitted in dot notation as `target.info`, `traces.target.info`, `traces.host.info`, `traces.span.metrics.calls`, and `traces.span.metrics.duration`; the Prometheus scrape path keeps the underscore names. For heap totals, sum `jvm.memory.used` by `jvm.memory.type` (the OBI-specific `obi.jvm.heap.used` metric is not emitted). The [OBI changelog](https://coralogix.com/docs/opentelemetry/instrumentation-options/ebpf-auto-instrumentation/release-notes.md) lists the metric renames this introduced. ### Span attributes[​](#span-attributes "Direct link to Span attributes") Database duration metrics carry the `db.namespace`, `db.response.status_code`, and `server.port` attributes, and messaging spans carry `messaging.operation.name`. HTTP spans emit `url.query` by default when present, and client HTTP spans include the query string in `url.full`. Sensitive parameters are automatically redacted; extend the redaction list with `OTEL_EBPF_SENSITIVE_QUERY_PARAMS_ADD`. Java route harvesting is disabled by default, so the `http.route` attribute on Java spans falls back to the raw URL path. ### Capture defaults[​](#capture-defaults "Direct link to Capture defaults") Database error response bodies are not included on SQL spans by default. Enable that capture explicitly if you need `sql.client.duration` spans to carry database error responses. DNS calls are excluded from span metrics and service-graph metrics by default. SQL operations (MySQL, PostgreSQL, MariaDB) are reported as `SERVER` spans, so aggregations or filters that key on span kind for SQL traffic should account for that. ## Attribute selection for OBI metrics[​](#attribute-selection-for-obi-metrics "Direct link to Attribute selection for OBI metrics") Metrics and attributes follow **OTEL dot.notation** but are converted to **Prometheus underscore\_notation** in Prometheus outputs. You can customize which attributes are exposed or hidden by using the `attributes->select` section in the configuration. ### Common attributes table[​](#common-attributes-table "Direct link to Common attributes table") | Metrics | Attribute Name | Default | | -------------------------------- | ---------------------------------------------------------------------------------------- | ---------------------------------- | | Application (all) | `http.request.method` | Shown | | Application (all) | `http.response.status_code` | Shown | | Application (all) | `http.route` | Shown if routes config exists | | Application (all) | `k8s.daemonset.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.deployment.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.namespace.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.node.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.owner.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.pod.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.container.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.pod.start_time` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.pod.uid` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.replicaset.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.statefulset.name` | Shown if Kubernetes metadata is on | | Application (all) | `k8s.cluster.name` | Shown if Kubernetes metadata is on | | Application (all) | `service.name` | Resource attribute only | | Application (all) | `service.namespace` | Resource attribute only | | Application (all) | `target.instance` | Shown | | Application (all) | `url.path` | Hidden | | Application (client) | `server.address` | Hidden | | Application (client) | `server.port` | Hidden | | Application (process) | various process-specific attributes (e.g., `process.command`, `process.executable.path`) | Shown if process metrics enabled | | Application RPC | various RPC attributes (`rpc.method`, `rpc.grpc.status_code`) | Shown | | Application (server) | `client.address` | Hidden | | Network | network-specific attributes (e.g., `src.address`, `dst.address`, `iface`) | Mostly hidden or conditional | | Network (TCP failed connections) | `network.tcp.handshake.role`: values `client` or `server` | Shown | Note: The `obi.network.inter.zone.bytes` metric shares the same attribute set as `obi.network.flow.bytes` but all are hidden by default except `k8s.cluster.name`, `src.zone`, and `dst.zone`. Note: `service.name` and `service.namespace` are no longer default metric labels. They are still reported as resource attributes (through `target_info` and the OTLP resource). To keep them on metric series, add them to the metric's `include` list under `attributes->select`. ## Internal OBI metrics[​](#internal-obi-metrics "Direct link to Internal OBI metrics") OBI provides several internal Prometheus-format metrics to help monitor its own health and activity: | Name | Type | Description | | ------------------------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `obi_ebpf_tracer_flushes` | Histogram | Size of trace groups flushed from eBPF to pipeline | | `obi_otel_metric_exports_total` | Counter | Total metric batches sent to remote OTEL collector | | `obi_otel_metric_export_errors_total` | CounterVec | Count of OTEL metric export errors by type | | `obi_otel_trace_exports_total` | Counter | Total trace batches sent to remote OTEL collector | | `obi_otel_trace_export_errors_total` | CounterVec | Count of OTEL trace export errors by type | | `obi_prometheus_http_requests_total` | CounterVec | Prometheus scrape endpoint request count by port and path | | `obi_instrumented_processes` | GaugeVec | Number of processes instrumented by Obi, labeled by process name | | `obi_instrumentation_errors_total` | CounterVec | Instrumentation errors, labeled by process executable name and error type | | `obi_avoided_services` | GaugeVec | Services OBI skipped because they already carry OpenTelemetry instrumentation, labeled by service name, namespace, and telemetry type. Emitted only when the avoided-services internal metric is enabled. | | `obi_internal_build_info` | GaugeVec | Obi version information including build timestamp and commit ID | `obi_otel_trace_exports_total` and `obi_otel_trace_export_errors_total` reflect the real outcome reported by the exporter rather than counting attempts. Internal metrics also carry host metadata as resource attributes. OBI can additionally report eBPF probe latency and map utilization, but that family is opt-in. It requires the `ebpf` metrics feature with a Prometheus endpoint, or `internal_metrics.bpf_metric_scrape_interval` set above zero. See the upstream [BPF metrics collection reference](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/blob/v0.13.0/devdocs/bpf-metrics-collection.md) for the metric names and labels.