Prometheus agent

Open in ChatGPTOpen in Claude

Prometheus Agent is an operational mode built into the Prometheus binary that does one job: scrape metrics and remote-write them. Querying and alerting are switched off, which is exactly what you want when Coralogix is where the querying happens.

Agent mode buys you:

• Scraping and forwarding only, with querying and alerting disabled.
• Easier horizontal scaling for ingestion than server mode.
• A purpose-built TSDB write-ahead log in place of local storage.

What you need

• A Coralogix account on the domain matching the region your data should be stored in. Sign up if you do not have one.
• A Coralogix Send-Your-Data API key.
• The Prometheus Operator installed and configured, at v0.59.0 or later. Agent mode collects ServiceMonitors and PodMonitors, which exist only under the Operator, and earlier Operator versions do not support agent mode at all.
• The Prometheus RemoteWrite endpoint for your domain. Use the domain selector at the top of this page to resolve it: https://ingress./prometheus/v1

Store the API key as a secret

Keeping the key in a Kubernetes secret keeps it out of your chart values and out of anything you commit.

kubectl create secret generic coralogix-keys \
  --from-literal=PRIVATE_KEY=<private-key>

The result looks like this:

apiVersion: v1
data:
  PRIVATE_KEY: <encrypted-private-key>
kind: Secret
metadata:
  name: coralogix-keys
  namespace: <the-release-namespace>
type: Opaque

Warning

The secret and the Helm release must live in the same namespace. A secret in another namespace is invisible to the chart, and the agent starts but never authenticates.

Install the chart

1.

Add the Coralogix Helm repository

Add the repository and refresh the local chart index:

helm repo add coralogix-charts-virtual https://cgx.jfrog.io/artifactory/coralogix-charts-virtual
helm repo update

2.

Create an override file

Set the remote write target to the Prometheus RemoteWrite endpoint for your domain:

---
# override.yaml
prometheus:
  prometheusSpec:
    remoteWrite:
    - authorization:
        credentials:
          name: coralogix-keys
          key: PRIVATE_KEY
      name: prometheus-agent-coralogix
      queueConfig:
        capacity: 2500
        maxSamplesPerSend: 1000
        maxShards: 200
      remoteTimeout: 120s
      url: https://ingress.eu2.coralogix.com/prometheus/v1

3.

Install

Install or upgrade the release:

helm upgrade prometheus-agent coralogix-charts-virtual/prometheus-agent-coralogix \
  --install \
  --namespace=<your-namespace> \
  --create-namespace \
  -f override.yaml

Use a persistent volume in production

The agent defaults to ephemeral volumes, which lose buffered samples on restart. Define a persistent volume in your values file instead.

prometheus:
  prometheusSpec:
    storageSpec:
      volumeClaimTemplate:
        spec:
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 50Gi

Omitting storageClassName makes Kubernetes use the cluster's default storage class.

Limits and quotas

These limits apply per request, regardless of the period the request covers.

Endpoint	Hard limit	Recommended
OpenTelemetry	10 MB	2 MB
Prometheus RemoteWrite	2,411,724 bytes	Anything below the hard limit

Related resources

Prometheus OperatorCoralogix endpointsSend-Your-Data API key