INTEGRATIONS

Kubernetes with Filebeat

k8s logs with filebeat

The following manual will help you integrate Coralogix with your Kubernetes cluster using Filebeat.

See other tutorials for:

Prerequisites

Before you will begin, make sure that you already have:

  • Installed Kubernetes Cluster
  • Enabled RBAC authorization mode support

Installation

First, you should create a Kubernetes secret with Coralogix credentials:

$ kubectl -n kube-system create secret generic filebeat-coralogix-account-secrets \
    --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX \
    --from-literal=COMPANY_ID=XXXX

You should receive something like:

secret "filebeat-coralogix-account-secrets" created

Then you need to create filebeat-coralogix-logger resources on your Kubernetes cluster with these manifests:

$ kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/filebeat/kubernetes/filebeat-coralogix-rbac.yaml
$ kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/filebeat/kubernetes/filebeat-coralogix-cm.yaml
$ kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/filebeat/kubernetes/filebeat-coralogix-secret.yaml
$ kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/filebeat/kubernetes/filebeat-coralogix-ds.yaml
$ kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/filebeat/kubernetes/filebeat-coralogix-svc.yaml

Output:

serviceaccount "filebeat-coralogix-service-account" created
clusterrole "filebeat-coralogix-service-account-role" created
clusterrolebinding "filebeat-coralogix-service-account" created
configmap "filebeat-coralogix-config" created
secret "filebeat-coralogix-certificate" created
daemonset "filebeat-coralogix-daemonset" created
service "filebeat-coralogix-service" created

Now filebeat-coralogix-logger collects logs from your Kubernetes cluster.

Here is an example of log record:

{
  "cloud": {
    "availability_zone": "projects/379343634745/zones/us-central1-a",
    "instance_id": "7653580772456904060",
    "instance_name": "gke-coralogix-test-default-pool-4d86c144-sbkd",
    "machine_type": "projects/379343634745/machineTypes/n1-standard-1",
    "project_id": "coralogix-test",
    "provider": "gce"
  },
  "kubernetes": {
    "container": {
      "name": "prometheus-to-sd"
    },
    "labels": {
      "k8s-app": "kube-dns",
      "pod-template-hash": "989689126"
    },
    "namespace": "kube-system",
    "node": {
      "name": "gke-coralogix-test-default-pool-4d86c144-sbkd"
    },
    "pod": {
      "name": "kube-dns-fdfbdf56b-jbbw2",
      "uid": "56584469-534d-11e9-8bcd-42010a800179"
    },
    "replicaset": {
      "name": "kube-dns-fdfbdf56b"
    }
  },
  "@timestamp": "2019-03-31T00:45:53.973Z",
  "@version": "1",
  "host": {
    "name": "filebeat-coralogix"
  },
  "beat": {
    "hostname": "filebeat-coralogix-daemonset-98wxr",
    "name": "filebeat-coralogix",
    "version": "6.7.0"
  },
  "message": "E0331 00:45:53.970719 1 stackdriver.go:58] Error while sending request to Stackdriver Post /v3/projects/coralogix-test/timeSeries?alt=json: unsupported protocol scheme \"\"",
  "tags": [
    "kubernetes",
    "containers",
    "beats_input_codec_plain_applied"
  ]
}

Uninstall

If you want to remove filebeat-coralogix-logger from your cluster, execute this:

$ kubectl -n kube-system delete secret filebeat-coralogix-account-secrets
$ kubectl -n kube-system delete secret filebeat-coralogix-certificate
$ kubectl -n kube-system delete svc,ds,cm,clusterrolebinding,clusterrole,sa \
     -l k8s-app=filebeat-coralogix-logger

Start solving your production issues faster

Let's talk about how Coralogix can help you better understand your logs

Managed, Scaled and Compliant ELK Stack

No credit card required

Get a personalized demo

Jump on a call with one of our experts and get a live personalized demonstration