Kubernetes with Filebeat

The following tutorial demonstrates how to integrate Coralogix with your Kubernetes cluster using Filebeat.

See other tutorials for:

• Kubernetes logs with FluentBit
• Kubernetes logs with FluentD

Prerequisites

Before you will begin, make sure that you already have:

• Installed Kubernetes Cluster
• Enabled RBAC authorization mode support

Installation

First, you should create Kubernetes secret with Coralogix credentials:

$ kubectl -n kube-system create secret generic coralogix-filebeat-account-secrets \
    --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX \
    --from-literal=COMPANY_ID=XXXX \
    --from-literal=CLUSTER_NAME=cluster.local

You should receive something like:

secret "coralogix-filebeat-account-secrets" created

Then you need to create filebeat-coralogix-logger resources on your Kubernetes cluster with this manifests:

$ kubectl create -k https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes

Output:

serviceaccount "coralogix-filebeat-service-account" created
clusterrole "coralogix-filebeat-cluster-role" created
clusterrolebinding "coralogix-filebeat-cluster-role-binding" created
configmap "coralogix-filebeat-config" created
secret "coralogix-filebeat-certificate" created
daemonset "coralogix-filebeat-daemonset" created
service "coralogix-filebeat-service" created

Now filebeat-coralogix-logger collects logs from your Kubernetes cluster.

Here is an example log record:

{
	"cloud": {
		"availability_zone": "projects/379343634745/zones/us-central1-a",
		"instance_id": "7653580772456904060",
		"instance_name": "gke-coralogix-test-default-pool-4d86c144-sbkd",
		"machine_type": "projects/379343634745/machineTypes/n1-standard-1",
		"project_id": "coralogix-test",
		"provider": "gce"
	},
	"kubernetes": {
		"container": {
			"name": "prometheus-to-sd"
		},
		"labels": {
			"k8s-app": "kube-dns",
			"pod-template-hash": "989689126"
		},
		"namespace": "kube-system",
		"node": {
			"name": "minikube",
			"labels": {
				"kubernetes_io/arch": "amd64",
				"kubernetes_io/os": "linux"
			},
			"hostname": "minikube"
		},
		"pod": {
			"name": "kube-dns-fdfbdf56b-jbbw2",
			"uid": "56584469-534d-11e9-8bcd-42010a800179"
		},
		"replicaset": {
			"name": "kube-dns-fdfbdf56b"
		}
	},
	"@timestamp": "2019-03-31T00:45:53.973Z",
	"@version": "1",
	"host": {
		"name": "filebeat-coralogix"
	},
	"beat": {
		"hostname": "filebeat-coralogix-daemonset-98wxr",
		"name": "filebeat-coralogix",
		"version": "6.7.0"
	},
	"message": "E0331 00:45:53.970719 1 stackdriver.go:58] Error while sending request to Stackdriver Post /v3/projects/coralogix-test/timeSeries?alt=json: unsupported protocol scheme \"\"",
	"tags": [
		"kubernetes",
		"containers",
		"beats_input_codec_plain_applied"
	]
}

To generate installation manifests for future customization, just execute:

$ kubectl kustomize https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes

Uninstall

If you want to remove filebeat-coralogix-logger from your cluster, execute this:

$ kubectl -n kube-system delete secret filebeat-coralogix-account-secrets
$ kubectl delete -k https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes