Learn more about Streama© – the foundational technology behind our stateful streaming data platform. Learn More

Kubernetes with Filebeat

The following manual will help you integrate Coralogix with your Kubernetes cluster using Filebeat.

See other tutorials for:

Prerequisites

Before you will begin, make sure that you already have:

  • Installed Kubernetes Cluster
  • Enabled RBAC authorization mode support

Installation

First, you should create Kubernetes secret with Coralogix credentials:

$ kubectl -n kube-system create secret generic coralogix-filebeat-account-secrets \
    --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX \
    --from-literal=COMPANY_ID=XXXX \
    --from-literal=CLUSTER_NAME=cluster.local

You should receive something like:

secret "coralogix-filebeat-account-secrets" created

Then you need to create filebeat-coralogix-logger resources on your Kubernetes cluster with this manifests:

$ kubectl create -k https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes

Output:

serviceaccount "coralogix-filebeat-service-account" created
clusterrole "coralogix-filebeat-cluster-role" created
clusterrolebinding "coralogix-filebeat-cluster-role-binding" created
configmap "coralogix-filebeat-config" created
secret "coralogix-filebeat-certificate" created
daemonset "coralogix-filebeat-daemonset" created
service "coralogix-filebeat-service" created

Now filebeat-coralogix-logger collects logs from your Kubernetes cluster.

Here is an example log record:

{
	"cloud": {
		"availability_zone": "projects/379343634745/zones/us-central1-a",
		"instance_id": "7653580772456904060",
		"instance_name": "gke-coralogix-test-default-pool-4d86c144-sbkd",
		"machine_type": "projects/379343634745/machineTypes/n1-standard-1",
		"project_id": "coralogix-test",
		"provider": "gce"
	},
	"kubernetes": {
		"container": {
			"name": "prometheus-to-sd"
		},
		"labels": {
			"k8s-app": "kube-dns",
			"pod-template-hash": "989689126"
		},
		"namespace": "kube-system",
		"node": {
			"name": "minikube",
			"labels": {
				"kubernetes_io/arch": "amd64",
				"kubernetes_io/os": "linux"
			},
			"hostname": "minikube"
		},
		"pod": {
			"name": "kube-dns-fdfbdf56b-jbbw2",
			"uid": "56584469-534d-11e9-8bcd-42010a800179"
		},
		"replicaset": {
			"name": "kube-dns-fdfbdf56b"
		}
	},
	"@timestamp": "2019-03-31T00:45:53.973Z",
	"@version": "1",
	"host": {
		"name": "filebeat-coralogix"
	},
	"beat": {
		"hostname": "filebeat-coralogix-daemonset-98wxr",
		"name": "filebeat-coralogix",
		"version": "6.7.0"
	},
	"message": "E0331 00:45:53.970719 1 stackdriver.go:58] Error while sending request to Stackdriver Post /v3/projects/coralogix-test/timeSeries?alt=json: unsupported protocol scheme \"\"",
	"tags": [
		"kubernetes",
		"containers",
		"beats_input_codec_plain_applied"
	]
}

To generate installation manifests for future customization, just execute:

$ kubectl kustomize https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes

Uninstall

If you want to remove filebeat-coralogix-logger from your cluster, execute this:

$ kubectl -n kube-system delete secret filebeat-coralogix-account-secrets
$ kubectl delete -k https://github.com/coralogix/integrations-docs/integrations/filebeat/kubernetes