Kubernetes with Fluent Bit (Without Helm)

This tutorial demonstrates how to send your Kubernetes logs to Coralogix using Fluent Bit. Find full instructions in our GitHub repository.

Prerequisites

• Installed Kubernetes Cluster
• Enabled RBAC authorization mode support

This document includes cluster dependent URL’s. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches your Coralogix domain. Replace the variable part of the URL with this entry.

Installation

First, you should create Kubernetes secret with Coralogix credentials:

kubectl -n kube-system create secret generic fluent-bit-coralogix-account-secrets \ 
--from-literal=CORALOGIX_API_URL=<your_account_api> \ 
--from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX 

You should receive something like:

secret "fluent-bit-coralogix-account-secrets" created

Then you need to create fluent-bit-coralogix-logger resources on your Kubernetes cluster with these commands:

kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-rbac.yaml -n monitoring
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-svc.yaml -n monitoring
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-cm.yaml -n monitoring
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-ds.yaml -n monitoring

Output

serviceaccount "fluent-bit-coralogix-service-account" created
clusterrole "fluent-bit-coralogix-service-account-role" created
clusterrolebinding "fluent-bit-coralogix-service-account" created
configmap "fluent-bit-coralogix-config" created
daemonset "fluent-bit-coralogix-daemonset" created
service "fluent-bit-coralogix-service" created

Now fluent-bit-coralogix-logger collects logs from your Kubernetes cluster.

Uninstall

To remove fluent-bit-coralogix-logger from your cluster, execute the following commands:

kubectl -n kube-system delete secret fluent-bit-coralogix-account-secrets
kubectl -n kube-system delete svc,ds,cm,clusterrolebinding,clusterrole,sa -l k8s-app=fluent-bit-coralogix-logger

Modifying applicationName and subsystemName

By default, we use the field kubernetes.namespace_name as the applicationName and kubernetes.container_name as the subsystemName.

Dynamic

To modify these values and use another value for applicationName and subsystemName, modify the fluent-bit-http-crxluascript config map, as demonstrated in the example:

{
	"kubernetes": {
		"container_name": "generator",
		"namespace_name": "default",
		"pod_name": "generator-app-589dbdc98-ghz8j",
		"container_image": "chentex/random-logger:latest",
		"container_image_id": "docker-pullable://chentex/random-logger@sha256:7cae589926ce903c65a853c22b4e2923211cc19966ac8f8cc533bbcff335ca39",
		"pod_id": "330ta782-a1ab-4daa-b3fa-5eb3f3d07fe0",
		"pod_ip": "177.17.0.4",
		"host": "minikube",
		"labels": {
			"app": "generator",
		}
	},
	"log": "2022-12-11T16:43:15+0000 DEBUG This is a debug log that shows a log that can be ignored.n",
	"stream": "stdout",
	"time": "2022-12-11T16:43:15.906733172Z",
}

The following example adopts the ‘app’ label from the Kubernetes object as subsystemName by modifying script.lua and supplying the wanted field in this format: record.json.<field_as_json_path>.

removed for brevity...
    new_record["subsystemName"] = record.json.kubernetes.labels.app
removed for brevity...

Note: As this script runs on all logs, make sure to use a field that is present in all the logs or add if/else logic to the lua script.

Static

To change these values and use a hard-coded value as the value of applicationName and subsystemName, modify the fluent-bit-http-crxluascript config map. The following example allows all logs to adopt the the applicationName ‘my-awesome-app’ with a modification of the script.lua.

removed for brevity...
    new_record["applicationName"] = "my-awesome-app"
removed for brevity...

Additional Resources

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].