Kubernetes with Fluent Bit (Without Helm)
This tutorial demonstrates how to send your Kubernetes logs to Coralogix using Fluent Bit. Find full instructions in our GitHub repository.
Prerequisites
-
Installed Kubernetes Cluster
-
Enabled RBAC authorization mode support
This document includes cluster dependent URL's. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches your Coralogix domain. Replace the variable part of the URL with this entry.
[table id=51 /]
Installation
First, you should create Kubernetes secret with Coralogix credentials:
kubectl -n kube-system create secret generic fluent-bit-coralogix-account-secrets \
--from-literal=CORALOGIX_API_URL=<your_account_api> \
--from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
You should receive something like:
Then you need to create fluent-bit-coralogix-logger
resources on your Kubernetes cluster with these commands:
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-rbac.yaml -n monitoring
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-svc.yaml -n monitoring
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-cm.yaml -n monitoring
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-ds.yaml -n monitoring
Output
serviceaccount "fluent-bit-coralogix-service-account" created
clusterrole "fluent-bit-coralogix-service-account-role" created
clusterrolebinding "fluent-bit-coralogix-service-account" created
configmap "fluent-bit-coralogix-config" created
daemonset "fluent-bit-coralogix-daemonset" created
service "fluent-bit-coralogix-service" created
Now fluent-bit-coralogix-logger
collects logs from your Kubernetes cluster.
Uninstall
To remove fluent-bit-coralogix-logger
from your cluster, execute the following commands:
kubectl -n kube-system delete secret fluent-bit-coralogix-account-secrets
kubectl -n kube-system delete svc,ds,cm,clusterrolebinding,clusterrole,sa -l k8s-app=fluent-bit-coralogix-logger
Modifying applicationName and subsystemName
By default, we use the field kubernetes.namespace_name
as the applicationName and kubernetes.container_name
as the subsystemName.
Dynamic
To modify these values and use another value for applicationName and subsystemName, modify the fluent-bit-http-crxluascript
config map, as demonstrated in the example:
{
"kubernetes": {
"container_name": "generator",
"namespace_name": "default",
"pod_name": "generator-app-589dbdc98-ghz8j",
"container_image": "chentex/random-logger:latest",
"container_image_id": "docker-pullable://chentex/random-logger@sha256:7cae589926ce903c65a853c22b4e2923211cc19966ac8f8cc533bbcff335ca39",
"pod_id": "330ta782-a1ab-4daa-b3fa-5eb3f3d07fe0",
"pod_ip": "177.17.0.4",
"host": "minikube",
"labels": {
"app": "generator",
}
},
"log": "2022-12-11T16:43:15+0000 DEBUG This is a debug log that shows a log that can be ignored.n",
"stream": "stdout",
"time": "2022-12-11T16:43:15.906733172Z",
}
The following example adopts the 'app' label from the Kubernetes object as subsystemName by modifying script.lua and supplying the wanted field in this format: record.json.<field_as_json_path>
.
removed for brevity...
new_record["subsystemName"] = record.json.kubernetes.labels.app
removed for brevity...
Note: As this script runs on all logs, make sure to use a field that is present in all the logs or add if/else logic to the lua script.
Static
To change these values and use a hard-coded value as the value of applicationName and subsystemName, modify the fluent-bit-http-crxluascript
config map. The following example allows all logs to adopt the the applicationName 'my-awesome-app' with a modification of the script.lua.
Additional Resources
External Documentation | GitHub |
Support
Need help?
Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.
Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].