Breaking News from AWS re:Invent
Coralogix receives AWS Rising Star award!
This tutorial demonstrates how to send your Kubernetes logs to Coralogix using Fluent Bit. Find full instructions in our GitHub repository.
This document includes cluster dependent URL’s. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches your Coralogix domain. Replace the variable part of the URL with this entry.
Coralogix Domain | Elasticsearch-API | SSL Certificates |
---|---|---|
coralogix.com | https://coralogix-esapi.coralogix.com:9443 | https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-EU.crt |
coralogix.in | https://es-api.app.coralogix.in:9443 | https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-IN.pem |
coralogix.us | https://esapi.coralogix.us:9443 | https://www.amazontrust.com/repository/AmazonRootCA1.pem |
coralogixsg.com | https://es-api.coralogixsg.com:9443 | https://www.amazontrust.com/repository/AmazonRootCA1.pem |
eu2.coralogix.com | https://es-api.eu2.coralogix.com:9443 | https://www.amazontrust.com/repository/AmazonRootCA1.pem |
First, you should create Kubernetes secret with Coralogix credentials:
kubectl -n kube-system create secret generic fluent-bit-coralogix-account-secrets \ --from-literal=CORALOGIX_API_URL=<your_account_api> \ --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
You should receive something like:
secret "fluent-bit-coralogix-account-secrets" created
Then you need to create fluent-bit-coralogix-logger
resources on your Kubernetes cluster with these commands:
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-rbac.yaml -n monitoring kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-svc.yaml -n monitoring kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-cm.yaml -n monitoring kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-ds.yaml -n monitoring
serviceaccount "fluent-bit-coralogix-service-account" created
clusterrole "fluent-bit-coralogix-service-account-role" created
clusterrolebinding "fluent-bit-coralogix-service-account" created
configmap "fluent-bit-coralogix-config" created
daemonset "fluent-bit-coralogix-daemonset" created
service "fluent-bit-coralogix-service" created
Now fluent-bit-coralogix-logger
collects logs from your Kubernetes cluster.
To remove fluent-bit-coralogix-logger
from your cluster, execute the following commands:
kubectl -n kube-system delete secret fluent-bit-coralogix-account-secrets
kubectl -n kube-system delete svc,ds,cm,clusterrolebinding,clusterrole,sa -l k8s-app=fluent-bit-coralogix-logger
By default, we use the field kubernetes.namespace_name
as the applicationName and kubernetes.container_name
as the subsystemName.
To modify these values and use another value for applicationName and subsystemName, modify the fluent-bit-http-crxluascript
config map, as demonstrated in the example:
{ "kubernetes": { "container_name": "generator", "namespace_name": "default", "pod_name": "generator-app-589dbdc98-ghz8j", "container_image": "chentex/random-logger:latest", "container_image_id": "docker-pullable://chentex/random-logger@sha256:7cae589926ce903c65a853c22b4e2923211cc19966ac8f8cc533bbcff335ca39", "pod_id": "330ta782-a1ab-4daa-b3fa-5eb3f3d07fe0", "pod_ip": "177.17.0.4", "host": "minikube", "labels": { "app": "generator", } }, "log": "2022-12-11T16:43:15+0000 DEBUG This is a debug log that shows a log that can be ignored.n", "stream": "stdout", "time": "2022-12-11T16:43:15.906733172Z", }
The following example adopts the ‘app’ label from the Kubernetes object as subsystemName by modifying script.lua and supplying the wanted field in this format: record.json.<field_as_json_path>.
removed for brevity... new_record["subsystemName"] = record.json.kubernetes.labels.app removed for brevity...
Note: As this script runs on all logs, make sure to use a field that is present in all the logs or add if/else logic to the lua script.
To change these values and use a hard-coded value as the value of applicationName and subsystemName, modify the fluent-bit-http-crxluascript
config map. The following example allows all logs to adopt the the applicationName ‘my-awesome-app’ with a modification of the script.lua.
removed for brevity... new_record["applicationName"] = "my-awesome-app" removed for brevity...
External Documentation | GitHub |
Need help?
Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.
Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].