The following manual will help you integrate your Kubernetes
logs to Coralogix using Fluent-Bit.
See other tutorials for:
Prerequisites
Before you will begin, make sure that you already have:
- Installed Kubernetes Cluster
- Enabled RBAC authorization mode support
This document includes cluster dependent URL’s. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.
.com | .in | |
---|---|---|
Elasticsearch-API | https://coralogix-esapi.coralogix.com:9443 | https://es-api.app.coralogix.in:9443 |
SSL Certificates | https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-EU.crt | https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-IN .pem |
Cluster URL | coralogix.com | app.coralogix.in |
Installation
First, you should create Kubernetes secret with Coralogix credentials:
kubectl -n kube-system create secret generic fluent-bit-coralogix-account-secrets --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
You should receive something like:
secret "fluent-bit-coralogix-account-secrets" created
If your account URL doesn’t end with .com use:
kubectl -n kube-system create secret generic fluentd-coralogix-account-secrets \ --from-literal=CORALOGIX_LOG_URL=https://api.Cluster URL/api/v1/logs \ --from-literal=CORALOGIX_TIME_DELTA_URL=https://api.Cluster URL/sdk/v1/time \ --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX \ --from-literal=APP_NAME=fluentd-coralogix-image \ --from-literal=SUB_SYSTEM=fluentd
Then you need to create fluent-bit-coralogix-logger
resources on your Kubernetes cluster with these manifests:
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-rbac.yaml
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-svc.yaml
Plugin based
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-cm.yaml
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-ds.yaml
Native based
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-cm.yaml
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-ds.yaml
Output:
serviceaccount "fluent-bit-coralogix-service-account" created clusterrole "fluent-bit-coralogix-service-account-role" created clusterrolebinding "fluent-bit-coralogix-service-account" created configmap "fluent-bit-coralogix-config" created daemonset "fluent-bit-coralogix-daemonset" created service "fluent-bit-coralogix-service" created
Now fluent-bit-coralogix-logger
collects logs from your Kubernetes cluster.
Here is the example of log record:
{ "log": "172.17.0.1 - - [05/Apr/2020:22:59:52 +0000] \"GET / HTTP/1.1\" 200 6 \"\" \"kube-probe/1.18\"\n", "stream": "stdout", "time": "2020-04-05T22:59:52.096035683Z", "kubernetes": { "pod_name": "dashboard-metrics-scraper-84bfdf55ff-l66cf", "namespace_name": "kubernetes-dashboard", "labels": { "k8s-app": "dashboard-metrics-scraper", "pod-template-hash": "84bfdf55ff" }, "annotations": { "seccomp.security.alpha.kubernetes.io/pod": "runtime/default" }, "host": "minikube", "container_name": "dashboard-metrics-scraper", "container_image": "kubernetesui/metrics-scraper:v1.0.2" } }
Uninstall
If you want to remove fluent-bit-coralogix-logger
from your cluster, execute this:
kubectl -n kube-system delete secret fluent-bit-coralogix-account-secrets kubectl -n kube-system delete svc,ds,cm,clusterrolebinding,clusterrole,sa -l k8s-app=fluent-bit-coralogix-logger