We're launching a new cybersecurity venture! Learn more about Snowbit >

Kubernetes with Fluent-Bit Kubernetes with Fluent-Bit

Last Updated: Mar. 30, 2022
kubernetes versions

The following manual will help you integrate your Kubernetes logs to Coralogix using Fluent-Bit.

See other tutorials for:

Prerequisites

Before you will begin, make sure that you already have:

  • Installed Kubernetes Cluster
  • Enabled RBAC authorization mode support

This document includes cluster dependent URL’s. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.

 Elasticsearch-APISSL CertificatesCluster URL
.comhttps://coralogix-esapi.coralogix.com:9443https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-EU.crtcoralogix.com
.ushttps://esapi.coralogix.us:9443https://www.amazontrust.com/repository/AmazonRootCA1.pemcoralogix.us
.inhttps://es-api.app.coralogix.in:9443https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-IN.pemapp.coralogix.in
.eu2.https://es-api.eu2.coralogix.com:9443https://www.amazontrust.com/repository/AmazonRootCA1.pemapp.eu2.coralogix.com
sg.comhttps://es-api.coralogixsg.com:9443https://www.amazontrust.com/repository/AmazonRootCA1.pemapp.coralogixsg.com

Installation

First, you should create Kubernetes secret with Coralogix credentials:

kubectl -n kube-system create secret generic fluent-bit-coralogix-account-secrets --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

You should receive something like:

secret "fluent-bit-coralogix-account-secrets" created

If your account URL doesn’t end with .com use:

kubectl -n kube-system create secret generic fluentd-coralogix-account-secrets \ --from-literal=CORALOGIX_LOG_URL=https://api.Cluster URL/api/v1/logs \ --from-literal=CORALOGIX_TIME_DELTA_URL=https://api.Cluster URL/sdk/v1/time \ --from-literal=PRIVATE_KEY=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX \ --from-literal=APP_NAME=fluentd-coralogix-image \ --from-literal=SUB_SYSTEM=fluentd

Then you need to create fluent-bit-coralogix-logger resources on your Kubernetes cluster with these manifests:

kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-rbac.yaml
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-svc.yaml

Plugin-based

kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-cm.yaml
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-coralogix-ds.yaml

Native-based

kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-cm.yaml
kubectl create -f https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/kubernetes/fluent-bit-native-coralogix-ds.yaml

Output

serviceaccount "fluent-bit-coralogix-service-account" created
clusterrole "fluent-bit-coralogix-service-account-role" created
clusterrolebinding "fluent-bit-coralogix-service-account" created
configmap "fluent-bit-coralogix-config" created
daemonset "fluent-bit-coralogix-daemonset" created
service "fluent-bit-coralogix-service" created

Now fluent-bit-coralogix-logger collects logs from your Kubernetes cluster.

Here is the example of log record:

{
    "log": "172.17.0.1 - - [05/Apr/2020:22:59:52 +0000] \"GET / HTTP/1.1\" 200 6 \"\" \"kube-probe/1.18\"\n",
    "stream": "stdout",
    "time": "2020-04-05T22:59:52.096035683Z",
    "kubernetes": {
        "pod_name": "dashboard-metrics-scraper-84bfdf55ff-l66cf",
        "namespace_name": "kubernetes-dashboard",
        "labels": {
            "k8s-app": "dashboard-metrics-scraper",
            "pod-template-hash": "84bfdf55ff"
        },
        "annotations": {
            "seccomp.security.alpha.kubernetes.io/pod": "runtime/default"
        },
        "host": "minikube",
        "container_name": "dashboard-metrics-scraper",
        "container_image": "kubernetesui/metrics-scraper:v1.0.2"
    }
}

Uninstall

If you want to remove fluent-bit-coralogix-logger from your cluster, execute this:

kubectl -n kube-system delete secret fluent-bit-coralogix-account-secrets
kubectl -n kube-system delete svc,ds,cm,clusterrolebinding,clusterrole,sa -l k8s-app=fluent-bit-coralogix-logger

On this page