Fluentd is a versatile data shipper with numerous available plugins and functionalities, playing a pivotal role as a logs shipper to our platform. Below are instructions on how to set up the Fluentd shipper along with the http output plugin to transmit logs to the Coralogix platform.
Before you begin, make sure you have:
To specify essential environment variables, create a configmap as shown below:
--- ## fluentd-env-cm.yaml: apiVersion: v1 kind: ConfigMap metadata: labels: app.kubernetes.io/name: fluentd app.kubernetes.io/instance: fluentd-http name: fluentd-env data: ENDPOINT: ingress.coralogix.com LOG_LEVEL: error
Notes:
Apply the configmap:
kubectl apply -f fluentd-env-cm.yaml
Next, apply the manifest files in this directory:
kubectl apply -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-cm.yaml kubectl apply -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-rbac.yaml kubectl apply -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-svc.yaml kubectl apply -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-ds.yaml
You should see the following output:
configmap/fluentd-prometheus-conf created configmap/fluentd-systemd-conf created configmap/fluentd-config created configmap/fluentd-main created daemonset.apps/fluentd-http created serviceaccount/fluentd-http created clusterrole.rbac.authorization.k8s.io/fluentd-http created clusterrolebinding.rbac.authorization.k8s.io/fluentd-http created servicemonitor.monitoring.coreos.com/fluentd-http created service/fluentd-http created
If you have the prometheus-operator installed, you can install this service monitor resource:
kubectl apply -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-svc-monitor.yaml
By default, we utilize the field kubernetes.namespace_name
as the applicationName and kubernetes.container_name
as the subsystemName.
To modify these values and use another field as the value of applicationName and subsystemName, modify the fluentd-config
configmap and specifically the coralogix.conf
key.
To modify these values and use a hard-coded value as the value of applicationName and subsystemName, modify the fluentd-config
configmap and specifically the coralogix.conf
key.
To remove all resources created with manifest files, use these commands:
kubectl delete -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-cm.yaml kubectl delete -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-rbac.yaml kubectl delete -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-svc.yaml kubectl delete -f https://raw.githubusercontent.com/coralogix/telemetry-shippers/master/logs/fluentd/k8s-manifest/fluentd-ds.yaml kubectl delete -f fluentd-env-cm.yaml
The output should be:
configmap "fluentd-prometheus-conf" deleted configmap "fluentd-systemd-conf" deleted configmap "fluentd-config" deleted configmap "fluentd-main" deleted daemonset.apps "fluentd-http" deleted serviceaccount "fluentd-http" deleted clusterrole.rbac.authorization.k8s.io "fluentd-http" deleted clusterrolebinding.rbac.authorization.k8s.io "fluentd-http" deleted servicemonitor.monitoring.coreos.com "fluentd-http" deleted service "fluentd-http" deleted configmap "fluentd-env" deleted
Region | Logs Endpoint |
---|---|
EU | ingress.coralogix.com |
EU2 | ingress.eu2.coralogix.com |
US | ingress.coralogix.us |
SG | ingress.coralogixsg.com |
IN | ingress.coralogix.in |
To disable the systemd logs, remove the fluentd-systemd-conf
configmap:
kubectl delete cm fluentd-systemd-conf
Under the dashboard
directory, you’ll find a Fluentd Grafana dashboard provided by Coralogix. To import the dashboard into Grafana, copy the JSON file content, navigate to Grafana, click on the Create
tab, then import
, and paste the copied JSON file.
To create an alert on Fluentd buffer in Coralogix, refer to the coralogix-alert doc.
External Documentation | GitHub repository |
Need help?
Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.
Contact us via our in-app chat or by emailing [email protected].