Fluent Bit

$ wget -o /fluent-bit/plugins/out_coralogix.so https://github.com/coralogix/integrations-docs/raw/master/integrations/fluent-bit/plugin/out_coralogix.so

Configuration

Common

Open your Fluent-Bit configuration file and add Coralogix output:

[INPUT]
    Name mem
    Tag memory

[OUTPUT]
    Name        coralogix
    Match       *
    Private_Key YOUR_PRIVATE_KEY
    App_Name    APP_NAME
    Sub_Name    SUB_NAME

The first four keys (Private_key, Company_id, App_name, Sub_name) are mandatory.

Application and subsystem name

If you wish to set your application and subsystem names to a fixed value, in your config file use App_Name and Sub_Name as described above. In case your input stream is a JSON object, you can extract APP_NAME and/or SUB_NAME from the JSON using the App_Name_Key and Sub_Name_Key options in your config file instead of App_Name and Sub_Name:

App_Name_Key APP_NAME_KEY
Sub_Name_Key SUB_NAME_KEY

For instance, with the bellow JSON App_Name_Key application will extract “testApp“ into Coralogix applicationName.

{
    "application": "testApp",
    "subsystem": "testSub",
    "code": "200",
    "stream": "stdout",
    "timestamp": "2016-07-20T17:05:17.743Z",
    "message": "hello_world",
}

*Note – nested JSONs are also supported so you can extract into App_Name_Key and/or Sub_Name_Key nested values, e.g. App_Name_Key log.application.

Record content

In case your input stream is a JSON object and you don’t want to send the entire JSON, rather just a portion of it, you can add the Log_Key parameter, in your Fluent-Bit configuration file–>output section, with the name of the key you want to send. For instance, with the above example, if you write:

Log_Key message

then only the key message will be sent. If you do want to send the entire JSON then you can just delete this parameter from your configuration file.

Timestamp

If you want to use some field as timestamp in Coralogix, you can use Time_Key option:

Time_Key timestamp

then you will see that logs records have timestamp from this field.

Note: We accept only logs that are not older than 24 hours.

Run

On host machine

If your Coralogix account top level domain is different than ‘.com’ set the following environment variable

CORALOGIX_LOG_URL=https://api.Cluster URL.in/logs/rest/singles

To start Fluent-Bit with Coralogix output plugin, execute:

$ fluent-bit -e /fluent-bit/plugins/out_coralogix.so -c /fluent-bit/etc/fluent-bit.conf

[PLUGINS]
    Path /fluent-bit/plugins/out_coralogix.so

Docker

Build a Docker image with your fluent-bit.conf:

If your Coralogix account top level domain is different than ‘.com’ set the following environment variable

CORALOGIX_LOG_URL=https://api.Cluster URL.in/logs/rest/singles

FROM golang:alpine AS builder
RUN apk add --no-cache gcc libc-dev git
WORKDIR /go/src/app
RUN wget https://raw.githubusercontent.com/fluent/fluent-bit/master/conf/plugins.conf && \
    echo "    Path /fluent-bit/plugins/out_coralogix.so" | tee -a plugins.conf
RUN wget https://raw.githubusercontent.com/coralogix/integrations-docs/master/integrations/fluent-bit/plugin/out_coralogix.go && \
    go get . && \
    go build -buildmode=c-shared -o out_coralogix.so .


FROM fluent/fluent-bit:1.4
MAINTAINER Coralogix Inc. <info@coralogix.com>
LABEL Description="Special Fluent-Bit image for Coralogix integration" Vendor="Coralogix Inc." Version="1.0.0"
COPY --from=builder /lib/libc.musl-x86_64.so* /lib/x86_64-linux-gnu/
COPY --from=builder /go/src/app/out_coralogix.so /fluent-bit/plugins/
COPY --from=builder /go/src/app/plugins.conf /fluent-bit/etc/
COPY fluent-bit.conf /fluent-bit/etc/

$ cd plugin/
$ make

{
  ...
  "text": "Geo-point data",
  "location": { 
    "lat": 41.12,
    "lon": -71.34
  }
  ...
}

{
  ...
  "text": "Geo-point data",
  "location_geopoint": { 
    "lat": 41.12,
    "lon": -71.34
  },
  ...
}

{
  ...
  "time": "2020-10-13T09:45:33.783441Z",
  ...
}

{
 ... 
 "time_custom_timestamp": "2020-10-13T09:45:33.783441Z",
 ... 
}

{
  ...
  "ip_addr": "192.168.1.1",
  ...
}

{
  ...
  "ip_addr_ipaddr": "192.168.1.1",
  ...
}