FluentD
- coralogix
- March , 2017,03
Coralogix provides a seamless integration with FluentD so you can send your logs from anywhere and parse them according to your needs.
Prerequisites
Have FluentD installed, for more information on how to implement: FluentD implementation docs
Usage
You must provide the following four variables when creating a Coralogix logger instance.
Private Key – A unique ID which represents your company, this Id will be sent to your mail once you register to Coralogix.
Application Name – The name of your environment, for example, a company named “SuperData” would probably insert the “SuperData” string parameter or if they want to debug their test environment they might insert the “SuperData– Test”.
SubSystem Name – Your application probably has multiple components, for example: Backend servers, Middleware, Frontend servers etc. in order to help you examine the data you need, inserting the subsystem parameter is vital.
Installation
td-agent:
td-agent-gem install fluent-plugin-coralogix
Ruby:
gem install fluent-plugin-coralogix
Configuration
Open your Fluentd configuration file and add coralogix output.
If you installed Fluentd using the td-agent packages, the config file is located at /etc/td-agent/td-agent.conf.
If you installed Fluentd using the Ruby Gem, the config file is located at /etc/fluent/fluent.conf.
<match *.*>
@type coralogix
config {"PRIVATE_KEY":"01010101-0101-0101-0101-0101010101010", "APP_NAME": "YOUR_APPLICATION_NAME", "SUB_SYSTEM": "YOUR_SUB_SYSTEM_NAME"}
log_key_name SOME_KEY_NAME
## In case your input stream is a JSON object
#is_json true
</match>
The fist two keys (type and config) are mandatory while the last one is optional.
In case your input stream is a JSON object, you can extract APP_NAME and/or SUB_SYSTEM from the JSON using the $ sign. For instance, in the bellow JSON $kubernetes. pod_name will extract “my name” value.
{
"context": "something",
"code": "200",
"stream": "stdout",
"docker": {
"container_id": "e518dc690e2bc3314842d5bd98b9e24ff7686daa573d063033ea023426c7f667"
},
"kubernetes": {
"namespace_name": "default",
"pod_id": "e061eb42-4e4b-11e6-9fd1-fa163edd44fd",
"pod_name": "my name",
"container_name": "some container",
"host": "myhost"
},
"k8scluster": "ci",
"@timestamp": "2016-07-20T17:05:17.743Z",
"message": "{"context":"something" , "code":"200" }\n",
"type": "k8s",
}
In case your input stream is a JSON object and you don’t want to send the entire JSON, rather just a portion of it, you can write the value of the key you want to send in the log_key_name.
For instance, in the above example, if you write log_key_name kubernetes then only the value of kubernetes key will be sent.
If you do want to send the entire message then you can just delete this key.
Restart fluentd.
