FluentD
- coralogix
- March , 2017,03
Coralogix provides a seamless integration with FluentD so you can send your logs from anywhere and parse them according to your needs.
Prerequisites
Have FluentD installed, for more information on how to implement: FluentD implementation docs
Usage
You must provide the following four variables when creating a Coralogix logger instance.
Company ID – A private key which is used to validate your authenticity, this key will be sent to your mail once you register to Coralogix and should not be transferred outside your company.
Private Key – A unique ID which represents your company, this Id will be sent to your mail once you register to Coralogix.
Application Name – The name of your environment, for example, a company named “SuperData” would probably insert the “SuperData” string parameter or if they want to debug their test environment they might insert the “SuperData– Test”.
SubSystem Name – Your application probably has multiple components, for example: Backend servers, Middleware, Frontend servers etc. in order to help you examine the data you need, inserting the subsystem parameter is vital.
Installation
td-agent:
td-agent-gem install fluent-plugin-coralogix
Ruby:
gem install fluent-plugin-coralogix
Configuration
Open your Fluentd configuration file and add coralogix output.
If you installed Fluentd using the td-agent packages, the config file is located at /etc/td-agent/td-agent.conf.
If you installed Fluentd using the Ruby Gem, the config file is located at /etc/fluent/fluent.conf.
<match *.*>
@type coralogix
config {"PRIVATE_KEY":"01010101-0101-0101-0101-0101010101010", "APP_NAME": "YOUR_APPLICATION_NAME", "SUB_SYSTEM": "YOUR_SUB_SYSTEM_NAME"}
log_key_name SOME_KEY_NAME
</match>
The fist two keys (type and config) are mandatory while the last one is optional.
In case your input stream is a JSON object, you can extract APP_NAME and/or SUB_SYSTEM from the JSON using the $ sign. For instance, in the bellow JSON $kubernetes. pod_name will extract “my name” value.
{
"context": "something",
"code": "200",
"stream": "stdout",
"docker": {
"container_id": "e518dc690e2bc3314842d5bd98b9e24ff7686daa573d063033ea023426c7f667"
},
"kubernetes": {
"namespace_name": "default",
"pod_id": "e061eb42-4e4b-11e6-9fd1-fa163edd44fd",
"pod_name": "my name",
"container_name": "some container",
"host": "myhost"
},
"k8scluster": "ci",
"@timestamp": "2016-07-20T17:05:17.743Z",
"message": "{"context":"something" , "code":"200" }\n",
"type": "k8s",
}
In case your input stream is a JSON object and you don’t want to send the entire JSON, rather just a portion of it, you can write the value of the key you want to send in the log_key_name.
For instance, in the above example, if you write log_key_name kubernetes then only the value of kubernetes key will be sent.
If you do want to send the entire message then you can just delete this key.
Restart fluentd.
