Syslog using Fluentd

This tutorial demonstrates how to use custom syslog to send your logs to Coralogix using Fluentd.

Overview

Syslog is a standard for message logging. It allows separation of the software that generates messages, the system that stores them, and the software that reports and analyzes them. Each message is labeled with a facility code, indicating the type of system generating the message, and is assigned a severity level.

When there is no support for custom syslog, an intermediate server is required in order to send the data to the Coralogix account.

View our other syslog integrations:

• Custom Syslog
• Rsyslog

Prerequisites

• Server to install Fluentd
• Static public IP allocated to the server for initial configuration

Deployment

STEP 1. Install Fluentd on the server.

STEP 2. Under /etc/td-agent/, edit the file configuration file called td-agent.conf and replace the content with the following configuration:

<system>
  log_level info
</system>

<source>
  @type syslog
  @label @CORALOGIX
  port 5140
  bind 0.0.0.0
  source_hostname_key true
  tag syslog.messages
</source>

<label @CORALOGIX>
  <filter **>
  @type record_transformer
  @log_level warn
  enable_ruby true
  auto_typecast true
  renew_record true
  <record>
    applicationName "application_name"
    subsystemName "subsystem_name"
    text ${record.to_json}
  </record>
  </filter>
  <match **>
    @type http
    @id http_to_coralogix
    endpoint "https://api.<coralogix domain>/logs/rest/singles" 
    headers {"private_key":"Your Coralogix account private key"}
    retryable_response_codes 503
    error_response_as_unrecoverable false	  
    <buffer>
      @type memory
      chunk_limit_size 10MB
      compress gzip
      flush_interval 1s
      retry_max_times 5
      retry_type periodic
      retry_wait 2
    </buffer>
    <secondary>
      #If any messages fail to send they will be send to STDOUT for debug.
      @type stdout
    </secondary>
  </match>
</label>

Replace the values for:

• applicationName & subsystemName: Application and subsystem names to be displayed in your Coralogix dashboard
• endpoint: Replace <coralogix domain> with your Coralogix domain
• private_key: Replace with your Coralogix private key

STEP 3. Save the td-agent.conf file.

STEP 4. Validate the configuration:

td-agent --dry-run

Note: port 514 is the default port for syslog. In some cases it can be changed, and the Fluentd configuration should be changed accordingly.

STEP 5. If the configuration file validation was successful, start the td-agent service:

sudo systemctl start td-agent.service

STEP 6. Check the status:

sudo systemctl status td-agent.service

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].