Learn more about Streama© – the foundational technology behind our stateful streaming data platform. Learn More

Vector

Vector

Coralogix provides seamless integration with Vector so you can send your logs from anywhere and parse them according to your needs.

Prerequisites

Have Vector installed, for more information on how to implement: https://vector.dev/docs/setup/installation/

This document includes cluster dependent URL’s. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.

 EUINUSA
Cluster URLcoralogix.comapp.coralogix.incoralogix.us
Team URL.coralogix.com.app.coralogix.in.app.coralogix.us

Usage

You must provide the following four variables when creating a Coralogix logger instance.

Private Key – A unique ID that represents your company. The private key can be found under ‘settings’->’ send your logs’. It is located in the upper left corner.

Application Name – The name of your environment, for example, a company named “SuperData” would probably insert the “SuperData” string parameter or if they want to debug their test environment they might insert the “SuperData– Test”.

SubSystem Name – Your application probably has multiple components, for example, Backend servers, Middleware, Frontend servers, etc. in order to help you examine the data you need, inserting the subsystem parameter is vital.

<Cluster_URL> – use the table above to find your Cluster URL

Configuration

After the installation there is a default configuration file vector.toml. Make a backup of the file if needed. Next, edit vector.toml and paste the following configuration below. Remember to update:

  • event.log.applicationName = “VECTOR_APP”
  • event.log.subsystemName = “VECTOR_SUBSYS”
  • uri = “https://api.<Cluster URL>/api/v1/logs/rest/singles”
  • headers.private_key = “<<YOUR_CORALOGIX_PRIVATE_KEY_HERE>>”
# This input is just for test or demo purposes and is supported only since Vector v0.12.0. Feel free to use any other input.
[sources.my_source_id]
  type = "generator" # required
  lines = ["{\"msg\":\"success\", \"code\":200}", "{\"msg\":\"error\", \"code\":500}"] # optional, no default, relevant when [`format`](#format) = `shuffle`
  sequence = false # optional, default, relevant when [`format`](#format) = `shuffle`
  batch_interval = 10
  format = "shuffle"

[transforms.prepare_for_coralogix]
  # General
  type = "lua" # required
  inputs = ["my_source_id"] # required
  version = "2" # required
  # Hooks
  hooks.process = '''
  function (event, emit)
    event.log = {
      applicationName = "VECTOR_APP",
      subsystemName = "VECTOR_SUBSYS",
      json = event.log,
      computerName = os.getenv("HOSTNAME"),
      timestamp = os.time() * 1000
    }
    emit(event)
  end
  '''

[sinks.coralogix]
  # General
  type = "http" # required
  inputs = ["prepare_for_coralogix"] # required
  compression = "none" # optional, default
  uri = "https://api.<Cluster URL>/logs/rest/singles" # required
  # Batch
  batch.max_bytes = 1049000 # optional, default, bytes
  batch.timeout_secs = 2 # optional, default, seconds
  # Encoding
  encoding.codec = "json" # required
  # Healthcheck
  #healthcheck.enabled = true # optional, default
  headers.Content-Type = "application/json"
  headers.private_key = "<<YOUR_CORALOGIX_PRIVATE_KEY_HERE>>"

That code will allow you to generate the test traffic.
After testing the connection you can add your own sources, for example:

[sources.my_source_id]
  type = "file"
  include = ["/var/log/access.log "]

You can read more about type of sources at the configuration here.

Remember to restart vector service after every change to load updated configuration.