Skip to content

Auto-Generated Custom Enrichment Service

The Coralogix Security Traffic Analyzer (STA) collects data using internal enrichment services, including AWS, Kubernetes, and/or geographical information.

It sends these enrichments to Coralogix as CSV files per service, allowing you to enrich any of your logs with additional fields, even if the source is not STA.

Connect all types of logs sent to Coralogix’s platform and improve readability, deepen your understanding of your environments, and reduce investigation time dramatically over large content of logs from different sources.

All custom enrichment files are available on Coralogix’s platform under Data Flow -> Data Enrichment section.

Enrichment Types

STA collects enrichments using several services running in the background.

To see the full list of STA enrichment modules, run the following command inside STA: sta-get-status-short

In the console, you’ll find all running services inside STA, and their current status. See the example below:

All rows with the prefix: coralogix.sta.enrichment represent different services that handle STA enrichments.

Currently the example above includes the following services:

  1. domains-tld-extract

  2. domain-stats

  3. dns-rbls

  4. unshorten-url

  5. domains-similarity

  6. freq-server

  7. geo

  8. nist-cpe

  9. cloud-context

  10. k8s-context

  11. stats-info

Note: We are constantly improving and extending the STA, and more services might be added/modified.

Configuration

To use this API you need to create a personal or team API key. It’s recommended to use permission presets, as they are automatically updated with all relevant permissions. Alternatively, you can manually add individual permissions.

Preset Action Description
Enrichments CLOUD-METADATA-ENRICHMENT:READCONFIG
CLOUD-METADATA-ENRICHMENT:UPDATECONFIG
GEO-ENRICHMENT:READCONFIG
GEO-ENRICHMENT:UPDATECONFIG
SECURITY-ENRICHMENT:READCONFIG
SECURITY-ENRICHMENT:UPDATECONFIG
TEAM-CUSTOM-ENRICHMENT:READDATA
TEAM-CUSTOM-ENRICHMENT:UPDATECONFIG
TEAM-CUSTOM-ENRICHMENT:UPDATEDATA
View AWS Enrichment Configuration
Manage AWS Enrichment Configuration
View Geo Enrichment Configuration
Manage Geo Enrichment Configuration
View Unified Threat Intelligence Enrichment Configuration
View Custom Enrichment Configuration
View Custom Enrichment Data
Manage Custom Enrichment Configuration
Manage Data for Custom Enrichment Ingestion
  1. Copy the API key.

  2. Copy the previously-configured Coralogix custom enrichment endpoint host. The endpoint list is available here.

  3. After you’ve copied the key and configured custom enrichment endpoint host, go to your sta.conf file that can be configured locally in STA or using Amazon S3 bucket,

  4. Search for "coralogix" JSON key (you'll see that you already provided there your "private_key").

  5. Add the following:

{
    "coralogix": {
        ...you will find here your private key specification and other params...,
        "coralogix_alerts_api_key": "<YOUR_COPIED_API_KEY>",
        "coralogix_custom_enrichment_endpoint_host" : "<YOUR_CUSTOM_ENRICHMENT_ENDPOINT_HOST>",
    }
}

Structure

The service configuration is also located in sta.conf with the following structure:

{
    "sensors": {
        "custom_enrichment_producer": {
            "log_level": "TRACE" | "DEBUG" | "INFO" | "WARN" | "ERROR",
            "file_modification_minutes_threshold": int,
            "max_csv_rows": int,
            "enable": bool,
            "customized_services": [
                {
                    "name": str,
                    "headers": [str]
                }
            ],
            "disabled_for_services": [str],
            "other_services_enabled": bool,
        }
    }
}

Default Values

log_level: INFO
file_modification_minutes_threshold: 10
max_csv_rows: 9000
enable: true
customized_services: []
disabled_for_services: []
other_services_enabled: true

Variables

Name Type Constraints Description
log_level Predefined string Possible values:TRACE, DEBUG, INFO, WARN, ERROR The lowest log level to be shown in the console
file_modification_minutes_threshold int 1 ≤ x ≤ 60 Number of minutes threshold of idleness before sending enrichments to Coralogix. This can happen when no new enrichments are found by the STA in a given period of time.
max_csv_rows int 2 ≤ x ≤ 10000 Number of lines threshold per CSV file. if this threshold is reached, the file is sent to Coralogix, and a new file is created afterward.
enable boolean enable/disable service. By default set to true.
customized_services Array of objects Please see 2 rows below for object representation Leave an empty array to include all services without specific configuration.
customized_services.name str Predefined service names. Please see the section "Enrichment Types". Names of the services to enable, if only the names are specified, only those services will be enabled. See "headers" below for additional configuration.
customized_services.headers Array of strings Be sure that you know what headers you want as once you specify, only those will be searched, and others will be dropped. Representing the headers extracted into the service’s enrichment CSV - and only those. Leave an empty array to include all possible headers.
disabled_for_services Array of strings Predefined service names. Please see the section "Enrichment Types". Defines what services to exclude from CSV files.
other_services_enabled boolean Enable other services that are not specified in the variable: customized_services. Find additional information and use cases below.

Use Cases

For a better understanding of how those variables interact with STA, let’s see some use cases below.

Let’s assume that we want to disable cloud-cotext enrichment service from sending CSVs to Coralogix. Our configuration should be as follows:

{
    "sensors": {
        "custom_enrichment_producer": {
            "disabled_for_services": ["cloud-context"],
        }
    }
}

As mentioned, the other values will receive their default values and only the specified service will be disabled.

Now let's assume that we want to configure geo with specific headers: country, zone, coordinate.x, coordinate.y. In addition, we want to enable only cloud-context and k8s-context services without configuration. Our configuration should be as follows:

{
    "sensors": {
        "custom_enrichment_producer": {
          "customized_services": [
            {
              "name":"geo",
              "headers": ["country","zone",".x", "coordinate.y"]
            },
            {
              "name":"cloud-context",
              "headers": []
            },
            {
              "name":"k8s-context",
              "headers": []
            }
          ],
        "disabled_for_services": [],
        "other_services_enabled": false,
        }
    }
}

Finally, let's assume again that we want to configure geo with specific headers: country, zone, coordinate.x, coordinate.y. In addition, we want all other services to enrich without configuration except the service nist-cpe which should be disabled. Our configuration should be as follows:

{
   "sensors": {
       "custom_enrichment_producer": {
          "customized_services": [
            {
              "name":"geo",
              "headers": ["country","zone","coordinate.x", "coordinate.y"]
            }
          ],
          "disabled_for_services": ["nist-cpe"],
          "other_services_enabled": true,
       }
   }
}