Reserved Fields

Overview

Coralogix ingests data documents containing multiple fields with specific data types. Fields are dynamically stored and indexed in the order they are ingested. Once the maximum number of stored fields is reached, additional fields may not be available for querying.

Use Reserved Fields to avoid dependency on ingestion order and gain more control over field creation and storage. This feature allows you to explicitly define fields of importance for querying and monitoring purposes, while still enabling Coralogix to dynamically add other fields as needed.

With Reserved Fields, you can:

Ensure important fields are always available for queries across the Coralogix platform.
Specify the data type for each field to make storage and querying more efficient.
Prevent indexed field limitations that could impact query performance.

How it works

When Coralogix ingests data, it automatically maps fields to an index using OpenSearch. This automatic mapping enables dynamic field additions but may result in missing or misclassified fields due to ingestion order. Reserved Fields allows you to override this behavior by predefining specific fields and their data types.

By default, reserved fields are added at 0:00 (midnight) UTC, when the daily index is created. This means that the user can utilize the reserved field the day after it has been added.

Data types

Each field has an associated data type, determining the kind of data it holds. The following types are supported:

Data type	Description	Example JSON usage
boolean	Represents a `true` or `false` value, commonly used for logical checks and conditions.	`true` or `false`
string	A sequence of characters, often used for text, identifiers, or alphanumeric data. Stored internally in UTF-8 encoding.	`"example text"`
number	A numeric value, either an integer or a floating-point number, used for calculations or measurements.	`123` or `45.67`

Coralogix tracks data types throughout the ingestion and query process, maintaining a data lineage that records type conversions and transformations. This ensures better validation, autocompletion, and query-building support within DataPrime.

Getting started

Access reserved fields by navigating to Data Flow > Reserved Fields from your Coralogix toolbar.

Adding reserved fields

Manually add detected fields

To reserve fields that have been ingested by Coralogix in the last 24 hours, take the following steps.

STEP 1. Click + Add field from the Reserved Fields screen.

STEP 2. Type the field name. A dropdown of detected fields will appear. Select the desired field.

STEP 3. If the data type is mismatched, you may override it by selecting a different datatype in the drop-down menu.

STEP 4. Click Save.

Manually add undetected fields

To reserve fields that have not been ingested by Coralogix and are undetected by the system, take the following steps.

STEP 1. Click + Add field from the Reserved Fields screen. Type the field name.

STEP 2. Select the field’s data type in the drop-down menu.

STEP 3. Click Save.

Select fields from a full list of detected fields

STEP 1. Click + Add field from the Reserved Fields screen.

STEP 2. Click on the search bar that appears.

STEP 3. In the dropdown menu, click View full list.

STEP 4. In the popup modal that appears, select the detected fields that you would like to reserve. You may modify the time picker to change the time period for which fields were detected.

STEP 5. Click Add to list.

STEP 6. Click Save.

Managing reserved fields

Search and filter

Use the search bar in the Reserved Fields Management screen to locate fields based on specific criteria.

Edit or remove fields

Edit: Update field properties directly in the Reserved Fields screen.
Remove: Remove fields no longer required.

Permissions

To view or manage reserved fields, users must have the required permissions:

Resource	Action	Description
`logs.reserved-fields`	`ReadConfig`	View-only access to reserved fields.
`logs.reserved-fields`	`Manage`	Full control, including creating, editing, and deleting reserved fields.

Archive Retention Policy

Next Getting Started