Manage Schemas with Reserved Fields

Overview

Coralogix ingests data documents containing multiple fields with specific data types. Fields are dynamically stored and indexed in the order they are ingested. Once the maximum number of stored fields is reached, additional fields may not be available for querying.

Use reserved fields to avoid dependency on ingestion order and gain more control over field creation and storage. This feature allows you to explicitly define fields of importance for querying and monitoring purposes, while still enabling Coralogix to dynamically add other fields as needed.

With reserved fields, you can:

• Ensure important fields are always available for queries across the Coralogix platform.

• Specify the data type for each field to make storage and querying more efficient.

• Prevent indexed field limitations that could impact query performance.

How it works

When Coralogix ingests data, it automatically maps fields to an index using OpenSearch. This automatic mapping enables dynamic field additions but may result in missing or misclassified fields due to ingestion order. Reserved fields allow you to override this behavior by predefining specific fields and their data types.

Index mapping

Reserved fields are applied automatically at 00:00 UTC each day when a new daily index is created. During this process:

• Fields explicitly defined as reserved are pre-mapped in the index.

• Any additional fields are mapped dynamically as data is ingested.

If needed, users can perform a manual index reset once per day. This allows newly added reserved fields to be mapped immediately, without waiting for the midnight split.

Data types

Each field has an associated data type, determining the kind of data it holds. The following types are supported:

Data type	Description	Example JSON usage
boolean	Represents a true or false value, commonly used for logical checks and conditions.	true or false
string	A sequence of characters, often used for text, identifiers, or alphanumeric data. Stored internally in UTF-8 encoding.	"example text"
number	A numeric value, either an integer or a floating-point number, used for calculations or measurements.	123 or 45.67

Coralogix tracks data types throughout the ingestion and query process, maintaining a data lineage that records type conversions and transformations. This ensures better validation, autocompletion, and query-building support within DataPrime.

Getting started

Access reserved fields by navigating to Data Flow > Reserved Fields from your Coralogix toolbar.

Adding reserved fields

Manually add detected fields

To reserve fields that have been ingested by Coralogix in the last 24 hours, take the following steps.

STEP 1. Click + Add field from the Reserved Fields screen.

STEP 2. Type the field name. A dropdown of detected fields will appear. Select the desired field.

STEP 3. If the data type is mismatched, you may override it by selecting a different datatype in the drop-down menu.

STEP 4. Click Save.

Manually add undetected fields

To reserve fields that have not been ingested by Coralogix and are undetected by the system, take the following steps.

STEP 1. Click + Add field from the Reserved Fields screen. Type the field name.

STEP 2. Select the field’s data type in the drop-down menu.

STEP 3. Click Save.

Select fields from a full list of detected fields

STEP 1. Click + Add field from the Reserved Fields screen.

STEP 2. Click on the search bar that appears.

STEP 3. In the dropdown menu, click View full list.

STEP 4. In the popup modal that appears, select the detected fields that you would like to reserve. You may modify the time picker to change the time period for which fields were detected.

STEP 5. Click Add to list.

STEP 6. Click Save.

Managing reserved fields

Search and filter

Use the search bar in the Reserved Fields Management screen to locate fields based on specific criteria.

Edit or remove fields

• Edit: Update field properties directly in the Reserved Fields screen.

• Remove: Remove fields no longer required.

Permissions

To view or manage reserved fields, users must have the required permissions:

Resource	Action	Description
logs.reserved-fields	ReadConfig	View-only access to reserved fields.
logs.reserved-fields	Manage	Full control, including creating, editing, and deleting reserved fields.

Previous Archive Retention Policy

Next Metric Explorer

Theme

Light

light

dark