Dataset management

Dataset Management is the central place to view, configure, and create datasets across your dataspaces. Navigate to Data Flow, then Dataset Management.

The page is organized into tabs by dataspace:

• Default — view predefined datasets and create user-defined datasets
• System — enable or disable Coralogix-managed system datasets

Default datasets

The Default tab lists all datasets in the default/ dataspace. The predefined logs and spans datasets are shown as placeholders and cannot be edited. User-defined datasets you create appear below them.

Hover over a dataset row to access the following actions:

• Edit the dataset description and access policy.
• Delete the dataset.
• View in Explore to open the dataset with the query source default/<dataset-name> pre-filled.

Create a user-defined dataset

Create a custom dataset to isolate a specific log stream, apply access policies, and route data to it via TCO Optimizer.

1. Select the Default tab.
2. Select Create Dataset.
3. In Dataset name, enter a unique name. Names must be unique within default/ and cannot be changed after creation.
4. In Description, enter an optional description.
5. In Bucket name, the storage bucket is inherited from your archive configuration. Optionally, enter a different bucket name.

6. Under Access Policy, optionally toggle Enabled Access Policy on to restrict access and configure exception rules.

   Warning

   When the access policy is disabled, everyone has read access to the dataset. When enabled, you can create exceptions to this default behavior.

7. Select Create.

For a full walkthrough including TCO routing and access control, see User-defined datasets.

System datasets

The system/ dataspace contains Coralogix-managed datasets that store internal metadata about your environment, including schema changes, alert history, and audit logs.

System datasets must be manually enabled before they start ingesting data.

Hover over a dataset row to access the following actions:

• Edit the access policy and quota limits.
• View in Explore to open the dataset with the query source system/<dataset-name> pre-filled.

Enable or disable a dataset

For both default and system datasets, toggle Write Access to start or stop data ingestion. Read access and other actions are determined by the access policy.

Once enabled:

• All users can query the dataset.
• The dataset counts toward your daily ingestion quota.
• Previously ingested data remains accessible even if you later disable the dataset.

Disabling a dataset stops ingestion but does not delete or archive stored data. Any configured quota limit is preserved and takes effect again when you re-enable the dataset.

Configure and edit a dataset

Hover over a system dataset row and select the edit icon to open the Edit Dataset dialog box. From here you can update the description, change the storage bucket, set a per-dataset quota limit, and configure the access policy.

1. In the Details section, optionally edit the Description and Bucket name. The Dataset name is fixed and cannot be changed.

   

2. Under Quota limit, optionally toggle Enable quota limit on to cap daily ingestion for the dataset. When ingestion reaches the limit, it halts for the rest of the calendar day. At 00:00 UTC, the counter resets and ingestion resumes. Previously ingested data remains accessible even after ingestion stops.

   Quota limits are based on the daily units a dataset can consume — its available daily quota. This pool comes from one of the following sources:

   • Units allocated to this dataset specifically, or to its entity type as a whole (such as logs), in Quota Rules.
   • The unallocated pool of units remaining after allocations to other entity types (such as metrics or spans) are subtracted.

   Select an input mode:

   • Percent — enter the limit as a percentage of the available daily quota. The slider hard-stops at the allocatable amount after subtracting reservations by other datasets in the same entity type.
   • Units — enter the limit as an absolute number of CX Units, shown as units/day.

   As you move the slider, the dialog displays the live conversion:

   {N}% of available daily quota ≈ {M} units/day (available quota depends on entity type allocations)

   For example, 11% ≈ 5 units/day, and 93% ≈ 43 units/day. The slider moves in integer steps (1% or 1 unit).

   

3. Under Access Policy, optionally toggle Enabled Access Policy on to restrict access and configure exception rules. For details, see Access policies.

4. Select Save.

The Quota column in the System datasets table shows current usage against the configured limit (for example, 5 units/day / 43 units/day). Rows with no quota configured display —.