Data model concepts
Coralogix organizes your data with six terms: team, dataspace, dataset, pillar, entity type, and record. Several of them are specific to Coralogix, so this page defines each one and shows how they nest.
How the concepts fit together
We start with what the data is, then move on to where Coralogix keeps it.
Read from the outside in, the hierarchy is:
Each dataset version holds data of one entity type, and so of one pillar. That is what lets a dataset promise a schema.
Terms
Team
The Coralogix account your data belongs to. Every dataspace sits inside a team. To find your team ID, see Team ID. A DataPrime query can read datasets from other teams you have access to with the teamId source parameter. See source.
Dataspace
A logical container for datasets, like a database. Coralogix provides the default, system, and frequentsearch dataspaces. You can't create your own dataspaces yet, but support is planned. See Dataspaces and datasets.
Dataset
A named collection of records inside a dataspace, like a table, such as default/logs or system/engine.queries. The first write to a dataset locks it to one entity type. Writing in Overwrite mode starts a new dataset version, which can have a different entity type. A query can read a specific version with the version parameter of source. See Dataspaces and datasets.
Pillar
The broad kind of data a record is: logs, spans, metrics, profiles, binary (files such as session recordings and source maps), or data (JSON documents with no time axis). DataPrime queries the logs, spans, and data pillars.
A pillar can guarantee some fields. Every record in the logs pillar, for example, has $m.timestamp, $m.severity, $m.cxEventId, $m.priorityClass, and $m.ingressTimestamp. See Entity types.
Entity type
A narrower kind of data within a pillar, which can add its own schema guarantees on top of the pillar's. Every engineQueries record, for example, has $d.clientInfo and $d.queryInfo in addition to the logs pillar fields. A record can carry more fields than its entity type guarantees. See Entity types. To see the schema of a dataset, open it in Dataset Management.
The fields a record is guaranteed to have are the pillar's guarantees plus its entity type's:
Record
A single log, span, or document. Every record is made of labels ($l), metadata ($m), and user data ($d). See Access mechanisms.
Inspect the concepts in a query
DataPrime has a function for each level, so a query can report where every record comes from. This query takes one record from each of three sources and returns its lineage:
source logs
| limit 1
| union (source spans | limit 1)
| union (source system/engine.queries | limit 1)
| choose
dataspace() as dataspace,
dataset() as dataset,
pillar() as pillar,
entityType() as entityType
| dataspace | dataset | pillar | entityType |
|---|---|---|---|
| default | logs | logs | logs |
| default | spans | spans | spans |
| system | engine.queries | logs | engineQueries |
Logs and engine queries share the logs pillar but have different entity types.
To see the same breakdown with record counts, group by all four instead of taking one record per source. This is useful when a list of datasets comes from somewhere else, such as a saved query or an AI agent, and you need to confirm what each one holds before you aggregate across them:
source logs
| union (source spans)
| union (source system/engine.queries)
| groupby dataset() as dataset, dataspace() as dataspace, entityType() as entityType, pillar() as pillar
agg count() as count
See dataspace, dataset, and recordLocation, which adds the archive file each record was read from.