Skip to content

Sources in DataPrime

The first component of any DataPrime query is the dataset, which acts as the data source. You can choose from logs, spans, and custom enrichments as a dataset.

Logs

Logs are records of events and messages that occur during the execution of an application or service.

# Query:
source logs | count 

# Output
78997003

# Explanation: This counts the number of logs for the UI time picker in the UI.

Spans

Spans are fundamental units of work that represent individual operations or events within a trace.

 # Query 
 source spans | filter $l.applicationName != 'dev'

 # Explanation: Allow access only to spans in which the applicationName is not "dev"

Custom enrichments

DataPrime allows the user to expose custom enrichments as fully queryable datasets. Coralogix’s Custom Enrichment feature allows you to enhance your logs by adding critical contextual data that may not be available at runtime. This enrichment is done by appending fields to your JSON logs based on specific matches using a custom data source you define.

source ... | enrich ... using my_enrichment | ... 

Handling custom time ranges

The user may specify an optional time range for every data source. Conceptually, each source coupled with a time range may be considered a separate data source.

Time ranges are created using the following constructs:

between <timestamp> and <timestamp>

# Query

source logs between @'2021-01-01T00:00:00Z' and @'2021-01-02T00:00:00Z'

# Explanation: This presents the logs between two timestamps.

last <interval>

# Query:
source logs last 4h| count 

# Output:
30832

# Explanation: This counts the logs for the time range of last 4 hours regardless of the time picker in the UI. 

after <timestamp>

# Query:
source logs after @'now' - 3h| count 

# Output:
4597

# Explanation: This counts all of the logs from the last 3 hours.

timeshifted <interval>

Time shifting allows you to compare data from different periods by adjusting timestamps. This technique is valuable for trend analysis, such as comparing this week’s metrics against those of the last week. The timeshifted expression allows you to shift events forward or backward in time relative to the current time range by a specified interval. A positive interval shifts events forward in time, whereas a negative interval shifts events backward in time.

# Query:
source logs timeshifted 1h

# Explanation: Shift events forward by 1 hour

# Query:
source logs timeshifted -1d

# Explanation: Shift events backwards by 1 day

See Working with time in DataPrime for a full review of time-based operations.