Understanding expressions

Goal

By the end of this guide, you should be able to write expressions that evaluate to values and use them effectively in commands like filter, create, and choose.

Why it matters

An expression in DataPrime is a self-contained unit of logic that evaluates to a single value.

You’ll use expressions everywhere in DataPrime: to compare fields, calculate durations, format messages, and cast between types. They're the backbone of every transformation, filter, and aggregation. Mastering expressions means mastering the language.

What you can do with expressions

You can use expressions to compute or compare any value. Every expression must return a result, like a string, number, boolean, timestamp, or array.

Expressions live inside the pipeline

You can think of each | (pipe) in a DataPrime query as a step in a pipeline, and at each step, you're writing an expression that gets applied to the incoming data.

For example:

source logs
| filter latency > 500
| create duration_s from latency / 1000
| choose duration_s

The filter command runs an expression: latency > 500
The create command runs an expression: latency / 1000
The choose command runs one or more expressions to define output fields

Each of these expressions returns a value. The value doesn't “persist” on its own. It flows downstream through the pipeline, much like output from a function in a traditional code block.

You can think of each piped section as a kind of mini code block—except that in DataPrime, the expression inside each command is focused on transforming, filtering, or calculating values from the document stream.

Here are the main kinds of expressions you’ll use in real-world queries:

Return a literal value

source spans
| choose $m.duration as how_long

Use a field reference

source spans
| filter $m.priorityClass == 'high'

Do math and comparisons

source spans
| filter duration / 1000 > 5
| create remaining from quota - usage

Combine conditions with booleans

source logs
| filter status == "error" && latency > 500
| filter environment == "prod" || cluster == "blue"

Use function calls

source logs
| create ts from fromUnixTime(timestamp_ms, 'ms')

Cast types

source logs
| filter status_code:number > 399

Format strings with interpolation

source logs
| create msg from `User {user_id} logged in from {ip}`

Perform time arithmetic

source logs
| create one_hour_ago from now() - 1h

Where expressions are used

Expressions are the value-producing part of most commands:
Command Expression role
filter Must return true or false
create Sets a new key to the result of the expression. Similar to instantiating a variable
choose Defines what values to keep
groupby Expressions define keys and aggregation calculations
replace Supplies the new value to insert

Example:

source spans
| filter $m.duration > 500                          // boolean expression
| create is_high_latency from $m.duration           // expression used as a predicate
| choose `Request at {$m.timestamp} took {is_high_latency}ms` as summary

Output:

{
summary:Request at 2025-07-17T12:14:32.193186+0000 took 550ms
}
{
summary:Request at 2025-07-17T12:14:32.121102+0000 took 664ms
}
{
summary:Request at 2025-07-17T12:14:31.885782+0000 took 36866ms
}

Common pitfalls

Expressions always return a value—but it might be null if the logic fails or types mismatch.
Commands expect specific types: filter needs a boolean, create can accept anything.
Expressions don’t persist unless you assign them with create or choose.

Understanding functions

Next Troubleshooting

Command	Expression role
`filter`	Must return `true` or `false`
`create`	Sets a new key to the result of the expression. Similar to instantiating a variable
`choose`	Defines what values to keep
`groupby`	Expressions define keys and aggregation calculations
`replace`	Supplies the new value to insert