Skip to content

DataPrime Expression Language (DPXL)

Overview

DataPrime Expression Language, or DPXL, is an expression language based on expression syntax. Leverage it across the Coralogix platform to define rich expression-based filters, such as when setting up Scopes.

General syntax

DPXL expressions are a subset of DataPrime expressions, such as those used in the filter command.

DPXL expressions are versioned to maintain predictability and stability of execution. Versioning allows enhancing DPXL over time without changing the semantics of existing expressions.

Each DPXL expression starts with a version identifier <vX>, currently <v1>. This is followed by the actual boolean expression, including literals, logical and comparison constructs, keypath access, and functions.

<v1> <boolean-expression>

Note that the UI provides the <v1> prefix automatically, allowing you to focus on the actual boolean expressions. However, when used in an API, a DPXL expression must start with the <v1> prefix.

Some of the examples in this document omit the <v1> prefix for brevity.

Examples

# A filter that returns true if my_text field has the value 'example'
<v1> $d.my_text == 'example'
# A filter that returns true if the event's timestamp is before the beginning of the year 2023
<v1> $m.timestamp < @'2023-01-01T00:00:00'
# A filter that returns true if the application name starts with 'dev-'
<v1> $l.applicationname.startsWith('dev-')
# A filter that returns true if the field region_id is us-east-1 or us-east-2
<v1> region_id:string.in('us-east-1', 'us-west-2')

Literals and data types

Here’s a list of data types and example literals:

Data Type Example Literals
string ‘us-east-1’ ’dev-’
number 23 -12.32
bool true false
timestamp @(’2023-01-01T00:00Z’) @’now’
Regular Expression /H.*o$/ /^prod-.*/
severity VERBOSE DEBUG INFO WARNING ERROR CRITICAL

In addition, there’s a null literal, which can be used with all other types.

Logical constructs

Operator Meaning Example Example Description
&& Logical AND cloud == ‘aws’ && region == ‘us-east-1’ Will return true if both the cloud is aws and the region is us-east-1
|| Logical OR age > 40 || country == ‘us’ Will return true if either the age is above 40, or the country is us
! Logical NOT !region.contains(’us-’) Will return true if the region does not contain us-

Ordering

You can control the evaluation order inside an expression using parenthesis. For example:

region.startsWith('us-') && (cloud == 'aws' || cloud == 'gcp')

(country == 'us' && (age > 40 || age < 10)) || (country == 'il' && age > 25)

Comparison constructs

Operator Meaning Examples
> Greater than duration > 40.5
$m.timestamp > @(’2023-01-01T00:00:00’)
>= Greater than or equals duration >= 40.5
< Smaller than age < 20
<= Smaller than or equals age <= 20
lastName <= ‘Smith’
== Equals cloud == 'aws'
!= Not equal to first_name != 'joe'

Keypath access

Keypaths are divided into three different parts, each with a separate prefix:

  • $m : Metadata

  • $l : Labels, such as applicationname or subsystemName

  • $d : Actual user data (default prefix)

$m - Metadata keypaths

Keypath Data Type Description
$m.timestamp timestamp Contains the timestamp of the event
$m.severity severity Contains the severity of the event

$l - Label keypaths

For logs:

Keypath Data Type
$l.applicationname string
$l.subsystemname string

For spans:

Keypath Data Type
$l.applicationname string
$l.subsystemName string
$l.serviceName string
$l.operationName string

$d - User Data keypaths

Any user keypath can be accessed using $d.<keypath> including nested keypaths.

$d is the default prefix. Any keypath that does not contain a prefix will be considered a user data field.

Functions

Functions provide additional capabilities within DPXL expressions.

Function Description Example
<s>.startsWith(<substr>):bool Checks if a string s starts with the specified substring substr region.startsWith('us-')
<s>.endsWith(<substr>):bool Checks if a string s ends with the specified substring substr firstName.endsWith(’Jo’)
<s>.contains(<substr>):bool Checks if a string s contains the specified substring substr stream.contains(’err’)
<s>.matches(<regex>):bool Checks if a string s matches the specified pattern provided by regex hostname.matches(/prod-.*/)
<value>.in(<value1>,<value2>,...) Checks if value is one of the provided values value1-valueN cloud.in(’aws’,’gcp’)

Data types

DPXL strives to infer the expected datatype of keypaths. For example, when writing age > 50, it will infer that age is expected to be a number. In cases where DPXL cannot infer the data type for a keypath, it will require a cast to provide the necessary information about the type. For example:

'123':number

region1:string == region2

my_key:number > my_other_key

Usage

Hit the ground running with DPXL based on these examples.

  # For the entity logs, allow access only to logs where the application name is "production"
  <v1> $l.applicationname == 'production'
  # For the entity logs, allow access only to logs in which app name starts with dev, or the field "r egion_id" in the data is us-east-1
  <v1> $l.applicationname.startsWith('dev-') && region_id == 'us-east-1'
  # For the entity logs, allow access only to logs in which the field "country" is not one of the listed below. 
  <v1> !$d.country:string.in('us','il','gr')
  # For the entity logs, allow access only to logs in which cloud is aws and the region field either ends with "-2" or is speficially set to "eu-west-1"
  <v1> cloud == 'aws' && (region == 'eu-west-1' || region.endsWith('-2'))
  # For the entity logs, allow access only to logs where the pod name matches the regex provided
  <v1> kubernetes.pod_name.matches(/^kafka-[0-9]+/)
  # For the entity logs, allow access only to logs that don't have a DEBUG severity
  <v1> $m.severity != DEBUG
  # For the entity logs, allow access only to logs in which some query duration is very large
  <v1> query_duration_seconds > 100
  # For the entity logs, allow access only to logs up to the beginning of the year 2023
  <v1> $m.timestamp < @'2023-01-01T00:00:00'
  # For the entity logs, disallow access to all logs entirely
  <v1> false
  # For the entity spans, allow access only to spans in which the applicationname is not "dev"
  <v1> $l.applicationname != 'dev'
  # For the entity spans, allow access only to spans that have a service name that starts with "alerts-"
  <v1> $l.serviceName.startsWith('alerts-')
  # For the entity spans, allow access only to spans that have the tag "db.system" set to mysql, and whose operationName contains the string "query"
  <v1> $l.operationName.contains('query') && tags['db.system'] == 'mysql'
  # For the entity spans, allow access to all spans
  <v1> true

Limitations

Expressions

  • Maximum number: 150

  • Maximum depth: 20

in Function type inference

The in function currently cannot automatically infer the expected type of a keypath. In order to use it, cast the keypath to the proper type. For example:

<v1> !$d.country:string.in('us','il','gr')

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by emailing [email protected].