Skip to content

Data types

Supported data types

DataPrime supports the following data types.

Data TypeExplanationReflected in JSON
TimestampA precise point in time in nanoseconds. Often used for tracking events or logs.Represented as a JSON number in nanoseconds. Example: 1609459200000000 (for a specific timestamp).
StringA sequence of characters. Commonly used for textual data, identifiers, or other alphanumeric data. Internally, the engine uses utf-8 encoding for all strings.Represented as a JSON string. Example: "example text".
NumberA numeric value, which can be an integer or a floating point. Used for calculations or measurements.Represented as a JSON number. Example: 123 or 45.67.
BooleanRepresents a true or false value. Common in logical checks and conditions.Represented as a JSON boolean. Example: true or false.
IntervalA time duration (e.g., 1m, 5h, 3h30m). Supported time units are: dhmsmsusns.Represented as a JSON string when shown as an interval. Example: "1h". Casted to a JSON number in nanoseconds (e.g., 3600000000000 for 1h).
NullRepresents the absence of a value. Commonly used to indicate missing or undefined data.Represented as JSON null. Example: null.
RegexpA regular expression used for pattern matching in strings.Represented as a JSON string containing the regular expression. Example: "/[a-z]+/". (Usually not used directly in JSON output).
ArrayA collection of values of a primitive type.Represented as a JSON array. Example: ["a", "b", "c"].

For intervals, although JSON doesn't natively support them, they can be displayed as strings or cast into a number when needed (e.g., in nanoseconds).

Data type tracking

The system tracks data types on raw data and at each stage of the query, based on the results of previous operations. This means the query engine and parsers maintain a record (or lineage) of data types and any conversions that occur during the query process. This tracking enables better validation, autocompletion, and suggestion capabilities when building queries.

Data type conversions

The system supports converting data types in several ways, using core constructs, operators and functions.

Casting

DataPrime users can use casting to transform the datatype of a given keypath. For example, in the following field, the user may change the type of status_code to a number, to perform a greater-than filter:

source logs | filter status_code:number > 299

Using the convert command

convert is a command for changing the type of a field. This is most useful when you wish to make multiple downstream calculations with a field after conversion, and you don’t wish to cast every time.

source logs | convert duration:number | agg max(duration), min(duration), avg(duration)

The datatypes clause

In various commands, there is a datatypes clause that can provide type information to the DataPrime query engine, which ensures that when values are parsed or extracted, they are done so into the correct type. For example, consider the following document:

{ 
  "msg": "query_type=fetch query_id=200 query_results_duration_ms=1001" 
}

We can use the extract command to pull out the values of these different fields, but query_results_duration_ms is clearly a numerical value, and we should inform DataPrime that it is. Using the extract command, the kv() parsing strategy and the datatypes clause, we can do everything in one command:

extract msg into query_data using kv() datatypes query_results_duration_ms:number

Conversion functions

Throughout the DataPrime Functions Reference, there are functions designed to help to parse or transform a given value. For example, interpreting a numerical value as a time interval:

create time_interval from 10.toInterval('s')