# Using arrays, strings, and complex structures with DataPrime

## Goal

By the end of this guide you should be able to manipulate arrays, transform and analyze strings, and access deeply nested fields. You’ll learn how to split strings, decode values, flatten complex objects, and troubleshoot keypath issues that can break queries.

______________________________________________________________________

## Why it matters

Real-world logs rarely come in clean. Arrays often need to be expanded, strings need parsing or decoding, and deeply nested or oddly named fields can get in the way of querying. This guide shows you how to handle that complexity with confidence using DataPrime.

______________________________________________________________________

## `explode` – Split array elements into rows

### Description

Use [`explode`](https://coralogix.com/docs/dataprime/language-reference/commands-reference/explode/index.md) when you need to analyze each element in an array as its own document. This is useful for permissions, tags, error lists, and other multi-valued fields.

### Syntax

```dataprime
explode <array_field> into <new_field> original preserve
```

### Example – Flatten `scopes` for permission analysis

#### Input data

```json
{ "user_id": "1", "scopes": ["read", "write"] }
```

#### Query

```dataprime
explode scopes into scope original preserve
```

#### Result

```text
{ user_id: 1, scope: read, scopes: [read, write] }
{ user_id: 1, scope: write, scopes: [read, write] }
```

______________________________________________________________________

## `arrayConcat` – Combine multiple arrays into one

### Description

Use [`arrayConcat`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/array/arrayconcat/index.md) to merge two or more arrays into a single array field. Ideal for combining values split across fields (e.g., job queues, error types).

### Syntax

```dataprime
create <new_field> from arrayConcat(array1, array2, ...)
```

### Example – Merge frontend and backend error arrays

#### Input data

```json
{
  "frontend_errors": ["missing_token", "timeout"],
  "backend_errors": ["db_error"]
}
```

#### Query

```dataprime
create all_errors from arrayConcat(frontend_errors, backend_errors)
```

#### Result

```text
{ all_errors: [missing_token, timeout, db_error] }
```

______________________________________________________________________

## `arrayAppend` – Add a value to the end of an array

### Description

Use [`arrayAppend`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/array/arrayappend/index.md) to add a value to the end of an array field. This is useful when the value is available, but stored separately from the array.

### Syntax

```dataprime
replace <array_field> with <array_field>.arrayAppend(<value>)
```

### Example 1 – Add a static job step

#### Input data

```json
{ "steps": ["extract", "transform"] }
```

#### Query

```dataprime
replace steps with steps.arrayAppend('finalize')
```

#### Result

```text
{ steps: [extract, transform, finalize] }
```

______________________________________________________________________

### Example 2 – Append a field value into the array

#### Input data

```json
{
  "jobs": ["job-1", "job-2"],
  "extra_job": "job-3"
}
```

#### Query

```dataprime
replace jobs with jobs.arrayAppend(extra_job)
```

#### Result

```text
{ jobs: [job-1, job-2, job-3] }
```

______________________________________________________________________

## `arrayContains` – Check if a value exists in an array

### Description

Use [`arrayContains`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/array/arraycontains/index.md) to determine if a specific value appears inside an array. Returns a boolean.

### Syntax

```dataprime
create <flag_field> from <array>.arrayContains(<value>)
```

### Example – Flag blocked IP addresses

#### Input data

```json
{
  "client_ip": "1.2.3.4",
  "blocked_ips": ["1.2.3.4", "5.6.7.8"]
}
```

#### Query

```dataprime
create is_blocked_ip from blocked_ips.arrayContains(client_ip)
```

#### Result

```text
{ is_blocked_ip: true }
```

______________________________________________________________________

## Parsing and transforming strings

### `arraySplit` – Split a string into parts

### Description

Use [`arraySplit`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/array/arraysplit/index.md) to break a string into parts using a delimiter. Often used for names, paths, versions, or tags.

### Syntax

```dataprime
<field>.arraySplit(<delimiter>)[<index>]
```

### Example – Split full name into first and last

#### Input data

```json
{ "name": "Joe-Nelson" }
```

#### Query

```dataprime
create first_name from name.arraySplit("-")[0]
| create last_name from name.arraySplit("-")[1]
```

#### Result

```text
{ first_name: Joe, last_name: Nelson }
```

______________________________________________________________________

## `arrayJoin` – Join array values into a string

### Description

Use [`arrayJoin`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/array/arrayjoin/index.md) to convert an array into a readable string using a delimiter.

### Syntax

```dataprime
<array_field>.arrayJoin(<delimiter>)
```

### Example – Format a user action log

#### Input data

```json
{ "users": ["Emma", "Ofri", "Zev"], "action": "LOGIN" }
```

#### Query

```dataprime
create msg from `Users {users.arrayJoin(', ')} performed action {action}`
```

#### Result

```text
{ msg: Users Emma, Ofri, Zev performed action LOGIN }
```

______________________________________________________________________

## `urlDecode` / `urlEncode` – Decode or encode URL-safe strings

### Description

Use [`urlDecode`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/url/urldecode/index.md) to make encoded strings readable. Use `urlEncode` when you need to safely transmit or store text.

### Syntax

```dataprime
<field>.urlDecode()
<field>.urlEncode()
```

### Example – Decode a query string parameter

#### Input data

```json
{ "query_string_parameters": { "name": "Tom%20Kosman" } }
```

#### Query

```dataprime
replace query_string_parameters.name with query_string_parameters.name.urlDecode()
```

#### Result

```text
{ query_string_parameters: { name: Tom Kosman } }
```

______________________________________________________________________

## `decodeBase64` – Decode base64 strings

### Description

Use [`decodeBase64`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/string/decodebase64/index.md) to convert encoded strings (e.g., compressed URLs or payloads) into readable values.

### Syntax

```dataprime
<field>.decodeBase64()
```

### Example – Decode a base64 URL

#### Input data

```json
{ "full_url_encoded": "aHR0cHM6Ly9jb3JhbG9naXguY29t" }
```

#### Query

```dataprime
create full_url from full_url_encoded.decodeBase64()
```

#### Result

```text
{ full_url: https://coralogix.com }
```

______________________________________________________________________

## `contains`, `startsWith` – Check for string patterns

### Description

Use [`contains`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/string/contains/index.md) or [`startsWith`](https://coralogix.com/docs/dataprime/language-reference/functions-reference/string/startswith/index.md) to detect substrings in a string field. Useful for filtering by prefix, domain, or label.

### Syntax

```dataprime
<field>.contains(<substring>)
<field>.startsWith(<substring>)
```

### Example – Identify log types or domains

#### Input data

```json
{ "url": "https://google.com/search", "msg": "ERROR timeout exceeded" }
```

#### Query

```dataprime
filter url.contains("google.com")
| create is_error_log from msg.startsWith("ERROR")
```

#### Result

```text
{ is_error_log: true }
```

______________________________________________________________________

## `choose` – Flatten deeply nested fields

### Description

Use [`choose`](https://coralogix.com/docs/dataprime/language-reference/commands-reference/choose/index.md) to extract a deeply nested field and bring it to the top level. Makes it easier to read and query.

### Syntax

```dataprime
choose <deep_field> as <alias>
```

### Example – Simplify metric access

#### Input data

```json
{
  "http_request": {
    "metrics": {
      "bytes_metrics": {
        "bytes_received": 1024
      }
    }
  }
}
```

#### Query

```dataprime
choose http_request.metrics.bytes_metrics.bytes_received as bytes_received
```

#### Result

```text
{ bytes_received: 1024 }
```

______________________________________________________________________

## `extract ... using kv()` – Parse key-value strings

### Description

Use `extract ... using kv()` to convert a structured string into a map of fields. Add `datatypes` to cast values into numbers or timestamps.

Note

There are several [extractor](https://coralogix.com/docs/dataprime/language-reference/commands-reference/extract/index.md) functions that can be used with the `extract` command.

### Syntax

```dataprime
extract <field> into <object> using kv(pair_delimiter, key_delimiter) datatypes ...
```

### Example – Parse and cast query parameters

#### Input data

```json
{ "msg": "query_id=200&duration_ms=1000" }
```

#### Query

```dataprime
extract msg into data using kv(pair_delimiter='&', key_delimiter='=') datatypes duration_ms:number
```

#### Result

```text
{ data: { query_id: 200, duration_ms: 1000 } }
```

______________________________________________________________________

## Bracket notation – Access special keypaths

### Description

Use bracket notation to access keys that contain dots, spaces, or special characters. Required in archive/compliance mode.

### Syntax

```dataprime
<root_field>['key.with.dot.or.special-character']
```

### Example – Filter by a key with dots

#### Input data

```json
{ "k8s.tags.process_id": 1234 }
```

#### Query

```dataprime
filter k8s['tags.process_id'] == 1234
```

#### Result

```text
{ k8s.tags.process_id: 1234 }
```

______________________________________________________________________

## Common pitfalls or gotchas

- **Exploding arrays removes other fields unless you preserve them.** Always use `original preserve` unless you want a minimal result.
- **String length limits can break string functions.** If a field is longer than 256 characters, some functions may silently return `null` in high-tier mode.
- **Always quote bracketed keypaths.** Even one missed bracket or dot can break your query.