union
Description
The union command concatenates the results from two or more datasets into one dataset. This allows users to combine results from multiple queries into one seamless dataset. One dataset can be a result set piped into the union command and then concatenated with another dataset.
Syntax
<query> | union (<query>)
Example 1
Combine logs from two different teams into a single dataset to analyze them together.
Example data
// Logs for Team 58942
{ "id": "111", "name": "John" , "team.id": "58942" }
{ "id": "222", "name": "Emily", "team.id": "58942" }
{ "id": "333", "name": "Alice", "team.id": "58942" }
// Logs for Team 98361
{ "userid": "111", "timestamp": "2022-01-01T12:00:00Z", "team.id": "98361" }
{ "userid": "111", "timestamp": "2022-01-01T12:30:00Z", "team.id": "98361" }
{ "userid": "222", "timestamp": "2022-01-01T13:00:00Z", "team.id": "98361" }
{ "userid": "222", "timestamp": "2022-01-01T13:00:00Z", "team.id": "98361" }
{ "userid": "222", "timestamp": "2022-01-01T13:00:00Z", "team.id": "98361" }
Example query
source logs(teamId='58942')
| union (source logs(teamId='98361'))
Example output
{ "id": "111", "name": "John" , "team.id": "58942" }
{ "id": "222", "name": "Emily", "team.id": "58942" }
{ "id": "333", "name": "Alice", "team.id": "58942" }
{ "userid": "111", "timestamp": "2022-01-01T12:00:00Z", "team.id": "98361" }
{ "userid": "111", "timestamp": "2022-01-01T12:30:00Z", "team.id": "98361" }
{ "userid": "222", "timestamp": "2022-01-01T13:00:00Z", "team.id": "98361" }
{ "userid": "222", "timestamp": "2022-01-01T13:00:00Z", "team.id": "98361" }
{ "userid": "222", "timestamp": "2022-01-01T13:00:00Z", "team.id": "98361" }
Example 2
Compare how many records you have for the same timeframe across storage tiers. Because frequentsearch datasets are queryable like any other source, you can union your High (Frequent Search) tier with your default data, then use `dataspace()` and `dataset()` to attribute each record to its origin and count per source in a single query.
Example query
source frequentsearch/logs
| union (source logs)
| create query_source.dataspace from dataspace()
| create query_source.dataset from dataset()
| groupby query_source.dataspace, query_source.dataset agg count() as count
Example output
{ "query_source": { "dataspace": "frequentsearch", "dataset": "logs" }, "count": 3996594 }
{ "query_source": { "dataspace": "default", "dataset": "logs" }, "count": 11272291 }
Last updated on