choose - Keep a subset of keypaths

The choose keyword will remove all keypaths that are not specified. This is a powerful way of extracting only the values needed from a larger log document.

Syntax

(choose|select) <keypath1> [as <new_keypath>],<keypath2> [as <new_keypath>],...

Example - Extracting only key data from HTTP Access Logs

Consider the following document:

status_code: 200,
user: "Chris",
path: "/home",
x-forwarded-for-header: "",
user_agent: "Mozilla...."

For the purposes of our analysis, we only care about path and status code. We can use the choose keyword to extract only these two values:

choose status_code, path

This results in a document that looks like this:

status_code: 200,
path: "/home"

Example - Extracting and flattening values from a complex log document

Some log documents are extremely complex and nested, and it can be challenging to work with them. While using the choose function to pull values out of these documents, we can also alias them to simpler keypaths:

choose http_request.metrics.bytes_metrics.bytes_received as bytes_received

This results in a document with a single value, bytes_received without the complex nested structure. This will make subsequent queries much simpler.

Example - Adding a constant value while extracting values

Sometimes it's important to add some constant value to a document, for example your username, if you're generating a report. This can also be done using the choose command:

choose http_request.metrics.bytes_metrics.bytes_received as bytes_received, 'Chris' as report_author

This will result in a document that looks like this:

{
    "bytes_received": 57819,
    "report_author": "Chris"
}

Example - Performing a calculation on the fly

The choose command is also good for producing new values on the fly. Consider the following documents:

{
    "response_bytes": 4065849304
},
{
    "response_bytes": 5573222
},
{
    "response_bytes": 990045767
},
{
    "response_bytes": 1287340
}

We can use the choose command to extract converted values for the response_bytes fields in an incredibly expressive way:

choose (response_bytes / 1024 / 1024) as response_mb

This will result in the following documents:

{
    "response_mb": 3877
},
{
    "response_mb": 5
},
{
    "response_mb": 944
},
{
    "response_mb": 1
}