dedupeby - Deduplicate events based on a combination of expressions.

dedupeby deduplicates documents based on the result of one or more expressions, and keeps N events for each distinct combination of the provided expressions.

NOTE: The content of the events are unchanged. This is like a very clever filter.

Syntax

dedupeby <expression1> [, <expression2> ...] keep N

Example - Deduplicating traces to see a sample of unique events

Consider the following application and endpoint:

Product Service
- Healthcheck Endpoint
- Index Endpoint

There are many requests going to both of these endpoints, and we're looking to see just a few of them. We can use dedupe to select 10 from the healthcheck endpoint, and 10 from the index endpoint, giving us a small sample for each, without forcing us to aggregate. We can do this with the following command:

dedupeby $l.operationName keep 10

The resulting documents will look something like this:

{
    "path": "/index",
    "latency": 632
    ...
},
{
    "path": "/index",
    "latency": 125
    ...
} ... x10,
{
    "path": "/healthcheck",
    "latency": 4000
    ...
},
{
    "path": "/healthcheck",
    "latency": 109
    ...
} ... x10

create

Next distinct