Advanced Guide to Kibana Timelion

Kibana Timelion is a time-series based visualization language that enables you to analyze time-series data in a more flexible way. compared to other visualization types that Kibana offers.

Instead of using a visual editor to create visualizations, Timelion uses a combination of chained functions, with a unique syntax, to depict any visualization, as complex as it may be.

The biggest value of using Timelion comes from the fact that you can concatenate any function on any log data. This means that you can plot a combination of functions made of different sets of logs within your index. It’s like creating a Join between uncorrelated logs except you don’t just fetch the entire set of data, but rather visualize their relationship. This is something no other Kibana visualization tool provides.

In this post, we’ll explore the variety of functions that Kibana Timelion supports, their syntax and options, and see some examples.

Kibana Timelion Functions

View the list of functions with their relevant arguments and syntaxes here.

View the list of functions with their relevant arguments and syntaxes here.

Tips

• You can enter the Timelion wizard either from the main page when entering Kibana or from the visualizations section. If you enter it from the main screen make sure you choose “save current expression as Kibana dashboard panel” if your goal is to add the Timelion visualization to a Dashboard.
• Use the index Argument in your .es() functions when building your Timelion expressions. By using it, any element you add, such as a metric, or a field will have auto-suggestions for names when starting to type. For example, .es(index=*:9466_newlogs*).
• If you enable the Coralogix Logs2metrics feature and start to collect aggregations of your logs, using .es(index=*:9466_log_metrics*) lets you visualize those metrics. Using both index patterns in the same expression lets you visualize separate data sources like your logs and metrics. No other Kibana visualization gives you such an option. It will look like this: .es(index=*:9466_newlogs*),.es(index=*:9466_log_metrics*).
• Use the Kibana Argument, with the true option in your .es() functions when building your Timelion expressions if you plan on integrating them with a Dashboard so that they’ll apply the filters in your dashboard. For example, .es(kibana=true).
  

Examples

Cache Status

.es(kibana=true,q='_exists_:cache_status.keyword', split=cache_status.keyword:5).divide(.es(kibana=true,q='_exists_:cache_status.keyword')).multiply(100).label(regex='.*cache_status\.keyword:(.*) > .*', label='$1%').lines(show=true,width=1).yaxis(1,min=0,max=100,null,'Upstream Cache Status (%)').legend(columns=5, position=nw).title(title="Cache Status"),
.es(kibana=true,q='_exists_:cache_status.keyword', split=cache_status.keyword:5).divide(.es(kibana=true,q='_exists_:cache_status.keyword')).multiply(100).label(regex='.*cache_status\.keyword:(.*) > .*', label='$1 avg').lines(show=true,width=1).yaxis(1,min=0,max=100,null,'Upstream Cache Status (%)').legend(columns=5, position=nw).title(title="Cache Status").aggregate(function=avg)

1. .es(q="_exists_:cache_status.keyword", split=cache_status.keyword:5) – query under q=; aggregate, per top 5 unique values of cache_status field, under split=
2. .divide() – divide the series by whatever is in parenthesis. In this case:  .es(q="_exists_:upstream_cache_status.keyword") is the same query above, but without aggregating different values (=all values together). Together, (1) and (2) provide the % of each value.
3. .multiply(100) – convert 0..1 to 0..100 for an easier view of %.
4. .label(regex='.*upstream_cache_status\.keyword:(.*) > .*', label='avg $1%') – change the label of the legend (match the original value with the regex and create a label with the result).
5. .lines(show=true,width=1) – This is the line styling.
6. .yaxis(1,min=0,max=100,null,'Upstream Cache Status (%)') the y-axis styling sets the min and max as constants for the %.
7. .legend(columns=5, position=nw) – sets 5 columns for the legend, as we have 5 splits in the above series, and place it at the northwest corner.
8. .title(title="Upstream Cache Status") – sets a title for the graph.
9. .aggregate(function=avg) – averages each of the different series (throughout the whole query timeframe).

Using two series, one for 1 through 8 (without averaging) and another series for 1 through 9 (with averaging), we can get each of the series along with its average in the same graph.

Response Size

.es(kibana=true,q='_exists_:response.header_size', metric=sum:response.header_size.numeric,split=request.protocol.keyword:5).add(.es(kibana=true,q='_exists_:response.body_size', metric=sum:response.body_size.numeric,split=request.protocol.keyword:5)).divide(1024).label(regex='.*request\.protocol\.keyword\:(.*) > .*', label='$1').lines(width=1.4,fill=0.5).legend(columns=5, position=nw).title(title="Total response size (KB) by protocol")

1. .es(kibana=true,q='_exists_:response.header_size', metric=sum:response.header_size.numeric,split=request.protocol.keyword:5) – query under q=; the metric to use (in this case sum of response header size) under metric=; aggregate, per top 5 unique values of request.protocol, under split=
2. .add() – adding to the series whatever is in parenthesis. In this case – .es(kibana=true,q='_exists_:response.body_size', metric=sum:response.body_size.numeric,split=request.protocol.keyword:5) is the same query above except we are summing the response body size rather than the header size. Together, (1) and (2) provide the total number of bytes for the response.
3. .divide(1024) – convert bytes to Kilobytes.
4. .label(regex='.*request\.protocol\.keyword\:(.*) > .*', label='$1') – change the label of the legend (match the original value with the regex, create a label with the result).
5. .lines(width=1.4,fill=0.5) – line styling.
6. .legend(columns=5, position=nw) – sets 5 columns for the legend, as we have 5 splits in the above series, and place it at the northwest corner.
7. .title(title="Total response size (KB) by protocol") – set a title for the graph.

Using the .add() function with the same .es() function for the response body size, we can get the full response size even though our log data includes the size of the header and size of the body separately.

Bytes Sent

.es(metric="percentiles:bytes_sent.numeric:5,25,50,75,95,99").log().lines(width=0.9,steps=true).label(regex="q:\* > percentiles\([^\.]+\.numeric\):([0-9]+).*",label="bytes sent $1th percentile").legend(columns=3,position=nw,timeFormat="dddd, MMMM Do YYYY, h:mm:ss a").title(title="Bytes sent percentiles, log scale")

1. .es(metric="percentiles:bytes_sent.numeric:5,25,50,75,95,99") – metric to use (in this case percentiles of bytes sent) under metric=
2. .log() – calculating log (with base 10 if not specified otherwise) for y-axis values of our expression.
3. .label(regex="q:\* > percentiles\([^\.]+\.numeric\):([0-9]+).*",label="bytes sent $1th percentile") – change the label of the legend (match the original value with the regex, create a label with the result).
4. .lines(width=0.9,steps=true) – line styling.
5. .legend(columns=3,position=nw,timeFormat="dddd, MMMM Do YYYY, h:mm:ss a") – sets 3 columns for the legend and place it at the northwest corner.
6. .title(title="Bytes sent percentiles, log scale") – set a title for the graph.

High severity logs

.es(q="coralogix.metadata.severity:(5 OR 6)",split=coralogix.metadata.subsystemName:5).lines(width=1.3,fill=2).label(regex=".*subsystemName:(.*) >.*",label="high severity logs count from subsystem $1").title(title="High severity logs count VS moving average per top 5 subsystems").legend(columns=2,position=nw),
.es(q="coralogix.metadata.severity:(5 OR 6)",split=coralogix.metadata.subsystemName:5).lines(width=1.3,fill=2).label(regex=".*subsystemName:(.*) >.*",label="high severity logs moving average from subsystem $1").mvavg(window=10,position=right)

1. .es(q="coralogix.metadata.severity:(5 OR 6)",split=coralogix.metadata.subsystemName:5) – query under q=; aggregate, per top 5 subsystems, under split=
2. .label(regex=".*subsystemName:(.*) >.*",label="high severity logs count from subsystem $1") – change the label of the legend for the 1st series (match the original value with the regex, create a label with the result).
3. .label(regex=".*subsystemName:(.*) >.*",label="high severity logs moving average from subsystem $1") – change the label of the legend for the 2nd series (match the original value with the regex, create a label with the result).
4. .lines(width=1.3,fill=2) – line styling.
5. .legend(columns=2, position=nw) – sets 2 columns for the legend, and place it at the northwest corner.
6. .title(title="High severity logs count VS moving average") – set a title for the graph.
7. .mvavg(window=10,position=right) – computes the moving average, sliding window of 10 points to the right of each computation point, for each of the different subsystems.

Using two series, 2nd series is the moving average for the 1st, we can get each of the series along with its moving average in the same graph.

5xx responses benchmark

.es(q="status_code.numeric:[500 TO 599]").if(operator=lt,if=3000,then=.es(q="status_code.numeric:[500 TO 599]")).color(color=red).points(symbol=circle,radius=2).label(label="5xx status log count above 3000"),
.es(q="status_code.numeric:[500 TO 599]",offset=-1w).if(operator=lt,if=3000,then=.es(q="status_code.numeric:[500 TO 599]")).color(color=purple).points(symbol=circle,radius=2).label(label="5xx status log count above 3000 a week ago"),
.es(q="status_code.numeric:[500 TO 599]").if(operator=gte,if=3000,then=null).color(color=blue).points(symbol=circle,radius=2).label(label="5xx status log count under 3000"),
.es(q="status_code.numeric:[500 TO 599]",offset=-1w).if(operator=gte,if=3000,then=null).color(color=green).points(symbol=circle,radius=2).label(label="5xx status log count under 3000 a week ago").title(title="5xx log count benchmark")

1. .es(q="status_code.numeric:[500 TO 599]") – query under q=;
2. .es(q="status_code.numeric:[500 TO 599]",offset=-1w) – query with an offset of 1w.
3. .if(operator=lt,if=3000,then=.es(q="status_code.numeric:[500 TO 599]")) – when chained to a .es() series, each point from the origin series is compared with the value under the ‘if’ Argument. According to the operator (in this case, lt=less than) if the result is true, the value is set to the value under the ‘then’ Argument for each point of the series
4. .if(operator=gte,if=3000,then=null) – opposite to [3].
5. .color(color=red) – color styling.
6. .points(symbol=circle,radius=2) – setting the result to be presented with dots (circle signs) instead of a line.
7. .label(label="5xx status log count above 3000") – change the label of the legend.
8. .title(title="5xx log count benchmark") – set a title for the graph.

Using 4 series, 2 pairs of same expressions only second pair with an offset and between them setting them with different colors from a certain threshold, gives us a nice benchmark comparing our 5xx status compared with a week earlier.

Need help? Check our website and in-app chat for quick advice from our product specialists.