Loggregation: Making Big Data Small

In this post, you will learn how to make the most of Coralogix’s very own automatic log clustering feature: Loggregation©.

Loggregation is an algorithm that condenses millions of log entries into a narrow set of patterns. It does so by automatically analyzing each log record sent to Coralogix, then separating the log constants from its variables.

Let’s have a look.

1) As you can see, the following query retrieved ~11M log entries:

before loggregation


2) Within seconds of clicking Loggregation, the 11M logs become grouped into their original patterns, thereby dramatically reducing the number of entries.

After loggregation

As you can see below, we went from 11M entries, to only 79 templates. There is also a display of how many times each pattern arrived, and a full visualization of all parameters. This serves as a major timesaver in terms of log analysis.

3) Have JSON format log data? no problem! Coralogix clusters your JSON’s into their unique appearances while allowing you to graphically view the different values including an automatic text clustering. 

coralogix JSON loggregation / clustering

For further analysis, you can click on any of the variables, which are conveniently color-coded. Doing so will immediately display a graph with relevant information.

Here’s a closer look at the types of graphs for each (colored) variable.

Clicking on the pink variables will display a categorical bar graph:

coralogix categorical parameter

Clicking on the green variables will display a numeric line graph:

coralogix numeric param

Clicking on a blue variable will display a free variable distribution graph:

coralogix free parameter

That is it! Now you know how you can a full day’s worth of data in a matter of minutes, including deep analyses of distributions and variables. Save time and resources with Coralogix!

In addition, clicking the pin icon at the top right corner of each parameter allows you to send that graph directly to your dashboard in seconds. 

Signup to Coralogix

WordPress Lightbox