Nowadays, as efficiency is the name of the game, companies that store great amounts of data are keeping a mindset of how to optimize data stores –raising the question of what data is more important to have available and what is less?
Coralogix Logs2Metrics enables you to do just that. You simply define a query and Coralogix will execute it every 5 minutes and store its aggregation (a count of matching logs) in a long-term (1 year) index. Metrics start to gather from the point in time in which they were defined. The available query time range for your Logs2Metrics indices is 90 days.
By activating Logs2Metrics, 5% of your daily quota will be used allowing you to create up to 30 metrics with a 1-year retention period.
- In Coralgix, Go to the TCO tab –> Logs2Metrics section and click on the New Metric button.
- Define your metric:
- Metric Name – The name you will choose will be the name of the field representing this metric in the long term index and will be used in Kibana visualizations.
- Metric Description – Describe your Metric.
- Search Query – Use a written text query. For example, status:[500 TO *] will store the amount of 5XX responses every x minutes which would allow long term analysis on your API stability. You can also set filters on applications, subsystems, and severities. If you want the query to match for any application, subsystem, and severity just check All in the relevant checkbox.
- Granularity – The time span for the aggregation of matching logs. You may choose between 1, 5, or 10 minutes.
- In Kibana –> Management –> Index Patterns you can see the newly created long term index for the aggregated metric data.
- Click on it to see the fields in your new Metrics index (you should expect to see fields with your metrics names). If you are not seeing all the expected fields, refresh your index pattern to show the most updated fields list.
- Now, you are ready to visualize your metrics. Let’s see an example. Here we’re showing how many logs originated from OpenVPN containers per a 1-minute time span.
Other Use Cases
You can create any metric using complex queries based on your log data. These are a few examples of common use cases:
- Enable long term analysis on your API stability: Store the amount of 5XX responses on your prod server.
- Query – status.numeric:[500 TO *] AND env:production
- Discover business trends: Store the number of successful purchases on your website.
- Query – message:”user completed purchase successfully”
- Discover trends in your application quality: Store the number of exceptions you have.
- Query – message:Exception and severity:ERROR
- Identify a trend and possible attacks: Store the amount of NXDOMAIN responses by your DNS resolver.
- Query – message:NXDOMAIN
Have any questions? check our website and in-app chat for quick help.