Have you ever found yourself late at night combing through a myriad of logs attempting to determine why your cluster went down? Yes, that’s a really stressful job, especially when you think about how much money your company loses as a result of these incidents. Gartner estimates that the revenue lost due to outages is around $5,600/minute, which amounts to more than $330K/hour.
To make matters worse, your boss is breathing down your neck asking for updates, and the logs you are working with look something like this:
Not very helpful, huh?
Please, do not despair! Coralogix Custom Data Enrichment to the rescue!
Using our Custom Data Enrichment feature, you can create one or more “translation” tables that add translated (e.g. enriched) fields to your logs. This allows you to unscramble any obscure data in the fields.
In this example, I created two very simple Custom Enrichments, which would translate the data from the following 2 log fields:
Please note that the above fields are case-sensitive. I have oversimplified my example to illustrate the essence of our Custom Data Enrichment feature, but in practice, your enrichment .csv files would contain more than one line (up to 10,000 rows, or approximately 0.5 MB of standard information).
Here is the content of the two Custom Data Enrichment .csv files I used for this example:
%$^&*(#@!!!!!,Pepe unplugged it!
^00^:(,The Power is off
Please note that the header lines in the .csv files are just placeholders (to remind you what kind of data the file contains) and are not referenced during the Custom Enrichment creation. What is important here to keep in mind is that the value to be enriched is the first one in each line, before the second comma-separated value.
Following are the two Custom Enrichments I created for this example:
Please note how the “hw-status”, and “errorcode” fields have been selected.
After creating the Custom Data Enrichment, and sending logs that would match the defined fields in the Enrichments, you will notice new fields added to your logs which are named using the original field’s name, with “_enriched” appended to it. Please take a look at what our example looks like after the Enrichment:
Isn’t that great? No more guessing what those values are…
We have gathered several .csv Enrichment files that have been used internally, and by our customers, and we are sharing them with you so you could use them to create your own enrichments.
They are available for download here:
|Custom Data Enrichment||Description||Download Link|
|area-code-to-state.csv||Maps US area codes to their corresponding state code(s).||Download Now|
|us-state-or-terrirory-code-to-state-or-country-name.csv||Maps US state/territory codes to their corresponding names.||Download Now|
|cisco-cipm-return-codes-to-description.csv||Maps Cisco IP Manager return codes |
to their corresponding descriptions.
|country-phone-code-to-country-name.csv||Maps country codes to their corresponding country name.||Download Now|
|country-territory-to-currency-symbol-code-unit.csv||Maps country names to their corresponding currency names, symbols, and ISO codes.||Download Now|
|country-to-currency-name-code-number-and-minor-unit.csv||Maps country names to their corresponding currency and alphabetic/numeric codes.||Download Now|
|cve-code-to-cve-description.csv||Maps CVE codes to their corresponding descriptions.||Download Now|
|date-to-day-of-the-week.csv||Maps dates to the corresponding day of the week.|
*Includes mapping from Friday, July 30th, 2021 to Sunday, February 7th, 2033.
|gmt-offset-to-civilian-code-during-standard-time.csv||Maps GMT Offset codes (ex. +0000) to their corresponding civilian codes (ex. GMT)||Download Now|
|iana-network-protocol-number-to-protocol-name.csv||Maps IANA network protocol numbers to their corresponding names.||Download Now|
|iana-network-protocol-number-to-protocol-keyword-to-protocol-name.csv||Maps IANA network protocol numbers to their corresponding keywords and names.||Download Now|
|iso-country-code-to-country-name.csv||Maps country names to their corresponding short and long ISO Codes.||Download Now|
|naics-industry-codes-to-description.csv||Maps NAICS industry codes to their corresponding titles (2017).||Download Now|
|cloudfront-locations.csv||Maps IATA Codes to AWS CloudFront CDN Edge Locations.||Download Now|
Let us know please about any other Custom Enrichments that may be useful and relevant to your environment. If you would like to contribute to our users’ repository with any .csv files that could be used to create new Custom log Enrichments, please send them our way:
The users’ repository will be updated as new .csv files are uploaded. Please check this page frequently.
We would also love to hear about your experience using the .csv enrichment files, as well as receiving any feedback you may have for us.
For more information about our Custom Data Enrichment feature, check out the complete tutorial here.