Heroku Logging

TL;DR – Coralogix Heroku logging addon: https://elements.heroku.com/addons/coralogix

Coralogix has recently released a logging add-on that forwards all Heroku output to a Coralogix account you specify.

provision coralogix on heroku

Or use the CLI with:

heroku addons:create coralogix:test
Then within a few seconds, any logs that your application generates will feed right into Coralogix, ready for use the high scale, high-speed search and analytics that Coralogix offers. Here’s a simple example, that shows that even with a small application generating simple log records, as soon it starts scaling, logs become hard to traverse and understand.

Sample Application

The sample application consists of a Python and Ruby application that randomly generates a cat from the Cat API every 2 seconds and logs the value.

I started with one instance of each, then scaled to three instances, here are the logs using heroku logs one one instance.

one worker sending logs to coralogix

And another example of mixed logs with messages from Heroku, debug messages and errors, already getting confusing.

two workers sending logs to Coralogix

Throw in two more workers and tracking down the source of messages is getting even more confusing. And the sample application has two applications running three workers each.

multiple workers on Coralogix

Let’s use Coralogix instead. You can open the dashboard page, or again use the CLI and see the initial display right after you start sending logs:

CLI:addons:open coralogix  

coralogix dashboard

view heroku logs in coralogix

After 24 hours, Coralogix’ Loggregation feature will kick in to automatically cluster log entries and make finding what you need faster and easier. In the screenshot below you can see that Loggregation has identified that despite being ‘random’, the CatAPI repeats itself a lot, outputting the same value over 1,000 times. In this case, it’s not important, but in a mission critical application, this helps you identify patterns clearly.

loggregation on heroku logs

 

Errors and broken flows are often introduced by new versions and builds of an application. Coralogix’s integration with Heroku includes an integration to Heroku Pipelines and offers an automatic status for new Heroku builds (or tags). Coralogix presents the suspicious and common errors introduced since that build as well as the alerts and anomalies which might be related to the new version release, thus allowing you to pinpoint issues to particular versions and points in time.

Heroku builds status coralogix

new logs created since heroku build

 

After 5 days of learning, Coralogix will start to send a daily report highlighting new anomalies (errors and critical messages), further helping you identify new errors, and not constantly bombard you with logging noise.

In addition to the machine learning aspect, Coralogix offers the entire set of Logging capabilities, including Log Query, a centralized live tail, user defined alerts to email or slack, and a fully hosted Kibana for data slicing and dashboards creation.

Provision the Coralogix addon and start enjoying 3rd generation Log analytics

WordPress Lightbox