Heroku Logs

Use our logging add-on to seamlessly forward all Heroku logging output to Coralogix.

This tutorial demonstrates how to:

• Create a sample application

• Configure your Heroku Shield private space to send logs to Coralogix

• Attach multiple Heroku apps to one Coralogix account

Overview

Logging add-on

Coralogix offers a logging add-on that forwards all Heroku output to a Coralogix account that you specify. Alternatively, you can use the CLI with the following:

heroku addons:create coralogix:test

Within seconds, any logs that your application generates will feed right into Coralogix, paving the way for using our high scale, high-speed search, and analytics.

Extension package

In addition, Coralogix offers an extension package for Heroku logging that includes a set of Heroku parsing rules to parse the unstructured data streaming from your Heroku app, a set of predefined Heroku alerts, and a predefined dashboard for your Heroku Router logs. What you will need to do to get those is go to Coralogix Extension Packages and deploy the Heroku extension package and then, with a click of a button, you will get tremendous value out of your Heroku Router logs.

Here's a basic example that shows that even with a small application generating simple log records, as soon it starts scaling, logs become hard to traverse and understand.

Sample application

The sample application consists of a Python and Ruby application that randomly generates a Cat from the Cat API every 2 seconds and logs the value.

I started with one instance of each, then scaled to three instances. Here are the logs using heroku logs on one instance.

Here's an example of mixed logs with messages from Heroku, debug messages, and errors. It's already getting confusing.

Throw in two more workers and tracking down the source of messages becomes even more confusing. The sample application has two applications running three workers each.

We can do this a lot more efficiently using Coralogix. To begin, open the dashboard page, or use the CLI and check the initial display right after you start sending logs:

CLI:addons:open coralogix 

After 24 hours, the Coralogix Loggregation feature automatically clusters log entries – which makes finding what you need faster and powers our ML-powered anomalies. In the screenshot below, you can see that Loggregation has identified that despite being 'random', the CatAPI repeats itself a lot, outputting the same value over 1,000 times. In this case, it's not important, but in a mission-critical application, this helps you identify patterns clearly.

Errors and broken flows are often introduced by new versions and application builds. The Coralogix Heroku integration includes integration to Heroku Pipelines and offers an automatic status for new Heroku builds (i.e. tags in Coralogix). Coralogix presents the suspicious and common errors introduced since that build as well as the alerts and anomalies which might be related to the new version release, thus allowing you to correlate issues to particular versions and points in time.

After 5 days of learning, Coralogix will start to send a daily report highlighting new anomalies (errors and critical messages), further helping you identify new errors, and not constantly bombard you with logging noise.

In addition to the machine learning aspect, Coralogix offers a complete set of logging capabilities, including Log Query, a centralized live tail, user-defined alerts to email or slack, and dashboard creation.

Provision the Coralogix addon and start enjoying 3rd generation Log analytics

Private space logging

You may configure your Heroku Shield private space to send logs to Coralogix by pointing the Heroku log drain to the appropriate Coralogix destination.

Prerequisites

• Coralogix Company ID
• Send-Your-Data API Key (Private Key)
• Choose the https://ingress. endpoint that corresponds to your Coralogix domain using the domain selector at the top of the page

Endpoint URL format

The log drain URL uses HTTP Basic Authentication with your Company ID as the username and your Private Key as the password:

https://<CompanyId>:<PrivateKey>@https://ingress./heroku/v1/private?appName=<applicationName>

Query parameters

Content-Type requirements

The log drain must send logs with one of the following Content-Type headers:

• application/logplex-1 (standard Heroku log drain format)
• text/plain

Adding and removing log drains

Add a log drain:

heroku drains:add "https://<CompanyId>:<PrivateKey>@https://ingress./heroku/v1/private?appName=<applicationName>" -a myapp

Remove a log drain:

heroku drains:remove "https://<CompanyId>:<PrivateKey>@https://ingress./heroku/v1/private?appName=<applicationName>" -a myapp

Find out how to find Heroku messages from a specific app here.

Custom fields

You can add custom fields to your log messages by including query parameters prefixed with cxf_. These fields are appended to the log message body as key-value pairs.

Constraints:

• Maximum 20 custom fields per request
• Field key (after cxf_ prefix): alphanumeric characters, underscores (_), and hyphens (-) only, maximum 64 characters
• Field value: maximum 256 characters

Example:

https://<CompanyId>:<PrivateKey>@https://ingress./heroku/v1/private?appName=myapp&cxf_env=production&cxf_region=us-east-1

This appends env=production region=us-east-1 to each log message.

Subsystem name override

By default, the subsystem name is derived from the Heroku process type (e.g., web.1 becomes subsystem web with thread ID 1). You can override this for all logs from a drain by setting the subsName parameter:

https://<CompanyId>:<PrivateKey>@https://ingress./heroku/v1/private?appName=myapp&subsName=my-custom-subsystem

​Attaching multiple apps to a single Coralogix account

You can add multiple apps to the same Coralogix account you created when provisioning your first Heroku app with the Coralogix add-on. All apps will have a unified billing and you will be able to filter by our tag of "application name".

To attach multiple apps to the same Coralogix account using CLI, follow this link. Or from your Heroku dashboard: configure addons -> open details for Coralogix -> attach another app.

Sending Heroku Metric Logs to Coralogix

Coralogix automatically parses Heroku Metric logs into JSON documents, to make analysis and parsing much easier. A more in depth explanation is available here.

Note

Removing your parent Heroku app, to which Coralogix was provisioned originally, will remove the Coralogix addon from all other apps that were attached to this Coralogix account in the past.

Additional resources

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].