# Heroku metric logs

Our Heroku integration is configured to automatically detect inbound platform and custom metric logs, and parse them into JSON. To avoid confusion, we'll show some before and after examples, as well as explaining how to parse and generate metrics from these values, once they appear in your account.

## Before & after

[Heroku metric logs](https://devcenter.heroku.com/articles/log-runtime-metrics) appear as key-value pairs, in [Syslog format](https://www.rfc-editor.org/rfc/rfc5424). Coralogix automatically extracts the Syslog metadata, and makes the body of the syslog message available.

The unparsed body of the log (without syslog metadata) appears like this:

```text
dyno=heroku.7af55a6f-b5eb-46e7-b05c-36d3dc04ef22.5df262fa-c95f-4e06-a6c1-6d189b9781e9 source=web.1 sample#load_avg_1m=0.05 sample#load_avg_5m=0.03 sample#load_avg_15m=0.02
```

While this is readable, it is difficult to query each individual field without complex regex parsing. To make this easier for our Heroku customers, we have enabled JSON transformation for this incoming data, so that when it arrives in your Coralogix account, the above log body is transformed into this:

```text
{"dyno":"heroku.7af55a6f-b5eb-46e7-b05c-36d3dc04ef22.5df262fa-c95f-4e06-a6c1-6d189b9781e9","sample#load_avg_15m":"0.02","sample#load_avg_1m":"0.05","sample#load_avg_5m":"0.03","source":"web.1"}
```

This means that, rather than extracting the values you need, customers can run direct queries on their incoming metric logs, for example, the following query can extract all metric samples where the number of active connections are greater than 0:

```text
filter $d['sample#active-connections']:num > 0
```

Of course, this also opens up reporting for Heroku customers too, who can build queries that compute average over time, using DataPrime:

```text
filter addon != null 
| groupby addon avg($d['sample#memory-percentage-used']:num) as avg_memory_utilization
```

## I am not seeing JSON parsed metrics

This feature has been enabled for new users only, to ensure backwards compatibility with our existing Heroku users. Contact our support team via the in-app chat if you wish to move to the new format. In time, this will be rolled out for all customers.

## Handling of units

Units are automatically extracted from the value of the metric, and suffixed into the key. For example, consider the following metric log, which comes from Heroku:

```text
source=DATABASE addon=postgresql-animate-79954 ... sample#memory-total=3944456kB sample#memory-free=84080kB sample#memory-percentage-used=0.97868 sample#memory-cached=3194564kB
```

Notice that the value of `sample#memory-total` is `3944456kB`. Our integration attempts to ensure that this is a calculable numeric value out of the box, so it converts this into `sample#memory-total-kB` with value `3944456`. This means that it is natively available for conversion using [Events2Metrics](https://coralogix.com/docs/user-guides/monitoring-and-insights/events2metrics/index.md).

Note that any pattern after the numeric value is considered a unit, and will be handled in this way, for example: `sample#my-metric=100foo` will become `{"sample#my-metric-foo": 100}`. This will also preserve the unit casing.

This may cause edge cases if undesired special characters appear as units in your logs. In this case, you should use [Parsing Rules](https://coralogix.com/docs/user-guides/data-transformation/parsing/log-parsing-rules/index.md) to change the value. See below for an example of how to handle these edge cases using parsing rules.

## Converting incoming metric logs into metrics

For long term retention, Coralogix works well for logs, but to maintain high performance over longer periods of time, conversion to metrics provides a low cost, simple method for constantly querying data over months. For example, if we consider the `sample#active-connections` field, we can define an [Events2Metric](https://coralogix.com/docs/user-guides/monitoring-and-insights/events2metrics/index.md) for this, that will automatically generate a timeseries metric for this field:

This practice is especially useful for users who wish to maintain dashboard performance over long periods of time for a series of important metrics. It is also important to label your metrics with useful dimensions on which you wish to query. For example, in the above example, we also extract a number of fields that function as labels:

This means that data can now be queried using `PromQL` in the Coralogix app, and the labels will be available on the metric. For example, the following query shows Postgres Read iOps over time, grouped by source & addon names:

```text
avg(postgres_read_iops{postgres_addon=~'{{postgres_addon}}',postgres_source=~'{{postgres_source}}', postgres_application=~'{{postgres_application}}'}) by (postgres_source, postgres_addon) or on() vector(0)
```

## Custom application metrics

Custom application metrics that conform to the expected format will also be converted. That is, they should follow `<metric-type>#<metric-name>=<value>`. For example, `counter#total_cpu_seconds=500`. If you wish for your metrics to be available for conversion into Coralogix metrics using Events2Metrics, then the **value MUST be numeric or decimal followed by a unit**.

### Custom application metrics embedded in other logs

Currently, custom metrics that are embedded inside other logs do not currently work. For example, if your application writes something like:

```text
2025-10-07 16:00:00 INFO v1.package.app.OrderController counter#total_cpu_seconds=500
```

This will **not** be parsed currently. This is because our engine is looking for a key value pattern in the log body, and while this *contains* a key value pattern, it is not entirely composed of them. For this reason, it will not be parsed. See the section entitled *Handling embedded metrics* below for a guide on how to solve this problem.

## Handling unit edge cases with parsing rules

If we consider the following incoming log that we want to extract:

```text
sample#my-metric=50%
```

This will be converted into JSON by our integration, so what arrives at your Coralogix account will look like this:

```text
{
    "sample#my-metric-%": 50
}
```

In its current form, this is valid JSON and will work just fine, although you may wish to alter this to be more readable. To do this, navigate to **Data Flow** > **Parsing Rules**.

From here, either add a rule to an existing rule group or select `Replace` from the rule options at the top of the page:

Then using the following Regular expression, we can replace the value with the desired suffix.

For easy editing, the regular expression is simple `-%` (the value we're looking to replace) and the replacement string is `-perc`, which is the replacement value. This will transform the above log into:

```text
{
    "sample#my-metric-perc": 50
}
```

## Handling embedded metrics

If we consider the following incoming log:

```text
2025-10-07 16:00:00 INFO v1.package.app.OrderController counter#total_cpu_seconds=500
```

We can see that *within* this log is our metric, but our integration engine will not yet automatically parse this value. In order to capture this value, we must use parsing rules to bring this metric out into a JSON key-value pair. Access **Data Flow** > **Parsing Rules**, and select the extract parsing rule:

To extract your desired value, it's very simple. You simply need to isolate the value and declare the key name using a regex capture group, like so:

We can see in the log, the name of the key we're interested in is `counter#total_cpu_seconds`. We also know that the value is entirely numeric, which will almost always be the case because these are metrics.

This is why at the start of our regular expression, we have the key name. This is informing our regex where to look in the string. We then have `=`, which is the character immediately before the value. We have a named capture group called `total_cpu_seconds` (which will become the name of the resultant JSON key) and this capture group is looking for `\d+` which means one or more numerical values.

```text
counter#total_cpu_seconds=(?<total_cpu_seconds>\d+)
```

### Handling embedded metrics with units

The idiomatic method for manually handling custom metrics with units is to parse the metric in the same way, but suffix the name of the resultant key with the unit. For example, considering the following input log:

```text
2025-10-07 16:00:00 INFO v1.package.app.OrderController sample#memory_usage=500kB
```

A simple way to preserve the unit while ensuring the value is numerical is:

```text
sample#memory_usage=(?<sample_memory_usage-kb>\d+)
```

Note that `-kb` has been suffixed onto the end of the metric.

### Handling multiple metrics in a single embedded log

Parsing rules can capture multiple metrics. If you know the expected order of your metrics (i.e they're always written in the same order) then handling them is simple:

```text
counter#total_cpu_seconds=(?<total_cpu_seconds>\d+) sample#memory_usage=(?<sample_memory_usage-kb>\d+)
```