Safe Use of Recording Rule–Based Metrics and Event2Metrics in SLOs

Coralogix SLOs can be powered by metrics derived from Prometheus recording rules or from Event2Metrics-generated metrics. When those metrics come from chained recording rules—where one rule depends on another—there’s a risk of inaccurate evaluations.

This guide explains:

• Why metric-generation processes (chained recording rules or E2M) can cause incomplete or premature SLO results.
• How to safely use recording rule–based metrics in your SLOs.
• How to apply the PromQL offset modifier to ensure evaluation accuracy.

Why chained recording rules can introduce errors

In chained setups, a downstream recording rule may run before its upstream recording rule or E2M has finished writing data. This timing mismatch can cause the downstream rule to persist incomplete values, which are then used in the SLO. The result: inaccurate or misleading SLO evaluations.

The solution: Apply an offset

To avoid this issue, apply a PromQL offset to your SLO query. The offset shifts the evaluation window backward, ensuring all dependent data is fully materialized before being used.

For example:

increase(my_recorded_metric offset 2m)

This is especially useful in situations where the most recent data might not yet be complete—such as with recording rules or E2M, which can have slight delays in updating. Applying an offset (for example, 2m) helps avoid premature or inaccurate evaluations, particularly in time-sensitive use cases like SLOs.

Query examples

The examples below are applicable to recording rule-based metrics within an SLO query (RR_based_metric), but can be implemented in the same manner also when using an E2M_based_metric.

Basic total events (no filter)

Without offset:

```promql
sum(increase(RR_based_metric)) by (service_name)
```

With offset:

```promql
sum(increase(RR_based_metric offset 2m)) by (service_name)
```

Good events (with filter)

Without offset:

```promql
sum(increase(RR_based_metric{status!="error"})) by (service_name)    
```

With offset:

```promql
sum(increase(RR_based_metric{status!="error"} offset 2m)) by (service_name)   
```

Compound query 1

Without offset:

```promql
sum(increase(RR_based_metric_successes)) by (service_name)
+
sum(increase(RR_based_metric_failures)) by (service_name)    
```

With offset:

```promql
sum(increase(RR_based_metric_successes offset 2m)) by (service_name)
+
sum(increase(RR_based_metric_failures offset 2m)) by (service_name)    
```

Compound query 2

Without offset:

```promql
sum(increase(RR_based_metric_successes) + increase(RR_based_metric_failures)) by (service_name)    
```

With offset:

```promql
sum(increase(RR_based_metric_successes offset 2m) + increase(RR_based_metric_failures offset 2m)) by (service_name)    
```