Recording Rules

Recording rules allow you to pre-process and derive new time series from existing ones. The rules are defined in a configuration file and are executed in the background at a regular interval, specified in the configuration file.

Overview

A recording rule is defined by:

1. a name
2. a query expression
3. an optional set of labels to add to the resulting time series.

The query expression is used to select the time series that the rule will operate on, and can include aggregation functions such as sum, min, max, etc. When the rule is executed, it will evaluate the query expression over a range of time, specified in the configuration file as evaluation_interval, and create a new time series for each unique combination of label values resulting from the evaluation.

Example:

Suppose you have a metric called request_duration_seconds that represents the duration of HTTP requests made to your web server, and you want to track the average request duration over the past minute. You could define a recording rule like this:

groups:
  - name: 
      rules:
        - record: service:request_duration_seconds:avg1m
          expr: avg(request_duration_seconds{namespace="frontend"}[1m]) by (service)

This rule would create a new time series called service:request_duration_seconds:avg1m for each unique value of the service label, with the value of the time series being the average of the request_duration_seconds time series over the past minute.

Recording-rules API

Coralogix provides an API that allows you to manage your recording rules. In order to use the Recording Rules API, you will need:

1. An authentication key from Data Flow –> API Keys -> Alerts, Rules, and Tags API key
2. Your Coralogix domain determines the API endpoint to be used. Check here to identify your Domain

NOTE:

1. The Alerts, Rules, and Tags API keys are visible only for admin users.
2. Rules that depend on other recording-rules are not supported.
3. Importing a new configuration file will replace the existing recording-rules. Therefore, new configuration must contain all your rules, not just updates/changes.
4. Alerting rules are not supported.

The actions supported by the API are listed below. These examples require that you have the grpcurl tool installed in your environment. Also note that these examples use the EU1 endpoint

Save RuleGroups

This action will Create a new rule-group. In case the rule-group exists, it will be overwritten.

grpcurl -H "Authorization: Bearer <ALERTS_RULES_TAGS_API_KEY>" -d @ ng-api-grpc.coralogix.com:443 rule_manager.groups.RuleGroups.Save <<EOF
{
  "name": "FrontendRuleGroup",
  "interval": 60,
  "limit": "0",
  "rules": [
    {
      "record": "service:request_duration_seconds:avg1h",
      "expr": "avg(request_duration_seconds{namespace=\"frontend\"}[1h]) by (service)",
      "labels": {}
    }
  ]
}
EOF

An empty response is returned on success.

{}

Fetch rule-groups

This action will retrieve the specified rule-group.

grpcurl -H "Authorization: Bearer <ALERTS_RULES_TAGS_API_KEY>" -d @ ng-api-grpc.coralogix.com:443 rule_manager.groups.RuleGroups.Fetch <<EOF
{
  "name": "HTTPRuleGroup"
}
EOF

A single rule-group is returned on success

{
  "ruleGroup": {
    "name": "HTTPRuleGroup",
    "interval": 60,
    "limit": "0",
    "rules": [
      {
        "record": "job:http_requests_total:sum",
        "expr": "sum(rate(http_requests_total{namespace=\"frontend\"}[5m])) by (job)"
      }
    ]
  }
}

List rule-groups

This action will list all existing rule-groups.

grpcurl -H "Authorization: Bearer <ALERTS_RULES_TAGS_API_KEY>" ng-api-grpc.coralogix.com:443 rule_manager.groups.RuleGroups.List

A list of rule-groups is returned on success.

{
  "ruleGroups": [{
  "name": "FrontendRuleGroup",
  "interval": 60,
  "limit": "0",
  "rules": [
    {
      "record": "service:request_duration_seconds:avg1m",
      "expr": "avg(request_duration_seconds{namespace=\"frontend\"}[1m]) by (service)"
    }]
  }, {
  "name": "HTTPRuleGroup",
  "interval": 60,
  "limit": "0",
  "rules": [{
    "labels": {},
    "record": "job:http_requests_total:sum",
    "expr": "sum(rate(http_requests_total{}[5m])) by (job)"
    }]		
  }]
}

Delete rule-groups

This action will delete the specified rule-groups.

grpcurl -H "Authorization: Bearer <ALERTS_RULES_TAGS_API_KEY>" -d @ ng-api-grpc.coralogix.com:443 rule_manager.groups.RuleGroups.Delete <<EOF
{
  "name": "HTTPRuleGroup"
}
EOF

An empty response is returned on success.

{}

Additional Resources

Coralogix Terraform Provider – Recording Rules

Prometheus Recording Rules

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].