Find out why a notification did not arrive
A notification that never arrives has a small number of causes, and they sit at different points in the chain. Work through them in order rather than guessing, because the first step tells you whether Coralogix tried to deliver at all.
What you need
- Permission to view routers and connectors. See Notification Center permissions.
- Access to the Notification Deliveries dataset.
Start with the delivery record
Query the Notification Deliveries dataset for the entity you expected a notification about. It records delivery attempts, the destination, and the failure category and error when an attempt fails.
The answer splits two ways:
- A record exists. Coralogix matched a router and tried to deliver. The problem is between Coralogix and the destination, so continue with Delivery failed at the destination.
- No record exists. Coralogix never sent anything, so the problem sits upstream of delivery. Continue with No router matched.
Case notifications record both successes and failures. Alert notifications record failures.
No router matched
No delivery record means no router matched, so work back through the chain that decides routing.
- The entity carries no routing labels. An alert carries the labels you set on it. A Case inherits its labels from the ownership tags on the infrastructure the problem came from, so infrastructure with no ownership tags produces Cases that carry nothing to match on. See Ownership.
- No router matches the labels. A router only fires when its conditions match. Check the conditions against the labels the entity actually carries, rather than the ones you expect it to carry.
- The trigger is not selected. A Case router notifies on the lifecycle events you select. A Case that becomes active again sends nothing if the rule only covers activation. See Cases.
Delivery failed at the destination
A delivery record with a failure tells you the destination rejected or dropped the notification. The error it carries is the place to start.
Slack stops accepting a burst
Slack applies a rate limit per workspace. When one alert fires many times in a short window, the notifications it produces can exceed that limit, and Slack rejects the excess. Coralogix retries, but a retry can only help while the limit resets, so a long enough burst ends with notifications discarded rather than delivered.
Nothing in Slack tells you this happened. The delivery record does.
A burst usually traces back to one noisy alert rather than to Notification Center. Narrow the alert, or group its permutations, so it produces fewer notifications. Routing the alert to a Case rather than to a per-notification destination also collapses many firings into one piece of work.
The connector points somewhere that is not ready
A connector can point at a destination that cannot accept what Coralogix sends. A ServiceNow instance without the Coralogix integration installed is the common case: the connector looks configured, and delivery fails at the far end.
When a connector is new, send a test notification before relying on it.
The destination expects a different method
The Generic HTTPS connector sends a POST. A destination that expects a different method rejects the request, and the notification never lands.