New Value Alerts
The New Value alert is triggered by the first occurrence of a new value within a time interval. All values are tested against a list that is being dynamically created while the alert is active. The alert is set by a specific query to identify a subset of logs (if needed), and is defined with a key of choice to track for new values within the desired interval.
In many use cases, this alert enables you to automatically detect a possible abnormal behavior within your system.
A few use cases examples for this alert type include:
Security: An alert may be triggered by a new domain connection. As Coralogix Cloud Security logs all security information across all network traffic, a new domain connection will result with the field ‘security.highest_registered_domain’ having a new value. This can point to a possible attack (Command & Control activity, Data ex-filtration, etc).
Monitoring: An alert may be triggered by a new application error code. Many applications send an ‘error_code’ field. A new value for this field indicates a new issue with the application.
Create New Value alerts
STEP 1. Create an alert.
There are 2 ways to creating an alert:
- Through the explore screen.
The advantage of creating an alert through the explore screen is that you can create your query, adjust the filters you want to alert on (application/subsystem, severity, fields..). Once you hit create Alert all the filters and query will be added automatically.
- Alerts > Alert Management tab.
With the Alert Management tab you are creating the alert from scratch.
- Click NEW ALERT on the top-right area of the UI.
STEP 2. Define alert details: Name, Description, Priority (P1, highest to P5, lowest), Labels (A new label or an existing one. Nest a label using key:value
.).
You can also select the Set as Security Alert checkbox to add the alert_type:security
label. This will help Security customers filter for this alert type in the Incidents screen.
STEP 3. Select New Value alert type.
STEP 4. [Optional] Choose to add a query, and adjust the application, subsystem, and severity of the logs you want to be considered for by the alert to trigger.
STEP 5. Define Conditions.
Key to track: A key from your logs that you want to track for new values(country, city name).
Notify on new value in the last: The duration you want keep tracking this key. You can track a key up to 3 months for new values.
Notify Every: This is used to tune the alert if the alert is noisy and triggers more often.
Note
When an alert is triggered, it won’t be triggered again until one of two things happens: either the Notify Every period passes or it is resolved. In the latter case, the Notify Every parameter is reset.
STEP 6. Define Notification settings.
By default, a single notification, aggregating all values matching an alert query and conditions, will be sent to your Coralogix Insights screen.
+ Add Webhook. Define additional alert recipient(s) and notification channels.
Notify Every. Sets the alert cadence. After an alert is triggered and a notification is sent, the alert will continue to work, but notifications will be suppressed for the duration of the suppression period.
Notify when resolved. Activate to receive an automatic update once an alert has ceased.
Phantom Mode. Toggle the Phantom Mode switch to silence the alert. In the Phantom mode, alerts can serve as building blocks for flow alerts without triggering independent notifications or creating an incident.
STEP 7. Set Schedule.
The schedule is a good option if you have 2 Teams in 2 different Time zones handling or collaborating on the same tasks. You can chose the days when Team "A" should be alerted and the same thing for Team "B".
STEP 8. Define Notification Content.
When a notification is sent, it contains a sample log. It is the newest log that matches the query and alert triggering key:value
pair.
Note
If the alert was set with an evaluation delay, the log’s timestamp might be newer, placing it beyond the boundaries of the alert’s evaluation timeframe.
Sample log size is limited to 1.5Kb.
You can control the sample log content by:
Choosing a specific JSON key(s) to include specific fields into the log sample and removing the rest of the log content.
Specifying a simple JSONPath as a filter.
Leaving blank to view the full log text.
Note
A new/updated alert will become active after the configured alert time window or 7 days (the shorter of the two). This is in order for Coralogix to train on the set of different values, capture a baseline as well as try to prevent false notifications.
The alert can track up to 50K unique values in the defined time window. When the captured values list gets to 50K, the alert will not be triggered until values are cleared from the list. A value will be cleared from the list when its age in the list is equal to the alert time window. The first detection of this value after it was deleted will trigger an alert.
The first 255 characters will be taken as the value (i.e if you have 2 values that have the same first 255 chars, they will be considered as the same value).
There is a 5 min "silence" period after the alert was triggered. During this time, new values will be added to the list but the alert will not be triggered.
Support
Need help?
Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.
Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].