Whether you are just starting your observability journey or already are an expert, our courses will help advance your knowledge and practical skills.
Expert insight, best practices and information on everything related to Observability issues, trends and solutions.
Explore our guides on a broad range of observability related topics.
Make data analysis easier with log normalization.
Managing and analyzing logs across various products presents a significant challenge due to the diverse structures and formats in which logs are stored. This diversity complicates searching for specific information across product logs, as each product may use different keys to store the same information. For example, the source IP address might be recorded under sourceIPAddress
, source.ip
, srcaddr
, or other keys, depending on the reporting product. This discrepancy makes data comparison and aggregation difficult, resulting in complex and inefficient data analysis processes.
Log normalization simplifies data analysis by giving standard names to common values in logs and organizing them using parsing rules.
Deploy one of many Coralogix extension packages to add the cx_security
Add a key to your product logs. This will normalize email addresses, IP addresses, and URLs across your logs into a single key name, even if the same data comes from different key names.
Keys created under cx_security
include source_ip
, destination_ip
, email
, url
, domain
, username
, userAgent
, event_name
, service_name
, result
, source_hostname
, destination_hostname
, resource
, event_result
, event_name
, target_email
, target_username
. Additional keys will be added from time to time.
Here are some examples:
{ cx_security: { email: "[email protected]" source_ip: "1.1.1.1" destination_ip: "2.2.2.2" url: "www.google.com" } sourceIPAddress: "1.1.1.1" destinationIPAddress: "2.2.2.2" website: "www.google.com" user: { id: "[email protected]" } } { cx_security: { source_ip: "1.1.1.1" destination_ip: "2.2.2.2" } network: { src: "1.1.1.1" dst: "2.2.2.2" } }
Get started with log normalization by deploying a Coralogix extension package. Once deployed, the log normalization parsing rules are automatically activated using parsing rules.
STEP 1. Navigate to Data Flow > Extensions from your Coralogix toolbar.
STEP 2. Choose an extension package bound to your integration. Select from any of our extension packages that include log normalization parsing rules:
Note that this list may not reflect the most updated list of extensions.
STEP 3. Define the parameters of the extension.
STEP 4. Deploy.
Utilize the Explore screen to execute queries based on cx_security
keys to retrieve all logs sharing the same value. For example, run the following to retrieve all activity logs associated with the user’s email [email protected]:
cx_security.email:”[email protected]”
In your Custom Dashboard, create widgets grouped by normalized keys under cx_security
.
For instance, create a horizontal bar chart grouping by cx_security.source_ip
to visualize the top source IPs across product logs.
Set up Flow Alerts to inform you know when a series of alerts with the same details happen in a row within a certain time. Grouping normalized keys under cx_security
allows you to configure your Flow Alert with base alerts sharing the same value for these normalized keys.
For example, to detect a suspicious sequence of actions taken by the same user, where each action is reported by a different integration or product, create a Flow Alert where every alert has the same user’s email in cx_security.email
.
Select the normalized keys to be enriched by Unified Threat Intelligence and Geo enrichments, rather than selecting each of the original keys per product log.
For instance, select the cx_security.url
key for the Unified Threat Intelligence enrichment and define an alert to detect the presence of the enriched cx_security.url_suspected
key across all normalized product logs containing a URL.ֿ
We recommend log normalization in parallel to deploying Google Workspace Users, which adds user context to the normalized log keys.
Need help?
Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.
Feel free to contact us via our in-app chat or by emailing [email protected].