Coralogix is a powerful tool for querying your logs. By mastering how to query you will be able to find specific events out of millions of logs generated by your applications.
With this skill, you will comfortably be able to investigate issues, create alerts, and visualize your data.
In the following section you will learn about:
In order to master how to query logs, it is important to first understand how Coralogix indexes your data after it has been analyzed. Indexing logs is important because it allows you to quickly retrieve (usually within a few seconds) matching logs using:
Tip: We recommend serializing your logs as JSON to get maximum value from Coralogix analytics features. Read more about parsing unstructured logs to JSON using parsing rules here.
Coralogix supports the following data types:
Text: This type represents unstructured, human-readable content that is analyzed into terms before indexing.
Keyword: This type represents text that does not pass through the analyzer before indexing. This makes it suitable for regular expressions, aggregation, and sorting.
The syntax to use the keyword data type in your query is: <fieldName>.keyword
Note: Coralogix does not create the keyword type when a field is longer than 256 characters.
Numeric: This type is suitable for range queries and arithmetic aggregations (avg, max, min, sum).
The syntax to use the numeric data type in your query is: <fieldName>.numeric
Date: This type enables you to filter by timestamp or plot time-series graphs. Values should be formatted as epoch milliseconds.
Geopoint: This type allows you to plot longitude and latitude pairs on a Kibana or Grafana map.
Object: This type represents a hierarchy. This means that it may contain fields of any other type (including objects).
To get started, navigate to the Explore screen. This screen allows you to query logs from the index or from your Amazon S3 archive.
The supported query languages are Lucene or DataPrime (for archive queries). This tutorial will cover how to use Lucene queries on Coralogix.
A Lucene query is composed of Terms and Operators. Terms are extracted from your log by the analyzer. There are 2 types of terms:
This tool helps you better understand how the Coralogix analyzer extracts terms from your Text fields.
Use this type of search to match terms in ANY field of your log.
Query | Results |
---|---|
a very interesting log message | Matches logs containing these terms. They may appear in any field and in any order |
“a very interesting log message” | Matches this exact phrase in any field |
Note: Text fields will pass through an analyzer before indexing. The analyzed text is separated into the “terms” used to index your logs.
Use this type of search to restrict which field MUST match your search term.
Examples:
Query | Results |
---|---|
msg:interesting | Matches logs containing this term in the msg field |
msg:“a very interesting log message!” | Matches this exact phrase in the msg field. |
msg.keyword:”a very interesting message!” | Matches logs that contain the phrase (including the !) |
Use this to query a range of matching numeric values
Examples:
Query | Results |
---|---|
status_code.numeric:[200 TO 299] | Matches status codes between 200 and 299 (including 200 and 299) |
status_code.numeric:{199 TO 300} | Matches status codes between 200 and 299 (excluding 199 and 300) |
status_code.numeric:[200 TO 300} | Matches status codes between 200 and 299 (including 200 but excluding 300) |
status_code.numeric:{199 TO 299] | Matches status codes between 200 and 299 (excluding 199 but including 299) |
Regular expressions are available to match patterns in your log. Coralogix supports Lucene regex engine standard operators
The regex pattern to be matched should be enclosed in forward slashes “/”.
Note: Whenever possible, we recommend using regex searches against keywords, because this data type is not passed through the analyzer.
Examples:
Query | Results |
---|---|
msg.keyword:/.*what an interesting message!.*/ | Matches logs that contain the pattern “what an interesting message!” (including the !) |
version.keyword:/.*v.[1-5].[0-9]{2}.*/ | Matches logs that contain the patterns like “v.1.24” or “v.5.69” in the version field |
The operators AND, OR, and NOT can be used to combine multiple filters and create more precise queries.
Parentheses “()” should be used to determine operator precedence whenever you have multiple operators in a query.
Examples:
Query | Results |
---|---|
msg:”failed transaction” AND level: “ERROR” NOT env:”staging” | Matches ERROR level logs that contain the phrase “failed transaction” |
(msg:”failed transaction” AND (cluster:”eu” OR cluster:”us”)) NOT env:”staging” | Matches logs from the “eu” or “us” clusters that contain the phrase “failed transaction” but not from the “staging environment” |
Still have questions? Reach out via our in-app chat for quick help.