Coralogix is a powerful tool for querying your logs. By mastering how to query you will be able to find specific events out of millions of logs generated by your applications.
With this skill, you will comfortably be able to investigate issues, create alerts, and visualize your data.
In the following section you will learn about:
Field types
Different types of queries
Concepts
Indexing and field mapping
In order to master how to query logs, it is important to first understand how Coralogix indexes your data after it has been analyzed. Indexing logs is important because it allows you to quickly retrieve (usually within a few seconds) matching logs using:
Free-text searches
Regular expressions
Field searches
Tip: We recommend serializing your logs as JSON to get maximum value from Coralogix analytics features. Read more about parsing unstructured logs to JSON using parsing rules here.
Data types
Coralogix supports the following data types:
Text: This type represents unstructured, human-readable content that is analyzed into terms before indexing.
Keyword: This type represents text that does not pass through the analyzer before indexing. This makes it suitable for regular expressions, aggregation, and sorting. The syntax to use the keyword data type in your query is: <fieldName>.keyword
Note: Coralogix does not create the keyword type when a field is longer than 256 characters.
Numeric: This type is suitable for range queries and arithmetic aggregations (avg, max, min, sum). The syntax to use the numeric data type in your query is: <fieldName>.numeric
Date: This type enables you to filter by timestamp or plot time-series graphs. Values should be formatted as epoch milliseconds.
Geopoint: This type allows you to plot longitude and latitude pairs on a Kibana or Grafana map.
Object: This type represents a hierarchy. This means that it may contain fields of any other type (including objects).
Notes
Coralogix has a default limit of 1000 mapped fields. You can view your teams mapped field statistics under Account settings → Mapping stats
Explicit mapping is supported for timestamps and geopoints. Appending _timestamp or _geopoint to your field name will map it as a date or geopoint respectively. For example, a field named duration_timestamp is mapped as a date.
Dynamic mapping is used for all other fields. This means that at the time of indexing, a new field’s value determines the mapped data type.
Arrays are valid JSON, however, there is no dedicated array data type in Coralogix. This means that:
A field may contain multiple values, and they should all be of the same data type, otherwise a Mapping Exception will occur.
The first value in an array determines the field mapping.
For an array of objects, it is not possible to query each object independently.
Each field in the log is mapped as 3 data types:
Text or Object or Date or Geopoint
Keyword
Numeric
Querying your logs
To get started, navigate to the Explore screen. This screen allows you to query logs from the index or from your Amazon S3 archive.
The supported query languages are Lucene or DataPrime (for archive queries). This tutorial will cover how to use Lucene queries on Coralogix.
Lucene query syntax reference
A Lucene query is composed of Terms and Operators. Terms are extracted from your log by the analyzer. There are 2 types of terms:
Single term: This is a word in your Text field.
Phrase: This is a group of words surrounded by double quotation marks (“)
This tool helps you better understand how the Coralogix analyzer extracts terms from your Text fields.
Free text search
Use this type of search to match terms in ANY field of your log.
Query
Results
a very interesting log message
Matches logs containing these terms. They may appear in any field and in any order
“a very interesting log message”
Matches this exact phrase in any field
Note:Text fields will pass through an analyzer before indexing. The analyzed text is separated into the “terms” used to index your logs.
Field search
Use this type of search to restrict which field MUST match your search term.
Examples:
Query
Results
msg:interesting
Matches logs containing this term in the msg field
msg:“a very interesting log message!”
Matches this exact phrase in the msg field.
msg.keyword:”a very interesting message!”
Matches logs that contain the phrase (including the !)
Range search
Use this to query a range of matching numeric values
Examples:
Query
Results
status_code.numeric:[200 TO 299]
Matches status codes between 200 and 299 (including 200 and 299)
status_code.numeric:{199 TO 300}
Matches status codes between 200 and 299 (excluding 199 and 300)
status_code.numeric:[200 TO 300}
Matches status codes between 200 and 299 (including 200 but excluding 300)
status_code.numeric:{199 TO 299]
Matches status codes between 200 and 299 (excluding 199 but including 299)
Regex search
Regular expressions are available to match patterns in your log. Coralogix supports Lucene regex engine standard operators
The regex pattern to be matched should be enclosed in forward slashes “/”.
Note: Whenever possible, we recommend using regex searches against keywords, because this data type is not passed through the analyzer.
Examples:
Query
Results
msg.keyword:/.*what an interesting message!.*/
Matches logs that contain the pattern “what an interesting message!” (including the !)
version.keyword:/.*v.[1-5].[0-9]{2}.*/
Matches logs that contain the patterns like “v.1.24” or “v.5.69” in the version field
Boolean Operators and Grouping
The operators AND, OR, and NOT can be used to combine multiple filters and create more precise queries.
Parentheses “()” should be used to determine operator precedence whenever you have multiple operators in a query.
Examples:
Query
Results
msg:”failed transaction” AND level: “ERROR” NOT env:”staging”
Matches ERROR level logs that contain the phrase “failed transaction”
(msg:”failed transaction” AND (cluster:”eu” OR cluster:”us”)) NOT env:”staging”
Matches logs from the “eu” or “us” clusters that contain the phrase “failed transaction” but not from the “staging environment”
Still have questions? Reach out via our in-app chat for quick help.
Oh no!We didn’t find anything for “”,
