Learn more about Streama© – the foundational technology behind our stateful streaming data platform. Learn More

Archive Query from Logs screen

Coralogix ‘Archive query’ is available to you via the Logs screen enabling the option of querying your past archived data along with the live stream of your data in the same screen side by side with all the familiar Logs screen utilities.

‘Archive query’ allows you to query your data directly from your S3 archive using any text or Elasticsearch syntax query (In the future, SQL queries with presto functions will also be supported), even if the data was never indexed and without the usage of your daily quota. This enables you to store more of your data in our monitoring and compliance priority levels (Read more here) and take advantage of Coralogix’s real-time analysis and remote storage search capabilities. This means you can use a shorter retention period and still be able to query all your data in less than 1 minute using the familiar ES syntax.

In order to use this feature make sure you have set Read/Write permission to your AWS S3 archive bucket (read more about enabling the Archive feature here)

If you don’t have such permission you will see the following screen:

Limitations

  • ‘Archive Query’ time range limit is up to 24 hours at a single query.
  • Some of the functionalities of the Logs screen are not available for the Archive queries.
  • Archived queries are slower than regular Logs screen queries.
  • Results shown on the logs grid are capped to 10K of logs. On the other hand, the statistics around the Archived query represent the entire set of data matching the executed query.
  • Reindexing the data fetched with the ‘Archive Query’ is currently not available from the Logs screen. To reindex archived data please create an ‘Archive query’ under the TCO tab –> Archive query to reindex archived data, visit here for more info.
  • Exporting the data in the logs grid father an archive query is limited to the top 20 pages (100 logs per page) so you can export a top of 2000 logs. In order to export all archive query logs, please create an ‘Archive query’ under the TCO tab –> Archive query.

For any questions, please don’t hesitate to approach us via the app chat. We are here to help.