Our next-gen architecture is built to help you make sense of your ever-growing data Watch a 4-min demo video!

Start Free Trial Request Demo
Back to All Docs

Archive Query from the Explore Screen Archive Query from the Explore Screen

Last Updated: Oct. 20, 2022

The ‘Archive query’ allows you to query the data in your S3 archive directly from the Explore screen with the live data streams on the same screen and using familiar functionalities.

Query your archive in the Coralogix UI using any text or Elasticsearch syntax query – even if the data was never indexed and without the usage of your daily quota.

This enables you to store more of your data in our monitoring and compliance priority levels (Read more here) and take advantage of Coralogix’s real-time analysis and remote storage search capabilities. It also means you can use a shorter retention period for frequent search and still be able to query all your data in less than 1 minute.

Find the Archive Query in the top-left portion of the Logs UI (see image below).

Connect S3 Bucket

In order to use the Archive Query feature from the Explore screen, make sure you have set Read/Write permission to your AWS S3 archive bucket (read more about enabling the Archive feature here)

If you don’t have such permission you will see the following screen:

CX-Data Archive Format

Note: This new archive format offers 5X better archive query performance.

In Setup Archive under the Data Flow tab, you will now see the option to ‘Configure cx-data Bucket’. Once you’ve configured a bucket in CX-Data format, you will be able to choose CX-Data in the dropdown as shown below.

Functionality

  • Query using any text or Elasticsearch syntax query.
  • Filter your logs using the familiar application, subsystem, and severity filters.
  • Use Actions to trigger 3rd party services/websites based on your search results and/or values under specific keys. Learn more about Actions.
  • View surrounding archived data through the context menu and copy permalink:
  • Add a value by clicking on the value and selecting “INCLUDE IN QUERY”
  • Coming soon! Highlight Log

Minor Limitations

  • ‘Archive Query’ time range limit is up to 72 hours at a single query.
  • Running queries on archived data is a bit slower than regular Explore screen queries. Note that if you store your data in CX-Data format, your queries will be much faster.
  • It is possible to use the same query syntax as queries run on the Archive Query page. For better performance, we strongly recommend querying your data using key-value syntax.
  • Some of the functionalities of the Explore screen are not available yet for the Archive queries.
  • Results shown on the logs grid are capped at 10,000 raw logs. While aggregations are available across all the data, we only pull up to 10,000 raw logs to display in the main logs grid. (Kibana for example limits that number to 500 logs).
  • Reindexing the data fetched with the ‘Archive Query’ is currently not available from the Explore screen. To reindex archived data please create an ‘Archive query’ under the Data Flow –> Archive Queries. Learn more about reindexing here.
  • Exporting the data in the logs grid from an archive query is limited to the top 20 pages (100 logs per page) so you can export a max of 2000 logs at a time. In order to export all archive query logs, please create an ‘Archive query’ under the Data Flow –> Archive Queries.

For any questions, please don’t hesitate to visit us via our Application Chat. We are here to help!

On this page