Coralogix ‘Reindex’ feature allows you to query your logs directly from your S3 archive using the Elasticsearch syntax. This enables you to store more of your log data in lower cost storage and take advantage of Coralogix’s real time data scanning capabilities.
In order to use this feature make sure you have set Read/Write permission to your AWS S3 archive bucket
If you don’t have such permission you will see the following screen:
Click on the appropriate button and you will be guided through the configuration process:
At this stage, when going into the Reindex screen, you will see an indication that there are no reindexed logs to show:
Click on the ‘Reindex archive’ archive button and the following dialogue box will open:
In the top section, you will fill the Reindex name and description.
In the following section, you can enter the exact Search query you choose to reindex from S3, note, we will not mount anything besides logs matching this query.
Choose the applications, subsystems, severity, and time frame criteria for the reindex.
After clicking “Reindex archived data’ you will be asked to verify your selection as it will affect your daily quota.
After clicking ‘CONFIRM’ you will be taken back to the main Reindexing window. The window will show Reindexing tasks. Each task can be in one of these states:
- Completed – Ready to view logs
- Failed – The reindexing process failed. You have the option to retry or create a new Reindex.
- Processing – ‘REINDEXING …’ is displayed while the reindex process is underway
After processing is finished you can click to go to the logs screen to view and analyze your reindexed logs.
Reindexed Logs, like any other of your logs, will be automatically deleted after the account’s retention period has passed. They differ from other logs by having their original timestamp (which might be out of the retention period) and by not being processed through rules, alerts, archiving, ML and anomalies, livetail and the enrichment engines. This keeps the operational integrity of the notifications, views and analysis provided to you by Coralogix.