Our next-gen architecture is built to help you make sense of your ever-growing data. Watch a 4-min demo video!

Back to All Docs

TCO Optimizer: Logs TCO Optimizer: Logs

Last Updated: May. 22, 2024

By allowing you to define the data pipeline for your logs based on the importance of that data to your business, our TCO Optimizer not only reduces your logging costs by up to two-thirds, but also improves your real-time analysis and alerting capabilities.

Find information on our Tracing TCO Optimizer here.

Overview

The TCO Optimizer allows you to assign different logging pipelines for each application and subsystem pair and log severity. In this way, it allows you to define the data pipeline for your logs based on the importance of that data to your business.

The feature enables you to get all of the benefits of an ML-powered logging solution at only a third of the cost, with an improved ability to query, monitor, and manage your data.

Find information on our Tracing TCO Optimizer here.

TCO Data Pipelines

Compliance Data (Low-Priority)

Log data that needs to be kept for compliance/post-processing reasons will be archived immediately.

Features available include:

Monitoring Data (Medium-Priority)

Logs that are used for monitoring or statistics will be fully available for those use cases. This allows you to define alerts, build dashboards, view statistics, query the live data stream, and receive proactive anomalies.

Features available include:

Frequent Search Data (High-Priority)

Typically high-severity or business-critical data, logs in this pipeline need to be individually queried and analyzed. These logs are stored on highly available SSDs, replicated, and ready to be queried within seconds.

Features available include:

TCO Optimizer Screen

The TCO Optimizer screen includes a Logs tab, split into three sections:

  • Distribution
  • Policy Criteria
  • Application and Policy Overrides

Distribution

The Distribution section presents the percentage of logs for each TCO pipeline: High – Frequent Search, Medium – Monitoring, and Low – Compliance. The Compliance pipeline includes low and blocked logs.

Coralogix TCO Optimizer

Policy Criteria

The Policy Criteria section includes current policies and allows for the creation of new ones. Policies are applied to combinations of applications, subsystems, and severities as the logs are ingested and will assign them the appropriate TCO pipeline based on the policy content. In this way, they simplify assigning TCO pipelines and capture any applicable future logs on ingestion. Each policy creates new default values for the logs for which the policy is applicable.

Notes:

  • The default policy for all logs is high.
  • If policies conflict, the first policy appearing on the screen will take precedence.


TCO Policy criteria

Create a New Log Policy

STEP 1. Click + ADD NEW POLICY.

STEP 2. Choose the relevant filters for applications, subsystems and severities.

STEP 3. Set the priority.

These will become the default TCO pipelines assignments for all matching logs that will be ingested into Coralogix.

STEP 4. Once you finish selecting filters and setting the priority, click Save.

Application and Policy Overrides

The Application and Policy Overrides section displays the usage of all applications and subsystems producing logs, sorted by the top consumers. You may use the top filters to easily locate a specific component. Clicking on any row will allow you to see a detailed view of the application-subsystem pair usage organized by level of severity and its default TCO pipeline assignments as created initially.

In this view, you are able to change the priority for an entire application-subsystem pair or to change the priority for specific severities within any application-subsystem pair. The priority will determine the data pipeline through which the data for this subsystem will be channelled. If there are different TCO pipelines for different severities, the policy status will be Multiple.
TCO Policy override

TCO Tune Severity


To begin overriding from the beginning, remove all overrides by clicking Reset All Overrides.

TCO - Reset all overrides

Additional Resources

DocumentationTracing TCO Optimizer
Archive Retention Policy
APITCO Optimizer API
Archive Setup gRPC API

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at support@coralogix.com.

On this page