How CRED moved their logging platform to Coralogix to improve error alerts, speed up debugging, and lower costs

CRED was using the popular tool Datadog as its primary tool for logging. Additionally, as a Fintech company, they had to adhere to data localization rules and hence needed to keep the logs for PCI-compliant microservices within the Amazon Web Services India region. They used Amazon CloudWatch logs for this use case. 

CRED identified four challenges that needed to be addressed: 

1. Tool sprawl: Multiple logging solutions to maintain across the incumbent observability tool and Amazon CloudWatch.
2. Scale: The log volumes increased as the company and tech teams grew. This impacted performance (basically searching a pattern inside logs from Amazon CloudWatch).
3. Cost: The scale also escalated overheads, and cost issues became a bottleneck.
4. Data Sovereignty: The popular observability provider did not have an India region to support data localization requirements.

CRED decided to look for an alternative solution after considering the above challenges of the incumbent solution. They found the Pattern Search in Amazon CloudWatch’s log stream particularly cumbersome. In cases where the data range is longer than seven days, it took a long time to process. Moreover, it did not have the Live Tail feature available.

CRED found the migration process very straightforward. They used Amazon ECS as their Container Orchestrator and AWS FireLens to send logs to the Coralogix platform. 

The Coralogix team helped a lot with the initial implementation, and then CRED converted the changes into an AWS Cloudformation template. Since their deployments were through Infrastructure as Code (IAC), the change was relatively easy to implement and took between 3-6 months to complete the migration. They also saw value in finding Log Patterns using Coralogix’s Loggregation©, which is an automatic log clustering feature that condenses millions of log entries into a narrow set of patterns using machine learning.

Today there are 450+ users of Coralogix at CRED. The main data sources hosting the primary application layer are Amazon ECS, Amazon EC2, and AWS Fargate containers, along with a small set of Amazon EKS.

As per the technical leads at CRED, they see value derived from Coralogix in 3 areas:

1. Coralogix TCO Optimizer – This allows CRED to suppress or prioritize certain log types, helping to control log volume while maintaining the desired service level.  This was important for cost optimization.
2. Coralogix Alerts feature – To identify high error rates or anomalies in the count for a specific log pattern.
3. Amazon S3 Direct Query – This option helped in querying archived logs directly without rehydrating.

After migrating to Coralogix, CRED now finds that log-based alerting is faster than the previous solution. They felt that Coralogix had enabled faster debugging, because of features like Live Tail and easy searches, without bloating up the cost. Most importantly, developers who work on multiple services do not have to switch to different consoles to observe service logs. CRED is now looking forward to exploring Coralogix as a metrics solution.