Google Cloud Operations, formerly known as Stackdriver, is relatively new to the observability space. That being said, its position in the GCP ecosystem makes the platform…
The modern enterprise has expanded its reach by using the power of cloud computing. However, with that power comes complexity in leveraging the multiple platforms needed to provide rich functionality. To achieve a seamless integration that involves multiple cloud infrastructures you need insightful and actionable data. You also need the right team to bring the clouds together in a seamless, effective, and efficient manner. Coralogix together with Capgemini provided the expertise and technology for a recent project that needed full multi-cloud observability across Heroku/Salesforce.
This project shows how with the right team and technology in place, even the most complex, multi-functional, cross-platform system can be made to mesh seamlessly. The customer, a large European automotive manufacturer with a global presence, required a rollout of their global CRM system, based on Heroku Platform-as-a-Service (PaaS). The platform needed to be dynamic scalability and would see significantly increased functionality; the goal, to weave together over 100 applications in a single deployment.
The system was mass-adopted with over 20,000 users as well as thousands of vendors in the supply chain. The key challenge of the project was a major undertaking: Existing sales data must dovetail with a new CRM (Customer Relationship Management) solution through a dedicated integration layer for all inbound and outbound data streams.
Several platforms were behind the customer’s ecosystem:
The team that successfully took on the challenge was made up of core specialists, Capgemini, Salesforce Heroku, and Coralogix. Working in collaboration, the team builds dedicated multi-cloud platforms for customers that meet exacting and complex needs.
To meet the requirements of this major and complex project, Coralogix met a list of criteria for integration across the multi-cloud platform. The core Coralogix functionality powering the observability project included:
Dynamic alerts offer a way to define the threshold for an alert trigger. Coralogix provides even finer granularity as thresholds can be time-based, changing over the course of a day. Ultimately, this reduces alert overload and creates greater focus and accuracy.
The project utilizes Coralogix rules that are used to provide structure to unstructured logs. These unstructured logs are formatted to JSON, which allows for easier and controllable queries and visualizations. The resulting alerts can be used with query-based detection to proactively identify issues and shorten resolution time.
Logs provide context for application lifecycle events. This offers a way to easily understand the impact of new features on the system. For example, during the project, when a problem was identified an automatic tag was created. This is then used to cross-reference against when troubleshooting, helping to reduce resolution time. Notably, one of the latest Coralogix features is the ability to verify an LDAP configuration dynamically, based on the previous 24 hours logs; this helps in finding the threshold for alerts.
ML capabilities empower Coralogix with the ability to decipher key information from data patterns. The Coralogix ML identifies statistically significant errors during a process flows sending out an alert if issues are detected.
Coralogix Machine learning is used for troubleshooting issues in the project, and to provide data for root cause analysis. Two key methods are used in the project:
One of the reasons for choosing Coralogix for the project was the deep integration of Coralogix with Heroku. The ‘out-of-the-box’ syncing between Heroku and Coralogix provides smooth scaling, easy configuration, and proactive monitoring. Configuration in Coralogix is automatic. Once integrated with Heroku a ‘sub-system’ is created in Coralogix. This deep integration facilitates:
Parsing logs: Coralogix is agnostic to data format. However, in this project, unstructured logs were parsed by Coralogix to extract structured data to allow for easier querying.
Alerts: Real-time verification on the accuracy of an alert can be easily configured. A threshold can be set, and rules used to send out alerts on an ‘If This, Then That’ basis; the threshold adding a layer of granularity to the rule to optimize the output.
Machine learning-based anomalies: When Coralogix ML detects an anomalous flow in log volume an email alert is sent. This email provides the user with an interface to drill down into more detail on the anomalous event. This detail includes insight into the logs that were created during the event.
The CI/CD pipeline process is optimized via Coralogix integration with Heroku. Coralogix machine learning capability is then used to spot trends, patterns, and anomalies. These data being mapped to a deployed version. In addition, anomalies can be tied to a specific upgrade. Coralogix is automatically added to the Heroku pipeline. Tags are used to benchmark releases, subsequently applied to analyze data across the software development lifecycle (SDLC).
Because of the rich functionality and deep integration capabilities of the collaboration between the team partners, the project moved in a consistent and focused manner.
The project was expedited by:
All of these features were used to optimize and inform successful deployment choices.
Metrics are expected to be deployed in the coming weeks to enhance long term deviation monitoring.
Coralogix prides itself on the versatility of the Coralogix platform. Coupled with deep integration capability with Heroku, the team delivered a specialized and highly collaborative offering that made the project a success.