Coralogix launches OpenAPI endpoints

Observability is about much more than dashboards and alerts. Extensible platforms that integrate into the user’s tech stack are fundamental parts of a great developer experience. This is why Coralogix has supported gRPC APIs for account management, data ingress & query, alert definition, dashboard creation, permissions management and more.

Today, Coralogix adds a new integration, with the launch of OpenAPI endpoints for all existing functionality. OpenAPI is a well understood format for consistent, self documenting APIs. By introducing this API, Coralogix users can seamlessly integrate downstream and upstream automation with the Coralogix platform. 

How do I get started?

Documentation is available in our newly hosted portal, which gives parameters and conventions for all API endpoints. Every endpoint strictly conforms to the OpenAPI spec, meaning any existing OpenAPI libraries that you have will work consistently. 

There are also code samples for common languages, like node, python, Javascript and PHP. This makes your integration process simple, and offers clear examples of how to integrate with the Coralogix APIs. 

Why should I use the Coralogix APIs?

The Coralogix platform is extremely flexible and via the user interface, most customers can achieve the outcomes they need. However, to get the absolute maximum value out of their telemetry, customers can also query telemetry and manipulate the Coralogix platform via the APIs. 

In addition, OpenAPI is a well known, publicly available standard. This allows users of any programming language to interface with the new Coralogix endpoints, greatly expanding the compatibility with tooling across our thousands of customers. 

By connecting running production code with the telemetry that they produce, it creates the opportunity for entirely new feedback loops, that not only allows the system to optimize itself for better production performance, but also allows customers to achieve a number of other interesting outcomes:

• Better cost optimization, by updating TCO rules when telemetry volumes shift
• Deploying a new suite of alerts when a new customer is onboarding into your platform
• Automatically expanding an Events2Metrics instance when it begins to run out of permutations.

This self-healing relationship, between production code and the underlying telemetry, allows automatic resolution of problems that may appear down the line.

Worked Example: Automatic Optimization of TCO Rules with Cloudflare & AWS Lambda

The Coralogix TCO Optimizer allows customers to route their data to different use cases, allowing them to ingest up to 10x the volume for the same unit cost. This is typically done via the UI, where users define policies. 

For example, a user can define a policy that ensures all debug and info level logs from their CDN are routed to Monitoring, meaning that the data is eligible for metric generation, direct query, updating dashboards, triggering alarms and more, without the overhead of indexing. This reduces ingest costs by around 65% for logs. 

Defining policies via the UI is powerful, but we can automate policy definition through the API, to make it possible for customers to make TCO decisions automatically, based on traffic volumes or error rates. 

Consider the following architecture, where Cloudflare is pushing telemetry data into Coralogix. Coralogix is then configured to push events to AWS Lambda, and Lambda is making updates via the Coralogix API.

The first signs of a DDoS Attack

Cloudflare WAF offers extremely sophisticated DDoS detection, and it surfaces this information beautifully in the generated WAF logs. These WAF logs offer scores and insights into the type of attack that is occurring, and the action that the Cloudflare firewall took to mitigate the impact. 

In our Coralogix account, we begin to see the following logs incoming from Cloudflare WAF. The WAFAttackScore field is around 30, indicating a high probability that this represents a malicious attack. 

{
   "BotScoreSrc":"Machine Learning",
   "ClientMTLSAuthStatus":"unknown",
   "WAFAttackScore":31,
   ...
}

There are many ways to detect a DDoS attack in Cloudflare, but a simple option here is to build a Coralogix alarm and look for a high volume of WAF actions where the WAFAttackScore is less than 35 and there are over 10,000 of such requests within a 10 minute period. 

Defining a self adjusting system

This alert is then configured to push events to a downstream AWS Lambda function via AWS EventBridge. This lambda function invokes the Coralogix API and makes a new policy. This lambda function can also look for the resolution, and automatically remove the TCO policy. Here is some example pseudocode, to give some idea of how this function may work:

export const handler = async(event) => {
    if (event.alert.triggered) {
    // Invoke the Coralogix OpenAPI endpoint to create our overflow TCO rule.

    client.send({
        "policy": {
            "id": "policy_id",
            "companyId": 1234,
            "name": "Overflow Policy",
            "description": "High Traffic Dynamic Policy",
            ...
        }
    })

    } else if (event.alert.resolved) {
      // Invoke the Coralogix OpenAPI endpoint to remove the overflow TCO rule.
    }

    else {
      // No op
    }
}

The full repository for this lambda function is available here in Github, and ready to deploy today! It is fully configurable and can be wired up for this exact use case. 

When WAF activity begins to spike, the alarm triggers and the lambda function creates a new policy, which subsequently defines a new dynamic policy within the TCO. This policy can automatically route Cloudflare WAF logs from Monitoring to Compliance, changing the cost per unit from a 60% reduction to an 80% reduction. Alternatively, the new policy can block the logs entirely to save on costs altogether. 

The power of dynamic observability

At Coralogix, we want our users to enjoy a dynamic, responsive platform that speaks their language – whether it’s button clicks, terraform, kubernetes resources, gRPC or HTTP. This easy integration opens the door for your observability platform to be more than simply dashboards or alarms – it becomes a full, decision making engine that deeply connects, in real time, with your business needs.