Get up to speed on everything that’s new and improved in the Coralogix platform!
Service Catalog is a new feature added under Coralogix’s APM offering. It includes a list of all the services in your system and you can view details such as service type, number of requests sent by the service, the error rate, and the P95 latency by specifying a time range.
You can drill down on each service to look at a service map in the left pane and additional details in the right pane. The service map shows all the services connected to a root service and the right pane provides tabs for a detailed overview, actions taken by the services, health status of resources and logs of each service.
Under standard user-defined alerts, if the alert conditions are set for ‘More than’, you can now choose a new type of evaluation window to define the queried time window. You can choose between the newly added Rolling Window and the existing Dynamic Creation.
Rolling Window is a fixed timeframe and doesn’t change with alert triggers. Dynamic Duration changes the queried time period when an alert is triggered. Rolling Window is now the default choice of evaluation window and is recommended to be used when using ‘Group-by’ with an alert.
Pie Chart is added as a new visualization widget under custom dashboards. It comes with abilities to group-by, aggregate and stack-by data sources and other advanced controls to customize the visuals. Pie Chart is the 4th visualization widget added to custom dashboards.
In addition, ‘Metrics’ is added as a new filter type to make dashboards customizable by the source of data. A ‘Save As’ button is added to clone existing dashboards easily, and a ‘Show Only Errors’ button is added to highlight only errors in span-based widgets.
With the new feature and design improvements, you can now do more with Extensions on Coralogix. Prometheus recording rules can now be deployed as part of an extension. Security and AWS enrichments can be deployed in addition to Geo and Custom Enrichments. Also, selective deployment of enrichments is now possible, allowing you to select only specific enrichment rules when deploying an extension.
A notification will be displayed if deploying the extension will take you over the maximum enrichment quota. Also, hover backgrounds are now shown on dashboard screenshots. And lastly, design and UI improvements have been made to Extensions.
The flow of alerts can now be grouped by the common group-by keys used in the defined alerts. This allows you to examine all alert stages in the context of a single field value. The group-by options available for each alert can be viewed in the Flow Builder by hovering over the alerts.
The group-by fields that can be selected are automatically pulled from the list of fields (from Log-based alerts) and tags (from Span-based alerts), and only ones that show up throughout all alerts will be pulled out automatically for selection.
The K8S dashboard in Coralogix now supports OTEL metrics if you are using an OpenTelemetry collector with a Kubernetes orchestration together with Prometheus to send your data to Coralogix. You can use this feature along with our other application performance monitoring (APM) features, to get a full picture of your system performance.
The K8S dashboard gives you a comprehensive view of your system’s clusters, nodes and pods. In addition, the dashboard also shows the health status of your resources and displays all Kubernetes events. This guide can help you collect Kubernetes events using OpenTelemetry.
Custom Dashboards are now more flexible. Use spans as a new data source, along with logs and metrics. The data source is available across all existing visualizations including Line Chart, Data Table and Gauge visualizations.
Spans can also be added as a data source for filters on any Custom Dashboard, allowing you to apply filters on parameters such as Span Service. Filters in Custom Dashboards also support Span Tags, which can be added as fields in the filter. Spans filters will only affect spans widgets on a dashboard, and will not affect any widgets based on logs or metrics.
Alert notifications can now be grouped into multiple ‘Notification Groups’ for different keys. You can add multiple webhooks to split the alert triggers and customize the notification frequency for each webhook.
You can further activate the ‘Notify when Resolved’ trigger to stay on top of alert notifications.
In addition, an individual notification is sent to each value of the Group By key when the unique query conditions are met within the specified timeframe.
For example, if you have used “region” as a Group By key and let’s say, it has 2 values i.e. Region A and Region B; individual notifications will be sent to both A and B respectively.
A new Content Column is added to the ‘Explore’ screen grid, to improve visibility for content fields and distinguish them from the labels fields. Most logs will have content fields and it is now easy to extract these fields from logs, according to a predefined order list (message, log, k8s.log
etc).
You can edit the content list and truncate long keys and values in the label fields and show them flattened instead of JSON. With this new feature, you can define important/favourite labels that will be sorted first, have a default favourites list and allow an extension to update it.
Admins can now configure multiple private API keys to send data to their Coralogix teams. You can further customize these API keys and activate/deactivate them as required. After creating the new key, it is visible in the Send Your Data section.
Customizations are made available to the private key management to create, modify, activate, deactivate, reactivate and delete the private API keys created.
To enable this feature please contact your TAM or Coralogix’s support.
Managing your extensions is now easier as you will be notified about the updates available for your extensions through a notification bar found at the top of the Extensions screen. Further, an ‘Update’ button is added, that can be used to update an already deployed extension.
For some older extensions, you may have to remove and redeploy them by selecting the required applications and sub-systems. This way you can switch between versions and stay up to date with the extensions you are actively using.
Under the Data Usage section, you can now download a detailed data usage report. The report
is exported as a CSV file and the generation of these reports is available by API.
The detailed usage report will contain multiple fields including the application, sub-systems, severity, TCO tier, type of data source, amount of GBs sent and amount of units sent.
Along with the Metrics Cardinality feature, the ‘Metrics Usage’ screen can offer you a visual representation of your metrics-related data. Expanding on this capability, we have now added a new visualization i.e. ‘Metrics Usage Trend’. This allows you to visualise 7 days of history for each metric value within the Metrics Usage screen.
The time history can help you identify the metrics that are using up your quota very frequently. Identifying such metrics can help you in cost optimization and get better insights into noisy or expensive applications.
Create unlimited, personalized custom dashboards. Use three new visualizations – Data Table, Line Chart, and Gauge – to define and create a dashboard catered to your specific observability needs. Then query across your widgets using our new Filter and Variable capabilities.
You can now generate metrics from spans in addition to log data! Get started in the platform under Data Flow > Events2Metrics.
In the Dashboard menu dropdown, find the new Service Map screen which shows a system overview on all services based on the traces being sent.
Service Map works out-of-the-box if you’ve already got tracing data in Coralogix!
DataPrime is now available across fully indexed, frequent search logs AND data that sits in your own archive.
Use DataPrime to explore your data, perform schema on read transformations, group and aggregate fields, extract data, and much more!
Control the length of archive retention for different groups of logs. Define different lifecycle policies in the Setup Archive page and configure permissions in your bucket.
In the TCO Optimizer, you can configure which logs are tagged with each retention. Only logs that are received AFTER the rule has been created will be tagged.
New column in ‘Explore’ screen grid automatically extracts the key with the meaningful textual content in the log (i.e. ’message’ / ‘msg’ / ‘info’…)
We’ve added some fine touches to the logs view to help you find what you’re looking for faster.
Lucene syntax highlighting makes it easier to read and write queries.
Highlighting is available everywhere in the platform where you can write Lucene queries. In the Explore Screen, terms and fields that match your query are also highlighted within the resulting logs.
You can now acknowledge, assign, and resolve alerts directly from the Insights UI. Use Coralogix as a task allocation and process control UI to extend and consolidate your monitoring and alerting workflow.
Visualize logical groupings of your alerts and their statuses for a powerful “at a glance” view of what’s going on in your systems.
For example, if you group your alerts by cluster and node, the Alerts Map will highlight which cluster-node pairings have one or more triggered alerts.
Query your tracing data with Lucene syntax in the tracing UI. You can now build complex searches for specific fields and phrases contained within traces and spans.
The dependency view is now the default tracing view with the metadata about the trace now shown on the right-hand side of the screen. Hovering over a given node in the graph will reveal a tooltip with information about the trace and associated span.
Organization admins can now create new teams from the “My Teams” tab in the Settings page. To create a new team, fill in the 3-step wizard with your new team name, unit allocation, and team admins.
If you need help getting started, reach out to our support team using the in-app chat!
Prometheus Recording Rules
Prometheus Recording Rules can now be pulled from Coralogix using cURL commands. Use recording rules to pre-compute new timeseries based on existing ones. Once defined, the corresponding recording rules are automatically created as additional metrics in Coralogix.
View Documentation >
AWS Lambda Metrics
Our Lambda extension now supports collection of AWS Lambda metrics for essential invocation measurements. Additionally, CX_Metadata is included alongside incoming documents on the Coralogix platform to provide context around cloud provider, account ID, cloud region and more.
View Documentation >
APM for Amazon EC2
Using Amazon Kineses Data Firehose, you can now send Amazon EC2 metrics to Coralogix and view them on your Coralogix dashboard correlated with relevant traces and spans.
View Documentation >
JumpCloud SCIM Identity Management
New SAML integration with JumpCloud allows you to manage groups and users from in JumpCloud. Once integrated, those changes will be automatically reflected in the Coralogix UI.
View Documentation >
Application Performance Monitoring (APM)
Coralogix now offers key features of Application Performance Monitoring (APM) for modern, cloud-native environments!
With expanded visibility into service performance, you can more effectively monitor latency and pinpoint components responsible for issues like performance degradation or an increase in errors.
Read more about our unique approach to APM and how we can help you turn 30-minute investigations into 30-second discoveries.
OpenTelemetry Support for Logs, Metrics & Traces
You can now use OpenTelemetry to send logs, metrics, and tracing data to Coralogix!
Leverage the popular vendor-neutral, open-source framework for instrumentation and collection of your telemetry data for analysis in the Coralogix platform.
AWS Resource Enrichment
New AWS Resource Enrichment allows you to enrich your logs with tags from Amazon Web Services (AWS) EC2 instances. Use this feature to connect your business and operation metadata from AWS and gain greater insight into your data.
To get started, visit our documentation for setup and configuration.
Lucene Query Auto-Complete
Auto-complete for Lucene queries is now available across the platform. Suggestions are provided for operators (including field range hints) and data type hints!
Extensions Improvements
The Coralogix Extensions lobby has been improved with search, label, and status filters.
In addition, individual extensions can now be deployed to specific applications and subsystems as part of their configuration.
Note that selecting “all applications” or “all subsystems” will apply the extension to all existing and future applications and subsystems created in the Coralogix platform.
Integrations
Our new Coralogix AWS Lambda Telemetry Exporter is now available in the AWS Serverless Application Repository as an open beta. We encourage you to try it out and welcome any feedback.
Use our new integration with Telegraf Operator to simplify metric collection in Kubernetes. With Telegraf Operator, you only need to define the input plugin configuration for Telegraf when creating the pod annotations. Telegraf Operator then sets the configuration for the entire cluster, avoiding the need to configure a metrics destination when deploying applications.
We’ve updated our integration with Google Cloud Pub/Sub to use a push subscription to send logs to Coralogix.
We recommend to use the updated integration with push subscription as it avoids running any additional software (i.e. functions) in your GCP account which can contribute to operational overhead and costs.
Dashboard Updates
Logs Graph Grouped by Severity
Previously, each column would display a single combined amount for Warning/Error/Critical logs. Now, each severity is stacked and a tooltip displays the breakdown of the severity count.
Clicking a section will navigate to the Explore tab with the relevant severity and timeframe filters applied.
Help Tooltips on Dashboard Widgets
New tooltips have been added to the dashboard widgets with an explanation of the information being displayed.
Top 3 Abnormal Errors – Above Normal / Newly Introduced
The Top 3 Abnormal Errors widget now differentiates between ‘Above Normal’ and ‘Newly Introduced’ errors. ‘Above Normal’ shows the amount above normal that is being seen (e.g. 3.2X normal levels), and ‘Newly Introduced’ shows the number of occurrences of the new errors.
Explore Updates
TCO Priority Added as Metadata
A Metadata field is added to each field indicating TCO Level. This can be seen in tooltips within the dashboard.
Tracking Logical Data Types
Logical data types (i.e. IP Addresses, URLs, Emails) are now being detected in addition to raw data types (i.e. String, Number, etc).
This data is also displayed in the column tooltip. If more than one data type has been detected in a field (e.g. IP and String), they will all be displayed in the tooltip.
Content Type Icons added to Column Headers
New icons indicating content type have been added to column headers.
Archive Query Improvements
DataFusion Query Engine Enhancements
The most recent round of improvements to the DataFusion query engine are supporting a conservative 5X increase in query speeds over AWS Athena.
With DataFusion there is no longer a hard limit to the number of partitions that can be scanned, it is restricted only by the resources assigned to it.
22 New Scalar Functions in DataPrime
22 new Scalar functions have been added to DataPrime, including:
Lucene Query Improvements
Autocomplete for Lucene queries has been added within the DataPrime archive query UI, and will soon be added to the standard log query fields across the platform.
Tracing Alert
Create Tracing alerts for high latency on specified Tags and Services. Tracing alerts can be grouped by different Tags and specified for a specific threshold of Latency and Spans.
‘Group By’ for ‘Less Than’ Alerts
Enhance your ‘Less Than‘ alerts using the ‘Group By’ option. Values under the ‘Group By’ fields are aggregated into a histogram, and an alert will trigger whenever the condition threshold is met for an aggregated value within the specified timeframe.
Enhancements to Logs Screen UI
A new warning icon in the Logs Screen indicates if there are any mapping exceptions in the selected time frame.
To streamline analysis and reduce the screen space each log takes, we added new Row Formats so you can choose how your logs are displayed.
Choose from the following options:
Organization and Admin Console
Some companies prefer separate teams to isolate data based on the environment it originates from like Dev, QA, and Production. While others prefer to isolate the data based on organizational units like Infrastructure, Security, and Application.
Coralogix supports multi-tenancy, allowing a single organization to contain multiple teams. The Org Admin can then manage quota, settings, users, and more at the organization level from a single interface.
Organizations can be created upon request. Please contact us through our in-app chat or via email at [email protected].
New Integrations
New & Improved Dashboard!
Our dashboard just got a refresh!
Get an overview of your system health – informed by all of your observability data – with more in-depth widgets showing a summary of your anomalies, alerts, and more.
Plus, use the new sidebar filters to seamlessly drill down and investigate further with a single click to the Explore UI.
DataPrime Archive Query Syntax
DataPrime is now officially GA!
Use DataPrime to parse and query unstructured data fields on the fly. Plus, generate synthetic fields and run calculations from your archived data. From the query UI, view your query history and access a complete cheat sheet with documentation of the query format and operators.
Explore UI Improvements
Log Screen
Improvements to the Top Graph in the Logs screen enable you to view and investigate your data more efficiently:
The state of columns in the Log Screen are persistent in the URL and custom views can be easily shared with additional team members.
Avg, Max, Min, and Sum aggregations can now be used to visualize log fields that contain numeric values!
Tracing Screen
Hovering over graphs in tracing will now show a crosshair for faster analysis.
DataMap Filtering & Actions
You can now filter hexagons in the DataMap using multiple Field Operators and Regex.
Create and access Actions from DataMap using metric labels ($l.<labelName>) to seamlessly connect DataMaps to external resources using metric variables and labels.
General UI Improvements
Insights UI
New ‘Go To Explore’ functionality allows you to jump from the Insights UI directly to the relevant query in the Explore tab.
Alerts Menu
You can now Clone and Delete alerts directly from the main alerts menu!
Webhooks
Added fields that can be shipped in the body of the payload.
Session Timeout & Force Logout
Admins can now configure a session length on the Setting page after which users will be forced to log back in with no dependency on activity.
New Integrations
Flow Alert
Define a sequence of alerts in our Drag & Drop Flow Builder UI that combines Logs, Metrics, Tracing, and Security information to create a single alert flow that will trigger based on multiple conditions within defined timeframes.
Use Flow Alerts to generate actionable insights from your observability data:
New Integrations
New Parsing Rules
This month we are introducing 2 new parsing rules to bring more value to customers who have many fields and nested fields in their log data.
The new Stringify JSON Field and Parse JSON Field rules enable you to parse escaped JSON values within a field to a valid JSON object and vice versa – stringify a JSON object to an escaped string.
DataMap Updates
The DataMap allows you to build custom mappings of your infrastructure using metric data for monitoring system health and quickly identifying issues.
In the Group Editor, you’ll find new options to:
In the DataMap display, use new ‘Compare to others’ functionality to compare an element with 10 others in the same group. Plus, expand and collapse specific groups to minimize the number of displayed elements.
Tracing Updates
New dynamic graphs and saved views in the Tracing UI enable it to serve as SLA dashboards for any application or service.
In addition to the original default graph for Max duration by Action, there are now two additional default graphs for Count by Service and Error Count by Service.
All three graphs can be customized, and aggregation operators have been added for 99, 95, and 50th percentiles to help deepen your ability to monitor business SLOs.
When investigating traces in the explore section, you can now save your current view and load saved views just like you do in the Logs UI.
*Note that the aggregation operators, as well as the Duration filter in the sidebar, are run over the Spans.
Archive Query Updates
Improvements to the archive query now allow timeframes up to 3 days for added accessibility to data in your remote bucket.
Additional updates to the Archive Query in Explore Screen include:
New Integrations
DataMap
Build custom mappings of your infrastructure, log-based, and business metrics to visualize and monitor your system health.
Pro-tip! Use tooltips to view additional information about the area of your system that you are looking at in the mapping visualization.
Tracing UI
Use our powerful Tracing UI to explore your data and streamline investigations and troubleshooting.
CX-DATA Archive Format
We’ve launched a new archive format based on Parquet that improves archive query performance by 5X!
In addition to the CSV format, supported today, this new CX-DATA format can be configured in your S3 Buckets Settings and selected in the Archive Query screen in the platform.
Snooze Alerts in Alerts Tab
The option to snooze an alert, which was previously only available from the Insights screen, will now be available from within the main Alerts screen. This allows for centralized management of alert statuses across your team.
Session Timeout Management
A new option on the Settings page gives users the ability to choose how much idle time will trigger a force logout from the system. In case of inactivity, a pop-up will appear to alert the user, prior to logout.
New Integrations