AWS CloudWatch Metrics
Overview
The Coralogix AWS Metrics integration offers a simple and easy way to ingest AWS metrics into Coralogix. It’s a cost-efficient alternative to Amazon Firehose. In addition, it accepts S3
metrics with 24-hour frequency that can’t be collected via Firehose.
The metrics data is collected by Amazon CloudWatch across all your AWS resources. Integrate AWS metrics to retrieve and ingest the data into Coralogix using CloudWatch API.
Poll intervals
By default, metrics are pulled every 10 minutes. To increase savings, you can configure a higher number.
Metric definitions are polled every 5 minutes.
Tag poll interval is 5 minutes for both resource group tagging and specialized APIs.
Prerequisites
STEP 0. Make sure you have STS endpoint enabled for the region where the integration is deployed. To check that it is enabled or to enable it, follow the instructions in the AWS documentation.
STEP 1. Create IAM policy and with CloudFormation template.
STEP 2. Pick AWSAccount
based on target Coralogix environment. Pay attention that the ['ExternalId'] value passed to --parameter-overrides
must be in the Key=Value
format.
aws cloudformation deploy --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM \
--template-file template.yaml \
--stack-name coralogix-cloudwatch-metrics-integration \
--parameter-overrides AWSAccount=EU2 RoleName=coralogix-cloudwatch-metrics-integration ExternalId=my_super-secret-Id-1234
STEP 3. Run it in:
A terminal with installed AWS CLI and configured credentials, or
AWS CloudShell.
STEP 4. Get ARN
of the role for the integration.
aws cloudformation describe-stacks --stack-name coralogix-cloudwatch-metrics-integration --query 'Stacks[0].Outputs[?OutputKey==`CoralogixAwsMetricsRoleArn`].OutputValue' --output text
AWS metrics integration deployment
Basic
STEP 1. From your Coralogix toolbar, go to Data Flow > Integrations.
STEP 2. From the Integrations section, select AWS Metrics.
STEP 3. Enter configuration parameters according to your application requirements (some of the fields are optional):
Integration name
- A meaningful name assigned to the integration for easier identification of this integration instance.AWS Role ARN
- The role ARN created as described in [Prerequisites].AWS Region
- Metrics collection region. To collect data from multiple regions, create multiple integrations.ExternalId
- External ID that the role depends on, so it can’t be assumed only with ARN.CloudWatch Metric Namespaces
- Predefined list of all known AWS metric namespaces for metrics collection.If
With Aggregations
is selected, the_min
,_max
and_avg
metrics will be collected too, if available. Note that you can get the average from the metric with a PromQL query.If
Enrich With Tags
is selected, the resource tags of the metrics related to those resources will be added to the metrics prefixed withcloud_tag
. Note that some metrics are related to specific resources, while others are not. For example, the metricCPUUtilization
inAWS/EC2
has labelInstanceId
, making the metric values relatable to a specific EC2 instance. On the other hand, the metricTotalBackupStorageBilled
inAWS/RDS
isn’t linked to a specific database instance that can have tags.
Advanced
Application Name
andSubsystem Name
fields definecx_application_name
andcx_subsystem_name
labels to be added to all imported metrics.Resource Tags Filter
allows to filter the resources on which to collect metrics. Note that some metrics are not related to specific resources (seeEnrich With Tags
above). This filter has no effect on these metrics. The filter is defined by a comma-separated list of key-value pairs. For example,team,eks:cluster-name=staging
will select all resources that either have the tagteam
set with any value or have tageks:cluster-name=staging
.
Note
The AWS API does not return tags for AWS/AutoScaling, AWS/DMS, AWS/EC2Spot, AWS/Prometheus and AWS/StorageGateway. As a result, the resources from these namespaces are excluded from filtering.
Tags Key Filter
allows selection of the tags to be added as labels to the metrics. If empty, all tags will be added, prefixed withcloud_tag
.Static Labels
allows entering a list of key-value pairs to be added as labels to all metrics in the integration.Poll Interval In Minutes
defines how often the metrics are collected. Shorter intervals mean lower latency and higher costs, and vice versa.Additional Statistics Configuration
allows defining additional metric statistics in JSON format. For example,[{"metric_selector":{"namespace":"AWS/RDS","metric_names":["CPUUtilization"]},"extended_statistics":["p99"]}]
.
Enhanced monitoring
STEP 4. Click Save to create the integration.
If the provided account ARN cannot be assumed by our service, an error message will be displayed. If the integration is created, metrics will start appearing in several minutes. The integration checks the list of available resources with a period of 10 minutes. As a result, it may take up to 10 minutes and above between the creation of a resource and the appearance of the metric. Resources that are created and removed between two scans may never show in the metrics.
Additional resources
Documentation | Get Started with Coralogix AWS CloudWatch Metric Streams with Amazon Data Firehose |
External links | What is CloudWatch View available metrics |
Support
Need help?
Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.
Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].