AWS Kinesis Firehose Integration to Coralogix

This template can be used to deploy an AWS Kinesis Firehose Integration to send resource metrics to Coralogix.

For a more detailed description of the settigns and architecture of this AWS Kinesis Data Firehose setup, please refer to the Coralogix documentation on AWS Kinesis Data Firehose – Metrics.

Prerequisites

• AWS account.
• Coralogix account.

Main Parameters

Parameter	Description	Default Value	Required
CoralogixRegion	The region of your Coralogix Account. If set to Custom, you must provide a CustomDomain otherwise url will be invalid.	Allowed Values: - Custom - EU1 - EU2 - AP1 - AP2 - AP3 - US1 - US2 - US3 Default: Custom	✅
CustomDomain	The Custom Coralogix domain. If set, will be the domain to send telemetry.
ApiKey	Your Coralogix Private Key. Required unless ApiKeySecretArn is set.		✅ (unless using ApiKeySecretArn)
ApiKeySecretArn	ARN of a Secrets Manager secret holding the Coralogix API key as JSON {"api_key": "..."}. When set, Firehose reads the key at runtime so rotating the secret takes effect without a stack update. Must be in the same region as the delivery stream. See AWS Firehose Secrets Manager.		✅ (unless using ApiKey)
ApiKeyKmsKeyArn	ARN of the KMS key used to encrypt the Secrets Manager secret. Required only when the secret uses a customer-managed key (CMK).
ApplicationName	Your Coralogix Application name
SubsystemName	Your Coralogix Subsystem name

Metrics Stream Parameters

Parameter	Description	Default Value	Required
IntegrationTypeMetrics	The data structure of the Firehose delivery stream for metrics. For CloudWatch_Metrics_OpenTelemetry070_WithAggregations, additional aggregations here are _min, _max, _avg recorded as gauges. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-metric-streams-formats-opentelemetry-translation.html	Allowed Values: - CloudWatch_Metrics_OpenTelemetry070 - CloudWatch_Metrics_OpenTelemetry070_WithAggregations Default: CloudWatch_Metrics_OpenTelemetry070_WithAggregations
OutputFormat	The output format of the cloudwatch metric stream	Allowed Values: - opentelemetry0.7 - json Default: opentelemetry0.7
IncludeNamespaces	A string comma-delimited list of namespaces to include to the metric stream e.g. AWS/EC2,AWS/EKS,AWS/ELB,AWS/Logs,AWS/S3
IncludeNamespacesMetricNames	A string json list of namespaces and metric_names to include to the metric stream. JSON stringify the input to avoid format errors. e.g. {"AWS/EC2":["CPUUtilization","NetworkOut"],"AWS/S3":["BucketSizeBytes"]}
AddtionalStatisticsConfigurations	A json list of additional statistics to include to the metric stream following MetricStream StatisticsConfiguration. JSON stringify the input to avoid format errors.	"p50","p75","p95","p99" of the following - AWS/EBS:[VolumeTotalReadTime,VolumeTotalWriteTime] - AWS/ELB:[Latency,Duration], - AWS/Lambda:[PostRuntimeExtensionsDuration] - AWS/S3:[FirstByteLatency,TotalRequestLatency]
EnableMetricsTagsProcessors	Enable the lambda metrics tags processor function. Set to false to remove the processor	true
IncludeLinkedAccountsMetrics	Enable cross-account observability to include metrics from linked source accounts (requires CloudWatch OAM setup between monitoring and source accounts)	false
CrossAccountEnabled	Enable Lambda cross-account tag enrichment. When true, Lambda assumes per-account roles from CrossAccountRoles.	false
CrossAccountRoles	JSON map of source account IDs to role ARNs used for tag enrichment. Example: {"123456789123":"arn:aws:iam::123456789123:role/CoralogixMetricsReader"}	{}

Optional Parameters

Parameter	Description	Default Value	Required
CloudwatchRetentionDays	Days of retention in Cloudwatch retention days	1

Notes:

• Provide either ApiKey or ApiKeySecretArn (or both; the secret wins). ApiKeySecretArn requires a JSON secret {"api_key": "..."} — a different format from plaintext secrets used with ApiKey or CloudFormation {{resolve:secretsmanager:...}} dynamic references.

• Switching an existing stack from ApiKey to ApiKeySecretArn replaces the Firehose delivery stream (the stream name becomes auto-generated). Update any CloudWatch Metric Streams that target the previous stream ARN after the update.

• When ApiKeySecretArn is set, the Coralogix integration status notifier is omitted (it cannot read the runtime secret).

• For cross-account enrichment, each linked account role (for example CoralogixMetricsReader) must trust the monitoring account Lambda role and allow sts:AssumeRole:

  • Principal: arn:aws:iam::<monitoring-account-id>:role/<stack-name>-lambda
  • Action: sts:AssumeRole

• LambdaMetricsTagsProcessors lambda function code is deployed to the following s3 regions: [ us-east-1 us-east-2 us-west-1 us-west-2 ap-south-1 ap-northeast-2 ap-southeast-1 ap-southeast-2 ap-northeast-1 ca-central-1 eu-central-1 eu-west-1 eu-west-2 eu-west-3 eu-north-1 eu-south-1 sa-east-1 ]. If you are using a different region, please contact Coralogix support.

Deploy the Cloudformation template using aws cli

With the aws cli installed and configured, run the following command:

aws cloudformation create-stack --stack-name <stack_name> --template-body template.yaml --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM CAPABILITY_NAMED_IAM --parameter-overrides ApiKey=<coralogix_api_key> CoralogixRegion=<region> ApplicationName=<application_name> SubsystemName=<subsystem_name> EnableLogsStream=<true/false> EnableMetricsStream=<true/false> 

or with a parameters json file example:

aws cloudformation create-stack --stack-name <stack_name> --template-body template.yaml --parameters parameters.json --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM CAPABILITY_NAMED_IAM