Firehose Logs Module
Firehose Logs module is designed to support AWS Firehose Logs integration with Coralogix.
Logs - Usage
Firehose Delivery Stream
Provision a firehose delivery stream for streaming logs to Coralogix - add this parameters to the configuration of the integration to enable to stream logs:
module "cloudwatch_firehose_logs_coralogix" {
source = "coralogix/aws/coralogix//modules/firehose-logs"
firehose_stream = var.coralogix_firehose_stream_name
api_key = var.api_key
coralogix_region = var.coralogix_region
integration_type_logs = "Default"
source_type_logs = "DirectPut"
}
Secrets Manager API key
Optionally pass api_key_secret_arn instead of api_key so Firehose fetches the key at runtime via SecretsManagerConfiguration. The secret value must be JSON: {"api_key": "..."}. Rotating the secret then takes effect without a Terraform apply. If the secret is encrypted with a customer managed KMS key, also set api_key_secret_kms_key_arn. When the module creates the Firehose IAM role, it grants secretsmanager:GetSecretValue (and kms:Decrypt when a CMK ARN is set). If you pass existing_firehose_iam, that role must already allow those actions.
module "cloudwatch_firehose_logs_coralogix" {
source = "coralogix/aws/coralogix//modules/firehose-logs"
firehose_stream = var.coralogix_firehose_stream_name
api_key_secret_arn = var.api_key_secret_arn
coralogix_region = var.coralogix_region
integration_type_logs = "Default"
source_type_logs = "DirectPut"
}
Dynamic Values Table for Logs
For application_name and/or subsystem_name to be set dynamically in relation to their integrationType's resource fields (e.g. CloudWatch_JSON's loggroup name, EksFargate's k8s namespace). The source's var has to be mapped as a string literal to the integrationType's as a DynamicFromField with pre-defined values:
| Field | Source var | Expected String Literal | Integration Type | Notes |
|---|---|---|---|---|
applicationName field in logs | applicationName | ${applicationName} | Default | need to be supplied in the log to be used |
subsystemName field in logs | subsystemName | ${subsystemName} | Default | need to be supplied in the log to be used |
| CloudWatch LogGroup name | logGroup | ${logGroup} | CloudWatch_JSON CloudWatch_CloudTrail | supplied by aws |
kubernetes.namespace_name field | kubernetesNamespaceName | ${kubernetesNamespaceName} | EksFargate | supplied by the default configuration |
kubernetes.container_name field | kubernetesContainerName | ${kubernetesContainerName} | EksFargate | supplied by the default configuration |
name part of the log.webaclId field | webAclName | ${webAclName} | WAF | supplied by aws |
As the parameter value expected is in string format of ${var}, it is required to be escaped with $$ in terraform to be interpreted as a string literal. For example, to set subsystem_name to the ${logGroup} variable would be subsystem_name = "$${logGroup}".
Note: RawText integrationType does not support dynamic values.
For more information - visit Kinesis Data Firehose - Logs.
Examples
Examples can be found under the firehose-logs examples directory
Override Coralogix applicationName and subsystemName
The application name and subsystem name by default is the firehose delivery stream arn and name, but it can be overriden by setting an environment variable called application_name and subsystem_name.
Coralogix account region
The Coralogix region variable accepts one of the following regions:
- EU1
- EU2
- AP1
- AP2
- AP3
- US1
- US2
Coralogix regions and endpoints
| Region | Domain | Endpoint |
|---|---|---|
| EU1 | eu1.coralogix.com | https://ingress.coralogix.com/aws/firehose |
| EU2 | eu2.coralogix.com | https://ingress.eu2.coralogix.com/aws/firehose |
| AP1 | ap1.coralogix.com | https://ingress.ap1.coralogix.com/aws/firehose |
| AP2 | ap2.coralogix.com | https://ingress.ap2.coralogix.com/aws/firehose |
| AP3 | ap3.coralogix.com | https://ingress.ap3.coralogix.com/aws/firehose |
| US1 | us1.coralogix.com | https://ingress.us1.coralogix.com/aws/firehose |
| US2 | us2.coralogix.com | https://ingress.us2.coralogix.com/aws/firehose |
Custom Domain
It is possible to pass a custom Coralogix domain by using the custom_domain variable.
{/* BEGIN_TF_DOCS */}
Requirements
Providers
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| coralogix_region | Coralogix account region: EU1, EU2, AP1, AP2, AP3, US1, US2 [exact] | any | n/a | yes |
| api_key | Coralogix account logs API key | any | n/a | yes |
| firehose_stream | AWS Kinesis firehose delivery stream name | string | n/a | yes |
| application_name | The name of your application in Coralogix | string | n/a | yes |
| subsystem_name | The subsystem name of your application in Coralogix | string | n/a | yes |
| cloudwatch_retention_days | Days of retention in Cloudwatch retention days | number | n/a | no |
| custom_domain | Custom domain for Coralogix firehose integration endpoint (private.coralogix.net:8443) | string | null | no |
| source_type_logs | The source_type of kinesis firehose: KinesisStreamAsSource or DirectPut | string | DirectPut | no |
| kinesis_stream_arn | If 'KinesisStreamAsSource' set as source_type_logs. Set the kinesis stream's ARN as the source of the firehose log stream | string | "" | no |
| integration_type_logs | The integration type of the firehose delivery stream: 'CloudWatch_JSON', 'WAF', 'CloudWatch_CloudTrail', 'EksFargate', 'Default', 'RawText' | string | Default | no |
| s3_backup_custom_name | Set the name of the S3 backup bucket, otherwise variable '{firehose_stream}-backup-logs' will be used | string | null | no |
| existing_s3_backup | Use an existing S3 bucket to use as a backup bucket. | string | n/a | no |
| govcloud_deployment | Enable if you deploy the integration in govcloud | bool | false | no |
| firehose_iam_custom_name | Set the name of the IAM role & policy, otherwise variable '{firehose_stream}-firehose-metrics-iam' will be used. | string | n/a | no |
| existing_firehose_iam | Use an existing IAM role to use as a firehose role. | string | n/a | no |
| user_supplied_tags | Tags supplied by the user to populate to all generated resources | map(string) | n/a | no |
| override_default_tags | Override and remove the default tags by setting to true | bool | false | no |
| s3_enable_secure_transport | Disable if you dont want bucket policy that complies with s3-bucket-ssl-requests-only rule | bool | true | no |
| content_encoding | Set encoding of data in firehose to GZIP or NONE | string | GZIP | no |
Region name mapping
| Coralogix region | AWS Region | Coralogix Domain |
|---|---|---|
Europe | eu-west-1 | eu1.coralogix.com |
Europe2 | eu-north-1 | eu2.coralogix.com |
India | ap-south-1 | ap1.coralogix.com |
Singapore | ap-southeast-1 | ap2.coralogix.com |
AP3 | ap-southeast-3 | ap3.coralogix.com |
US | us-east-2 | us1.coralogix.com |
US2 | us-west-2 | us2.coralogix.com |
Outputs
{/* END_TF_DOCS */}