Skip to content

Firehose Logs Module

Firehose Logs module is designed to support AWS Firehose Logs integration with Coralogix.

Logs - Usage

Firehose Delivery Stream

Provision a firehose delivery stream for streaming logs to Coralogix - add this parameters to the configuration of the integration to enable to stream logs:

module "cloudwatch_firehose_logs_coralogix" {
  source                         = "coralogix/aws/coralogix//modules/firehose-logs"
  firehose_stream                = var.coralogix_firehose_stream_name
  api_key                        = var.api_key
  coralogix_region               = var.coralogix_region
  integration_type_logs          = "Default"
  source_type_logs               = "DirectPut"
}

Dynamic Values Table for Logs

For application_name and/or subsystem_name to be set dynamically in relation to their integrationType's resource fields (e.g. CloudWatch_JSON's loggroup name, EksFargate's k8s namespace). The source's var has to be mapped as a string literal to the integrationType's as a DyanamicFromFrield with pre-defined values:
FieldSource varExpected String LiteralIntegration TypeNotes
applicationName field in logsapplicationName${applicationName}Defaultneed to be supplied in the log to be used
subsystemName field in logssubsystemName${subsystemName}Defaultneed to be supplied in the log to be used
CloudWatch LogGroup namelogGroup${logGroup}CloudWatch_JSON
CloudWatch_CloudTrail
supplied by aws
kubernetes.namespace_name fieldkubernetesNamespaceName${kubernetesNamespaceName}EksFargatesupplied by the default configuration
kubernetes.container_name fieldkubernetesContainerName${kubernetesContainerName}EksFargatesupplied by the default configuration
name part of the log.webaclId fieldwebAclName${webAclName}WAFsupplied by aws

As the parameter value expected is in string format of ${var}, it is required to be escaped with $$ in terraform to be interpreted as a string literal. For example, to set subsystem_name to the ${logGroup} variable would be subsystem_name = "$${logGroup}".

Note: RawText integrationType does not support dynamic values.

For more information - visit Kinesis Data Firehose - Logs.

Examples

Examples can be found under the firehose-logs examples directory

Override Coralogix applicationName and subsystemName

The application name and subsystem name by default is the firehose delivery stream arn and name, but it can be overriden by setting an environment variable called application_name and subsystem_name.

Coralogix account region

The coralogix region variable accepts one of the following regions: * EU1 * EU2 * AP1 * AP2 * AP3 * US1 * US2

Coralogix Regions & Description.

RegionDomainEndpoint
EU1eu1.coralogix.comhttps://ingress.coralogix.com/aws/firehose
EU2eu2.coralogix.comhttps://ingress.eu2.coralogix.com/aws/firehose
AP1ap1.coralogix.comhttps://ingress.ap1.coralogix.com/aws/firehose
AP2ap2.coralogix.comhttps://ingress.ap2.coralogix.com/aws/firehose
AP3ap3.coralogix.comhttps://ingress.ap3.coralogix.com/aws/firehose
US1us1.coralogix.comhttps://ingress.us1.coralogix.com/aws/firehose
US2us2.coralogix.comhttps://ingress.us2.coralogix.com/aws/firehose

Custom Domain

It is possible to pass a custom coralogix domain by using the custom_domain variable.

Requirements

NameVersion
aws~> 4.17.1
terraform>= 1.6.0

Providers

NameVersion
aws~> 4.17.1

Inputs

NameDescriptionTypeDefaultRequired
coralogix_regionCoralogix account region: EU1, EU2, AP1, AP2, AP3, US1, US2 [exact]anyn/ayes
api_keyCoralogix account logs API keyanyn/ayes
firehose_streamAWS Kinesis firehose delivery stream namestringn/ayes
application_nameThe name of your application in Coralogixstringn/ayes
subsystem_nameThe subsystem name of your application in Coralogixstringn/ayes
cloudwatch_retention_daysDays of retention in Cloudwatch retention daysnumbern/ano
custom_domainCustom domain for Coralogix firehose integration endpoint (private.coralogix.net:8443)stringnullno
source_type_logsThe source_type of kinesis firehose: KinesisStreamAsSource or DirectPutstringDirectPutno
kinesis_stream_arnIf 'KinesisStreamAsSource' set as source_type_logs. Set the kinesis stream's ARN as the source of the firehose log streamstring""no
integration_type_logsThe integration type of the firehose delivery stream: 'CloudWatch_JSON', 'WAF', 'CloudWatch_CloudTrail', 'EksFargate', 'Default', 'RawText'stringDefaultno
s3_backup_custom_nameSet the name of the S3 backup bucket, otherwise variable '{firehose_stream}-backup-logs' will be usedstringnullno
existing_s3_backupUse an existing S3 bucket to use as a backup bucket.stringn/ano
govcloud_deploymentEnable if you deploy the integration in govcloudboolfalseno
firehose_iam_custom_nameSet the name of the IAM role & policy, otherwise variable '{firehose_stream}-firehose-metrics-iam' will be used.stringn/ano
existing_firehose_iamUse an existing IAM role to use as a firehose role.stringn/ano
user_supplied_tagsTags supplied by the user to populate to all generated resourcesmap(string)n/ano
override_default_tagsOverride and remove the default tags by setting to trueboolfalseno
s3_enable_secure_transportDisable if you dont want bucket policy that complies with s3-bucket-ssl-requests-only rulebooltrueno
content_encodingSet encoding of data in firehose to GZIP or NONEstringGZIPno

Coralgoix regions

Coralogix regionAWS RegionCoralogix Domain
Europeeu-west-1eu1.coralogix.com
Europe2eu-north-1eu2.coralogix.com
Indiaap-south-1ap1.coralogix.com
Singaporeap-southeast-1ap2.coralogix.com
AP3ap-southeast-3ap3.coralogix.com
USus-east-2us1.coralogix.com
US2us-west-2us2.coralogix.com

Outputs

NameDescription
firehose_stream_arnARN of the Firehose Delivery Stream
firehose_stream_nameName of the Firehose Delivery Stream
firehose_iam_role_arnARN of the Firehose IAM role
s3_backup_bucket_arnARN of the Firehose S3 Backup Bucket