Our next-gen architecture is built to help you make sense of your ever-growing data Watch a 4-min demo video!

Back to All Docs

AWS S3 Log Collection via SNS Trigger AWS S3 Log Collection via SNS Trigger

Last Updated: Aug. 01, 2023

Like our S3 Log collection integration, this integration will process logs from you S3 buckets, but is triggered by an SNS notification. For easy setup, use our app in the AWS Serverless Application Repository.

Prerequisites

  • Ready-made SNS Topic
  • Ready-made S3 bucket with configured Event Notifications to above SNS Topic
  • AWS permissions to create Lambdas and IAM roles

Installation

STEP 1. Navigate to the application page and search for Coralogix-S3-via-SNS.

STEP 2. Fill in the required parameters.

STEP 3. Click Deploy.

Parameters & Descriptions

VariableDescription
Application NameStack name of the application created via AWS CloudFormation.
If your log is JSON format, use its dynamic value.
Example: $.level1.level2.value
NotificationEmailFailure notification email address
ApplicationNameApplication name as it appears in your Coralogix UI
BlockingPatternIf you wish to block some of the logs, adding a substring will act as a selector.
Default is empty to send all logs.
BufferSizeBuffer size for logs in the Lambda function
CoralogixRegionCoralogix region associated with your Coralogix domain
CustomDomainCoralogix custom domain. Leave empty if you do not use a custom domain.
DebugCoralogix logger debug mode
FunctionArchitectureFunction supports x86_64 or arm64
FunctionMemorySizeMax memory for the function itself
FunctionTimeoutMaximum time in seconds the function may be allowed to run
NewlinePatternPattern for lines splitting. Default is (?:\r\n|\r|\n).
PrivateKeyCoralogix Send-Your-Data API Key
S3BucketNameName of the S3 bucket to watch
SNSTopicArnARN of SNS topic to subscribe
SamplingRateSets the sampling rate
The rate is set to 1 by default, meaning that it collects every log message from the S3 bucket. Increase it to change the sampling rate [i.e. increase it to 2 to ship 1 of every 2 logs, etc].
SubsystemNameSubsystem name as it appears in your Coralogix UI.
If your log is JSON format, can use its dynamic value, for example: $.level1.level2.value.

Notes:

  • To perform filtering based on prefix or suffix of the S3 object, filters need to be placed on the S3 bucket’s Event Notification. Documentation for this process can be found here.

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

On this page