Integrations

S3 Log Collection

Description

S3 to Coralogix lambda allows you to send your logs from your S3 bucket to Coralogix.

The preferred and easiest integration method will be to use our aws Serverless Application Repository. Search for ‘coralogix’. Don’t forget to check the ‘Show apps that create custom IAM roles or resource policies’ box located just under the search field to see all available applications. Select your application of choice and click on it. You will see detailed instructions in the readme section on the left.

The rest of this document describes a manual configuration of this integration and the Lambda associated with it and should be used if there is a need for special customization.

Usage

This document includes cluster dependent URL’s. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.

 .com.in
Elasticsearch-APIhttps://coralogix-esapi.coralogix.com:9443https://es-api.app.coralogix.in:9443
SSL Certificateshttps://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-EU.crthttps://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-IN
.pem
Cluster URLcoralogix.comapp.coralogix.in

You must add the following environment variables when using this lambda.

 

1. Create an “author from scratch” Node.js 10.x runtime lambda with an S3 read permissions 

2. At “Code entry type” choose “Upload a ZIP file” and upload “s3ToCoralogix.zip”

https://s3-eu-west-1.amazonaws.com/coralogix-public/tools/s3ToCoralogix.zip

3. Add the mandatory environment variables: private_key, app_name, sub_name

env vars for coralogix S3 lambda

If your Coralogix account top level domain is not ‘.com’, add the following variable:

CORALOGIX_LOG_URL=https://api.Cluster URL/api/v1/logs

4. Increase Memory to 1024MB and Timeout to 3 min

5. Choose the S3 bucket you want to get triggered by and change the event type from “PUT” to “Object Created(All)”

S3 bucket for coralogix

6. Multiline pattern: Coralogix supports multiline pattern by default, you can define a custom pattern with an environment variables, for example:

newline_pattern [\s(?={)|(?<=})\s,\s(?={)|(?<=})\s\]. 

custom multiline in S3

6. Click ‘Save’

Your all set! data would be streaming to Coralogix, need help? ping us on our in-app chat for support@coralogix.com. 

Start solving your production issues faster

Let's talk about how Coralogix can help you

Managed, scaled, and compliant monitoring, built for CI/CD

Get a demo

No credit card required

Get a personalized demo

Jump on a call with one of our experts and get a live personalized demonstration