AWS Kinesis with Lambda Function

Coralogix provides a predefined Lambda function to forward your Kinesis stream straight to Coralogix.

Usage

The preferred and easiest integration method will be to use our AWS Serverless Application Repository. Search for ‘coralogix’. Don’t forget to check the ‘Show apps that create custom IAM roles or resource policies’ box located just under the search field to see all available applications. Select your application of choice and click on it. You will see detailed instructions in the readme section on the left.

Automation

You can include SAM (Serverless Application Model) in your automation frameworks. If you need access to the latest and greatest Lambda code go to https://github.com/coralogix/coralogix-aws-serverless/tree/master/src.

The rest of this document describes a manual configuration of this integration and the Lambda associated with it and should be used if there is a need for special customization.

This document includes cluster dependent URL’s. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.

1. Create an “author from scratch” Node.js 12.x runtime lambda with the following permissions(Create execution role tutorial):

• kinesis:DescribeStream
• kinesis:DescribeStreamSummary
• kinesis:GetRecords
• kinesis:GetShardIterator
• kinesis:ListShards
• kinesis:ListStreams
• kinesis:SubscribeToShar

2. At “Code entry type” choose “Edit code inline” and paste this function

3. Add the mandatory environment variables:

• Private Key – Your private key is a unique ID that represents your company.
• Application Name – Used to separate your environment, e.g. SuperApp-test/SuperApp-prod.
• If your Coralogix account top level domain is not ‘.com’, add the following environment variable:CORALOGIX_URL=https://api.Cluster URL/api/v1/logs
• SubSystem Name – Your application probably has multiple subsystems, for example, Backend servers, Middlewar, Frontend servers etc.

4.   Go to Add triggers and add Kinesis:

5.  Configure the trigger, select the desired “Kinesis stream” and “Consumer”, change “Batch size” equals to 10:

6.  Increase Memory to 1024MB and Timeout to 1 min.

7.  Click “Save”.