Request Demo

Google Cloud Storage


Coralogix provides a predefined function to forward your logs from Google Cloud Storage straight to Coralogix.



Create a Cloud Function in your Google Cloud Console with the following settings:

Function settings


  1. Increase “Memory allocated“ to “1 GB“.
  2. Change “Trigger“ to “Cloud Storage“.
  3. Change “Event Type“ to “Finalize/Create“.
  4. Select your GCS Bucket.
  5. Change “Runtime“ to “Python 3.7“.
  6. Paste the following code to

# -*- coding: utf-8 -*-

Coralogix GCP function
Author: Coralogix Ltd.

import os
import re
import sys
import gzip
import logging
from import storage
from coralogix.handlers import CoralogixLogger

__name__ = 'gcsToCoralogix'
__author__ = 'Coralogix Ltd.'
__email__ = ''
__copyright__ = 'Copyright 2019, Coralogix Ltd.'
__credits__ = ['Ariel Assaraf', 'Amnon Shahar', 'Eldar Aliiev']
__license__ = 'Apache Version 2.0'
__version__ = '1.0.0'
__maintainer__ = 'Coralogix Ltd.'
__date__ = '1 April 2019'
__status__ = 'Stable'

# Get function parameters
PRIVATE_KEY = os.environ.get('private_key')
APP_NAME = os.environ.get('app_name', 'NO_APPLICATION')
SUB_SYSTEM = os.environ.get('sub_name', 'NO_SUBSYSTEM')
NEWLINE_PATTERN = os.environ.get('newline_pattern', '(?:\r\n|\r|\n)')

# Function entrypoint
def to_coralogix(event, context):
    Function entrypoint
    :param event: event metadata
    :type event: dict
    :param context: event context
    :type context: dict

    def get_severity(message: str) -> int:
        Extract severity from message text
        :param message: log record text
        :type message: str
        :return: severity value
        :rtype: int
        severity = 3
        if 'Warning' in message or 'warn' in message:
            severity = 4
        if 'Error' in message or 'Exception' in message:
            severity = 5
        return severity

    # Initialize GCS client
    client = storage.Client()

    # Initialize Coralogix logger
    logger = CoralogixLogger(
    )"Processing file {event['name']}")

    # Get file content
    bucket = client.get_bucket(event['bucket'])
    blob = bucket.get_blob(event['name'])
    content = blob.download_as_string()

    # Check if file is compressed
    if event['contentType'] == 'application/gzip' or \
       event['name'].endswith('.gz'):"Uncompress file {event['name']}")
            # Decompress file
            content = gzip.decompress(content)
        except Exception as exc:
            logging.fatal(f"Cannot uncompress file {event['name']}: ", exc)

    # Split file into line and remove empty lines
    logs = list(filter(None, re.split(NEWLINE_PATTERN, content.decode('utf-8'))))"Number of logs: {len(logs)}")

    # Send logs to Coralogix
    for log in logs:


7. Paste the following packages to requirements.txt:


8. Increase “Timeout“ to “60 seconds“.

9. Add the mandatory environment variables: private_key, app_name, sub_name:

  • Private Key – A unique ID that represents your company, this Id will be sent to your mail once you register to Coralogix.
  • Application Name – Used to separate your environment, e.g. SuperApp-test/SuperApp-prod.
  • SubSystem Name – Your application probably has multiple subsystems, for example, Backend servers, Middleware, Frontend servers etc.

10. Multiline pattern: Coralogix supports multiline pattern by default, you can define a custom pattern with environment variables, for example:

newline_pattern [\s(?={)|(?<=})\s,\s(?={)|(?<=})\s\].


11. Click “Create”.

gcloud CLI

To set up the function, execute this:

$ curl -sSL -o
$ unzip -d gcsToCoralogix/
$ gcloud functions deploy gcsToCoralogix \
--region=us-central1 \
--runtime=python37 \
--memory=1024MB \
--timeout=60s \
--entry-point=to_coralogix \
--source=gcsToCoralogix \
--trigger-resource=YOUR_BUCKET_NAME \ \



Here is the Terraform module to deploy the Cloud Function.

Add this module to your manifest and change its options:

provider "google" {
region = "us-central1"

module "gcs_to_coralogix" {
source = "git::"

private_key = "YOUR_PRIVATE_KEY"
app_name = "APP_NAME"
sub_name = "SUB_NAME"
bucket_name = "YOUR_BUCKET_NAME"


Download the module and apply these changes:

$ terraform init
$ terraform apply


Start solving your production issues faster

Let's talk about how Coralogix can help you

Managed, scaled, and compliant monitoring, built for CI/CD

Get a demo

No credit card required

Get a personalized demo

Jump on a call with one of our experts and get a live personalized demonstration