Our next-gen architecture is built to help you make sense of your ever-growing data Watch a 4-min demo video!

Back to All Integrations

Google Cloud Pub/Sub Google Cloud Pub/Sub

Last Updated: Nov. 28, 2022

Coralogix offers a number of different approaches for collecting logs from your Google Cloud environments including using GCP Log Explorer and Google Cloud Storage.

In this tutorial, we will describe how to configure a Logs router to send logs to a Pub/Sub topic and deliver them to Coralogix using a push subscription on the topic.

The main advantage of using a push subscription is that it avoids running any additional software (i.e. functions) in your GCP account which can contribute to operational overhead and costs.

Requirements

  • A Coralogix account
  • GCP account with permissions to configure Logging and Pub/Sub services

Query Params in the Endpoint URL

When you create your subscription, you will need to set the correct Endpoint URL using the following parameters:

  • domain – the region-specific endpoint associated with your Coralogix Account. You can find your Domain here.
  • key – Your private key can be found in your Coralogix account under Settings>Send Your Data. Learn more about your private key.
  • application (optional) – overrides the default application name
  • subsystem (optional) – overrides the default subsystem name
  • computer (optional) – sets the computer name (otherwise there is no computer name)

Configuration

To configure the ingestion of GCP log data to Coralogix, we will first create a new topic in Google Cloud Pub/Sub. Then we will configure the topic as a sink in the logs router and configure a subscription to push the data to Coralogix.

  1. Log in to GCP console
  2. Go to Pub/Sub / Topics and create a Topic
    • Note – Uncheck the ‘Add a default subscription’ checkbox” We will create a subscription in a later step.
  3. Go to Logging / Logs Router and create a Sink
    • Select sink service: Cloud Pub/Sub topic
    • Select the Cloud Pub/Sub topic created in the previous step
    • Optionally: Choose which logs should be included/excluded
gcp sink details
  1. Go to Pub/Sub / Subscriptions and create a Subscription
    • Select the topic created in step 2, and make the following adjustments:
      • Set Delivery type to ‘Push’
      • Set Endpoint URL to ‘https://gcp-ingress.<domain>/api/v1/gcp/logs?key=<private_key>’
      • Set message retention to ‘1 day’
      • Set Retry policy to ‘Retry after exponential backoff delay’ (keep the default backoff values)
  2. Note: if you like to pass additional attributes to Coralogix, set the endpoint to ‘https://gcp-ingress.<domain>/api/v1/gcp/logs?key=<private_key>&?application=<application_name>&?subsystem=<subsystem_name>’
new subscription gcp
subscription details gcp

On this page