Our next-gen architecture is built to help you make sense of your ever-growing data Watch a 4-min demo video!

Back to All Integrations

Google Cloud Platform (GCP) Pub/Sub Google Cloud Platform (GCP) Pub/Sub

Last Updated: Mar. 01, 2023

Coralogix offers a number of different approaches for collecting logs from your Google Cloud environments including using GCP Log Explorer and Google Cloud Storage.

The tutorial describes how to configure a Logs router to send logs to a Pub/Sub topic and deliver them to Coralogix using a push subscription on the topic.

The main advantage of using a push subscription is that it avoids running any additional software (i.e. functions) in your GCP account which can contribute to operational overhead and costs.

Requirements

  • Coralogix account
  • GCP account with permissions to configure Logging and Pub/Sub services

Query Parameters: Endpoint URL

When you create your subscription, you will need to set the correct Endpoint URL using the following parameters:

  • domain – Your domain can be found by matching your Coralogix account URL to the table here.
  • key – Your private key can be found in your Coralogix account under Settings>Send Your Data. Learn more about your private key.
  • application (optional) – overrides the default application name
  • subsystem (optional) – overrides the default subsystem name
  • computer (optional) – sets the computer name (otherwise there is no computer name)
  • log_message (optional): controls how GCP Audit logs are transformed
    • Defaults to LogEntry if not specified
    • Payload: parse only payload field content of the log entry as log text
    • LogEntry: parse whole log entry, for example:
{
     "spanId": "abcdefghijk",
     "severity": "INFO",
     ...
     "jsonPayload": {
        ...
     },
     ...
     "receiveTimestamp": "2023-02-27T10:16:23.544899312Z",
     "timestamp": "2023-02-27T10:16:20.035098Z",
 }

Configuration

To configure the ingestion of GCP log data to Coralogix, we will first create a new topic in Google Cloud Pub/Sub. Then we will configure the topic as a sink in the logs router and configure a subscription to push the data to Coralogix.

  1. Log in to GCP console
  2. Go to Pub/Sub / Topics and create a Topic
    • Note – Uncheck the ‘Add a default subscription’ checkbox” We will create a subscription in a later step.
  3. Go to Logging / Logs Router and create a Sink
    • Select sink service: Cloud Pub/Sub topic
    • Select the Cloud Pub/Sub topic created in the previous step
    • Optionally: Choose which logs should be included/excluded
gcp sink details
  1. Go to Pub/Sub / Subscriptions and create a Subscription
    • Select the topic created in step 2, and make the following adjustments:
      • Set Delivery type to ‘Push’
      • Set Endpoint URL to ‘https://gcp-ingress.<domain>/api/v1/gcp/logs?key=<private_key>’
      • Set message retention to ‘1 day’
      • Set Retry policy to ‘Retry after exponential backoff delay’ (keep the default backoff values)
  2. Note: if you like to pass additional attributes to Coralogix, set the endpoint to ‘https://gcp-ingress.<domain>/api/v1/gcp/logs?key=<private_key>&application=<application_name>&subsystem=<subsystem_name>’
new subscription gcp
subscription details gcp

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

On this page