Our next-gen architecture is built to help you make sense of your ever-growing data Watch a 4-min demo video!

Start Free Trial Request Demo
Back to All Docs

Google Workspace Data Ingestion – GCP Google Workspace Data Ingestion – GCP

Last Updated: Jun. 13, 2023

The following tutorial demonstrates how to successfully integrate Google Workspace with Coralogix and send us your logs using Filebeat and Google report API.

This document includes cluster dependent URLs. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.

Coralogix DomainElasticsearch-APISSL Certificates
coralogix.comhttps://coralogix-esapi.coralogix.com:9443https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-EU.crt
coralogix.inhttps://es-api.app.coralogix.in:9443https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-IN.pem
coralogix.ushttps://esapi.coralogix.us:9443https://www.amazontrust.com/repository/AmazonRootCA1.pem
coralogixsg.comhttps://es-api.coralogixsg.com:9443https://www.amazontrust.com/repository/AmazonRootCA1.pem
eu2.coralogix.comhttps://es-api.eu2.coralogix.com:9443https://www.amazontrust.com/repository/AmazonRootCA1.pem

Set up a GCP Service Account

Follow the official Google tutorial for setting up a service account.

Grant access to the Admin SDK API

Follow the official Google Workspace tutorial for granting access to the Admin API.

Note: to gather Google Alert Center, enable the Google Workspace Alert Center API on the GCP project.

Delegate domain-wide authority to your service account

  • Go to your Google Workspace domain’s Admin console.
  • Go to Main menu > Security > Access and data control >  API controls.
  • Under  the Domain-wide delegation pane, select Manage Domain Wide Delegation.
  • Click Add new, and fill in the details:
    • Client ID – Enter the service account’s Client ID – you can find it in the service account’s details under Unique ID. It is also found in the client_id field of the credentials file that was auto-downloaded when you created a new key for your service account.
    • OAuth Scopes – Enter https://www.googleapis.com/auth/admin.reports.audit.readonly
    • Click Authorize to confirm your changes.

Once done with the configuration of Google Workspace, now we should start with filebeat.

Filebeat Setup

Please follow the link for Coralogix filebeat setup (Make sure to use the latest version)

Please make sure to have at least version 7.12 and up for filebeat.

Filebeat Google Workspace Configuration

ignore_older: 3h
filebeat.modules:
- module: google_workspace
  saml:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  user_accounts:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  login:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  admin:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  drive:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"
  groups:
    enabled: true
    var.jwt_file: "path to the service accounts credentials file"
    var.delegated_account: "email of the primary admin Google Workspace user"

fields_under_root: true
fields:
  PRIVATE_KEY: "Send-Your-Data API Key"
  COMPANY_ID: your_company_Id
  APP_NAME: "App_name for example google"
  SUB_SYSTEM: "Sub_system_name"

processors:
- drop_fields:
    fields:
    - event.original
    ignore_missing: true

logging:
  level: debug
  to_files: true
  files:
  path: /var/log/filebeat
  name: filebeat.log
  keepfiles: 10
  permissions: 0644

output.logstash:
  enabled: true
  hosts: ["logstashserver.Cluster URL:5015"] 
  tls.certificate_authorities: ["/etc/filebeat/ssl/coralogix.crt"]
  ssl.certificate_authorities: ["/etc/filebeat/ssl/coralogix.crt"]

After applying the changes. Start your filebeat service.

SAMLView users’ successful and failed sign-ins to SAML applications.
User AccountsAudit actions carried out by users on their own accounts including password changes, account recovery details and 2-Step Verification enrollment.
LoginTrack user sign-in activity to your domain.
AdminView administrator activity performed within the Google Admin console.
DriveRecord user activity within Google Drive including content creation in such as Google Docs, as well as content created elsewhere that your users upload to Drive such as PDFs and Microsoft Word files.
GroupsTrack changes to groups, group memberships and group messages.

If you want to learn more about the Google Workspace plugin please see the link.

Please note that Google Workspace defaults to a 2-hour polling interval because Google reports can go from some minutes up to 3 days of delay.
For more details on this, you can read more here.

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

On this page