Our next-gen architecture is built to help you make sense of your ever-growing data Watch a 4-min demo video!

Back to All Docs

G Suite Data Ingestion – Google Cloud Platform (GCP) G Suite Data Ingestion – Google Cloud Platform (GCP)

Last Updated: Feb. 01, 2023

The following tutorial demonstrates how to successfully integrate G Suite with Coralogix and send us your logs using Filebeat and Google report API.

This document includes cluster dependent URLs. Each URL has a variable part (in Italic). Please match this part with a row entry within the following table. Copy the table row entry located under the column that matches the top level domain of your Coralogix account (.com, .in etc.). Replace the variable part of the URL with this entry.

Elasticsearch-APISSL CertificatesCluster URL
.comhttps://coralogix-esapi.coralogix.com:9443https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-EU.crtcoralogix.com
.ushttps://esapi.coralogix.us:9443https://www.amazontrust.com/repository/AmazonRootCA1.pemcoralogix.us
.inhttps://es-api.app.coralogix.in:9443https://coralogix-public.s3-eu-west-1.amazonaws.com/certificate/Coralogix-IN.pemapp.coralogix.in
.eu2.https://es-api.eu2.coralogix.com:9443https://www.amazontrust.com/repository/AmazonRootCA1.pemapp.eu2.coralogix.com
sg.comhttps://es-api.coralogixsg.com:9443https://www.amazontrust.com/repository/AmazonRootCA1.pemapp.coralogixsg.com

Set up a G Suite Service Account

Follow the official G Suite tutorial for setting up a service account.

Grant access to the Admin SDK API

Follow the official G Suite tutorial for granting access to the Admin API.

Delegate domain-wide authority to your service account

  • Go to your G Suite domain’s Admin console.
  • Go to Main menu > Security > API controls.
  • Under  the Domain-wide delegation pane, select Manage Domain Wide Delegation.
  • Click Add new, and fill in the details:
    • Client ID – Enter the service account’s Client ID – you can find it in the service account’s details under Unique ID. It is also found in the client_id field of the credentials file that was auto-downloaded when you created a new key for your service account.
    • OAuth Scopes – Enter https://www.googleapis.com/auth/admin.reports.audit.readonly
    • Click Authorize to confirm your changes.

Once done with the configuration of G Suite, now we should start with filebeat.

Filebeat Setup

Please follow the link for Coralogix filebeat setup (Make sure to use the latest version)

Please make sure to have at least version 7.12 and up for filebeat.

Filebeat G Suite Configuration

ignore_older: 3h
filebeat.modules:
- module: gsuite
  saml:
    enabled: true
    var.jwt_file: "path to the credentials file"
    var.delegated_account: "email address of the admin G Suite user"
  user_accounts:
    enabled: true
    var.jwt_file: "path to the credentials file"
    var.delegated_account: "email address of the admin G Suite user"
  login:
    enabled: true
    var.jwt_file: "path to the credentials file"
    var.delegated_account: "email address of the admin G Suite user"
  admin:
    enabled: true
    var.jwt_file: "path to the credentials file"
    var.delegated_account: "email address of the admin G Suite user"
  drive:
    enabled: true
    var.jwt_file: "path to the credentials file"
    var.delegated_account: "email address of the admin G Suite user"
  groups:
    enabled: true
    var.jwt_file: "path to the credentials file"
    var.delegated_account: "email address of the admin G Suite user"

fields_under_root: true
fields:
  PRIVATE_KEY: "your_company_private_key"
  COMPANY_ID: your_company_Id
  APP_NAME: "App_name for example google"
  SUB_SYSTEM: "Sub_system_name"

processors:
- drop_fields:
    fields:
    - event.original
    ignore_missing: true

logging:
  level: debug
  to_files: true
  files:
  path: /var/log/filebeat
  name: filebeat.log
  keepfiles: 10
  permissions: 0644

output.logstash:
  enabled: true
  hosts: ["logstashserver.Cluster URL:5015"] 
  tls.certificate_authorities: ["/etc/filebeat/ssl/coralogix.crt"]
  ssl.certificate_authorities: ["/etc/filebeat/ssl/coralogix.crt"] 

After applying the changes. Start your filebeat service.

SAMLView users’ successful and failed sign-ins to SAML applications.
User AccountsAudit actions carried out by users on their own accounts including password changes, account recovery details and 2-Step Verification enrollment.
LoginTrack user sign-in activity to your domain.
AdminView administrator activity performed within the Google Admin console.
DriveRecord user activity within Google Drive including content creation in such as Google Docs, as well as content created elsewhere that your users upload to Drive such as PDFs and Microsoft Word files.
GroupsTrack changes to groups, group memberships and group messages.

If you want to learn more about the G Suite plugin please see the link.

Please note that G Suite defaults to a 2-hour polling interval because Google reports can go from some minutes up to 3 days of delay.
For more details on this, you can read more here.

On this page