Our next-gen architecture is built to help you make sense of your ever-growing data Watch a 4-min demo video!

Back to All Integrations

Cloudflare Cloudflare

Last Updated: Feb. 09, 2023

Cloudflare Enterprise customers have access to Logpush service, which allows you to forward logs to cloud service providers like AWS. This tutorial demonstrates how to send your logs to Coralogix.

Send logs Directly

Prerequisites

  • Coralogix private key
  • Cloudflare ZoneID / AccountID – Find the relevant id in the cloudflare dashboard under ‘Websites’ -> ‘<Your-site>’ -> ‘API (scroll down)’.

Configuration

To start sending data directly to coralogix we first need to pick the type of logs (data-sets) and fields that will be sent, Cloudflare Data sets.

To create the logpush job we need to call the API using the terminal.

Please note that you need to specify the correct API endpoint for your cluster to match the last few characters of your Team’s name (see “destination_conf” field below after this table):

ClusterAPI Endpoint
.coralogix.comhttps://cdn-ingress.coralogix.com/api/v1/cloudflare/logs
.app.coralogix.ushttps://cdn-ingress.coralogix.us/api/v1/cloudflare/logs
.app.coralogix.inhttps://cdn-ingress.app.coralogix.in/api/v1/cloudflare/logs
.app.eu2.coralogix.comhttps://cdn-ingress.eu2.coralogix.com/api/v1/cloudflare/logs
.app.coralogixsg.comhttps://cdn-ingress.coralogixsg.com/api/v1/cloudflare/logs

curl -s https://api.cloudflare.com/client/v4/zones/<zone_id>/logpush/jobs -X POST \
-H "Content-Type:application/json" \
-d '{
  "name": "logpush-to-coralogix",
  "logpull_options": "fields=RayID,EdgeStartTimestamp&timestamps=unixnano",
  "destination_conf": "https://cdn-ingress.coralogix.com/api/v1/cloudflare/logs?header_Authorization=Bearer%20<Send_your_data_key>&header_timestamp-format=UnixNano&header_dataset=HTTPRequests",
  "max_upload_bytes": 5000000,
  "max_upload_records": 1000,
  "dataset": "http_requests",
  "enabled": true,
  "frequency": "low"
}' \
-H "X-Auth-Email: <Your_Auth_Email>" \
-H "X-Auth-Key: <Your_API_Key>"

Notes:

  • Replace <zone_id> with your site zone id, <Send_your_data_key> with your coralogix private key and ‘X-Auth-Email’ ‘X-Auth-Key’ with your cloudflare credentials.
  • To change the dataset sent change the ‘dataset’ field and `header_dataset` inside the ‘destination_conf’ field using the table below.
  • To change the fields sent change the ‘logpull_options’ field, each dataset has different fields.
  • Each dataset has a different ‘timestamp’ key, providing it in the ‘logpull_options’ field is required, also using the unixnano format for the timestamp is also required.
  • To configure Account-scoped datasets use ‘https://api.cloudflare.com/client/v4/accounts/<Account_ID>/logpush/jobs’.
  • By default, the integration will set application_name as Cloudflare, and subsystem_name as the data set name. To overwrite these parameters, add the following:
    • header_CX-Application-Name – application name override
    • header_CX-Subsystem-Name – subsystem name override
Dataset nameHeader name‘Timestamp’ keyScope
dns_logsDNSLogsTimestampZone
firewall_eventsFirewallEventsDatetimeZone
http_requestsHTTPRequestsEdgeStartTimestampZone
nel_reportsNELReportsTimestampZone
spectrum_eventsSpectrumEventsTimestampZone
audit_logsAuditLogsWhenAccount
gateway_dnsGatewayDNSDatetimeAccount
gateway_httpGatewayHTTPDatetimeAccount
gateway_networkGatewayNetworkDatetimeAccount
network_analytics_logsNetworkAnalyticsLogsDatetimeAccount
access_requestsAccessRequestsCreatedAtAccount
casb_findingsCASBFindingsDetectedTimestampAccount
dns_firewall_logsDNSFirewallLogsTimestampAccount
magic_ids_detectionsMagicIDSDetectionsTimestampAccount
workers_trace_eventsWorkersTraceEventsEventTimestampMsAccount

After creating the logpush, view it in the terminal.

curl -s https://api.cloudflare.com/client/v4/zones/<Zone_ID>/logpush/jobs -X GET \
-H "X-Auth-Email: <Your_Auth_Email>" \
-H "X-Auth-Key: <Your_Auth-Key>"

Or in the dashboard itself under ‘Websites’ -> ‘<Your-site>’ -> ‘Analytics’ -> ‘Logs’.

Send Logs via S3 Bucket

Prerequisites

AWS S3 bucket – Follow the tutorial to send logs from the S3 bucket to Coralogix: https://coralogix.com/integrations/data-collection-s3/

Configuration

To enable the Clouflare Logpush service:

  1. Log in to the Cloudflare dashboard
  2. Ensure the Enterprise domain you want to use with Logpush is selected
  3. Select the Analytics app in the top menu
  4. Select the Logs section in the secondary menu
  5. Select Connect a service. A modal window opens where you will need to complete several steps.
  6. Under Select service, pick your Amazon S3 and click Next.
  7. Enter or select the following:
    • Bucket path
    • Daily subfolders
    • Bucket region
  8. Copy the policy from the field below “Grant Cloudflare access to upload files to your bucket”
  9. In new window go to S3 > your bucket name > Permissions > Bucket Policy
  10. Click on the Edit button at the Bucket Policy.
  11. Paste the copied policy and save changes.
  12. In the Cloudflare dashboard, click Validate access.
  13. Follow the on-screen instructions to enter the Ownership token (included in a file Cloudflare sends to your Amazon S3 bucket).

    Open a file with token on your Amazon S3 bucket and paste the token into Ownership token field:
  14. Click the “Prove ownership” button.
  15. Next in the Customize log screen, select the data set in which you’re interested (currently HTTP requests or Spectrum events).

    You can keep the default fields to include in your log or make changes. You can add or remove fields at a later time by modifying your settings in Logs > Logpush (select the wrench icon).
  16. To finish enabling Logpush, click Save and Start Pushing

Once connected, Cloudflare lists the provider you just configured under Logs > Logpush. This is where you can make changes or remove the provider.

If all steps were executed properly, you should see files in your S3 bucket and also in Coralogix.

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

On this page