Cloudflare

Cloudflare Enterprise customers have access to Logpush service which allows you to forward logs to cloud service providers like AWS. In this tutorial, you will find all steps to send logs to Coralogix.

Send logs directly

Prerequisites

• A Private Key – You can find your key in your Coralogix account under ‘Data Flow’ -> ‘API Keys’ -> ‘Send your logs’.
• Cloudflare ZoneID / AccountID – Find the relevant id in the cloudflare dashboard under ‘Websites’ -> ‘<Your-site>’ -> ‘API (scroll down)’.

Configuration

To start sending data directly to coralogix we first need to pick the type of logs (data-sets) and fields that will be sent, Cloudflare Data sets.

To create the logpush job we need to call the API using the terminal.

Please note that you need to specify the correct API endpoint for your cluster to match the last few characters of your Team’s name (see “destination_conf” field below after this table):

curl -s https://api.cloudflare.com/client/v4/zones/<zone_id>/logpush/jobs -X POST \
-H "Content-Type:application/json" \
-d '{
  "name": "logpush-to-coralogix",
  "logpull_options": "fields=RayID,EdgeStartTimestamp&timestamps=unixnano",
  "destination_conf": "https://cdn-ingress.coralogix.com/api/v1/cloudflare/logs?header_Authorization=Bearer%20<Send_your_data_key>&header_timestamp-format=UnixNano&header_dataset=HTTPRequests",
  "max_upload_bytes": 5000000,
  "max_upload_records": 1000,
  "dataset": "http_requests",
  "enabled": true,
  "frequency": "low"
}' \
-H "X-Auth-Email: <Your_Auth_Email>" \
-H "X-Auth-Key: <Your_API_Key>"

Notes:

• Replace <zone_id> with your site zone id, <Send_your_data_key> with your coralogix private key and ‘X-Auth-Email’ ‘X-Auth-Key’ with your cloudflare credentials.
• To change the dataset sent change the ‘dataset’ field and `header_dataset` inside the ‘destination_conf’ field using the table below.
• To change the fields sent change the ‘logpull_options’ field, each dataset has different fields.
• Each dataset has a different ‘timestamp’ key, providing it in the ‘logpull_options’ field is required, also using the unixnano format for the timestamp is also required.
• To configure Account-scoped datasets use ‘https://api.cloudflare.com/client/v4/accounts/<Account_ID>/logpush/jobs’ .

After creating the logpush you can view it in the terminal

curl -s https://api.cloudflare.com/client/v4/zones/<Zone_ID>/logpush/jobs -X GET \
-H "X-Auth-Email: <Your_Auth_Email>" \
-H "X-Auth-Key: <Your_Auth-Key>"

Or in the dashboard itself under ‘Websites’ -> ‘<Your-site>’ -> ‘Analytics’ -> ‘Logs’.

Send logs via S3 bucket

Prerequisites

AWS S3 bucket – Follow the tutorial to send logs from the S3 bucket to Coralogix: https://coralogix.com/integrations/data-collection-s3/

Configuration

To enable the Clouflare Logpush service:

1. Log in to the Cloudflare dashboard
2. Ensure the Enterprise domain you want to use with Logpush is selected
3. Select the Analytics app in the top menu
4. Select the Logs section in the secondary menu
   
5. Select Connect a service. A modal window opens where you will need to complete several steps.
   
6. Under Select service, pick your Amazon S3 and click Next.
   
7. Enter or select the following:
   • Bucket path
   • Daily subfolders
   • Bucket region
     
8. Copy the policy from the field below “Grant Cloudflare access to upload files to your bucket”
9. In new window go to S3 > your bucket name > Permissions > Bucket Policy
10. Click on the Edit button at the Bucket Policy.
11. Paste the copied policy and save changes.
12. In the Cloudflare dashboard, click Validate access.
13. Follow the on-screen instructions to enter the Ownership token (included in a file Cloudflare sends to your Amazon S3 bucket).
    
    Open a file with token on your Amazon S3 bucket and paste the token into Ownership token field:
    
14. Click the “Prove ownership” button.
15. Next in the Customize log screen, select the data set in which you’re interested (currently HTTP requests or Spectrum events).
    
    You can keep the default fields to include in your log or make changes. You can add or remove fields at a later time by modifying your settings in Logs > Logpush (select the wrench icon).
    
16. To finish enabling Logpush, click Save and Start Pushing

Once connected, Cloudflare lists the provider you just configured under Logs > Logpush. This is where you can make changes or remove the provider.

If all steps were executed properly, you should see files in your S3 bucket and also in Coralogix.

Contact us on via chat if you would like us to add Cloudflare Dashboards.