Our next-gen architecture is built to help you make sense of your ever-growing data Watch a 4-min demo video!

Back to All Integrations

Fastly Logs via HTTPS Streaming Fastly Logs via HTTPS Streaming

Last Updated: Sep. 12, 2022

Fastly’s Real-Time Log Streaming feature provides the ability to send Fastly logs to any HTTPS endpoint. This will allow you to optimize your Fastly services.

Fastly supports real-time log streaming of data that passes through it. Fastly supports a number of protocols that allow you to stream logs to a variety of locations, including third-party services, for storage and analysis. The following tutorial will cover how to set up your HTTPS endpoint to Coralogix from your Fastly account. For more details, on Log streaming through HTTPS, you can find the official documentation here: https://docs.fastly.com/en/guides/log-streaming-https.

NOTE: The logging endpoint for HTTPS streaming is disabled by default. To enable the endpoint for your account, you will need to contact [email protected] to request it.

Configuration

https://docs.fastly.com/en/guides/log-streaming-coralogix

After logging into your Fastly account, you will see a listing of all configured services. Click on Configure at the top.

coralogix fastly integration, list of all configured services

Choose which service you wish to provision. You can switch between existing services or create a new one. Click on “View active configuration” to set a new endpoint.

coralogix fastly integration choose service to provision

Click on “Clone” to access editing.

coralogix fastly integration clone button

On the left panel, click on “Logging”.

coralogix fastly integration click logging

Click on “Create Endpoint”.

coralogix fastly integration create endpoint

Scroll down the page and choose HTTPS, and click on the associated “Create Endpoint” button.

Next you will configure your endpoint.

coralogix fastly integration endpoint configuration

Here, enter the relevant information for each of the requested fields.

  • Name: Name your endpoint
  • Log format (Recommended): (enter the following data)
{
"timestamp":%{time.start.msec}V,
"applicationName":"fastly",
"subsystemName":"coralogix.com",
"severity": 3,
"json": {
"time": {
"start":"%{begin:%Y-%m-%dT%H:%M:%S%Z}t",
"end":"%{end:%Y-%m-%dT%H:%M:%S%Z}t",
"elapsed":%D
},
"cdn_server": {
"ip_ipaddr":"%A",
"code":"%{server.datacenter}V",
"hostname":"%{server.hostname}V",
"region_code":"%{server.region}V",
"is_cacheable":%{if(fastly_info.state ~"^(HIT|MISS)$", "true", "false")}V,
"cache_status":"%{regsub(fastly_info.state, "^(HIT-(SYNTH)|(HITPASS|HIT|MISS|PASS|ERROR|PIPE)).*", "\\2\\3")}V",
"is_h2":%{if(fastly_info.is_h2, "true", "false")}V,
"is_h2_push":%{if(fastly_info.h2.is_push, "true", "false")}V,
"h2_stream_id":"%{fastly_info.h2.stream_id}V"
},
"client": {
"city_name":"%{client.geo.city.utf8}V",
"country_code":"%{client.geo.country_code}V",
"country_name":"%{client.geo.country_name}V",
"continent_code":"%{client.geo.continent_code}V",
"region":"%{client.geo.region}V",
"ip_ipaddr":"%h",
"name":"%{client.as.name}V",
"number":"%{client.as.number}V",
"connection_speed":"%{client.geo.conn_speed}V",
"location_geopoint": {
"lat":%{client.geo.latitude}V,
"lon":%{client.geo.longitude}V
}
},
"response": {
"status":%>s,
"content_type":"%{Content-Type}o",
"age":"%{Age}o",
"cache_control":"%{Cache-Control}o",
"expires":"%{Expires}o",
"last_modified":"%{Last-Modified}o",
"tsv":"%{TSV}o",
"header_size":%{resp.header_bytes_written}V,
"body_size":%B
},
"request": {
"host":"%{req.http.host}V",
"is_ipv6":%{if(req.is_ipv6, "true", "false")}V,
"backend":"%{req.backend}V",
"service_id":"%{req.service_id}V",
"url":"%{cstr_escape(req.url)}V",
"url_ext":"%{req.url.ext}V",
"header_size":%{req.header_bytes_read}V,
"body_size":%{req.body_bytes_read}V,
"method":"%m",
"protocol":"%H",
"referer":"%{Referer}i",
"user_agent":"%{User-Agent}i",
"accept_content":"%{Accept}i",
"accept_language":"%{Accept-Language}i",
"accept_encoding":"%{Accept-Encoding}i",
"accept_charset":"%{Accept-Charset}i",
"connection":"%{Connection}i",
"dnt":"%{DNT}i",
"forwarded":"%{Forwarded}i",
"via":"%{Via}i",
"cache_control":"%{Cache-Control}i",
"x_requested_with":"%{X-Requested-With}i",
"x_att_device_id":"%{X-ATT-Device-Id}i",
"x_forwarded_for":"%{X-Forwarded-For}i"
},
"socket": {
"cwnd":%{client.socket.cwnd}V,
"pace":%{client.socket.pace}V,
"nexthop":"%{client.socket.nexthop}V",
"tcpi_rcv_mss":%{client.socket.tcpi_rcv_mss}V,
"tcpi_snd_mss":%{client.socket.tcpi_snd_mss}V,
"tcpi_rtt":%{client.socket.tcpi_rtt}V,
"tcpi_rttvar":%{client.socket.tcpi_rttvar}V,
"tcpi_rcv_rtt":%{client.socket.tcpi_rcv_rtt}V,
"tcpi_rcv_space":%{client.socket.tcpi_rcv_space}V,
"tcpi_last_data_sent":%{client.socket.tcpi_last_data_sent}V,
"tcpi_total_retrans":%{client.socket.tcpi_total_retrans}V,
"tcpi_delta_retrans":%{client.socket.tcpi_delta_retrans}V,
"ploss":%{client.socket.ploss}V
}
}
}

The first five fields are mandatory:

  • Timestamp – The format should not change.
  • applicationName – Enter the name of the application.
  • subsystemName – Enter the name of the subsystem. 
  • Severity – Apply the severity to all logs, using the following choices: 1-debug, 2-verbose, 3-info, 4-warning, 5-error, 6-critical. This can be changed later using an extract rule, as described below.
  • JSON (object) – Fields can be added or removed. Static fields can be added.  Nested JSON formats are supported including any fields described in the Fastly VCL reference: https://docs.fastly.com/vcl/variables/.

The response.status field sends the request status. This is a recommended field. Then, using the Coralogix parsing rules, you may set a JSON extract rule to extract the status code from the request into Coralogix severity. Define the severity to automatically determine the importance of the type of log. Note: in Coralogix we automatically map HTTP status codes into a severity tag as appropriate. For example, status code 200 will set the Coralogix severity as “INFO”, status code 4xx will set Coralogix severity as “ERROR”, etc.

Coralogix rule configuration

This is how the rule will look in Coralogix.

coralogix fastly integration json extract rule
ClusterAPI Endpoint
.coralogix.com https://api.coralogix.com/logs/rest/singles
.app.coralogix.ushttps://api.coralogix.us/logs/rest/singles
.app.coralogix.inhttps://api.app.coralogix.in/logs/rest/singles
.app.eu2.coralogix.comhttps://api.eu2.coralogix.com/logs/rest/singles
.app.coralogixsg.comhttps://api.coralogixsg.com/logs/rest/singles
  • Maximum logs: Leave as default
  • Maximum bytes: 2000000

Under advanced options, enter the following data.

coralogix fastly integration advanced options

Set the options as:

Content type – application/json

Custom header name – private_key

Custom header value – YOUR CORALOGIX PRIVATE KEY (your unique private key, which can be found under settings → send your logs, at the top left of the screen)

Method – Choose “POST” (default)

JSON log entry format – Choose “Array of JSON”

Select a log line format – Choose “Blank” (default)

Placement – Choose “Format Version Default” (default)

Leave the rest of the options empty and click on “Create” (or “Update” if you are updating an endpoint). To finish setting up your service, click on the green “Activate” button on the right to activate the new/updated endpoint.

After activation, under “Domains” on the left panel,  click on “Test domain” to verify your configuration. A test log should appear in your Coralogix account.

coralogix fastly integration test domain

This is how it should appear in Coralogix.

coralogix fastly integration log example in coralogix

If you see your test log in Coralogix, it means that you have successfully configured the integration.

To get all the Coralogix dashboards and alerts, contact our support on our website/in-app chat. We reply in under 2 minutes!

Still, have questions? Check our website and use the in-app chat for quick help from a Coralogix professional.

On this page