--- title: 'Log streaming: Coralogix' summary: null url: >- https://www.fastly.com/documentation/guides/integrations/logging-endpoints/log-management-and-observability/log-streaming-coralogix --- Fastly's [Real-Time Log Streaming](https://www.fastly.com/documentation/guides/integrations/streaming-logs/about-fastlys-realtime-log-streaming-features) feature can send log files to [Coralogix](https://coralogix.com/). Coralogix provides an analytics platform that allows you to detect abnormal behavior via [dynamic alerts](https://coralogix.com/docs/dynamic-alerts/), [ratio alerts](https://coralogix.com/docs/ratio-alerts/), [flow anomaly detection](https://coralogix.com/docs/what-is-coralogix-pattern-anomaly/), and threat discovery. > **NOTE:** > > Fastly does not provide direct support for third-party services. Read [Fastly's Terms of Service](https://www.fastly.com/terms) for more information. > > ## Prerequisites If you don't already have a Coralogix account, you'll need to register for one by following the [signup instructions](https://signup.coralogix.com/) on the Coralogix website. Once you've signed up, navigate to the **Send Your Logs** area in the **Settings** section of your Coralogix dashboard and make note of your unique private key. Coralogix uses this to associate data you send them with your account. You'll need it when you set up your endpoint with Fastly. > **HINT:** Consider reading [Coralogix's documentation on integrating with Fastly](https://coralogix.com/docs/fastly-logs-integration-through-https-streaming/). ## Adding Coralogix as a logging endpoint Follow these instructions to add Coralogix as a logging endpoint: ### Cdn Services 1. Review the information in our guide to [setting up remote log streaming](/guides/integrations/streaming-logs/setting-up-remote-log-streaming). 2. In the HTTPS area, click **Create endpoint**. 3. Fill out the **Create an HTTPS endpoint** fields as follows: - In the **Name** field, enter a human-readable name for the endpoint. - In the **Placement** area, select where the logging call should be placed in the generated VCL. Valid values are **Format Version Default** and **None**. Read our guide on [changing log placement](/guides/integrations/streaming-logs/changing-log-placement) for more information. - In the **Log format** field, replace the placeholder log format and make the appropriate changes as shown in our [log format and recommendations](https://www.fastly.com/documentation/guides/integrations/logging-endpoints/log-management-and-observability/log-streaming-coralogix#log-format-and-field-setting-recommendations) section below. - In the **URL** field, enter the [Coralogix REST API Singles Endpoint](https://coralogix.com/docs/coralogix-endpoints/#coralogix-rest-api-singles) for your [Coralogix Domain](https://coralogix.com/docs/coralogix-domain/). For example, you could use `https://ingress.coralogix.us/logs/v1/singles` for the `coralogix.us` domain. - In the **Maximum logs** field, leave the default value of `0`. - In the **Maximum bytes** field, enter `2000000`. - *(Optional)* From the **Processing region** menu, select a geographic region where logs are processed before being sent to the logging endpoint. Our guide on [regional log aggregation](/guides/integrations/streaming-logs/setting-up-regional-log-aggregation) provides more information. 4. Click **Advanced options** and fill out the fields as follows: - In the **Content type** field, enter `application/json`. - In the **Custom header name** field, enter `Authorization`. - In the **Custom header value** field, enter `Bearer `. - From the **Method** controls, select **POST**. - From the **JSON log entry format** controls, select **Array of JSON**. - Leave the **Select a log line format** controls set to the defaults. - Leave the remaining fields blank. 5. Click **Create** to create the new logging endpoint. 6. From the **Activate** menu, select **Activate on Production** to deploy your configuration changes. > **HINT:** For Coralogix, you do not need to configure anything to satisfy the [HTTPS Proof of domain ownership requirement](https://www.fastly.com/documentation/guides/integrations/logging-endpoints/protocol-based-and-self-hosted/log-streaming-https#prerequisites), and you can safely ignore any warning about it. ### Log format and field setting recommendations Use the following log format: ```plain { "timestamp":%{time.start.msec}V, "applicationName":"fastly", "subsystemName":"%{req.service_id}V", "severity": 3, "json": { "time": { "start":"%{begin:%Y-%m-%dT%H:%M:%S%Z}t", "end":"%{end:%Y-%m-%dT%H:%M:%S%Z}t", "elapsed":%D }, "cdn_server": { "ip_ipaddr":"%A", "code":"%{server.datacenter}V", "hostname":"%{server.hostname}V", "region_code":"%{server.region}V", "response_state":"%{fastly_info.state}V", "is_h2":%{if(fastly_info.is_h2, "true", "false")}V, "is_h2_push":%{if(fastly_info.h2.is_push, "true", "false")}V, "h2_stream_id":"%{fastly_info.h2.stream_id}V" }, "client": { "city_name":"%{client.geo.city.utf8}V", "country_code":"%{client.geo.country_code}V", "country_name":"%{client.geo.country_name}V", "continent_code":"%{client.geo.continent_code}V", "region":"%{client.geo.region}V", "ip_ipaddr":"%h", "name":"%{client.as.name}V", "number":"%{client.as.number}V", "connection_speed":"%{client.geo.conn_speed}V", "location_geopoint": { "lat":%{client.geo.latitude}V, "lon":%{client.geo.longitude}V } }, "response": { "status":%>s, "content_type":"%{Content-Type}o", "age":"%{Age}o", "cache_control":"%{Cache-Control}o", "expires":"%{Expires}o", "last_modified":"%{Last-Modified}o", "tsv":"%{TSV}o", "header_size":%{resp.header_bytes_written}V, "body_size":%B }, "request": { "host":"%{req.http.host}V", "is_ipv6":%{if(req.is_ipv6, "true", "false")}V, "backend":"%{req.backend}V", "service_id":"%{req.service_id}V", "url":"%{cstr_escape(req.url)}V", "url_ext":"%{req.url.ext}V", "header_size":%{req.header_bytes_read}V, "body_size":%{req.body_bytes_read}V, "method":"%m", "protocol":"%H", "referer":"%{Referer}i", "user_agent":"%{User-Agent}i", "accept_content":"%{Accept}i", "accept_language":"%{Accept-Language}i", "accept_encoding":"%{Accept-Encoding}i", "accept_charset":"%{Accept-Charset}i", "connection":"%{Connection}i", "dnt":"%{DNT}i", "forwarded":"%{Forwarded}i", "via":"%{Via}i", "cache_control":"%{Cache-Control}i", "x_requested_with":"%{X-Requested-With}i", "x_att_device_id":"%{X-ATT-Device-Id}i", "x_forwarded_for":"%{X-Forwarded-For}i" }, "socket": { "cwnd":%{client.socket.cwnd}V, "pace":%{client.socket.pace}V, "nexthop":"%{client.socket.nexthop}V", "tcpi_rcv_mss":%{client.socket.tcpi_rcv_mss}V, "tcpi_snd_mss":%{client.socket.tcpi_snd_mss}V, "tcpi_rtt":%{client.socket.tcpi_rtt}V, "tcpi_rttvar":%{client.socket.tcpi_rttvar}V, "tcpi_rcv_rtt":%{client.socket.tcpi_rcv_rtt}V, "tcpi_rcv_space":%{client.socket.tcpi_rcv_space}V, "tcpi_last_data_sent":%{client.socket.tcpi_last_data_sent}V, "tcpi_total_retrans":%{client.socket.tcpi_total_retrans}V, "tcpi_delta_retrans":%{client.socket.tcpi_delta_retrans}V, "ploss":%{client.socket.ploss}V } } } ``` The first five fields of the recommended format are required: - `timestamp`: Leave the format of this field unchanged. - `applicationName`: Enter the name of the application in this field. - `subsystemName`: Enter the name of the subsystem in this field. This is used to separate components. We use `req.service_id` in the example, which isn't particularly human readable. Use whatever subsystem name makes sense that helps you identify the subsystem. - `severity`: Specify the severity and apply it to all logs using the following choices: 1 (debug), 2 (verbose), 3 (info), 4 (warning), 5 (error), 6 (critical). This can be changed later using an extract rule as described below. - `json (object)`: Add or remove fields as necessary. Static fields can be added. Nested JSON formats are supported including any fields described in the [Fastly VCL reference](https://www.fastly.com/documentation/reference/vcl/variables/). Specifying a nested `response.status` field is a useful way to identify the status for servicing the request. Using [the Coralogix parsing rules](https://coralogix.com/docs/log-parsing-rules/), you can set a JSON Extract rule to use the status code value from the log to populate the severity field in the Coralogix interface. Specifically, you can automatically map an HTTP status code to a severity value. For example, status code `2xx` will set the Coralogix severity as **“INFO”** and status code `4xx` will set Coralogix severity as **“ERROR”**. In the Coralogix web interface, it will look like this: ![Creating a new Coralogix Rule](/img/create-new-coralogix-rule.png) ### Compute Services 1. Review the information in our guide to [setting up remote log streaming for Compute](/guides/integrations/streaming-logs/setting-up-remote-log-streaming-for-compute). Additionally, our developer documentation provides more [information about logging](/guides/integrations/non-fastly-services/developer-guide-logging/) with Compute code written in our [supported languages](/reference/compute/sdks/). 2. In the HTTPS area, click **Create endpoint**. 3. Fill out the **Create an HTTPS endpoint** fields as follows: - In the **Name** field, enter a human-readable name for the endpoint. - In the **Placement** area, select where the logging call should be placed in the generated VCL. Valid values are **Format Version Default** and **None**. Read our guide on [changing log placement](/guides/integrations/streaming-logs/changing-log-placement) for more information. - In the **URL** field, enter the [Coralogix REST API Singles Endpoint](https://coralogix.com/docs/coralogix-endpoints/#coralogix-rest-api-singles) for your [Coralogix Domain](https://coralogix.com/docs/coralogix-domain/). For example, you could use `https://ingress.coralogix.us/logs/v1/singles` for the `coralogix.us` domain. - In the **Maximum logs** field, leave the default of `0`. - In the **Maximum bytes** field, enter `2000000`. 4. Click **Advanced options** and fill out the fields as follows: - In the **Content type** field, enter `application/json`. - In the **Custom header name** field, enter `private_key`. - In the **Custom header value** field, enter your Coralogix private key. - From the **Method** controls, select **POST**. - From the **JSON log entry format** controls, select **Array of JSON**. - Leave the **Select a log line format** controls set to the defaults. - Leave the **TLS hostname**, **TLS CA certificate**, **TLS client certificate**, and **TLS client key** fields blank. 5. Click **Create** to create the new logging endpoint. 6. From the **Activate** menu, select **Activate on Production** to deploy your configuration changes. > **HINT:** For Coralogix, you do not need to configure anything explicitly to satisfy the [HTTPS Proof of domain ownership requirement](https://www.fastly.com/documentation/guides/integrations/logging-endpoints/protocol-based-and-self-hosted/log-streaming-https#prerequisites), and you can safely ignore any warning about it. ### Recommended log format Data sent to Coralogix must be serialized in a way [conforming to Coralogix's expectations](https://coralogix.com/docs/fastly-logs-integration-through-https-streaming/). If your logs are not formatted properly, attempts at processing your logs by your Coralogix endpoint may fail. Here's an example format string for sending data to Coralogix: ```json { "timestamp": 1653088964764, "applicationName": "fastly", "subsystemName": "wasm", "severity": 3, "json": { "message": "Request happened", "response": { "status":200 } } } ``` You can follow the general JSON structure above regardless of the chosen language for your Compute service. The following fields are required: - `timestamp`: The format of this field is in milliseconds. - `applicationName`: Enter the name of the application. - `subsystemName`: Enter the name of the subsystem. This field is used to separate components. Use whatever subsystem name makes sense that helps you identify the subsystem. - `severity`: The severity of the log. You can specify the severity to all logs using the following choices: 1 (debug), 2 (verbose), 3 (info), 4 (warning), 5 (error), 6 (critical). This can be changed later using an extraction rule as described in the field below. - `json (object)`: Used to specify additional log details as necessary. Nested JSON formats are supported. Specifying a nested `response.status` field is a useful way to identify the status for servicing the request. Using [the Coralogix parsing rules](https://coralogix.com/docs/log-parsing-rules/), you can set a JSON Extract rule to use the status code value from the log to populate the severity field in the Coralogix interface. Specifically, you can automatically map an HTTP status code to a severity value. For example, status code `2xx` will set the Coralogix severity as **“INFO”** and status code `4xx` will set Coralogix severity as **“ERROR”**. In the Coralogix web interface, it will look like this: ![Creating a new Coralogix Rule](/img/create-new-coralogix-rule.png) ## Configuring Coralogix dashboards and alerting Coralogix provides [tutorials](https://coralogix.com/blog/fastly-logs-insights/) for integrating their service with Fastly via dashboards and alerting. This includes examples of [data dashboards](https://github.com/coralogix-resources/Dashboards/blob/master/Fastly%20Kibana%20Dashboards/export.ndjson) created using Fastly data, including one for a general service overview, a visitor summary, and quality of service. Their tutorials also describe how to set up [user-defined alerts](https://coralogix.com/docs/coralogix-user-defined-alerts/) for situations like no logs being received from Fastly, outages at your origin, elevated error ratios and cache misses, unusual or suspicious requests of various types, as well as potential website defacement attempts. ## Related content - [API reference: HTTPS log streaming](https://www.fastly.com/documentation/reference/api/logging/https/) - [CLI reference: HTTPS log streaming](https://www.fastly.com/documentation/reference/cli/logging/https/)