---
title: 'Log streaming: Google Cloud Storage'
summary: null
url: >-
  https://www.fastly.com/documentation/guides/integrations/logging-endpoints/object-and-cloud-storage/log-streaming-google-cloud-storage
---

Fastly's [Real-Time Log Streaming](https://www.fastly.com/documentation/guides/integrations/streaming-logs/about-fastlys-realtime-log-streaming-features) feature can send log files to [Google Cloud Storage](https://cloud.google.com/storage/) (GCS). GCS is an online file storage service used for storing and accessing data on Google's infrastructure. One advantage of using GCS is that you can use [Google BigQuery](https://cloud.google.com/bigquery/) to analyze the log files.

> **NOTE:** 
>
> Fastly does not provide direct support for third-party services. Read [Fastly's Terms of Service](https://www.fastly.com/terms) for more information.
>
>

## Prerequisites

Before adding GCS as a logging endpoint for Fastly services, you will need to:

- Register for a GCS account.
- Create a bucket and service account on Google's website.
- Use Google IAM service account impersonation or key-based access to configure the access method that Fastly uses to send logs.
- Enable the Google Cloud Storage JSON API.

### Creating a GCS bucket

You can create a new GCS bucket to hold the logs, or you can use an existing bucket. Be sure to note the name of the bucket as you will need it later. To learn how to create a GCS bucket, refer to Google's guide on [creating a bucket](https://cloud.google.com/storage/docs/quickstart-console#create_a_bucket).

### Creating a service account

Google Cloud Platform (GCP) uses [service accounts](https://cloud.google.com/iam/docs/service-account-overview) for third-party application authentication and role-based access to Google Cloud resources. To create a new service account, follow the instructions in the [Google Cloud documentation](https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating).

When creating the service account for logging to GCS, the service account must be assigned the `Storage Object Creator` role for the bucket you created above. Refer to the [Storage Object Roles](https://cloud.google.com/iam/docs/understanding-roles#storage.objectCreator) for details.

![Create service account panel](/img/service_account_gcs.png)

### Configure access for the service account

Fastly requires access to write logs to your GCS bucket. You can use either Google IAM service account impersonation or key-based access to allow Fastly to send logs to your GCS bucket

#### Configuring Google IAM service account impersonation

If you elect to use Google IAM service account impersonation to avoid storing keys with Fastly, you'll need to specify a service account name and a project ID when adding GCS as your logging endpoint. The service account ID comes before the `@` in the service account email and the project ID immediately after. For example, if your service account email is `my-name@projectid.iam.gserviceaccount.com`, the service account ID is `my-name` and the project ID is `projectid`. Our guide to [creating an Google IAM role](https://www.fastly.com/documentation/guides/integrations/streaming-logs/configuring-google-iam-service-account-impersonation-for-fastly-logging) provides specific details on configuring this feature.

#### Configuring key-based access

If you elect to use key-based access, refer to Google's guide on [generating a service account credential](https://cloud.google.com/iam/docs/keys-create-delete#creating_service_account_keys).

When configuring key-based access, you must select the JSON format. Upon saving the configuration, a JSON file will be downloaded to your computer, containing the credentials for the GCS service account you just created. Open the file with a text editor and make a note of the `private_key` and `client_email`.

### Enabling the Google Cloud Storage JSON API

To ensure the Fastly logs are sent to your GCS bucket, you need to enable the Google Cloud Storage JSON API. For more information, refer to Google's instructions for [activating the API](https://support.google.com/googleapi/answer/6158841).

## Adding GCS as a logging endpoint

Follow these instructions to add GCS as a logging endpoint. As part of configuration, you can elect to configure Google IAM role-based service account impersonation to avoid storing secrets. Read our guide on [creating a Google IAM role](https://www.fastly.com/documentation/guides/integrations/streaming-logs/configuring-google-iam-service-account-impersonation-for-fastly-logging) for more information on this feature.

### Cdn Services

1.   Review the information in our guide to [setting up remote log streaming](/guides/integrations/streaming-logs/setting-up-remote-log-streaming).

2. In the Google Cloud Services area, click **Create endpoint**.
3. Fill out the **Create a Google Cloud Storage (GCS) endpoint** fields as follows:
   -   In the **Name** field, enter a human-readable name for the endpoint.

   -   In the **Placement** area, select where the logging call should be placed in the generated VCL. Valid values are **Format Version Default** and **None**. Read our guide on [changing log placement](/guides/integrations/streaming-logs/changing-log-placement) for more information.

   -   In the **Log format** field, optionally enter an Apache-style string or VCL variables to use for log formatting. Consult the [example format section](#example-format) for details.

   -   *(Optional)* In the **Timestamp format** field, enter a timestamp format for log files. The default is an `strftime` compatible string. Our guide on [changing where log files are written](/guides/integrations/streaming-logs/changing-where-log-files-are-written) provides more information.

   - In the **Email** field, enter the `client_email` address listed in the JSON file associated with the service account you created on Google's website.
   - In the **Bucket name** field, enter the name of the GCS bucket in which to store the logs.
   - In the **Access Method** area, select how Fastly will access Google resources for purposes of log delivery. Valid values are **User Credentials** and **IAM Role**.
   - If you selected **User Credentials**, fill out the following fields:
     - In the **Email** field, enter the `client_email` address listed in the JSON file associated with the service account you created on Google's website.
     - In the **Secret key** field, enter the [`private_key`](https://www.fastly.com/documentation/guides/integrations/logging-endpoints/object-and-cloud-storage/log-streaming-google-cloud-storage#configuring-key-based-access) value listed in the JSON file associated with the service account you created on Google's website. We strip out the JSON newline escape characters for you so don't worry about removing them.
   - If you selected **IAM Role**, fill out the following field:
     - In the **Service Account Name** field, enter the service account ID which was created for you from the service account name you entered when configuring Google IAM service account impersonation. The service account ID comes before the `@` in the service account email. For example, if your service account email is `my-name@projectid.iam.gserviceaccount.com`, the service account ID is `my-name`.
     - In the **Project ID** field, enter the ID of your Google Cloud Platform project.
   -   *(Optional)* In the **PGP public key** field, enter a PGP public key that Fastly will use to encrypt your log files before writing them to disk. You will only be able to read the contents by decrypting them with your private key. The PGP key should be in [PEM (Privacy-Enhanced Mail) format](https://en.wikipedia.org/wiki/Privacy-enhanced_Electronic_Mail). Read our guide on [log encryption](/guides/integrations/streaming-logs/encrypting-logs) for more information.

   -   *(Optional)* In the **Period** field, enter an interval (in seconds) to control how frequently your log files are rotated. Rotation entails the finalization of one file object and the start of a new one, never removing any previously created file object. This value defaults to `3600` seconds.

   -   *(Optional)* From the **Processing region** menu, select a geographic region where logs are processed before being sent to the logging endpoint. Our guide on [regional log aggregation](/guides/integrations/streaming-logs/setting-up-regional-log-aggregation) provides more information.

4. Click **Advanced options** and fill out the fields as follows:
   -   *(Optional)* In the **Path** field, enter the path within the bucket to store the files. The path ends with a trailing slash. If this field is left empty, the files will be saved in the bucket's root path. Our guide on [changing where log files are written](/guides/integrations/streaming-logs/changing-where-log-files-are-written) provides more information.

   -   In the **Select a log line format** area, select the log line format for your log messages. Our guide on [changing log line formats](/guides/integrations/streaming-logs/changing-log-line-formats) provides more information.

   -   *(Optional)* In the **Compression** field, select the compression format you want applied to the log files. Our guide on [changing log compression options](/guides/integrations/streaming-logs/changing-log-compression-options) provides more information.

5.   Click **Create** to create the new logging endpoint.

6.   From the **Activate** menu, select **Activate on Production** to deploy your configuration changes.

### Example format

The following is an example format string for sending data to GCS. Our discussion of [format strings](https://www.fastly.com/documentation/guides/integrations/streaming-logs/custom-log-formats) provides more information.

```plaintext
{
  "timestamp": "%{strftime(\{"%Y-%m-%dT%H:%M:%S%z"\}, time.start)}V",
  "client_ip": "%{req.http.Fastly-Client-IP}V",
  "geo_country": "%{client.geo.country_name}V",
  "geo_city": "%{client.geo.city}V",
  "host": "%{if(req.http.Fastly-Orig-Host, req.http.Fastly-Orig-Host, req.http.Host)}V",
  "url": "%{json.escape(req.url)}V",
  "request_method": "%{json.escape(req.method)}V",
  "request_protocol": "%{json.escape(req.proto)}V",
  "request_referer": "%{json.escape(req.http.referer)}V",
  "request_user_agent": "%{json.escape(req.http.User-Agent)}V",
  "response_state": "%{json.escape(fastly_info.state)}V",
  "response_status": %{resp.status}V,
  "response_reason": %{if(resp.response, "%22"+json.escape(resp.response)+"%22", "null")}V,
  "response_body_size": %{resp.body_bytes_written}V,
  "fastly_server": "%{json.escape(server.identity)}V",
  "fastly_is_edge": %{if(fastly.ff.visits_this_service == 0, "true", "false")}V
}
```

### Compute Services

1.   Review the information in our guide to [setting up remote log streaming for Compute](/guides/integrations/streaming-logs/setting-up-remote-log-streaming-for-compute). Additionally, our developer documentation provides more [information about logging](/guides/integrations/non-fastly-services/developer-guide-logging/) with Compute code written in our [supported languages](/reference/compute/sdks/).

2. In the Google Cloud Services area, click **Create endpoint**.
3. Fill out the **Create a Google Cloud Storage (GCS) endpoint** fields as follows:
   -   In the **Name** field, enter the endpoint name you specified in your Compute code. For example, in our [Rust code example](/guides/compute/developer-guides/rust/#logging), the name is `my_endpoint_name`.

   -   *(Optional)* In the **Timestamp format** field, enter a timestamp format for log files. The default is an `strftime` compatible string. Our guide on [changing where log files are written](/guides/integrations/streaming-logs/changing-where-log-files-are-written) provides more information.

   - In the **Bucket name** field, enter the name of the GCS bucket in which to store the logs.
   - In the **Access Method** area, select how Fastly will access Google resources for purposes of log delivery. Valid values are **User Credentials** and **IAM Role**.
   - If you selected **User Credentials**, fill out the following fields:
     - In the **Email** field, enter the `client_email` address listed in the JSON file associated with the service account you created on Google's website.
     - In the **Secret key** field, enter the `private_key` value listed in the JSON file associated with the service account you created on Google's website. We strip out the JSON newline escape characters for you so don't worry about removing them.
   - If you selected **IAM Role**, fill out the following field:
     - In the **Service Account Name** field, enter the service account ID which was created for you from the service account name you entered when configuring Google IAM service account impersonation. The service account ID comes before the `@` in the service account email. For example, if your service account email is `my-name@projectid.iam.gserviceaccount.com`, the service account ID is `my-name`.
     - In the **Project ID** field, enter the ID of your Google Cloud Platform project.
   -   *(Optional)* In the **PGP public key** field, enter a PGP public key that Fastly will use to encrypt your log files before writing them to disk. You will only be able to read the contents by decrypting them with your private key. The PGP key should be in [PEM (Privacy-Enhanced Mail) format](https://en.wikipedia.org/wiki/Privacy-enhanced_Electronic_Mail). Read our guide on [log encryption](/guides/integrations/streaming-logs/encrypting-logs) for more information.

   -   *(Optional)* In the **Period** field, enter an interval (in seconds) to control how frequently your log files are rotated. Rotation entails the finalization of one file object and the start of a new one, never removing any previously created file object. This value defaults to `3600` seconds.

4. Click **Advanced options** and fill out the fields as follows:
   -   *(Optional)* In the **Path** field, enter the path within the bucket to store the files. The path ends with a trailing slash. If this field is left empty, the files will be saved in the bucket's root path. Our guide on [changing where log files are written](/guides/integrations/streaming-logs/changing-where-log-files-are-written) provides more information.

   -   In the **Select a log line format** area, select the log line format for your log messages. Our guide on [changing log line formats](/guides/integrations/streaming-logs/changing-log-line-formats) provides more information.

   -   *(Optional)* In the **Compression** field, select the compression format you want applied to the log files. Our guide on [changing log compression options](/guides/integrations/streaming-logs/changing-log-compression-options) provides more information.

5.   Click **Create** to create the new logging endpoint.

6.   From the **Activate** menu, select **Activate on Production** to deploy your configuration changes.

### Recommended log format

Log messages can take on any format you choose as long as they can be processed from GCS.

## Related content

- [Using GCS as an origin](https://www.fastly.com/documentation/guides/integrations/non-fastly-services/google-cloud-storage)
- [API reference: GCS log streaming](https://www.fastly.com/documentation/reference/api/logging/gcs/)
- [CLI reference: GCS log streaming](https://www.fastly.com/documentation/reference/cli/logging/gcs/)