---
title: 'Log streaming: Google Cloud Pub/Sub'
summary: null
url: >-
  https://www.fastly.com/documentation/guides/integrations/logging-endpoints/data-streaming-and-message-queues/log-streaming-google-cloud-pubsub
---

Fastly's [Real-Time Log Streaming](https://www.fastly.com/documentation/guides/integrations/streaming-logs/about-fastlys-realtime-log-streaming-features) feature can send log files to [Cloud Pub/Sub](https://cloud.google.com/pubsub/), Google's global messaging and event data ingestion product.

> **NOTE:** 
>
> Fastly does not provide direct support for third-party services. Read [Fastly's Terms of Service](https://www.fastly.com/terms) for more information.
>
>

## Prerequisites

Before adding Cloud Pub/Sub as a logging endpoint for Fastly services, you will need to register for a [Google Cloud Platform](https://cloud.google.com/) (GCP) account and then:

- Create a [service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts) on Google's website.
- Navigate to the Pub/Sub section of the Google Cloud console. Follow the prompts to enable the API.
- Create a Pub/Sub topic.
- Use Google IAM service account impersonation or key-based access to configure the access method that Fastly uses to send logs.

### Creating a service account

Google Cloud Platform (GCP) uses [service accounts](https://cloud.google.com/iam/docs/service-account-overview) for third-party application authentication and role-based access to Google Cloud resources. To create a new service account, follow the instructions in the [Google Cloud documentation](https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating).

When creating the service account for logging to Google Pub/Sub, the service account must be assigned the `Pub/Sub Publisher` role to publish to the topic you created for Fastly logging.

![Create service account panel](/img/service_account_pubsub.png)

### Configure access for the service account

Fastly requires access to write logs to your Pub/Sub topic. You can use either role-based, service account impersonation, or key-based access to allow Fastly to send logs to Google Pub/Sub.

#### Configuring Google IAM service account impersonation

If you elect to use Google IAM service account impersonation to avoid storing keys with Fastly, you'll need to specify a service account name and a project ID when adding GCS as your logging endpoint. The service account ID comes before the `@` in the service account email and the project ID immediately after. For example, if your service account email is `my-name@projectid.iam.gserviceaccount.com`, the service account ID is `my-name` and the project ID is `projectid`. Our guide to [creating an Google IAM role](https://www.fastly.com/documentation/guides/integrations/streaming-logs/configuring-google-iam-service-account-impersonation-for-fastly-logging) provides specific details on configuring this feature.

#### Configuring key-based access

If you elect to use key-based access, refer to Google's guide on [generating a service account credential](https://cloud.google.com/iam/docs/keys-create-delete#creating_service_account_keys).

When configuring key-based access, you must select the JSON format. Upon saving the configuration, a JSON file will be downloaded to your computer, containing the credentials for the GCS service account you just created. Open the file with a text editor and make a note of the `private_key` and `client_email`.

> **HINT:** Read more about Cloud Pub/Sub in [Google’s documentation](https://cloud.google.com/pubsub/docs/).

## Adding Cloud Pub/Sub as a logging endpoint

Follow these instructions to add Cloud Pub/Sub as a logging endpoint. As part of configuration, you can elect to configure Google IAM role-based service account impersonation to avoid storing secrets. Read our guide on [creating a Google IAM role](https://www.fastly.com/documentation/guides/integrations/streaming-logs/configuring-google-iam-service-account-impersonation-for-fastly-logging) for more information on this feature.

### Cdn Services

1.   Review the information in our guide to [setting up remote log streaming](/guides/integrations/streaming-logs/setting-up-remote-log-streaming).

2. In the Google Cloud Pub/Sub area, click **Create endpoint**.
3. Fill out the **Create a Google Cloud Pub/Sub endpoint** fields as follows:
   -   In the **Name** field, enter a human-readable name for the endpoint.

   -   In the **Placement** area, select where the logging call should be placed in the generated VCL. Valid values are **Format Version Default** and **None**. Read our guide on [changing log placement](/guides/integrations/streaming-logs/changing-log-placement) for more information.

   - In the **Log format** field, enter the data to send to Google Cloud Pub/Sub. See the [example format section](https://www.fastly.com/documentation/guides/integrations/logging-endpoints/data-streaming-and-message-queues/log-streaming-google-cloud-pubsub#example-format) for details.
   - In the **Project ID** field, enter the ID of your Google Cloud Platform project.
   - In the **Email** field, enter the email address of the service account configured for your Pub/Sub topic.
   - In the **Topic** field, enter the Pub/Sub topic to which logs should be sent.
   - In the **Access Method** area, select how Fastly will access Google resources for purposes of log delivery. Valid values are **User Credentials** and **IAM Role**.
   - If you selected **User Credentials**, fill out the following fields:
     - In the **Email** field, enter the email address of the service account configured for your Pub/Sub topic.
     - In the **Secret Key** field, enter the exact value of the [`private_key`](https://www.fastly.com/documentation/guides/integrations/logging-endpoints/data-streaming-and-message-queues/log-streaming-google-cloud-pubsub#configuring-key-based-access) associated with the service account configured for your Pub/Sub topic.
   - If you selected **IAM Role**, enter fill out following field:
     - In the **Service Account Name** field, enter the name of the service account email address you selected when configuring Google IAM service account impersonation.
   -   *(Optional)* From the **Processing region** menu, select a geographic region where logs are processed before being sent to the logging endpoint. Our guide on [regional log aggregation](/guides/integrations/streaming-logs/setting-up-regional-log-aggregation) provides more information.

4.   Click **Create** to create the new logging endpoint.

5.   From the **Activate** menu, select **Activate on Production** to deploy your configuration changes.

### Example format

Data sent to Cloud Pub/Sub must be serialized as a JSON object, and every field in the JSON object must map to a string in your table's schema. The JSON can have nested data in it (e.g., the value of a key in your object can be another object). Here's an example format string for sending data:

```plaintext
{
  "timestamp": "%{strftime(\{"%Y-%m-%dT%H:%M:%S%z"\}, time.start)}V",
  "client_ip": "%{req.http.Fastly-Client-IP}V",
  "geo_country": "%{client.geo.country_name}V",
  "geo_city": "%{client.geo.city}V",
  "host": "%{if(req.http.Fastly-Orig-Host, req.http.Fastly-Orig-Host, req.http.Host)}V",
  "url": "%{json.escape(req.url)}V",
  "request_method": "%{json.escape(req.method)}V",
  "request_protocol": "%{json.escape(req.proto)}V",
  "request_referer": "%{json.escape(req.http.referer)}V",
  "request_user_agent": "%{json.escape(req.http.User-Agent)}V",
  "response_state": "%{json.escape(fastly_info.state)}V",
  "response_status": %{resp.status}V,
  "response_reason": %{if(resp.response, "%22"+json.escape(resp.response)+"%22", "null")}V,
  "response_body_size": %{resp.body_bytes_written}V,
  "fastly_server": "%{json.escape(server.identity)}V",
  "fastly_is_edge": %{if(fastly.ff.visits_this_service == 0, "true", "false")}V
}
```

### Compute Services

1.   Review the information in our guide to [setting up remote log streaming for Compute](/guides/integrations/streaming-logs/setting-up-remote-log-streaming-for-compute). Additionally, our developer documentation provides more [information about logging](/guides/integrations/non-fastly-services/developer-guide-logging/) with Compute code written in our [supported languages](/reference/compute/sdks/).

2. In the Google Cloud Pub/Sub area, click **Create endpoint**.
3. Fill out the **Create a Google Cloud Pub/Sub endpoint** fields as follows:
   -   In the **Name** field, enter the endpoint name you specified in your Compute code. For example, in our [Rust code example](/guides/compute/developer-guides/rust/#logging), the name is `my_endpoint_name`.

   - In the **Project ID** field, enter the ID of your Google Cloud Platform project.
   - In the **Topic** field, enter the Pub/Sub topic to which logs should be sent.
   - In the **Access Method** area, select how Fastly will access Google resources for purposes of log delivery. Valid values are **User Credentials** and **IAM Role**.
   - If you selected **User Credentials**, fill out the following fields:
     - In the **Email** field, enter the email address of the service account configured for your Pub/Sub topic.
     - In the **Secret Key** field, enter the exact value of the private key associated with the service account configured for your Pub/Sub topic.
   - If you selected **IAM Role**, fill out the following field:
     - In the **Service Account Name** field, enter the name of the service account email address you selected when configuring Google IAM service account impersonation.
4.   Click **Create** to create the new logging endpoint.

5.   From the **Activate** menu, select **Activate on Production** to deploy your configuration changes.

## Related content

- [API reference: Google Cloud Pub/Sub log streaming](https://www.fastly.com/documentation/reference/api/logging/google-pubsub/)
- [CLI reference: Google Cloud Pub/Sub log streaming](https://www.fastly.com/documentation/reference/cli/logging/googlepubsub/)