1. Home
  2. Guides
  3. Security
  4. Bot Management
  5. Pre-cache inspection (ContentGuard)

About ContentGuard

ContentGuard helps you identify and manage requests from bots before they reach your cache layer, allowing you to control bot access to cached and uncached resources. For example, you can block persistent scrapers from harvesting your content, allow beneficial search engine crawlers, and write custom VCL to handle specific scenarios, such as serving stale pricing data to marketing bots.

Prerequisites

Before setting up ContentGuard, you must:

Quick start

After deploying Bot Management using pre-cache inspection, Fastly immediately starts monitoring your traffic for bots and logging results. Fastly does not initially block any bot activity because the Protection mode's default value is Log (Monitor only). The Protection mode determines whether Fastly monitors traffic and enforces the actions you've configured (e.g., block headless bots but allow search engines).

For Fastly to actively protect your web applications from unwanted bots, you must change the Protection mode to Block (Active enforcement). To prepare for and then enable active enforcement, you should:

  1. Review the bot controls
  2. Monitor your bot traffic
  3. Change the Protection mode
  4. (Optional) Add detection for headless bots

Review and adjust the bot category and identity controls

To get started, review and adjust the action assigned to each bot category or identity based on your business requirements. Bot Management groups bots into categories (e.g., search engines, AI crawlers). Within each category, individual bots are tracked as identities (e.g., Googlebot). You can set actions at either level.

  1. Log in to the Fastly control panel.
  2. Go to Security > Bot Management > Protection.
  3. From the services menu, select the appropriate service.
  4. In the row of the bot category that you'd like to configure protection for, do one of the following:
    • From the Action menu, select the action Fastly should take for all bots that fall into that category.
    • Click Configure bots and then from the Action menu for each bot identity, select the action Fastly should take for all requests from that specific bot. When you change the value for a bot identity, the value of the Action menu for the category automatically changes to Custom.

Monitor your bot traffic and adjust the controls

Next, observe how your bot traffic is being handled and adjust the actions for any categories or identities that aren't behaving as expected.

  1. Log in to the Fastly control panel.
  2. Go to Security > Bot Management > Dashboards.
  3. Use the Overview dashboard to monitor the volume and types of bot traffic directed at your service.
  4. Go to Security > Bot Management > Protection.
  5. View the number of requests for each bot category and identity.
  6. Adjust the bot category and identity controls as needed until you're comfortable with the decisions Fastly is making.

Change the Protection mode

Finally, when you're ready for Fastly to enforce your configurations, change the Protection mode to Block (Active enforcement).

  1. Log in to the Fastly control panel.
  2. Go to Security > Bot Management > Settings.
  3. From the services menu, select the appropriate service.
  4. From the Protection mode menu, select Block (Active enforcement). Bot Management starts enforcing the bot category and identity actions that are selected on the Protection page.

Add detection for headless bots

By default, Bot Management doesn't detect bots that leverage headless browsers (e.g., headless Chrome). You can optionally extend detection for headless bots by adding the following line within the <head> section of your HTML. This sets up the Advanced client-side detection feature.

<script src="/_fs-ch-1T1wmsGaOgGaSxcX/assets/script.js"></script>

Related content

Fastly
© Fastly 2026