Securing your origin against automated threats shouldn’t compromise your visibility into marketing attribution. When deploying edge-based bot challenges, a frequent complication arises: the original HTTP Referer is often lost, skewing Google Analytics insights and disrupting conversion paths.
This guide details a technical approach using Fastly’s Next-Gen WAF (NGWAF) and VCL to implement an embedded challenge while ensuring the original referer remains available for Google Tag Manager (GTM) and Google Analytics.
The Challenge: Addressing the "Synthetic Blind Spot"
When legitimate users encounter an NGWAF-triggered challenge, the typical edge workflow involves:
The NGWAF issues a browser challenge.
The browser completes the challenge and reloads the page
The browser loads the content from the origin through Fastly
The complication occurs during that final reload. The Referer header recognized by your origin and your analytics scripts frequently defaults to the URL of the challenge page. This hides the true external source, such as organic search or social channels, leading to inflated "Direct" traffic stats.
The Solution: Maintaining Referer Persistence
By leveraging NGWAF signaling alongside VCL header manipulation, we insert the Google Tag Manager logic into an synthetic response with a highly customizable embedded challenge page.
Step 1: Signal the Challenge (NGWAF)
Configure an NGWAF rule to identify requests requiring verification. Suspicious traffic triggers a 456 status code, serving as the functional bridge between the WAF and your VCL logic. The rule should include logic that checks for signal CHALLENGE-TOKEN-INVALID so that the user will not end up in a loop when the challenge is completed successfully.
Step 2: Serve the Challenge with GTM Integration
We manage the 456 status in vcl_fetch and deliver a custom HTML page via vcl_error. This synthetic response includes the Bot Management Embedded Challenge, Advanced Client-Side Detection, GTM tags, and the logic to handle the stateful reload.
The following VCL snippet is placed in init:
# vcl init
# Handle 456 status from NGWAF
sub vcl_fetch {
if (beresp.status == 456) {
# Restart to enter vcl_error and serve synthetic content
error 801 "Synthetic 456";
}
}
sub vcl_error {
if (obj.status == 801) {
set obj.status = 200;
set obj.response = "OK";
set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-YOUR_ID_HERE"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-YOUR_ID_HERE');
</script>
<!-- Fastly Bot Management Challenge Script -->
<script src="/_fs-ch-1T1wmsGaOgGaSxcX/challenge.js" defer></script>
<script src="/_fs-ch-1T1wmsGaOgGaSxcX/assets/script.js"></script>
<title>Verification Required</title>
<style>
body { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; display: flex; flex-direction: column; align-items: center; justify-content: center; min-height: 100-vh; margin: 0; background-color: #f7f7f7; }
.container { background: white; padding: 2rem; border-radius: 8px; box-shadow: 0 4px 6px rgba(0,0,0,0.1); text-align: center; max-width: 400px; width: 100%; }
h1 { color: #333; margin-bottom: 1rem; }
p { color: #666; margin-bottom: 2rem; }
.fastly-challenge { margin-top: 1rem; }
/* Optional styles based on challenge status */
.fastly-challenge[data-challenge-status="complete"]::after {
content: "✓ Verification Successful";
color: green;
font-weight: bold;
display: block;
margin-top: 10px;
}
</style>
</head>
<body>
<div class="container">
<h1>Verification Required</h1>
<p>Please wait while we verify your request...</p>
<!-- Fastly Challenge Widget -->
<div class="fastly-challenge"></div>
<script>
// Automatically reload the page once the challenge is complete
const challengeObserver = new MutationObserver((mutations) => {
mutations.forEach((mutation) => {
if (mutation.type === 'attributes' && mutation.attributeName === 'data-challenge-status') {
const status = mutation.target.getAttribute('data-challenge-status');
console.log('Challenge status changed:', status);
if (status === 'complete') {
console.log('Verification successful. Reloading...');
setTimeout(() => {
window.location.reload();
}, 1000);
}
}
});
});
const challengeEl = document.querySelector('.fastly-challenge');
if (challengeEl) {
challengeObserver.observe(challengeEl, { attributes: true });
}
</script>
</div>
</body>
</html>
"};
return (deliver);
}
}Strategic Benefits
1. Fluid User Journey: Visitors remain in context, completing the challenge and proceeding automatically to their destination.
2. Data Precision: GA captures the authentic traffic source despite the intermediate challenge.
3. Edge Performance: This logic executes entirely at the Fastly edge, shielding your origin from unauthorized requests while conserving resources.
Conclusion
Tailoring your bot defense ensures you don't have to choose between security and analytics accuracy. Fastly’s VCL and NGWAF provide the control necessary to defend your infrastructure while providing marketing teams with the clean data they rely on.
Ready to implement? Explore our documentation on embedding challenges or reach out if you have questions.