Teach Your robots.txt a New Trick (for AI)
Senior Security Strategist, Fastly
Control how AI bots like Google-Extended and Applebot-Extended use your website content for training. Update your robots.txt file with simple Disallow rules.
Staying Secure in 2025
All blog posts
-
Teach Your robots.txt a New Trick (for AI)
Brooks Cunningham
Control how AI bots like Google-Extended and Applebot-Extended use your website content for training. Update your robots.txt file with simple Disallow rules.
CDN & Delivery+ 3 more
-
Fastly + Scalepost: Extending the Fastly platform to manage AI Crawlers
John Agger
See when and how AI chatbots use your content. With Fastly and ScalePost, publishers finally gain visibility into how their work shows up in AI-generated answers.
-
DDoS in August
Liam Mayron, David King
August 2025 DDoS attack trends: Hyperscale clouds are the source for 70% of attacks. Get insights on the latest application DDoS trends to strengthen security.
Security+ 2 more
-
Control and Monetize Your Content with the RSL Standard
Simon Wistow
AI crawlers are scraping the web, often ignoring rules and costing publishers resources. The new RSL Standard lets you block, allow, or even charge AI for access to your content. Here’s how it works.
CDN & DeliveryProduct
-
Sustainability dashboard: Shine a light on your digital carbon footprint
Eoghan Kelly
Fastly's Sustainability dashboard provides instant access to electricity-related Scope 2 & 3 emissions data. Understand and optimize your digital carbon footprint with ease.
CDN & Delivery+ 3 more
-
CISO Perspective: Q2 2025 Threat Insights Report
Marshall Erwin
Explore Fastly's Q2 2025 Threat Report through our CISO, Marshall Erwin's eyes. Uncover bot traffic insights and key security practices.
SecurityIndustry insights
-
Bridging the real-time testing gap: Fanout support in local development for Fastly Compute
Katsuyuki Omuro
Fastly Compute now supports local Fanout testing, letting you build and validate real-time features without deploying to production.
Engineering+ 2 more -
Powering PyPI with Advanced Traffic Engineering
Joe Williams, Stephen Strowes
PyPI serves millions daily. See how Fastly’s Individual Provider Anycast unlocks faster, smarter routing for Python’s package index.
CDN & Delivery+ 4 more
-
Vibe Shift? Senior Developers Ship nearly 2.5x more AI Code than Junior Counterparts
Alina Lehtinen-Vela
Fastly’s survey shows senior developers trust gen AI tools enough to ship 2.5x more AI code, while juniors stick to traditional coding and caution.
Industry insightsEngineering
-
Fastly DDoS Protection wins SiliconANGLE TechForward Cloud Security Award
David King
Fastly DDoS Protection wins SiliconANGLE TechForward Cloud Security Award after rigorous analysis by 32 industry peers.
Security
-
Why Paying Copyright Holders for AI Training is Essential
Simon Wistow, John Agger
AI and creator rights don’t need to clash. A fair, consent-based model can drive innovation without exploiting creative work.
Industry insights -
The Truth About Blocking AI, And How Publishers Can Still Win
Simon Wistow
Many AI crawlers aren’t following the rules, and robots.txt can’t stop them. Blocking Google’s AI means killing your SEO, but publishers aren’t completely out of options. Edge control is becoming their last real defense.
SecurityIndustry insights
-
AI Bots in Q2 2025: Trends from Fastly's Threat Insights Report
Matthew Mathur, David King, + 1 more
Fastly's Q2 2025 Threat Insights Report uncovers how Meta, OpenAI, and others are shaping web traffic and what organizations need to do to stay in control.
SecurityIndustry insights
-
DDoS in July
Liam Mayron, David King
July 2025 DDoS attack trends: Fastly's report reveals infrequent but massive enterprise attacks & insights on attack volume, industries targeted, and company size.
SecurityIndustry insights
-
Request Collapsing Demystified
Stephen Crim
Boost website performance with request collapsing! Learn how it improves efficiency, reduces origin load, and optimizes caching for a snappy user experience.
Performance+ 2 more -
Maximizing Compute Performance with Log Explorer & Insights
Namit Shivaram
Monitor and troubleshoot Fastly Compute services with Log Explorer & Insights. Gain granular insights, optimize performance, and debug faster for efficient applications.
Compute+ 2 more
-
Demystifying Fastly’s Defense Against HTTP Desynchronization Attacks
Brian Haberman, Sandra Escandor-O’Keefe, + 1 more
Learn how Fastly's robust architecture and strict protocol parsing defend against HTTP desynchronization attacks, ensuring your web applications are secure.
Security+ 3 more
-
Trust at Scale with Fastly Image Optimizer and C2PA
James Sherry
Fastly Image Optimizer now supports C2PA, enabling verifiable content authenticity. Combat misinformation and build trust with secure image provenance at scale.
CDN & Delivery+ 2 more
-
Fastest Sites Run on Fastly
Leon Brocard
Make your site 25%+ faster with Fastly’s programmable edge. See why the fastest media sites — from Vox.com to Business Insider — run on Fastly.
Performance+ 2 more
-
Fastly's Resilience to HTTP/1.1 Desynchronization Attacks
Frederik Deweerdt, Kazuho Oku, + 2 more
Discover why Fastly's architecture protects against HTTP/1.1 desynchronization attacks, unlike other CDNs. Protect your applications with Fastly's secure platform.
Security+ 4 more
