You appear to be offline. Some site functionality may not work.

Creating standards for CDNs

By  Mark Nottingham, Principal Engineer, February 20, 2019 in PerformanceStandardization

Content delivery networks (CDNs) have been around for a long time, but they’re not all built in the same way. While Fastly’s edge cloud platform goes beyond traditional content delivery networks by moving things to the edge, there are more fundamental differences; it’s not uncommon for CDNs to make up their own rules about how they serve web traffic, since CDNs didn’t exist when HTTP was defined. To improve this, we’re working alongside other platforms to standardize basic protocol handling for CDNs.

A while back, security researchers noticed what had concerned engineers working on content delivery networks CDNs for some time; it’s possible to point competing CDNs at each other to bring them down. From the abstract of Forwarding-Loop Attacks in Content Delivery Networks:

Malicious customers can attack the availability of Content Delivery Networks (CDNs) by creating forwarding loops inside one CDN or across multiple CDNs. Such forwarding loops cause one request to be processed repeatedly or even indefinitely, resulting in undesired resource consumption and potential Denial-of-Service attacks. To evaluate the practicality of such forwarding-loop attacks, we examined 16 popular CDN providers and found all of them are vulnerable to some form of such attacks.

Because of the massive scale of many CDNs — with terabits per second of link capacity available to each, worldwide — this is a potentially very scary problem. Whether it’s an intentional attack or an accidental misconfiguration, it could take large parts of the internet down with the CDNs in question. In fact, one CDN engineer I was discussing this with recently said it was one of the problems that “kept him awake at night.”

Many CDNs, and Fastly’s edge cloud platform, already have loop protection mechanisms in place, usually by using a header to identify requests that have already been seen. The problem is that these solutions aren’t coordinated between each other, so that one CDN might be configured to delete another CDN’s loop detection headers — either intentionally or accidentally.

We wanted something better, so we started talking to our colleagues at other CDNs as well as content platforms. The result was a small specification in the HTTP Working Group for the CDN-Loop request header, which was approved for publication as a standards-track RFC by the Internet Engineering Steering Group (IESG) two weeks ago.

This is a very simple mechanism. For example, Fastly’s looks like this:

GET /index.html HTTP/1.1
Host: www.example.com
CDN-Loop: Fastly

Implementing CDNs are required to add it on each request they make, and to protect it from accidental (or not-so-accidental) modifications, so that they can detect and mitigate such loops more reliably — even when a loop involves multiple CDNs.

We’re very happy about this small step forward because it’s a sign of change; since their early beginnings, there hasn’t been much coordination between CDNs about issues like this, much less in providing a consistent experience to our customers. So, while small, it’s an important step, as it signifies an industry that’s willing to work together.

We’re also not done; Fastly is contributing to a number of other community efforts to build better, more consistent experiences across vendors, including:

  • The HTTP Working Group has started work on the Cache response header to create a more consistent debugging experience for cache behaviour, both in CDNs and other HTTP caches.
  • We’re also contributing to a new, so-far unadopted draft on the Proxy-Status response header, to make CDN and reverse proxy debugging easier when something goes wrong.
  • The core HTTP specifications are being revised (again). This time, we’re paying special attention to how CDNs behave to make sure that they don’t need to violate the specifications on a regular basis.
  • That core work is being informed by an emerging set of common test cases for both browser and intermediary HTTP caches.
  • We’re looking at more advanced features like Variants to give intermediaries (including CDNs) more information about the content they’re serving, so they can make more intelligent caching decisions.

Once we get these and a few other basic building blocks for CDN behaviour standardised, we hope to move on to more exciting things — like standardising how an origin can discover that it’s behind a CDN, send purge requests to it, and even use surrogate keys without lots of configuration.

We believe that having one standard way to interact with a CDN makes our customers’ lives easier. It also makes it easier for content platforms like WordPress and Drupal to seamlessly work with CDNs when they’re in front of such a site — making things better for everybody.

Performance Standardization

You may also like:

Subscribe to our newsletter

Subscribe to our newsletter

Testing new ideas with Fastly Fiddle

Quickly experiment, debug and try out ideas on Fastly’s platform, without signing up, or even logging in.

Introducing Quick Value Packages

Keeping your digital presence continuously tuned, optimized, and secure to align with changing business and technical requirements can be time consuming. That’s why we’ve put together our Quick Value Packages — a collection of expert…

Towards ever faster websites with early hints and priority hints

Websites still load too slowly. During the most critical time in the page load lifecycle, your connection is often almost totally idle. At Fastly, we’re watching a new technology that hopes to make better use…

Author

Mark Nottingham | Principal Engineer

Mark Nottingham is a Principal Engineer at Fastly. He has had a role in developing the Web and the Internet since the late 90’s. Mark has written, edited or substantially contributed to more than twenty IETF RFCs and W3C Recommendations about topics like HTTP, caching, linking, Web architecture and security.

He has also chaired the HTTP Working Group since 2007 and the QUIC Working Group since 2016, and has been a member of the Internet Architecture Board since 2017. Before that, he served on the W3C Technical Architecture Group.

Beyond standards work, Mark helped deploy a precursor to “enterprise” CDNs in 1998, designed HTTP APIs and owned a caching platform at Yahoo!, and has contributed to several Open Source projects. Currently, he’s part of the Office of the CTO at Fastly.