--- title: How caching with Fastly works summary: null url: >- https://www.fastly.com/documentation/guides/full-site-delivery/caching/how-it-works --- The Fastly edge cache is an enormous pool of storage across the platform's network. While the server hosting your content (your origin) may be far away from your users, causing latency when they visit your site, Fastly caches your content closer to users. By delivering content directly to users instead of having to fetch from origin each request, caching with Fastly can help you reduce data transfer costs and make your site more efficient and scalable. ## How caching with Fastly works To cache content with Fastly, you [create a service](https://www.fastly.com/documentation/guides/getting-started/services/about-services/), which defines the caching rules and behaviors for your website or application. Once your service is configured to deliver your site or application through Fastly, the following takes place whenever a user makes a request for content: 1. If the content is in the cache, Fastly will return it without visiting the origin. 2. If the content is not in cache, Fastly will fetch it from the origin and store it in cache (assuming the content is cacheable). ### Caching features Caching use cases vary widely. Fastly has several built-in features that help with both simple and complex scenarios: - [**HTTP caching semantics**](https://www.fastly.com/documentation/guides/full-site-delivery/caching/caching-best-practices/#optimize-your-cache-control-headers) let HTTP responses carry caching instructions, such as the `Cache-Control` header, that the Fastly cache uses to decide how they should be cached, as defined by the [HTTP Caching standard (RFC 9111)](https://httpwg.org/specs/rfc9111.html). - [**Request collapsing**](https://www.fastly.com/documentation/guides/concepts/cache/request-collapsing) identifies multiple simultaneous requests for the same resource and makes a single backend fetch, using the resulting response to populate the cache and satisfy all waiting clients. - [**Range collapsing**](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests) merges requests for separate byte ranges of a backend object into a single backend fetch for the entire object, using the resulting response to populate the cache and fulfill future requests for any byte range while managing the lifetime of the entire object. - [**Streaming miss**](https://www.fastly.com/documentation/guides/full-site-delivery/performance/streaming-miss) writes a response stream to cache and to an end user at the same time. - [**Client revalidation**](https://www.fastly.com/documentation/guides/concepts/cache/cache-freshness/) evaluates conditional headers from a client to determine whether its cached copy is still valid, sending the response body only if necessary, and may forward the request to the backend to refresh the cached object depending on cache state. - [**Backend revalidation**](https://www.fastly.com/documentation/guides/full-site-delivery/performance/serving-stale-content/) adds conditional headers when forwarding a request for a stale cached object to a backend, allowing the backend to extend the object's lifetime without resending the body if the cached content is still valid. - [**Purging**](https://www.fastly.com/documentation/guides/full-site-delivery/purging/) expunges cache entries ahead of their normal expiry, so that changes to the source content can be reflected at the edge immediately. > **IMPORTANT:** All data stored in the Fastly cache is _ephemeral_ and will expire. It may be evicted by the platform before it expires depending on how frequently the data is used. If you require persistent storage at the edge, consider using dynamic configurations like [dictionaries](https://www.fastly.com/documentation/guides/full-site-delivery/dictionaries/about-dictionaries), [access control lists](https://www.fastly.com/documentation/guides/security/access-control-lists/about-acls/), or [data stores](https://www.fastly.com/documentation/guides/compute/edge-data-storage/about-edge-data-stores) instead. ## About the cache interfaces Fastly provides three interfaces for interacting with the cache: readthrough (HTTP), simple, and core. The interface you choose depends on your service type and caching requirements. ### About the readthrough (HTTP) cache The readthrough (HTTP) cache is the most commonly used cache interface. It automatically caches HTTP responses according to [HTTP caching semantics](https://www.fastly.com/documentation/guides/concepts/cache/cache-freshness) as requests pass through your Fastly service. It is enabled by default for both [CDN](https://www.fastly.com/documentation/guides/full-site-delivery/fastly-vcl) and [Compute](https://www.fastly.com/documentation/guides/compute) services and is the only cache interface available for CDN services. ### CDN In a CDN service, the readthrough interface works without any configuration or code required. ### Rust In a Compute service, calling `Request::send()` with a backend name invokes the readthrough cache: ```rust use fastly::{Error, Request, Response}; #[fastly::main] fn main(req: Request) -> Result { Ok(req.send("my_backend_name")?) } ``` ### Javascript In a Compute service, calling `fetch()` with a backend name invokes the readthrough cache: ```js /// addEventListener("fetch", event => event.respondWith(handler(event))); function handler(event) { return fetch(event.request, { backend: "my_backend_name" }); } ``` ### Go In a Compute service, calling `func (*Request) Send()` with a backend name invokes the readthrough cache: ```go package main import ( "context" "fmt" "io" "github.com/fastly/compute-sdk-go/fsthttp" ) func main() { fsthttp.ServeFunc(func(ctx context.Context, w fsthttp.ResponseWriter, r *fsthttp.Request) { resp, err := r.Send(ctx, "my_backend_name") if err != nil { w.WriteHeader(fsthttp.StatusBadGateway) fmt.Fprintln(w, err.Error()) return } w.Header().Reset(resp.Header) w.WriteHeader(resp.StatusCode) io.Copy(w, resp.Body) }) } ``` ### Cpp In a Compute service, calling `Request::send()` with a backend name invokes the readthrough cache: ```cpp #include int main() { // section visible auto req = fastly::Request::from_client(); auto res = req.send("my_backend_name"); if (!res) { fastly::Response::from_status(500).send_to_client(); return 1; } // section-end visible res->send_to_client(); } ``` > **NOTE:** Because network operations can fail, the `.send()` method returns a `fastly::expected` container. Your application should handle this error and fail gracefully. For more details, refer to [Handling failures](https://www.fastly.com/documentation/guides/compute/developer-guides/cpp#handling-failures) in the C++ Getting Started guide. ### About the simple cache The simple cache interface is available exclusively for Compute services and provides straightforward programmatic access to the cache through a `getOrSet` operation. Use it to cache data directly from your Compute application as volatile key-value data. Common use cases include caching authentication flow state or A/B test flags. Simple cache has always-on request collapsing: if two operations try to populate the same key simultaneously, the setter callback runs only once. Values are treated as opaque data with no headers or metadata, so simple cache does not support staleness, revalidation, or variation. ### CDN The simple cache interface is not available for CDN services. ### Rust In a Compute service written in Rust, the simple cache is supported via the [`fastly:cache:simple`](https://docs.rs/fastly/latest/fastly/cache/simple/index.html) module. ```rust use { fastly::{ cache::simple::{get_or_set_with, CacheEntry}, mime, Body, Error, Request, Response, }, std::{thread, time::Duration}, }; #[fastly::main] fn main(req: Request) -> Result { let path = req.get_path().to_owned(); let value = get_or_set_with(path.clone(), || { Ok(CacheEntry { value: expensive_render_operation(&path), ttl: Duration::from_secs(60), }) }) .unwrap() .expect("closure always returns `Ok`, so we have a value"); Ok(Response::from_body(value).with_content_type(mime::TEXT_PLAIN_UTF_8)) } fn expensive_render_operation(path: &str) -> Body { // expensive/slow function which constructs and returns the contents for a given path thread::sleep(Duration::from_secs(1)); return path.into(); } ``` ### Javascript In a Compute service written in JavaScript, the simple cache is supported via the `SimpleCache` object exported from the [`fastly:cache`](https://js-compute-reference-docs.edgecompute.app/docs/fastly:cache/SimpleCache/) module. ```js /// addEventListener('fetch', event => event.respondWith(app(event))); async function app(event) { const path = new URL(event.request.url).pathname; const content = SimpleCache.getOrSet(path, async () => { return { value: await expensiveRenderOperation(path), ttl: 60 } }); return new Response(content, { headers: { 'content-type': 'text/plain;charset=UTF-8' } }); } async function expensiveRenderOperation(path) { // expensive/slow function which constructs and returns the contents for a given path await new Promise(resolve => setTimeout(resolve, 10000)); return path; } ``` ### Go In a Compute service written in Go, the simple cache is supported via the [`cache/simple`](https://pkg.go.dev/github.com/fastly/compute-sdk-go/cache/simple) package. ```go package main import ( "context" "io" "strings" "time" "github.com/fastly/compute-sdk-go/cache/simple" "github.com/fastly/compute-sdk-go/fsthttp" ) func main() { fsthttp.ServeFunc(func(ctx context.Context, w fsthttp.ResponseWriter, r *fsthttp.Request) { rc, err := simple.GetOrSet([]byte(r.URL.Path), func() (simple.CacheEntry, error) { return simple.CacheEntry{ Body: render(r.URL.Path), TTL: time.Minute, }, nil }) if err != nil { fsthttp.Error(w, err.Error(), fsthttp.StatusInternalServerError) return } defer rc.Close() w.Header().Set("Content-Type", "text/plain") io.Copy(w, rc) }) } func render(path string) *strings.Reader { time.Sleep(10 * time.Second) return strings.NewReader(path) } ``` ### Cpp In this version of the Fastly Compute C++ SDK, the simple cache interface is not available. For complete documentation on the simple cache interface, refer to the reference for the [Compute SDK](https://www.fastly.com/documentation/reference/compute/sdks) of your choice. ### About the core cache The core cache interface is available exclusively for Compute services. It provides low-level programmatic access to the cache, with manual control over cache metadata and behavior. Use it for advanced caching requirements or to build custom higher-level caching abstractions. Items cached via this interface consist of: - **A cache key:** up to 4KiB of arbitrary bytes identifying a cached item. Since a key may not uniquely identify an item, headers can further distinguish items that share a key. See [LookupBuilder::header()](https://docs.rs/fastly/latest/fastly/cache/core/struct.LookupBuilder.html#method.header) in the Rust SDK documentation for details. - **General metadata:** expiry data, including item age, expiration time, and surrogate keys for purging. - **User-controlled metadata:** arbitrary bytes stored alongside the cached content, updatable during revalidation. - **The object itself:** arbitrary bytes, read via `Body` and written via `StreamingBody`. For complete documentation on the core cache interface, refer to the reference for the [Compute SDK](https://www.fastly.com/documentation/reference/compute/sdks) of your choice. ### CDN The core cache interface is not available for CDN services. ### Rust In a Compute service written in Rust, in the simplest cases, the top-level [`insert`](https://docs.rs/fastly/latest/fastly/cache/core/fn.insert.html) and [`lookup`](https://docs.rs/fastly/latest/fastly/cache/core/fn.lookup.html) functions are used for one-off operations on a cached item, and are appropriate when request collapsing and revalidation capabilities are not required. The core cache also supports more complex uses via the concept of a transaction, which can collapse concurrent lookups to the same item, including coordinating revalidation. The following example demonstrates a lookup that inserts a cache transaction: ```rust compile_fail const TTL: Duration = Duration::from_secs(3600); // perform the lookup let lookup_tx = Transaction::lookup(CacheKey::from_static(b"my_key")) .execute() .unwrap(); if let Some(found) = lookup_tx.found() { // a cached item was found; we use it now even though it might be stale, // and we'll revalidate it below use_found_item(&found); } // now we need to handle the "must insert" and "must update" cases if lookup_tx.must_insert() { // a cached item was not found, and we've been chosen to insert it let contents = build_contents(); let (mut writer, found) = lookup_tx .insert(TTL) .surrogate_keys(["my_key"]) .known_length(contents.len() as u64) // stream back the object so we can use it after inserting .execute_and_stream_back() .unwrap(); writer.write_all(contents).unwrap(); writer.finish().unwrap(); // now we can use the item we just inserted use_found_item(&found); } else if lookup_tx.must_insert_or_update() { // a cached item was found and used above, and now we need to perform // revalidation let revalidation_contents = build_contents(); if let Some(stale_found) = lookup_tx.found() { if should_replace(&stale_found, &revalidation_contents) { // use `insert` to replace the previous object let mut writer = lookup_tx .insert(TTL) .surrogate_keys(["my_key"]) .known_length(revalidation_contents.len() as u64) .execute() .unwrap(); writer.write_all(revalidation_contents).unwrap(); writer.finish().unwrap(); } else { // otherwise update the stale object's metadata lookup_tx .update(TTL) .surrogate_keys(["my_key"]) .execute() .unwrap(); } } } ``` ### Javascript In a Compute service written in JavaScript, the core cache is supported via the `CoreCache` object exported from the [`fastly:cache`](https://js-compute-reference-docs.edgecompute.app/docs/fastly:cache/SimpleCache/) module. In the simplest cases, the [`CoreCache.insert`](https://js-compute-reference-docs.edgecompute.app/docs/fastly:cache/CoreCache/insert) and [`CoreCache.lookup`](https://js-compute-reference-docs.edgecompute.app/docs/fastly:cache/CoreCache/lookup) functions are used for one-off operations on a cached item, and are appropriate when request collapsing and revalidation capabilities are not required. The core cache also supports more complex uses via the concept of a "transaction", which can collapse concurrent lookups to the same item, including coordinating revalidation. The following example demonstrates a lookup/insert cache transaction: ```javascript /// addEventListener("fetch", event => event.respondWith(handleRequest(event))); async function handleRequest(event) { const path = (new URL(event.request.url)).pathname; const entry = CoreCache.transactionLookup(path); if (entry.state().mustInsertOrUpdate()) { const [writer, reader] = entry.insertAndStreamBack({ maxAge: 60 * 1000 }); writer.append(`hello from ${path}`); writer.close(); return new Response(reader.body(), { headers: { "x-cache": "MISS", "x-cache-hits": 0, }, }); } else { return new Response(entry.body(), { headers: { "x-cache": "HIT", "x-cache-hits": entry.hits(), }, }); } } ``` ### Go In a Compute service written in Go, in the simplest cases, the [`core.Insert`](https://pkg.go.dev/github.com/fastly/compute-sdk-go/cache/core#Insert) and [`core.Lookup`](https://pkg.go.dev/github.com/fastly/compute-sdk-go/cache/core#Lookup) functions are used for one-off operations on a cached item, and are appropriate when request collapsing and revalidation capabilities are not required. The core cache also supports more complex uses via the concept of a "transaction", which can collapse concurrent lookups to the same item, including coordinating revalidation. The following example demonstrates a lookup/insert cache transaction: ```go import ( "time" "github.com/fastly/compute-sdk-go/cache/core" ) func ExampleTransaction() { // Users of the transactional API should at a minimum anticipate // lookups that are obligated to insert an object into the cache, // and lookups which are not. If the stale-while-revalidate // parameter is set for a cached object, the user should also // distinguish between the insertion and revalidation cases. useFoundItem := func(f *core.Found) { // Do something with the found item } buildContents := func() []byte { // Build the contents of the cached item return []byte("hello world!") } shouldReplace := func(f *core.Found, contents []byte) bool { // Determine whether the cached item should be replaced with // the new contents return true } tx, err := core.NewTransaction([]byte("my_key"), core.LookupOptions{}) if err != nil { panic(err) } defer tx.Close() // f is a core.Found value, representing a found cache item. // core.ErrNotFound is returned if the item is not cached. f, err := tx.Found() switch err { case nil: // A cached item was found, though it might be stale. useFoundItem(f) // Perform revalidation, if necessary. if tx.MustInsertOrUpdate() { contents := buildContents() if shouldReplace(f, contents) { // Use Insert to replace the previous object w, err := tx.Insert(core.WriteOptions{ TTL: time.Hour, SurrogateKeys: []string{"my_key"}, Length: uint64(len(contents)), }) if err != nil { panic(err) } if _, err := w.Write(contents); err != nil { panic(err) } if err := w.Close(); err != nil { panic(err) } } else { // Otherwise update the stale object's metadata if err := tx.Update(core.WriteOptions{ TTL: time.Hour, SurrogateKeys: []string{"my_key"}, }); err != nil { panic(err) } } } case core.ErrNotFound: // The item was not found. if tx.MustInsert() { // We've been chosen to insert the object. contents := buildContents() w, f, err := tx.InsertAndStreamBack(core.WriteOptions{ TTL: time.Hour, SurrogateKeys: []string{"my_key"}, Length: uint64(len(contents)), }) if err != nil { panic(err) } if _, err := w.Write(contents); err != nil { panic(err) } if err := w.Close(); err != nil { panic(err) } useFoundItem(f) } else { panic(err) } default: // An unexpected error panic(err) } } ``` ### Cpp In this version of the Fastly Compute C++ SDK, the core cache interface is not available. ### Compare the cache interfaces The following table summarizes the availability, behavior, and capabilities of each cache interface. | | Readthrough (HTTP cache) | Simple | Core | | ------------------------ | ------------------------ | ------------------------ | -------------------- | | **Service type** | CDN and Compute | Compute | Compute | | **Use it for...** | Automatic caching | Simple key-value caching | Complex requirements | | **Cache freshness** | HTTP semantics | Explicit | Explicit | | **Request collapsing** | Heuristic | Always-on | Manual control | | **Range collapsing** | ✅ (automatic) | ❌ | ✅ (manual) | | **Streaming miss** | ✅ (automatic) | ❌ | ✅ (manual) | | **Client revalidation** | ✅ (automatic) | ❌ | ✅ (manual) | | **Backend revalidation** | ✅ (automatic) | ❌ | ✅ (manual) | | **Surrogate keys** | ✅ | ❌ | ✅ | | **Purging** | ✅ | ✅ | ✅ | ### Interoperability All three cache interfaces store data in the same namespace, but interoperability is limited: - The core cache interface can read and overwrite objects inserted via the simple cache interface. - The simple cache can read (but cannot overwrite existing) objects inserted using core cache but provides only the body of the object. - The readthrough cache interface is not interoperable with other cache interfaces and cannot read data written through another interface, nor can it write data that is visible to other cache interfaces. - Interoperability also affects [purging](https://www.fastly.com/documentation/guides/concepts/cache/purging/). ## Limitations and constraints The following limitations apply across all cache interfaces: - Variants (created explicitly in the core cache interface or when the readthrough cache processes the `Vary` HTTP response header) are limited differently depending on platform. ### CDN services In CDN services, the number of variants is limited to 50 per cache object, regardless of the number of `Vary` rule permutations. ### Compute Services In Compute services, the number of variants is not limited but the number of distinct vary rules is limited to 8 per cache object. - In the core cache interface, writes target only the primary storage node for a given cache address. If an existing object is overwritten, replicated copies may continue to be served until their TTL expires or the object is purged. - Purges are asynchronous while writes are synchronous. Performing a purge immediately before a write can create a race condition where the purge clears the primary object _after_ the write completes. For platform-specific limits, refer to [CDN service constraints](https://www.fastly.com/documentation/reference/vcl/constraints-and-limitations/) and [Compute service constraints](https://www.fastly.com/documentation/guides/compute/getting-started-with-compute/#limitations-and-constraints). ## What's next Start exploring the different [caching features](https://www.fastly.com/documentation/guides/full-site-delivery/caching/how-it-works#caching-features) available in Fastly. Or, check out our [tutorials](https://www.fastly.com/documentation/solutions/tutorials/full-site-delivery/) to find and implement solutions for common caching scenarios.