How caching with Fastly works

The Fastly edge cache is an enormous pool of storage across the platform's network. While the server hosting your content (your origin) may be far away from your users, causing latency when they visit your site, Fastly caches your content closer to users. By delivering content directly to users instead of having to fetch from origin each request, caching with Fastly can help you reduce data transfer costs and make your site more efficient and scalable.

How caching with Fastly works

To cache content with Fastly, you create a service, which defines the caching rules and behaviors for your website or application. Once your service is configured to deliver your site or application through Fastly, the following takes place whenever a user makes a request for content:

1. If the content is in the cache, Fastly will return it without visiting the origin.
2. If the content is not in cache, Fastly will fetch it from the origin and store it in cache (assuming the content is cacheable).

Caching features

Caching use cases vary widely. Fastly has several built-in features that help with both simple and complex scenarios:

• HTTP caching semantics let HTTP responses carry caching instructions, such as the Cache-Control header, that the Fastly cache uses to decide how they should be cached, as defined by the HTTP Caching standard (RFC 9111).
• Request collapsing identifies multiple simultaneous requests for the same resource and makes a single backend fetch, using the resulting response to populate the cache and satisfy all waiting clients.
• Range collapsing merges requests for separate byte ranges of a backend object into a single backend fetch for the entire object, using the resulting response to populate the cache and fulfill future requests for any byte range while managing the lifetime of the entire object.
• Streaming miss writes a response stream to cache and to an end user at the same time.
• Client revalidation evaluates conditional headers from a client to determine whether its cached copy is still valid, sending the response body only if necessary, and may forward the request to the backend to refresh the cached object depending on cache state.
• Backend revalidation adds conditional headers when forwarding a request for a stale cached object to a backend, allowing the backend to extend the object's lifetime without resending the body if the cached content is still valid.
• Purging expunges cache entries ahead of their normal expiry, so that changes to the source content can be reflected at the edge immediately.

About the cache interfaces

Fastly provides three interfaces for interacting with the cache: readthrough (HTTP), simple, and core. The interface you choose depends on your service type and caching requirements.

About the readthrough (HTTP) cache

The readthrough (HTTP) cache is the most commonly used cache interface. It automatically caches HTTP responses according to HTTP caching semantics as requests pass through your Fastly service. It is enabled by default for both CDN and Compute services and is the only cache interface available for CDN services.

use fastly::{Error, Request, Response};

#[fastly::main]
fn main(req: Request) -> Result<Response, Error> {
    Ok(req.send("my_backend_name")?)
}

/// <reference types="@fastly/js-compute" />

addEventListener("fetch", event => event.respondWith(handler(event)));

function handler(event) {
  return fetch(event.request, { backend: "my_backend_name" });
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

package main

import (
  "context"
  "fmt"
  "io"

  "github.com/fastly/compute-sdk-go/fsthttp"
)

func main() {
  fsthttp.ServeFunc(func(ctx context.Context, w fsthttp.ResponseWriter, r *fsthttp.Request) {
    resp, err := r.Send(ctx, "my_backend_name")
    if err != nil {
      w.WriteHeader(fsthttp.StatusBadGateway)
      fmt.Fprintln(w, err.Error())
      return
    }

    w.Header().Reset(resp.Header)
    w.WriteHeader(resp.StatusCode)
    io.Copy(w, resp.Body)
  })
}

4
5
6
7
8
9

L1 : L3
 
 
    auto req = fastly::Request::from_client();
    auto res = req.send("my_backend_name");
    if (!res) {
        fastly::Response::from_status(500).send_to_client();
        return 1;
    }
L10 : L11
 
 

About the simple cache

The simple cache interface is available exclusively for Compute services and provides straightforward programmatic access to the cache through a getOrSet operation. Use it to cache data directly from your Compute application as volatile key-value data. Common use cases include caching authentication flow state or A/B test flags.

Simple cache has always-on request collapsing: if two operations try to populate the same key simultaneously, the setter callback runs only once. Values are treated as opaque data with no headers or metadata, so simple cache does not support staleness, revalidation, or variation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

use {
    fastly::{
        cache::simple::{get_or_set_with, CacheEntry},
        mime, Body, Error, Request, Response,
    },
    std::{thread, time::Duration},
};

#[fastly::main]
fn main(req: Request) -> Result<Response, Error> {
    let path = req.get_path().to_owned();
    let value = get_or_set_with(path.clone(), || {
        Ok(CacheEntry {
            value: expensive_render_operation(&path),
            ttl: Duration::from_secs(60),
        })
    })
    .unwrap()
    .expect("closure always returns `Ok`, so we have a value");

    Ok(Response::from_body(value).with_content_type(mime::TEXT_PLAIN_UTF_8))
}

fn expensive_render_operation(path: &str) -> Body {
// expensive/slow function which constructs and returns the contents for a given path
    thread::sleep(Duration::from_secs(1));
    return path.into();
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

/// <reference types="@fastly/js-compute" />

import { SimpleCache } from 'fastly:cache';

addEventListener('fetch', event => event.respondWith(app(event)));

async function app(event) {
  const path = new URL(event.request.url).pathname;
  const content = SimpleCache.getOrSet(path, async () => {
    return {
      value: await expensiveRenderOperation(path),
      ttl: 60
    }
  });
  return new Response(content, {
    headers: {
      'content-type': 'text/plain;charset=UTF-8'
    }
  });
}

async function expensiveRenderOperation(path) {
// expensive/slow function which constructs and returns the contents for a given path
  await new Promise(resolve => setTimeout(resolve, 10000));
  return path;
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

package main

import (
  "context"
  "io"
  "strings"
  "time"

  "github.com/fastly/compute-sdk-go/cache/simple"
  "github.com/fastly/compute-sdk-go/fsthttp"
)

func main() {
  fsthttp.ServeFunc(func(ctx context.Context, w fsthttp.ResponseWriter, r *fsthttp.Request) {

    rc, err := simple.GetOrSet([]byte(r.URL.Path), func() (simple.CacheEntry, error) {
      return simple.CacheEntry{
        Body: render(r.URL.Path),
        TTL:  time.Minute,
      }, nil
    })
    if err != nil {
      fsthttp.Error(w, err.Error(), fsthttp.StatusInternalServerError)
      return
    }
    defer rc.Close()

    w.Header().Set("Content-Type", "text/plain")
    io.Copy(w, rc)
  })
}

func render(path string) *strings.Reader {
  time.Sleep(10 * time.Second)
  return strings.NewReader(path)
}

For complete documentation on the simple cache interface, refer to the reference for the Compute SDK of your choice.

About the core cache

The core cache interface is available exclusively for Compute services. It provides low-level programmatic access to the cache, with manual control over cache metadata and behavior. Use it for advanced caching requirements or to build custom higher-level caching abstractions.

Items cached via this interface consist of:

• A cache key: up to 4KiB of arbitrary bytes identifying a cached item. Since a key may not uniquely identify an item, headers can further distinguish items that share a key. See LookupBuilder::header() in the Rust SDK documentation for details.
• General metadata: expiry data, including item age, expiration time, and surrogate keys for purging.
• User-controlled metadata: arbitrary bytes stored alongside the cached content, updatable during revalidation.
• The object itself: arbitrary bytes, read via Body and written via StreamingBody. For complete documentation on the core cache interface, refer to the reference for the Compute SDK of your choice.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

const TTL: Duration = Duration::from_secs(3600);
// perform the lookup
let lookup_tx = Transaction::lookup(CacheKey::from_static(b"my_key"))
    .execute()
    .unwrap();
if let Some(found) = lookup_tx.found() {
// a cached item was found; we use it now even though it might be stale,
// and we'll revalidate it below
    use_found_item(&found);
}
// now we need to handle the "must insert" and "must update" cases
if lookup_tx.must_insert() {
// a cached item was not found, and we've been chosen to insert it
    let contents = build_contents();
    let (mut writer, found) = lookup_tx
        .insert(TTL)
        .surrogate_keys(["my_key"])
        .known_length(contents.len() as u64)
// stream back the object so we can use it after inserting
        .execute_and_stream_back()
        .unwrap();
    writer.write_all(contents).unwrap();
    writer.finish().unwrap();
// now we can use the item we just inserted
    use_found_item(&found);
} else if lookup_tx.must_insert_or_update() {
// a cached item was found and used above, and now we need to perform
// revalidation
    let revalidation_contents = build_contents();
    if let Some(stale_found) = lookup_tx.found() {
        if should_replace(&stale_found, &revalidation_contents) {
// use `insert` to replace the previous object
            let mut writer = lookup_tx
                .insert(TTL)
                .surrogate_keys(["my_key"])
                .known_length(revalidation_contents.len() as u64)
                .execute()
                .unwrap();
            writer.write_all(revalidation_contents).unwrap();
            writer.finish().unwrap();
        } else {
// otherwise update the stale object's metadata
            lookup_tx
                .update(TTL)
                .surrogate_keys(["my_key"])
                .execute()
                .unwrap();
        }
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

/// <reference types="@fastly/js-compute" />
import { CoreCache } from "fastly:cache";

addEventListener("fetch", event => event.respondWith(handleRequest(event)));

async function handleRequest(event) {
    const path = (new URL(event.request.url)).pathname;
    const entry = CoreCache.transactionLookup(path);
    if (entry.state().mustInsertOrUpdate()) {
        const [writer, reader] = entry.insertAndStreamBack({
            maxAge: 60 * 1000
        });
        writer.append(`hello from ${path}`);
        writer.close();
        return new Response(reader.body(), {
            headers: {
                "x-cache": "MISS",
                "x-cache-hits": 0,
            },
        });
    } else {
        return new Response(entry.body(), {
            headers: {
                "x-cache": "HIT",
                "x-cache-hits": entry.hits(),
            },
        });
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106

import (
  "time"

  "github.com/fastly/compute-sdk-go/cache/core"
)

func ExampleTransaction() {
// Users of the transactional API should at a minimum anticipate
// lookups that are obligated to insert an object into the cache,
// and lookups which are not. If the stale-while-revalidate
// parameter is set for a cached object, the user should also
// distinguish between the insertion and revalidation cases.

  useFoundItem := func(f *core.Found) {
// Do something with the found item
  }

  buildContents := func() []byte {
// Build the contents of the cached item
    return []byte("hello world!")
  }

  shouldReplace := func(f *core.Found, contents []byte) bool {
// Determine whether the cached item should be replaced with
// the new contents
    return true
  }

  tx, err := core.NewTransaction([]byte("my_key"), core.LookupOptions{})
  if err != nil {
    panic(err)
  }
  defer tx.Close()

// f is a core.Found value, representing a found cache item.
// core.ErrNotFound is returned if the item is not cached.
  f, err := tx.Found()
  switch err {
  case nil:
// A cached item was found, though it might be stale.
    useFoundItem(f)

// Perform revalidation, if necessary.
    if tx.MustInsertOrUpdate() {
      contents := buildContents()
      if shouldReplace(f, contents) {
// Use Insert to replace the previous object
        w, err := tx.Insert(core.WriteOptions{
          TTL:           time.Hour,
          SurrogateKeys: []string{"my_key"},
          Length:        uint64(len(contents)),
        })
        if err != nil {
          panic(err)
        }

        if _, err := w.Write(contents); err != nil {
          panic(err)
        }

        if err := w.Close(); err != nil {
          panic(err)
        }
      } else {
// Otherwise update the stale object's metadata
        if err := tx.Update(core.WriteOptions{
          TTL:           time.Hour,
          SurrogateKeys: []string{"my_key"},
        }); err != nil {
          panic(err)
        }
      }
    }

  case core.ErrNotFound:
// The item was not found.
    if tx.MustInsert() {
// We've been chosen to insert the object.
      contents := buildContents()
      w, f, err := tx.InsertAndStreamBack(core.WriteOptions{
        TTL:           time.Hour,
        SurrogateKeys: []string{"my_key"},
        Length:        uint64(len(contents)),
      })
      if err != nil {
        panic(err)
      }

      if _, err := w.Write(contents); err != nil {
        panic(err)
      }

      if err := w.Close(); err != nil {
        panic(err)
      }

      useFoundItem(f)
    } else {
      panic(err)
    }

  default:
// An unexpected error
    panic(err)
  }
}

Compare the cache interfaces

The following table summarizes the availability, behavior, and capabilities of each cache interface.

Interoperability

All three cache interfaces store data in the same namespace, but interoperability is limited:

• The core cache interface can read and overwrite objects inserted via the simple cache interface.
• The simple cache can read (but cannot overwrite existing) objects inserted using core cache but provides only the body of the object.
• The readthrough cache interface is not interoperable with other cache interfaces and cannot read data written through another interface, nor can it write data that is visible to other cache interfaces.
• Interoperability also affects purging.

Limitations and constraints

The following limitations apply across all cache interfaces:

• Variants (created explicitly in the core cache interface or when the readthrough cache processes the Vary HTTP response header) are limited differently depending on platform.

  1. CDN services
  2. Compute services

  In CDN services, the number of variants is limited to 50 per cache object, regardless of the number of Vary rule permutations.

• In the core cache interface, writes target only the primary storage node for a given cache address. If an existing object is overwritten, replicated copies may continue to be served until their TTL expires or the object is purged.

• Purges are asynchronous while writes are synchronous. Performing a purge immediately before a write can create a race condition where the purge clears the primary object after the write completes.

For platform-specific limits, refer to CDN service constraints and Compute service constraints.

What's next

Start exploring the different caching features available in Fastly. Or, check out our tutorials to find and implement solutions for common caching scenarios.