Lifetime and revalidation

When a client request matches a stale object, the readthrough cache interface processes the object using the following steps:

1. If the object is within a stale-while-revalidate (SWR) period, then it will be served immediately to the client. Additionally, an asynchronous fetch to the backend will be scheduled to revalidate or update the stale object. See background revalidation process flow for details on this process.
2. Otherwise, the cache interface sends the request to the backend using a blocking fetch.
   • If this backend request was successful:
     • The default behavior is for the cache interface to use the response to revalidate or possibly update the stale object, and then return the response to the client.
     • If the stale object is in a stale-if-error (SIE) period, it is possible to use edge code to return the stale content instead. The cached object is not updated in this case. See explicitly serving stale content for details.
   • If the backend request failed:
     • If the stale object is within an SIE period, the stale object is returned to the client.
     • Otherwise, an error describing the failure is returned to the client.

When a Compute application's code makes a request to a backend that matches a stale object, the readthrough cache interface processes the object using the following steps:

1. If the stale object is within a stale-while-revalidate (SWR) period, then it is returned immediately to the application. Additionally, an asynchronous fetch to the backend will be scheduled to revalidate or update the stale object. See background revalidation process flow for details on this process.
2. Otherwise, the cache interface sends the request to the backend using a blocking fetch.
   • If this fetch was successful:
     • The default behavior is for the cache interface to use the response to revalidate or possibly update the stale object, and then return the response to the application.
     • If the stale object is in a stale-if-error (SIE) period, it is possible to return an error from the after-send hook to return the stale content instead. The cached object is not updated in this case. See explicitly serving stale content for details.
   • If this fetch failed:
     • If the stale object is within an SIE period, the stale object is returned to the application.
     • Otherwise, an error describing the failure (including any override error returned from your after-send hook) is returned to the Compute application.

No other aspects of the existing cached response will be modified. This means that after a successful revalidation, future client requests will receive the headers from the original backend response that populated the cache, rather than the headers returned in the 304 revalidation response.

This kind of response does not trigger vcl_fetch.

In a Compute service, the readthrough cache interface invokes the after-send callback, passing in a "candidate" response object that represents a complete response except for the body. This object has the status code, response headers, and cache policy based on the existing cached response, and reflects any headers and cache policy updates included in the backend revalidation response (for example, to extend the object's lifetime). These values are further merged with any changes you make during the after-send callback, and then finally used to update the cached response object. As a revalidation response does not contain a body, these updates occur "in-place", without invoking the body-transform callback, and without changing the cached response body.

Any response to a revalidation request other than 304 will be processed normally, trigger vcl_fetch, and (if cacheable) will replace the stale object in cache.

In a Compute service, any response to a revalidation request other than 304 will be processed normally, trigger the after-send and body-transform callbacks, and (if cacheable) will replace the stale object in cache.

To disable revalidation entirely, remove the ETag and Last-Modified headers from the response when it's received from the backend. This can be done in vcl_fetch:

unset beresp.http.etag;
unset beresp.http.last-modified;

Alternatively, you can enable revalidation between client and Fastly's cache, whilst effectively disabling it between Fastly's cache and the backend, by modifying the value of the ETag header:

set beresp.http.etag = beresp.http.etag "-fastly";
unset beresp.http.last-modified;

To disable revalidation entirely, remove the ETag and Last-Modified headers from the response when it's received by from the backend. This can be done in the after-send callback:

req.set_after_send(|resp| {
  resp.remove_header("ETag");
  resp.remove_header("Last-Modified");
  Ok(())
});

1
2
3
4
5
6
7
8

res = fetch(url, {
  cacheOverride: new CacheOverride({
    afterSend(resp) {
      resp.headers.delete('ETag');
      resp.headers.delete('Last-Modified');
    }
  })
});

r.CacheOptions.AfterSend = func(cr *CandidateResponse) error {
  cr.DelHeader("ETag")
  cr.DelHeader("Last-Modified")
  return nil
}

Alternatively, you can enable revalidation between the client and the Fastly edge platform, whilst effectively disabling it between Fastly's cache and the backend, by modifying the value of the ETag header:

req.set_after_send(|resp| {
  if let Some(etag) = resp.get_header_str("ETag") {
    resp.set_header("ETag", format!("{etag}-fastly"));
  }
  resp.remove_header("Last-Modified");
  Ok(())
});

1
2
3
4
5
6
7
8
9
10
11

res = fetch(url, {
  cacheOverride: new CacheOverride({
    afterSend(resp) {
      const etag = resp.headers.get('ETag');
      if (etag != null) {
        resp.headers.set('ETag', `${etag}-fastly`);
      }
      resp.headers.delete('Last-Modified');
    }
  })
});

r.CacheOptions.AfterSend = func(cr *CandidateResponse) error {
  if _, err := cr.Header("ETag"); err == nil {
    cr.DelHeader("Last-Modified")
  }
  return nil
}

When a CDN service handles a request that matches a stale object still within its SWR window, the readthrough cache interface executes a two-step process:

1. Immediate Response: The cache immediately delivers the stale object to both the leader and all follower requests to avoid latency.
2. Asynchronous Refresh: The cache schedules the leader request to fork and execute a background fetch to revalidate, or if deemed cacheable, replace the object.
   • Cacheability: The background revalidation process will deem the response to be cacheable if the origin fetch succeeds and beresp.cacheable is set to true at the end of vcl_fetch.
   • Variable Behavior: VCL subroutines executing within this background revalidation context are explicitly identifiable via the req.is_background_fetch VCL variable evaluating to true.

Each step invokes VCL subroutines as follows:

When a Compute application's code makes a request that matches a stale object still within its SWR window, the readthrough cache interface executes a two-step process:

1. Immediate Response: The cache immediately delivers the stale object to both the leader and all follower requests to avoid latency.
   • Hook Behavior: Because these requests are fulfilled directly from the cache, they do not invoke before-send or after-send hooks.
2. Asynchronous Refresh: The cache schedules the leader request to execute a background fetch to revalidate, or if deemed cacheable, replace the object.
   • Hook Behavior: Because this background fetch actively traverses the cache layer, it will invoke any before-send and after-send hooks.

In a CDN service, whenever a request is made through the readthrough cache interface during an SIE window:

• If the origin is sick (i.e. is currently failing a health check), the stale content will be served automatically.
• If the origin is erroring (i.e. responding with a valid HTTP response such as a 503 "service unavailable"), the Fastly platform invokes vcl_fetch and sets stale.exists to true. The stale content may be used by explicitly selecting it.
• If the origin is down/unreachable, the Fastly platform invokes vcl_error and sets stale.exists to true. The stale content may be used by explicitly selecting it.

When a request matches a stale object within its stale-if-error window, the cache interface attempts to contact the backend for a fresh copy, falling back to the stale asset only if the backend fetch fails.

1. Blocking Fetch: The interface initiates a blocking fetch to the backend.
   • Hook Behavior: This fetch traverses the cache, meaning the leader request invokes any before-send and after-send hooks. Follower requests pause and wait for the leader's outcome.
2. Outcome Evaluation: The cache processes the results based on the fetch outcome, as described below.

Fetch success case

If the origin returns an HTTP response (including 5xx status codes), it is considered successful. If the response is deemed cacheable, the cache interface updates the stale object in the cache. The leader and all follower requests receive the fresh backend Response.

If the response is a 5xx error or otherwise not usable according to the application, it may be preferable to use the stale content instead. To do this, use the after-send hook to check for the existence of stale content and then explicitly return an error result. The fetch is considered to have failed, and the cache follows the failure case below, blocking the cache update and returning the stale content to the Compute application.

Fetch failure case

The interface falls back and delivers the stale object to the application. The stale object is not updated in the cache.

Failures typically occur due to:

• Network issues: The backend is unreachable or a network error occurs.
• Unhealthy backend: The backend is currently failing health checks.
• Synthetic error: Your custom code explicitly returns an error from the after-send hook.

When your application has received the response, you can discern whether it is a fresh copy received from the backend or a stale copy returned from SIE by examining whether a "masked error" is attached to the Response, distributed as follows:

• The leader: Receives the actual error returned by the fetch (or the custom override error if one is generated by your after-send hook).
• The followers: Receive a distinct RequestCollapse error code, signaling that the leader's fetch failed and the cache fallback path was chosen.

• Stale content can be explicitly selected if a cached object exists and is within a stale-if-error (SIE) period, in the following subroutines:

  • In vcl_fetch: if the origin returns a response which is valid HTTP, then Fastly's cache will by default serve the received object, and if cacheable, use it to replace the stale object in cache. However, if the response is nonsensical or an error, you may prefer in that scenario to serve the stale content instead, by using return(deliver_stale).
  • In vcl_error: if, during a fetch to origin, Fastly's cache encounters a network level error, such as finding the origin unreachable or being unable to negotiate an acceptable TLS session, an error will be triggered and VCL control flow will be moved to vcl_error directly, without running vcl_fetch. By default, this will result in serving an error page generated by the Fastly platform to the end user, but if stale content exists in cache you can opt to use this instead by using return(deliver_stale) from vcl_error.
  • In vcl_miss: it is also possible to switch to a stale object in vcl_miss, but there is unlikely to be a reason to do so.

• The existence of stale content can be checked during the above subroutines using the stale.exists variable, which will only be true if a cached object exists and is within an SIE period. Stale content in the expired state cannot be used from VCL.

• If a backend fetch returns an HTTP response that includes a stale-while-revalidate or stale-if-error directive, those values can be read and overridden in the after-send hook.

• If a backend fetch returns an HTTP response (including 5xx status codes), the cache interface returns the response to the application, and if cacheable, uses it to replace the stale object in the cache.

  Stale content can be explicitly selected if a cached object exists and is within a stale-if-error (SIE) period. To do this, return an error from the after-send hook. The stale content is returned to the Compute application. The cached object is not updated and retains its status for subsequent requests.

• The existence of stale content can be checked during the after-send hook using the stale_if_error_available() function, which will be true if a cached object exists and is within an SIE period.

To summarize, these are the four possible freshness states that a piece of cached content can be in when it is matched by an incoming request:

• Fresh: Fastly's cache has a cached copy of the content, and it's within its initial freshness lifetime
• Stale-while-revalidate (SWR): Fastly's cache has a stale version of the content, and it's within a SWR period
• Stale-if-error (SIE): Fastly's cache has a stale version, and it doesn't qualify for SWR (or that period has already expired), but it's within a SIE period, allowing it to be used automatically if an origin is sick.
• None: Fastly's cache doesn't have the content or it's expired (content in this state will still allow for conditional fetches if it has an ETag or Last-Modified date)

And there are also four possible states that an origin server can be in:

• Healthy: The backend is up and working
• Erroring: The backend is returning syntactically valid HTTP responses with response status codes in the 5xx range (e.g., 503 "service unavailable")
• Down: The backend is unreachable, or is unable to negotiate a TCP connection
• Sick: Fastly's cache has marked this origin as unusable because it has consistently been unable to fetch from a health check endpoint. Origins in a down or erroring state that have a health check will eventually be transitioned to sick by the health check.

This results in 16 possible permutations, whose outcomes can be visualized as a grid:

The three possible outcomes are that the user will see the content they want served from the edge (😀), they'll get the content but including a blocking fetch to origin (😴), or they'll see an unfiltered error (😡), which could be either something generated by Fastly's platform or whatever your origin server returned.

To summarize, these are the four possible freshness states that a piece of cached content can be in when it is matched by an incoming request:

• Fresh: Fastly's cache has a cached copy of the content, and it's within its initial freshness lifetime
• Stale-while-revalidate (SWR): Fastly's cache has a stale version of the content, and it's within a SWR period
• Stale-if-error (SIE): Fastly's cache has a stale version, and it doesn't qualify for SWR (or that period has already expired), but it's within a SIE period, allowing it to be used automatically if an origin is sick.
• None: Fastly's cache doesn't have the content or it's expired (content in this state will still allow for conditional fetches if it has an ETag or Last-Modified date)

And there are also four possible states that an origin server can be in:

• Healthy: The backend is up and working
• Erroring: The backend is returning syntactically valid HTTP responses with response status codes in the 5xx range (e.g., 503 "service unavailable")
• Down: The backend is unreachable, or is unable to negotiate a TCP connection
• Sick: Fastly's cache has marked this origin as unusable because it has consistently been unable to fetch from a health check endpoint. Origins in a down or erroring state that have a health check will eventually be transitioned to sick by the health check.

This results in 16 possible permutations, which can be visualized as a grid to show where good and bad things happen:

The three possible outcomes are that the user will see the content they want served from the edge (😀), they'll get the content but including a blocking fetch to origin (😴), or they'll see an unfiltered error (😡), which could be either something generated by Fastly's platform or whatever your origin server returned.

The inadvertent caching of stale content at the edge POP due to these edge cases can be prevented in CDN services by disabling the use of stale content for asynchronous revalidation when a POP is acting as a shield:

if (fastly.ff.visits_this_service > 0) {
  set req.max_stale_while_revalidate = 0s;
}

This code will continue to allow stale content to be used when an origin is sick. To disable that as well, set req.max_stale_if_error to 0s.

The inadvertent caching of stale content at the edge POP due to these edge cases can be prevented in Compute services by disabling the use of stale content for asynchronous revalidation when a POP is acting as a shield:

1
2
3
4
5
6
7
8
9
35
36
37
38
39
40
41

let shield = match Shield::new("<YOUR SHIELD POP>") {
  Ok(v) => v,
  Err(e) => return Ok(Response::from_status(StatusCode::INTERNAL_SERVER_ERROR)
      .with_body(format!("Could not find shield '<YOUR SHIELD POP>': {:?}", e))),
};

if !shield.running_on() {
  // If we're not running on the shield POP, look up the encrypted
  // tunnel to that backend, and try to forward it on.
L10 : L34
 
 
}
// if we get here, then we're running on the shield!

let resp = req
    .with_stale_while_revalidate(0) // Set stale-while-revalidate to 0 seconds
    .send(backend)?;
return resp;

1
2
3
4
5
6
7
15
16
17
18
19
20
21
22
23
24
25
26

import { Shield } from "fastly:shielding";

async function handleRequest(event) {
    const shield = new Shield('<YOUR SHIELD POP>');
    if (!shield.runningOn()) {
        // If we're not running on the shield POP, look up the encrypted
        // tunnel to that backend, and try to forward it on.
L8 : L14
 
 
    }
    // if we get here, then we're running on the shield!

    let resp = fetch(event.req, {
            backend
            cacheOverride: new CacheOverride({
                staleWhileRevalidate: 0, // Set stale-while-revalidate to 0 seconds
            }),
        });

    return resp
}
L27 : L28
 
 

1
7
8
10
11
12
13
14
15
16
17
18
19
20
21
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

import (
L2 : L6
 
 
    "github.com/fastly/compute-sdk-go/shielding"
)
L9 : L9
 
 
func main() {
    fsthttp.ServeFunc(func(ctx context.Context, w fsthttp.ResponseWriter, r *fsthttp.Request) {
        shield, err := shielding.ShieldFromName("<YOUR SHIELD POP>")
        if err != nil {
            w.WriteHeader(fsthttp.StatusInternalServerError)
            fmt.Fprintln(w, err.Error())
            return
        }

        if !shield.IsRunningOn() {
            // If we're not running on the shield POP, look up the encrypted
            // tunnel to that backend, and try to forward it on.
L22 : L44
 
 
        }
        // if we get here, then we're running on the shield!

        r.CacheOptions.StaleWhileRevalidate = 0 // Set stale-while-revalidate to 0 seconds

        resp, err := r.Send(ctx, backend)
        if err != nil {
            w.WriteHeader(fsthttp.StatusBadGateway)
            fmt.Fprintln(w, err.Error())
            return
        }

        w.Header().Reset(resp.Header)
        w.WriteHeader(resp.StatusCode)
        io.Copy(w, resp.Body)
    })
}

Note that the above code will continue to allow stale content to be used when an origin is sick.

To disable that as well, also set the stale-if-error lifetime to 0s:

38
39
40
41
42

L1 : L37
 
 
let resp = req
    .with_stale_while_revalidate(0) // Set stale-while-revalidate to 0 seconds
    .with_stale_if_error(0)         // Set stale-if-error to 0 seconds
    .send(backend)?;
return resp;

18
19
20
21
22
23
24
25
26

L1 : L17
 
 
    let resp = fetch(event.req, {
            backend
            cacheOverride: new CacheOverride({
                staleWhileRevalidate: 0, // Set stale-while-revalidate to 0 seconds
                staleIfError: 0,         // Set stale-if-error to 0 seconds
            }),
        });

    return resp
L27 : L29
 
 

48
49
50
51

L1 : L47
 
 
        r.CacheOptions.StaleWhileRevalidate = 0 // Set stale-while-revalidate to 0 seconds
        r.CacheOptions.StaleIfError  = 0        // Set stale-if-error to 0 seconds

        resp, err := r.Send(ctx, backend)
L52 : L62
 
 

• Specify a short stale-while-revalidate and a long stale-if-error value. If your origin is working, you don't want to subject users to content that is significantly out of date. But if your origin is down, you're probably much more willing to serve something old if the alternative is an error page.
• Always include a validator (an ETag or Last-Modified header) on responses from origin.
• Use shielding to increase the cache hit ratio and increase the probability of having stale objects to serve.
• Always use soft purges to ensure stale versions of objects aren’t also evicted.

• Specify a short stale-while-revalidate and a long stale-if-error value. If your origin is working, you don't want to subject users to content that is significantly out of date. But if your origin is down, you're probably much more willing to serve something old if the alternative is an error page.
• Always include a validator (an ETag or Last-Modified header) on responses from origin.
• Always use soft purges to ensure stale versions of objects aren’t also evicted.