---
title: Stateful waiting room
summary: >-
  You have regular large volumes of traffic and need to limit the rate at which
  users can start new sessions. Once a user has been allowed in, they should
  retain access.
url: >-
  https://www.fastly.com/documentation/solutions/tutorials/compute/stateful-waiting-room
---

The concept of a waiting room is similar to other kinds of rate limiting, but differs in that it is applied at a session level, and users must earn access to the site by waiting some amount of time. Waiting rooms can get complicated - especially if you want people to have a numbered position in the queue, or if you want to allow people access based on availability of a resource, such as allowing only a fixed number of baskets to enter the checkout stage at a time. Fastly also offers the more common form of [rate limiting](https://www.fastly.com/documentation/guides/concepts/rate-limiting) in VCL services if you want to prevent individual users issuing large volumes of requests in a short period.

For this solution we'll show how you can create a stateful, ordered queue as a waiting room at the edge of the network, holding back eager users and ensuring that you don't overwhelm your infrastructure, and allowing them to be granted access in the order in which they arrived at the website.

## Instructions

The waiting room principle we're demonstrating here is like a ticket system at a shop counter. The system needs to know two numbers: the number of the user who has just reached the front of the queue (think of this as the "now serving" digital sign), and the number of the user at the back of the queue (which is like the number of the next ticket to come out of the ticket machine). The people in the waiting room need to know what number they are, but the system doesn't need to remember that information, as long as we make sure people can't cheat.

To implement such a mechanism, you need a state store that can atomically increment a counter. One of the most popular choices for this is [Redis](https://redis.io), which is a powerful in-memory data store. As the Redis network protocol is not HTTP-based, in order to be able to speak to Redis from a Compute service you need a HTTP interface on top of it. [Upstash](https://upstash.com) is a Redis-as-a-service provider that offers such an interface along with a generous free tier, so we'll use that for this example.

Redis will store two entries representing the two numbers we want to track - position markers for the beginning and end of the queue. When a new user that we've never seen before makes a request, we increment the back-of-queue counter, and serve them a 'holding' page (from the edge) along with a tamper-proof cookie that stores their position in the queue. When that user makes subsequent requests (with their queue token in the cookie), we can check whether the front-of-queue counter has yet reached their number. If it has, we forward their request to the backend.

Some things we want from such a solution are:

- Support changing the secret used for signing visitor positions, so the queue can be reset
- Make it hard for users to get multiple spots in the queue
- Allow requests and responses to stream through Fastly without buffering
- Ensure visitors end up accessing the site in the same order they joined the queue (i.e. it is fair)
- Allow the flow rate of the queue to be adjusted quickly

Let's dive in.

### Create a Compute project

This solution is also available as a [starter kit](https://github.com/fastly/compute-starter-kit-javascript-queue) but for the purposes of this tutorial we will assume you want to start from scratch. Create an empty JavaScript project:

```term
$ mkdir waiting-room && cd waiting-room
$ fastly compute init --from https://github.com/fastly/compute-starter-kit-javascript-empty

Creating a new Compute project (using --from to locate package template).

Press ^C at any time to quit.

Name: [waiting-room]
Description: A waiting room powered by JavaScript at the edge
Author: My Name <me@example.com>

✓ Initializing...
✓ Fetching package template...
✓ Updating package manifest...
✓ Initializing package...

SUCCESS: Initialized package waiting-room
```

Open `src/index.js` in your code editor of choice and you should see a small Compute program with an empty request handler.

```js title="src/index.js"
/// <reference types="@fastly/js-compute" />

addEventListener("fetch", (event) => event.respondWith(handleRequest(event)));

async function handleRequest(event) {
  return new Response("OK", { status: 200 });
}
```

### Define some configuration

There are a bunch of things about the way your queue behaves that you may need to change easily, so it's a nice idea to create a configuration file to hold these properties. Copy this configuration code into a file named `config.js` in your `src` directory:

Since this code fetches some parameters from a [dictionary](https://www.fastly.com/documentation/guides/full-site-delivery/dictionaries/about-dictionaries), you need to configure a dictionary for the local testing server to use. Create a file named `devconfig.json` with the following fields:

> **IMPORTANT:** Make sure to add this file to `.gitignore` so you don't accidentally commit secrets to your repository.

And make sure to refer to it in your `fastly.toml` so the local testing server can fetch the configuration:

> **HINT:** A benefit to storing the JWT signing key in a dictionary is that if you have granted access to too many users, and you want a mechanism to apply an 'emergency reset', changing the secret would terminate the access of all users whose sessions have been signed with that key.

### Create a queue page

If you block a visitor's request, it would be confusing for them to see a browser error. Instead, you should serve a webpage to indicate that they should wait.

![A branded waiting room, showing the header "Sometimes we have to restrict the number of people who can access our website at the same time, so that everything works properly." with a subtitle "There are 3288 people ahead of you in the queue."](/img/queue.png)

Since the page needs to include some dynamic content (visitor's current queue position), we'll whip up a rudimentary page templating function (you could also use an off-the-shelf dependency here like Mustache or Handlebars).

You can add this to your `index.js` or even split it out into a separate file. Here's an example:

Now build a template that uses this interpolation syntax:

You could serve this page from your origin server but we find that customers prefer to host the waiting room content purely at the edge. You're going to be serving these responses a lot, and they are the first line of defense against surges of traffic.

> **HINT:** Remember that for the same reason you want to serve the waiting room HTML page from the edge, you'll want to serve any images, stylesheets and scripts from the edge too. Don't accidentally overload your origin by having your waiting room page load an uncached stylesheet from your origin server!

### Handle permitted and blocked requests

Before writing the logic to decide if a visitor should be given access, make sure the service can pass traffic to the origin as well as serve the queue page.

Back in the main file, write a function that passes a permitted request to the origin, making sure to update the backend name to match your service:

And another function that returns a queue page when the request is unauthorized:

Note that here we're using the `processView` method written earlier to interpolate dynamic data into the queue page template. This code also sends a `Refresh` header in the response which will cause the browser to reload the page after the given interval.

### Deal with decisions that don't require a token

Sometimes you'll want to allow requests regardless of whether the requestor has waited in the queue. For example, on the queue page, you might want to allow requests to a static asset from the backend such as a CSS file (as discussed earlier, make sure these assets are highly cacheable!).

For this purpose, build an allowlist of paths that you want to permit:

At the top of the `handleRequest` function, parse the request URL and check if the request path is in the allowlist:

### Extract data from cookie

To determine if a request should be permitted, you need to be able to determine the visitor's position in the queue.

Write a function to extract the cookie from the request headers:

Install the `jws` package to decode the cookie:

```term
$ npm install jws
```

And make sure to import it at the top of `index.js`

Then use the `getQueueCookie` function to extract the cookie from the request headers. Place this code after the allowlist check:

When parsing the JWT, this validates that the signature is valid and that the cookie has not expired.

### Connect to Redis

In order to determine if the visitor should be permitted to make requests to the origin, you need to know their position in the queue, which is the difference between their assigned number and the current queue cursor.

For the purposes of this tutorial, this state will be stored in Upstash, a Redis-as-a-service provider that offers a REST API, perfect for running commands from the Compute platform.

If you haven't already, [sign up](https://console.upstash.com/) for Upstash and create a Redis service.

Next, install the Upstash JavaScript client:

```term
$ npm install @upstash/redis
```

You can now write some helper functions to fetch and set data within the store. It's recommended to put this in a separate file, such as `store.js`:

After your existing logic in `index.js`, initialize the store:

### Deal with invalid tokens

If the cookie either does not exist, is not valid, or has expired, `isValid` will be set to false. In this case you need to create a new cookie for the visitor.

Create another function, this time to set a cookie:

Then use the `setQueueCookie` function to set the cookie in the response, incrementing a counter within Redis to get the next queue position:

When sending a response later, you will check for the existence of `newToken` and if it is set, set the cookie in the response.

### Fetch the current queue cursor

Using the data from the store, the service can now make a decision about whether to permit the incoming request:

Make sure to set a new cookie, if required, before returning the response:

This is a great point to stop and try out what you've built.

```term
$ fastly compute serve --watch
```

Go to <http://127.0.0.1:7676/> and you should be greeted by the queue page. However, you will find that you are queuing indefinitely and your requests will not pass to the origin.

### Controlling the queue

The next step is to build a mechanism to let visitors in. There are a few ways to do this:

- **Manual:** A human manually lets visitors in.
- **Scheduled:** Visitors are let in at a fixed interval.
- **Origin managed:** Your server calls an endpoint to let people in when its load is below a certain threshold.

For the purposes of this tutorial, we will implement capabilities that allow for all three of these methods:

#### Create an admin panel

The most obvious way to keep visitors flowing in to your site is to manually increment the queue cursor. However, you definitely don't want any random visitor to be able to do this.

[HTTP Basic Auth](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication) is a really easy-to-implement authentication mechanism that should work just fine for this purpose. To support this, add a new section to the configuration:

Keep in mind that credentials are base64-encoded when using HTTP Basic Auth, so you will need a library to encode/decode them, unless you want to hardcode a base64 string:

```term
$ npm install base-64
```

Make sure to import the library so it is available:

Back in the main function, check if the request path is within the admin section. You should do this in the same place that you check for allowlisted paths, as there is no point in making yourself queue for the admin endpoints.

Now you can write a function to handle requests made to the admin endpoints. First, validate that the password has been provided.

Now implement an interface for letting a specified number of visitors in:

Finally, serve a web page with a form that allows the above endpoint to be `POST`ed to.

Create a new HTML template:

Import it into your JavaScript:

Then serve it in the `handleAdminRequest` function:

This works really well if you have a backend that can constantly ensure that the queue cursor is kept up to date, but if you are trying to protect a backend that you don't have much control over, this can be unfeasible. Instead, the waiting room should automatically let visitors in every few seconds.

#### Enable automatic mode

The Compute platform does not allow you to schedule a task to run at a specific time. However, a website in need of a waiting room can assume that there will be requests coming in regularly. You can take advantage of this by rounding the current time, and checking it against a counter of requests within that given time period. If a given request is the first one within a period, you can run some extra code to let extra visitors in.

The first step is to write some configuration for this functionality. Add the `automatic` and `automaticQuantity` fields to the configuration:

You also need some extra logic in the `store.js` file to handle tracking the requests for each time period:

Now, when a request is received but not permitted, you can check if there have been any requests in the current period and if not, allow the visitor in.

If you refresh, the queue should now automatically progress and eventually let you access the origin.

### Log the request and response

Waiting rooms are complex and you're probably going to want to do some logging.

Not only does this help with debugging during the development process, but you can feed this data to a monitoring tool such as Grafana or New Relic to monitor your queue in real-time once you deploy to Fastly.

![A dashboard on New Relic showing the current amount of visitors waiting, a graph of visitors waiting plotted against visitors permitted, and request logs](/img/monitoring.png)

Add a `logging.js` file to allow for the output of structured log messages for each request:

This function is collecting as much relevant information about the request and response as possible, including information about the Fastly cache node that the service is running on, and the visitor's geolocation data.

Define a [log endpoint](https://www.fastly.com/documentation/guides/integrations/non-fastly-services/developer-guide-logging) to send the logs to:

Make sure to call the `log` function before returning the response:

## Next steps

- Consider how you could harden this queue by limiting entries per IP address or requiring a CAPTCHA or other challenge on the queue page.
- If you have globally distributed backends, you could split the queue up into smaller regional queues.
- If using automatic mode, calculate the estimated time that a visitor will remain waiting.
- Provide a way for VIP visitors to jump the queue with a one-time-use code or URL.