Building Scalable Waiting Rooms with Fastly Compute

Imagine your ecommerce site is running a flash sale, or your ticketing platform is releasing seats for a stadium concert. Thousands of users flood your endpoints simultaneously, threatening to overwhelm your origin servers and create a poor customer experience that often leads to frustrated users staring at 404 errors or broken pages.

Waiting rooms are a perfect solution for controlling the flow of visitors to your site, maintaining site stability, and optimizing the shopping experience. In the case of ticket sales for high-demand events, waiting rooms provide a way to fairly queue customers and prevent overselling. This blog will provide an overview of a cool project that focused on building a waiting room app on top of Fastly Compute and how you can leverage it for your own use case.

Goals for our Waiting Room Project:

• Queue Management: The app should be able to efficiently handle large numbers of concurrent users, placing them in a virtual queue to prevent server overload.

• Real-time Updates: Users should be able to see their position in the queue and estimated wait time, which should update automatically.

• Configurable Settings: Allow for the ability to easily adjust queue settings like refresh intervals and automatic user admission rates.

• Anti-Fraud Measures: Include built-in security features to prevent queue manipulation and ensure fair access.

• Multi-Queue Support: Offer the ability to handle different paths or events with separate queues.

How Fastly Compute Powers Scalable Waiting Rooms

The waiting room app uses Fastly Compute in conjunction with Upstash's Redis database. Upstash has an API integration with Fastly, and using this type of stateful database with atomic operations ensures that every operation that is written to it is guaranteed to succeed in the order in which it was received. Here's a high-level overview:

1. When a user hits a protected endpoint, the app checks if a queue is active for that path. If active, the user is assigned a unique identifier and placed in the queue.

2. The app uses Redis to track queue position, length, and cursor (next position to be served).

3. Users receive a cookie with their queue information, which refreshes periodically to maintain their spot.

The app can automatically admit users at a set rate or allow manual admission in batches.

Figure 1: Waiting Room user request flow (see steps below)

1. A user makes a request to the site (which is fronted by Fastly).

2. The request is routed to the Fastly Compute service. (This is the main application logic running at the edge. It contains the code that decides whether to queue the user or admit them.)

3. The Compute logic checks if the Waiting Room is ACTIVE and if the user has a Valid Token (cookie).

4. If no valid token exists, Compute interacts with Upstash Redis to get the current queue length, assigns the user a new queue position, and issues a token cookie.

5. If the user is not yet admitted, Compute uses a page template stored in the Fastly KV Store to immediately serve the Waiting Room page from the edge, ensuring the origin is never hit.

6. If the token is valid and the user has reached the front of the queue (Token Live? = Yes), Compute forwards the request to the Origin Server.

Ways to Customize the Waiting Room:

1. Align the template to reflect your brand: Customize the waiting room page to match your brand using Fastly's KV Store.

2. Geo-Targeting: Configure country-specific queues for localized events.

3. Multiple Queues: Set up different queues for various sections of your site or different events.

4. Admission Strategies: Implement custom logic for how users are admitted from the queue.

Building Your Fastly Compute Waiting Room

To build your own waiting room with Fastly Compute:

1. Set up a Fastly Compute service

2. Create an Upstash Redis database

3. Configure Fastly's KV store for queue settings and page templates

4. Implement the core logic using JavaScript, Rust, or Go

5. Test thoroughly with simulated traffic

Protect Your Infrastructure and Enhance the User Experience

By leveraging Fastly Compute and a fast, distributed database, you can create robust solutions for managing high-traffic scenarios. We encourage you to explore this pattern and adapt it to your specific use cases.

Whether you're handling ticket sales, product launches, or any high-demand event, a well-implemented waiting room can significantly improve the user experience and protect your infrastructure. Happy coding, and let us know what amazing things you build with Fastly Compute!