serve

Support for reusable sandboxes.

Normally, each incoming HTTP request spins up a new Compute instance. This provides good reproducibility properties... but it means that if an instance has expensive initialization, that initialization must be repeated for each request. If a service has an expensive initialization routine, reusable instances can provide an efficiency improvement (and a cost improvement!)

Timeouts

A Compute instance is not a general-purpose server; it does not and should not indefinitely wait for new requests. Therefore, the time spent waiting for a new request is bounded in three different ways.

• At the platform level, there is a maximum time over which a RequestPromise can be unfulfilled. The timer starts when the RequestPromise is created. After this timeout, RequestPromise::wait returns NextRequestError::EndOfSession, and the promise is no longer valid.

  This timeout is not publicly disclosed, and may change.

• When a RequestPromise is created, the promise-level timeout can be configured in NextRequestOptions::timeout. The effective promise-level timeout is the minimum of the provided value and the platform-level timeout.

• Separately from the promise-level timeout, wait_timeout can be used to time out a particular wait call. If wait_timeout times out while the promise is still valid, wait_timeout will return RequestPromise::Timeout, indicating the promise can be retried.

Billing

When a WASM instance waits on a RequestPromise, it continues to consume memory and wall-clock time. The memory is only freed when the WASM instance exits, as usual.

If your service is billed by WASM memory * wall-clock time, you will be billed for the usage while waiting for a RequestPromise, even if the RequestPromise does not resolve to a request (i.e. times out). You can limit this usage by timing out the RequestPromise, as described above.

A WASM instance that is blocked waiting on a RequestPromise does not consume vCPU time.

Examples

1
2
3
4
5
6
7
8
9
10
11
12

fn handler(_req: Request) -> Result<Response, Error> {
    Ok(Response::from_body("hello")
        .with_header("hello", "world!")
        .with_status(200))
}

fn main() -> Result<(), Error> {
    Serve::new()
        .with_max_requests(5)
        .run(handler)
        .into_result()
}