---
title: Gateway
summary: The <a href="https://www.fastly.com/documentation/reference/api/ai-runtime-control">AI Runtime Control</a> gateway forwards your requests to AI model providers. You authenticate to the gateway with a <a href="https://www.fastly.com/documentation/reference/api/ai-runtime-control/virtual-keys">virtual key</a>, and the gateway uses the credentials configured on that key to reach the provider for you. Most gateway endpoints accept the same requests and return the same responses as the provider APIs OpenAI Chat Completions, OpenAI Responses APIs, Anthropic Messages API, and Amazon Bedrock Runtime <code>InvokeModel</code> and <code>InvokeModelWithResponseStream</code> APIs. This means you can keep using your existing provider client and change only its base URL and credentials. Specify models by name, for example <code>gpt-4</code> or <code>claude-sonnet-4-20250514</code>. Bedrock endpoints work differently. They put the Bedrock model ID or ARN in the request path instead of a <code>model</code> field, and authenticate with a <code>Bearer</code> token instead of signing requests with AWS Signature Version 4 (SigV4). Each endpoint follows its provider's request and response format, so see that provider's documentation for the full list of supported fields.
url: https://www.fastly.com/documentation/reference/api/ai-runtime-control/gateway
---

The <a href="https://www.fastly.com/documentation/reference/api/ai-runtime-control">AI Runtime Control</a>
gateway forwards your requests to AI model providers. You authenticate to the gateway with a
<a href="https://www.fastly.com/documentation/reference/api/ai-runtime-control/virtual-keys">virtual key</a>,
and the gateway uses the credentials configured on that key to reach the provider for you.
Most gateway endpoints accept the same requests and return the same responses as the provider
APIs OpenAI Chat Completions, OpenAI Responses APIs, Anthropic Messages API, and Amazon
Bedrock Runtime <code>InvokeModel</code> and <code>InvokeModelWithResponseStream</code> APIs. This means you can keep
using your existing provider client and change only its base URL and credentials.
Specify models by name, for example <code>gpt-4</code> or <code>claude-sonnet-4-20250514</code>. Bedrock endpoints
work differently. They put the Bedrock model ID or ARN in the request path instead of a <code>model</code> field,
and authenticate with a <code>Bearer</code> token instead of signing requests with AWS Signature Version 4 (SigV4).
Each endpoint follows its provider's request and response format, so see that provider's
documentation for the full list of supported fields.

## Data model

| Field | Type | Description |
|-------|------|-------------|
| `error` | object |  |
| `messages` | array | The conversation so far. |
| `model` | string | The model to use, addressed by its bare model name. The provider is determined by the virtual key's configured provider. Failover targets are only tried if that primary request fails. |
| `stream` | boolean | Whether to stream the response as server-sent events. |
| `choices` | array |  |
| `input_tokens` | integer | The number of input tokens the request would consume. |
| `max_tokens` | integer | The maximum number of tokens to generate. |
| `input` | string | The text to synthesize into speech. |
| `speed` | number | The speech speed multiplier. |
| `voice` | string | The voice to use. |
| `file` | string | The audio file to transcribe (up to 25 MB). |
| `language` | string | The language of the audio, as an ISO 639-1 code. |
| `text` | string | The transcribed text. |

## API endpoints

### Create chat completion

**POST** `/v1/chat/completions`

Creates a chat completion using the OpenAI Chat Completions wire format. Supports both
streaming (server-sent events) and non-streaming responses. See the
<a href="https://platform.openai.com/docs/api-reference/chat">OpenAI Chat Completions API reference</a>
for the full request and response shape.

### Create response

**POST** `/v1/responses`

Creates a response using the OpenAI Responses API wire format. Supports both streaming
(server-sent events) and non-streaming responses. See the
<a href="https://platform.openai.com/docs/api-reference/responses">OpenAI Responses API reference</a>
for the full request and response shape.

### Create message (Anthropic format)

**POST** `/v1/messages`

Creates a message using the Anthropic Messages API wire format. Accepts either an
<code>Authorization: Bearer</code> header or an <code>x-api-key</code> header for authentication. Supports
both streaming (server-sent events) and non-streaming responses. See the
<a href="https://docs.claude.com/en/api/messages">Anthropic Messages API reference</a> for the
full request and response shape.

### Count tokens

**POST** `/v1/messages/count_tokens`

Counts the input tokens a Messages API request would consume, without sending the
request to the provider.

### Create speech

**POST** `/v1/audio/speech`

Generates audio from text input, using the OpenAI text-to-speech wire format. Returns
raw audio bytes in the requested format.

### Create transcription

**POST** `/v1/audio/transcriptions`

Transcribes an audio file to text, using the OpenAI transcription wire format. Accepts
multipart form data with an audio file (up to 25 MB).

### List models

**GET** `/v1/models`

Lists the AI models available through your configured providers and provider connections.

**Parameters:**

| Name | Type | In | Required | Description |
|------|------|----|---------|--------------|
| `provider` | string | query | No | Filter results by provider name. |

### Bedrock InvokeModel

**POST** `/model/{modelId}/invoke`

A non-streaming, Amazon Bedrock Runtime <code>InvokeModel</code>-compatible endpoint. Request and
response bodies use the Anthropic Messages API format. See the
<a href="https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html">Bedrock Runtime InvokeModel reference</a>
for the wrapping request/response envelope.

**Parameters:**

| Name | Type | In | Required | Description |
|------|------|----|---------|--------------|
| `modelId` | string | path | Yes | The Bedrock model ID or ARN. Percent-encode it when building the request path. Most HTTP client libraries do this automatically. |

### Bedrock InvokeModelWithResponseStream

**POST** `/model/{modelId}/invoke-with-response-stream`

A streaming, Amazon Bedrock Runtime <code>InvokeModelWithResponseStream</code>-compatible endpoint.
The request body uses the Anthropic Messages API format. The response is an AWS
EventStream binary stream carrying base64-wrapped Anthropic streaming events. See the
<a href="https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html">Bedrock Runtime InvokeModelWithResponseStream reference</a>
for the streaming envelope.

**Parameters:**

| Name | Type | In | Required | Description |
|------|------|----|---------|--------------|
| `modelId` | string | path | Yes | The Bedrock model ID or ARN. Percent-encode it when building the request path. Most HTTP client libraries do this automatically. |