WebSocket support for OpenAI Responses API on AI Gateway
AI Gateway now supports WebSocket mode for the OpenAI Responses API, allowing persistent connections to send incremental input instead of re-sending full context on each turn. This enables up to 40% faster end-to-end execution for agentic rollouts with multiple tool calls.
now supports WebSocket mode for the OpenAI Responses API. You can keep a persistent connection open and continue each turn by sending only new input items plus , instead of re-sending the full context over a fresh HTTP request every turn.AI Gatewayprevious_response_id
up to ~40% faster end-to-end execution on WebSockets for agentic rollouts with 20 or more tool calls.OpenAI reports
The Responses route opens at and accepts raw frames over a persistent WebSocket connection. The mode is compatible with and , and follows OpenAI's specification upstream.GET /v1/responsesresponse.createstore=falseZero Data RetentionWebSocket mode
See the for setup and the full route list.AI Gateway documentation
Route Responses API traffic over WebSocket
Source: original entry ↗