Customize timeouts for faster automatic failover on AI Gateway
AI Gateway now supports per-inference timeouts to enable faster failover to alternative providers if the initial provider doesn't respond within your configured time limit. This beta feature is available for BYOK credentials with support for system provider timeouts coming soon.
AI Gateway now supports per-inference timeouts for faster failover than the provider default. If a provider doesn't start responding within your configured timeout, AI Gateway aborts the request and falls back to the next available provider.provider
Provider timeouts are available in beta for BYOK (Bring Your Own Key) credentials only, with support for system provider timeouts coming soon. Note that some providers don't support stream cancellation, so you may still be charged for timed-out requests depending on the provider.
Basic usage
Set timeouts per provider in milliseconds using in .providerTimeoutsproviderOptions.gateway
Advanced usage with multiple providers and failover
Use with to control both the provider sequence and failover speed.order
For more information, read the custom provider timeouts .documentation
Source: original entry ↗
More from Vercel
Follow Vercel to get its new changes in your feed and email digest.
OpenAI Decisions API now available on AI Gateway
OpenAI's Decisions API is now accessible through Vercel's AI Gateway with an OpenAI-compatible endpoint, enabling decision models to answer typed questions and return probabilities, choices, and scores for routing, triage, and guardrails use cases. Support is available across the OpenAI SDK, AI SDK, HTTP API, and CLI with the latest versions.
Timestamp attributes now supported in Vercel Flags
Vercel Flags now supports timestamp attributes for entities, allowing you to create time-based targeting rules. Use this feature to run limited-time campaigns, show content between specific dates, or target users based on registration date.
Glyph Cluster now available in stealth on AI Gateway
Glyph Cluster, a reasoning model for coding and long-context analysis, is now available as a stealth model on Vercel's AI Gateway for Pro and Enterprise plan teams with purchased AI Gateway credits at no cost during the stealth period. The model supports function calling, streams responses, and can be accessed via AI SDK, OpenAI-compatible APIs, and coding agents.