Sort providers by cost, latency, or throughput on AI Gateway
AI Gateway now supports sorting providers by cost, time to first token (TTFT), or throughput (TPS), allowing explicit control over which provider is selected for each request. Ranking is computed at request time so price changes and latency shifts flow through automatically without code changes.
You can now sort the providers behind a model by cost, time to first token (TTFT), or throughput (TPS) in .AI Gateway
The default provider order blends provider reliability, quality of model output, cost, and speed of response. You can now use for explicit control over ranking criteria.sort
For models with many providers and noticeable cost or speed variation, you can use to optimize on your dimension of choice. Ranking is computed at request time, so newly added providers, price changes, and shifts in observed latency or throughput flow through automatically without any code changes.sort
Set on to one of the three values:
sortproviderOptions.gateway
Use to ensure optimizing for your metric of choice.sort
In this example, AI Gateway has over five providers for with different prices, so sorting by cost is a useful option for requests that want to route through the lowest price provider.GPT OSS 120B
Providers are tried in sort order. Fallback to the next provider only happens when the higher-ranked one is unavailable.
is compatible with other gateway routing options like Zero Data Retention (ZDR).sort
The example below uses for an interactive request where latency and data retention matter: AI Gateway filters to only providers for that have zero data retention, and then sorts the remaining providers by time to first token (TTFT).deepseek/deepseek-v4-proDeepseek V4 Pro
also composes with : providers listed in are promoted to the front, and the remaining providers follow the requested sort criterion.sortorderorder
See exactly why each request landed where it did. Every response includes a block in the routing metadata showing which providers were considered, the metric values used to rank them, the order they were attempted, and any that were deprioritized due to degraded health.sort
For more information on sorting via AI Gateway, read the .documentation
Value | Description | Direction | When to use |
| Sort by the provider's listed input price per million tokens | Lowest price first | High-volume, cost-sensitive work |
| Sort by median time to first token, in ms | Lowest latency first | Latency-sensitive workloads where response speed matters |
| Sort by median tokens per second throughput | Highest first | Long-output generation where total response time matters most |
Basic usage
Combine with other routing controls
Inspecting routing decisions
Source: original entry ↗
More from Vercel
Follow Vercel to get its new changes in your feed and email digest.
OpenAI Decisions API now available on AI Gateway
OpenAI's Decisions API is now accessible through Vercel's AI Gateway with an OpenAI-compatible endpoint, enabling decision models to answer typed questions and return probabilities, choices, and scores for routing, triage, and guardrails use cases. Support is available across the OpenAI SDK, AI SDK, HTTP API, and CLI with the latest versions.
Timestamp attributes now supported in Vercel Flags
Vercel Flags now supports timestamp attributes for entities, allowing you to create time-based targeting rules. Use this feature to run limited-time campaigns, show content between specific dates, or target users based on registration date.
Glyph Cluster now available in stealth on AI Gateway
Glyph Cluster, a reasoning model for coding and long-context analysis, is now available as a stealth model on Vercel's AI Gateway for Pro and Enterprise plan teams with purchased AI Gateway credits at no cost during the stealth period. The model supports function calling, streams responses, and can be accessed via AI SDK, OpenAI-compatible APIs, and coding agents.