AI Gateway adds request timeouts and retries

AI Gateway adds additional ways to handle requests - Request Timeouts and Request Retries, making it easier to keep your applications responsive and reliable.

Timeouts and retries can be used on both the Universal Endpoint or directly to a supported provider.

Request timeouts A request timeout allows you to trigger fallbacks or a retry if a provider takes too long to respond.

To set a request timeout directly to a provider, add a cf-aig-request-timeout header.

Provider-specific endpoint examplebash

curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/workers-ai/@cf/meta/llama-3.1-8b-instruct \
 --header 'Authorization: Bearer {cf_api_token}' \
 --header 'Content-Type: application/json' \
 --header 'cf-aig-request-timeout: 5000'
 --data '{"prompt": "What is Cloudflare?"}'

Request retries A request retry automatically retries failed requests, so you can recover from temporary issues without intervening.

To set up request retries directly to a provider, add the following headers:

• cf-aig-max-attempts (number)
• cf-aig-retry-delay (number)
• cf-aig-backoff ("constant" | "linear" | "exponential)