Guides
Retries
What the gateway retries for you, and how to replay safely on your side.
The gateway retries internally before it ever returns a failure. When an error reaches your code, the gateway has already exhausted its own attempts for that request.
What the gateway retries
| Situation | Handling |
|---|---|
| Provider 429 / rate limit | Retry with exponential backoff, up to 2 attempts |
| Provider 5xx or gateway timeout | Retry once on a different regional edge if available |
| Timeout while waiting for the first token | Fail fast with a retryable error |
| Your request body invalid | Never retried — a 422 comes straight back |
These defaults are per request. They are not configurable in v1; per-client overrides are on the roadmap.
Provider fallback
When routing is automatic and a provider fails, the gateway may complete your request on another provider instead of retrying the same one. The response reports the actual provider so you always know who answered.
Forced providers skip fallback
If you force provider: "anthropic", requests never fall back to another provider — the failure surfaces instead.
Retrying on your side
async function chatWithRetry(client, messages, attempts = 3) {
for (let i = 0; i < attempts; i++) {
try {
return await client.chat({ messages });
} catch (err) {
if (!err.retryable) throw err;
const backoff = Math.min(1000 * 2 ** i, 8000);
await sleep(backoff + Math.random() * 250);
}
}
throw new Error("still failing after retries");
}Don't double-retry
One layer of retries
Because the gateway already retries, keep client retries count low (2–3) or the same slow request will multiply its attempts across both layers.