ReferenceReference
Limits and timeouts
Body size, request timeouts, concurrency, and rate limits.
By the end: You can choose client settings that never trip a gateway limit.
| Field | Value | Notes |
|---|---|---|
| Body size | 1 MiB | BODY_LIMIT_BYTES. Larger bodies get 413. |
| Request timeout | 120 s | UPSTREAM_TIMEOUT_MS, for non-streaming requests. |
| Stream timeout | 600 s | UPSTREAM_STREAM_TIMEOUT_MS, for streaming requests. |
| Concurrent requests | 8 per account | An open stream holds one slot. |
| Requests per minute | 60 per account, 30 per key | Configurable per deployment. |
| Key length | 128 characters | Longer values are rejected without a lookup. |
Why these exist
Timeouts and body limits protect the gateway and the upstream service from requests that cannot succeed. They are not billing controls — max_tokens is.
Choosing client settings
- Set a client timeout below the gateway timeout so you learn about a slow request first.
- Stream anything that may exceed a minute.
- Send attachments by reference where the provider supports it, rather than inline.