Troubleshooting·Troubleshooting

Streaming drops and missing usage

Why a stream can end early or arrive without token counts.

The stream ends early

A stream closes when the upstream finishes, when the timeout is reached, or when the client disconnects. A reverse proxy that buffers will make a stream appear to hang and then arrive all at once; the gateway sends x-accel-buffering: no to prevent that, but a proxy you add in front can reintroduce it.

Usage is missing from a stream

Some providers only emit usage when asked. For OpenAI-shaped streams, send stream_options: { "include_usage": true }. Anthropic reports usage on its own events, so no option is needed.

{
  "model": "gpt-5.6-sol",
  "stream": true,
  "stream_options": { "include_usage": true },
  "messages": [{ "role": "user", "content": "Count to five" }]
}

If a provider never reports usage, the gateway estimates it from bytes and records the same figure it charges — so the ledger still reconciles.

Timeouts

Non-streaming requests time out at 120 seconds and streams at 10 minutes by default. A long generation should stream, both for responsiveness and to stay inside the limit.