Timeouts and Deadlines

Why a missing timeout is dangerous

Most HTTP clients default to no read timeout, or one measured in minutes. That means a single stalled connection can pin a thread or async task indefinitely. Under load, a handful of stuck requests becomes a queue, the queue becomes resource exhaustion, and a minor upstream blip becomes a full outage on your side.

Setting an explicit, modest timeout converts an open-ended hang into a fast, recoverable error. A request that fails in two seconds can be retried, queued, or surfaced to the user. A request that hangs for five minutes simply consumes resources and confuses everyone watching the logs.

Three timeouts, not one

Separate the phases. A connect timeout bounds how long you wait to establish a TCP and TLS connection — keep it short, since healthy connections form quickly. A read timeout bounds waiting for response bytes once connected. An overall deadline caps the entire operation, including any retries.

A common, sensible starting point is a one to two second connect timeout and a read timeout sized to your endpoint's behaviour. Tune from there using real data, not intuition.

client = HttpClient(
  connect_timeout='2s',
  read_timeout='10s',
  overall_deadline='20s'
)

Set all three explicitly rather than relying on library defaults, which are frequently absent or far too generous. The connect timeout catches a server you cannot reach; the read timeout catches one that accepted your connection but then stalled; the overall deadline caps the whole operation so retries cannot quietly extend it without bound. Each guards a distinct failure, and leaving any of them unset leaves that failure mode open.

Size timeouts from percentiles

Pick values from observed latency, not round numbers. If an endpoint's 99th-percentile response time is 800 ms, a 10 second read timeout gives generous headroom while still catching genuine stalls. A timeout set far below the normal tail will cause needless failures; one set far above defeats the purpose.

Re-check these numbers periodically. Latency shifts as data volumes, payload sizes, and infrastructure change. A timeout that was comfortable last quarter may be too tight, or too loose, today. Our status page publishes operational context you can factor into your own thresholds.

Propagate deadlines down the chain

When one request triggers several downstream calls, share a single deadline rather than giving each call a fresh full-length timeout. If a user-facing request must complete in three seconds, each downstream hop should see the remaining budget, not three fresh seconds of its own.

Pass the remaining time as you descend, and stop early when it runs out. This prevents the situation where every layer waits its full timeout and the total far exceeds what the user will tolerate. Combined with failure-aware design, deadline propagation keeps slow requests from cascading.