Request Timeouts and Deadlines

Why timeouts are mandatory

Every call that leaves your process can hang. A network can stall, a server can wedge, a connection can be silently dropped. Without a timeout, a single stuck call ties up a thread or connection indefinitely, and enough of them exhaust your resources and take your service down.

The default in most libraries is to wait forever, which is almost never what you want. An explicit timeout converts an indefinite hang into a bounded, handleable failure. Treat a missing timeout as a bug: every outbound call should have one. The question is never whether to set a timeout but what value is appropriate for that call.

Request vs deadline

There are two related limits. A per-request timeout bounds a single attempt. An overall deadline bounds an entire operation, including any retries and intermediate steps.

The distinction matters once retries are involved. A 10-second per-attempt timeout with three retries could still consume far longer in total. An overall deadline says this whole operation must finish within X, regardless of how many attempts it took. Use both: the per-request timeout stops any one call hanging, and the deadline stops the combined sequence running away. The deadline is what a waiting user or upstream caller actually cares about.

Choosing values

Timeout values are a balance. Too short, and you abandon calls that would have succeeded, manufacturing failures. Too long, and a genuinely stuck call holds resources far longer than necessary.

• Base it on observed latency — set above the normal high percentile, not the average.
• Tighten interactive paths — a user waiting needs a shorter limit than a background job.
• Separate connect and read — failing to connect should give up faster than a slow response.

Review timeouts against real latency data periodically; a value that was right at launch may be wrong as traffic and dependencies change.

Propagate and handle

When one request triggers a chain of downstream calls, propagate the remaining deadline through them. If only two seconds are left, no downstream call should be allowed to wait ten. Passing the deadline down prevents work continuing on a request the caller has already abandoned.

remaining = deadline - now()
if remaining <= 0:
    raise DeadlineExceeded()
downstream.call(request, timeout=remaining)

Treat a timeout as its own outcome, distinct from a clean failure. Crucially, a timeout does not tell you whether the work happened — the request may have succeeded with a lost response. That is exactly why timed-out writes need idempotency before they are retried.