Handling Partial Failures

Success, failure, and unknown

A local function returns or it throws. A network call has a third outcome: unknown. The request may have reached the server, been processed, and the response lost on the way back — leaving you unsure whether the work happened. Treating unknown as plain failure is a common and costly mistake, because a blind retry can repeat an action that already succeeded.

Design for all three states. When a call times out or the connection drops mid-flight, your code does not know the server's outcome and must recover safely without assuming either result. This ambiguity is the heart of partial-failure handling.

Idempotency makes retries safe

The cleanest answer to the unknown case is idempotency: design operations so performing them twice has the same effect as once. Then, after an ambiguous result, you can simply retry without fear of duplication. A common technique is an idempotency key — a unique value the client sends so the server recognises and de-duplicates a repeat.

POST /charges
Idempotency-Key: order-7741-attempt
# retried with the SAME key returns the
# original result, no second charge

Where an API supports such keys, use them on every non-idempotent write so retries are always safe.

Reconcile when you cannot tell

Sometimes you cannot retry safely and must instead discover what actually happened. Reconciliation means checking the authoritative state — querying the resource to see whether it was created, or comparing your records against the server's — and correcting any drift. A scheduled reconciliation job can catch the rare cases that slip through real-time handling.

This matters most for operations with real-world effects, such as payments or record creation, where a duplicate or a missed action has genuine consequences. Reconciliation is the safety net beneath retries: it assumes some ambiguity will always escape and provides a way to clean up afterwards.

Beware naive retry loops

A retry loop written without idempotency is a duplication engine. If the first attempt actually succeeded but its response was lost, each retry creates another record or repeats another side effect. The user sees one action; the system records several. These bugs are nasty because they only appear under the exact timing of a lost response.

Guard every retry with idempotency or a reconciliation check, and cap the number of attempts. Combined with sensible timeouts and deadlines, this turns the unknown outcome from a source of corruption into a routine, recoverable event.