The Idempotent Consumer
Make repeated requests safe so a retry or duplicate delivery never causes double effects.
What you'll learn
- Define idempotency and why it matters in integrations
- Use idempotency keys to deduplicate write operations
- Build a consumer that ignores already-processed messages
- Distinguish naturally idempotent operations from unsafe ones
- Combine idempotency with retries safely
7 min
What idempotency means
An operation is idempotent if performing it many times has the same effect as performing it once. Reading a record is naturally idempotent. Setting a value to a fixed amount is idempotent. Incrementing a balance is not — do it twice and you get the wrong number.
This matters because retries and at-least-once message delivery are facts of distributed life. A timeout might mean your request never arrived, or it arrived and the response was lost. You cannot always tell. Idempotency lets you retry safely without worrying which case you are in, because a duplicate simply has no additional effect.
Idempotency keys
For write operations that are not naturally idempotent, the standard tool is an idempotency key: a unique value the client generates and attaches to a request. The server records the key with the result. If the same key arrives again, the server returns the stored result instead of doing the work twice.
POST /resource
Idempotency-Key: 9f1c-create-2026-06-29
{ 'amount': 100, 'reference': 'INV-42' }Generate one key per logical operation and reuse it across retries of that same operation. A new operation gets a new key. This turns an unsafe write into something you can retry with confidence.
The consumer side
When you consume messages or webhooks, assume each may arrive more than once. An idempotent consumer remembers what it has already handled and skips repeats. The usual mechanism is a deduplication store keyed by a stable message identifier.
- Record the identifier of every message you process.
- Check before acting — if seen, acknowledge and stop.
- Make the check and the work atomic so a crash between them cannot cause a gap or a duplicate.
Expire old identifiers after a sensible retention window so the store does not grow forever. The identifier need only outlive the realistic redelivery period.
Pairing with retries
Idempotency and retries are partners. Retries provide resilience; idempotency makes that resilience safe. Without idempotency, an aggressive retry policy can duplicate charges, records or notifications — exactly the failures that erode trust.
The discipline is simple to state: before you retry any write, ask whether repeating it is harmless. If not, give it an idempotency key or route it through an idempotent consumer first. With that in place, you can retry freely, knowing the worst case of a duplicate request is a no-op rather than a costly mistake. Build this in early; retrofitting it after a duplication incident is painful.
Key takeaways
- Idempotent operations produce the same result no matter how often they run
- Reads are naturally safe; many writes are not
- Idempotency keys let the server deduplicate repeated write requests
- Idempotent consumers record message identifiers and skip repeats
- Idempotency is what makes aggressive retries safe to use
FAQ
What is an idempotency key?
A unique value the client attaches to a write request. The server stores it with the outcome, so a repeat of the same key returns the stored result instead of performing the work a second time.
Do I need keys for read operations?
No. Reads are naturally idempotent — fetching the same data twice changes nothing. Keys are for operations that create or modify state, where a duplicate would otherwise have a real and unwanted effect.
How long should I keep processed identifiers?
Long enough to cover the realistic redelivery window for your source, then expire them. Keeping them forever wastes storage; expiring too soon risks reprocessing a late duplicate.
Ready to build?
Read the API reference, grab the OpenAPI spec, and ship a resilient integration.