Idempotent Receiver
Table of Contents
This page defines the service architecture pattern Idempotent Receiver in the pattern language. A receiver handles a repeated message so that the second delivery has no further effect. In ORE Studio every workflow step must be one, because the engine delivers a step at least once.
1. Also known as
- Idempotent Consumer, in Richardson's catalogue.
2. Context
A message can arrive twice: a restart re-dispatches unfinished work, a broker redelivers an unacknowledged message, a caller retries after a lost reply.
3. Problem
How does a receiver make a repeated message safe?
4. Forces
- Exactly-once delivery is not available across services.
- Some effects are expensive: a second compute batch runs the engine twice.
- A guard that asks another service can itself fail.
- The second delivery may arrive while the first is still running.
5. Solution
Key the work on a stable identity: the step id. Before acting, ask whether that identity already has an outcome, and replay it if so. Make the write itself converge, so a repeat meets its own earlier effect: an upsert on business keys, a batch adopted by its external reference, an object written under the same key.
Figure 1: Idempotent Receiver: replay a recorded outcome, otherwise do the work with converging writes.
Source: [[./pattern_idempotent_receiver.puml][pattern_idempotent_receiver.puml]]. Regenerate with
plantuml -tpng doc/knowledge/service_architecture_patterns/pattern_idempotent_receiver.puml.
6. Failure handling
| Concern | What the pattern does |
|---|---|
| Errors | An error is recorded against the step identity, and a replay returns it. |
| Timeouts | The replay guard has a short timeout and fails open, so the work must still converge. |
| Retries | A retry under the same identity runs the work once. |
| Idempotency | This is the pattern's purpose: the step identity is the key. |
| Observability | A replayed outcome is logged as a replay, so a repeat is visible. |
7. Pitfalls
- A guard without a converging write. The guard fails open, so two deliveries in flight both do the work.
- The correlation id used as the key. Two different steps of one action share it.
- An append in place of an upsert. A repeat adds a second row.
- Report phases that do not check. Report execution orchestration records that the report handlers do not yet use the replay guard.
8. Security
No security consequence: the identity decides repetition, not access.
9. Resulting context
- Benefits
- At-least-once delivery becomes safe, and recovery needs no person.
- Liabilities
- Every writer must find a converging form, and the guard adds a call.
10. In ORE Studio
The curve construction, Bootstrapping a curve: from quotes to a published set, relies on it: a republish over the same series and as-of converges on the same natural key.
The surface construction, Constructing a volatility surface: from quotes to a published set, relies on it in the same way.
The market data path, a public fixing, a curve and a report, relies on it at the publish gate, which applies one approved request once, and at the ingest, which tolerates a repeated tick.
- Idempotency in a workflow step — the replay guard and the converging write.
- Report execution orchestration — the convergence each report phase owes the engine.
- Tenant provisioning — every step is idempotent, so a retry is safe.
- Approval request expiry — the sweep states its rule over the deadline rather than over an interval, so a repeated run and a run after an outage converge.
- How the compute grid queues and claims work — a repeated or late submit for a result that is retired, done or claimed by another host changes nothing.
- Ticks reach the store through sharded binary COPY in bounded batches — the unique point key makes a redelivered tick a no-op in the store.
11. Related patterns
- Process Manager — delivers each step at least once.
- Retry — safe only against an idempotent receiver.
- Competing Consumers — redelivery to another member.
12. NATS patterns
The NATS service patterns hub states how each of these is written in
ores.nats code:
- Idempotent Message Identity — the message identity and the receiver's own duplicate guard.
- Acknowledge What Was Handled — the redelivery that makes a receiver see a repeat.
13. Sources
- Gregor Hohpe and Bobby Woolf, Enterprise Integration Patterns, Addison-Wesley, 2003, Idempotent Receiver.
- Chris Richardson, Pattern: Idempotent Consumer, microservices.io.
14. See also
- Service architecture patterns — the structure note that orders this pattern language, and where to read this page in it.