Competing Consumers
Table of Contents
This page defines the service architecture pattern Competing Consumers in the pattern language. Several replicas subscribe to one channel and each message goes to exactly one of them. In ORE Studio every service subscribes in a queue group named for the service, and compute workers share a queue group per platform.
1. Also known as
- Queue group, the NATS name.
- Work queue.
2. Context
A service runs as several replicas for capacity or availability. Messages for it arrive on one subject.
3. Problem
How do several replicas share the work on one channel without doing it twice?
4. Forces
- Each message must be handled once, not once per replica.
- Adding a replica must add capacity with no change to the sender.
- A replica can die while it holds a message.
- Messages may be handled out of order.
5. Solution
Every replica subscribes to the subject in the same queue group. The bus delivers each message to one member. Core NATS delivers at most once; a JetStream consumer holds the message until a member acknowledges it, so a member that dies mid-message does not lose it.
Figure 1: Competing Consumers: three replicas in one queue group each receive a different message.
Source: [[./pattern_competing_consumers.puml][pattern_competing_consumers.puml]]. Regenerate with
plantuml -tpng doc/knowledge/service_architecture_patterns/pattern_competing_consumers.puml.
6. Failure handling
| Concern | What the pattern does |
|---|---|
| Errors | A member that fails a message replies with the error; the others are unaffected. |
| Timeouts | A JetStream message not acknowledged in time is redelivered to another member. |
| Retries | Redelivery is a retry; the work must be safe to repeat. |
| Idempotency | A redelivered message may reach a member that already started it; see Idempotent Receiver. |
| Observability | Each member logs with its host, so a message's handler can be found. |
7. Pitfalls
- State in one replica's memory. A cache or a session held by one member is not seen by the others.
- Order assumed. Two messages may be handled in parallel by two members.
- A poison message. A message that always fails is redelivered forever under JetStream unless a limit stops it.
- Two queue group names for one service. Each group receives every message, so the work runs twice.
8. Security
Every member verifies the token on each message; membership grants nothing.
9. Resulting context
- Benefits
- Capacity grows with replicas, and a replica's failure costs no message under JetStream.
- Liabilities
- No ordering, and per-replica state diverges.
10. In ORE Studio
The market data path, a public fixing, a curve and a report, uses it at every service that takes messages: the adapter and the curve service each bind a queue group.
The convention is in ores.nats/service/client.hpp: the queue group is the service name, for example ores.iam.service.
- Anatomy of a Service — every service subscribes its subjects in a queue group named for the service.
- Compute job lifecycle — worker nodes share a durable, queue-grouped consumer per platform.
- Work is pushed to the grid, and an idle grid costs one claim per wrapper per timeout — the wrappers of a platform race for each job a notice announces, and the claim's lock decides.
11. Related patterns
- Request-Reply — one member answers each request.
- Bulkhead — the pools the members belong to.
- Queue-Based Load Leveling — the queue the members drain.
12. NATS patterns
The NATS service patterns hub states how each of these is written in
ores.nats code:
- Queue-Group Subscription — the queue group that shares one subject across replicas.
- Durable Subscription — a durable consumer holds the message when no member is up.
- Acknowledge What Was Handled — what a member acknowledges, and what it lets be redelivered.
- Own the Subscription Handle — how the group's subscriptions are kept alive and drained.
13. Sources
- Gregor Hohpe and Bobby Woolf, Enterprise Integration Patterns, Addison-Wesley, 2003, Competing Consumers.
- Microsoft, Azure Architecture Center, Cloud design patterns, Competing Consumers pattern.
- NATS documentation, Queue Groups.
14. See also
- Service architecture patterns — the structure note that orders this pattern language, and where to read this page in it.