Delivery threads enqueue and a bounded pool runs the handlers
Table of Contents
1. Summary
A NATS delivery thread decodes a message and puts it on its service's queue; it never waits for the database. A fixed pool of workers, as many as the service's database connections, takes work from the queues and runs the handlers. Queues are bounded and every entry has a deadline. A full queue answers a request with busy, or drops a message its policy allows to be lost, and a worker skips an entry whose deadline has passed. A service's thread count is set by its configuration, not by the number of its subscriptions.
2. Detail
2.1. Delivery
The subscriptions of a connection share a few delivery threads. A delivery thread does the work that cannot wait: it decodes, checks the deadline and enqueues. Nothing it does blocks on another service or on the store.
2.2. Queues by class of work
| Queue | Holds | When full |
|---|---|---|
| Requests | Calls that expect a reply | Reply busy, with the time to wait |
| Events | Core NATS events that may be lost | Drop the oldest and count it |
| Durable deliveries | JetStream messages | Answer with a delayed redelivery |
Each class has its own queue, so a burst of events cannot starve the requests behind it.
2.3. Deadlines
A request's deadline is its caller's timeout. A worker that takes an entry whose deadline has passed skips it, because nobody waits for its answer.
2.4. Size
The worker pool is as large as the connection pool (A service holds a few long-lived connections and never connects per query), so a worker never waits for a connection, and a request waits only in its queue, where its wait is counted (Every queue and every session reports its own numbers).
3. Patterns used
- Bulkhead — each class of work has its own queue, so one class cannot exhaust another's capacity.
- Throttling — a full queue answers busy or drops by policy instead of growing.
4. See also
- The data path — the structure note that orders this cluster.
- Message Queue — the bus whose delivery threads feed these queues.