Timeout
Table of Contents
This page defines the service architecture pattern Timeout in the pattern language. Every wait has a stated limit, so a silent or slow service fails the waiter instead of holding it. In ORE Studio requests have a default timeout and every workflow step states a deadline.
1. Also known as
- Deadline, for a budget that spans several calls.
2. Context
A caller waits for another service: a request, a workflow step, a grid batch. The other service can be slow, wedged or gone, and the bus does not say which.
3. Problem
How long should a caller wait, and what happens when it stops waiting?
4. Forces
- Waiting forever holds a thread, a connection or a run.
- Giving up too early fails work that would have finished.
- The callee may finish after the caller gave up.
- One number is wrong for both a write and an import.
5. Solution
Every wait states its limit where the operation is defined, not at each call site. When the limit passes, the caller fails the call with the subject and what it knows, and logs the correlation id. A budget that covers several calls is larger than any one of them, so the inner limit fails first and names the cause.
Figure 1: Timeout: the caller gives up after its limit and fails the call; the callee may still finish.
Source: [[./pattern_timeout.puml][pattern_timeout.puml]]. Regenerate with
plantuml -tpng doc/knowledge/service_architecture_patterns/pattern_timeout.puml.
6. Failure handling
| Concern | What the pattern does |
|---|---|
| Errors | The timeout is a failure with the subject and the elapsed time. |
| Timeouts | This is the pattern itself. |
| Retries | A retry after a timeout is safe only if the work is idempotent; see Retry. |
| Idempotency | A late answer after the timeout is ignored and logged as late. |
| Observability | The engine states what it knew of the service that owned the silent step: absent, not reporting, or reporting. |
7. Pitfalls
- No timeout. A step with no deadline waits forever, and every waiter behind it gives up with no reason.
- A timeout read as failure of the work. The callee may have finished; never compensate on a timeout.
- Equal limits nested. The outer wait expires with the inner and reports the wrong cause.
- A timeout per call site. Callers of one operation then disagree on how long it takes.
8. Security
No security consequence beyond availability: a limit stops a slow callee from holding the caller's resources.
9. Resulting context
- Benefits
- A silent service becomes a visible failure with a cause.
- Liabilities
- A slow but healthy service can be failed, and its late result must be handled.
10. In ORE Studio
Requests default to 30 seconds in ores.nats/service/timeouts.hpp. Steps read their budgets from data_step_timeout and write_step_timeout, and the engine checks them every thirty seconds.
- Failure, compensation and retry — every step states a deadline, and the engine fails a silent step.
- Waiting, deadlines and cancellation — the budget belongs to the operation's definition, and expiry is a failure.
- How the compute grid queues and claims work — the lease limits how long a node may hold a job in silence.
11. Related patterns
- Retry — what follows a timeout.
- Saga — a timed-out step stops the run instead of compensating it.
- Request-Reply — every request carries one.
12. NATS patterns
The NATS service patterns hub states how each of these is written in
ores.nats code:
- Bounded Request — the one default deadline every request shares.
- Classify Transport Failure — telling an expired deadline from no responders.
13. Sources
- Michael T. Nygard, Release It!, second edition, Pragmatic Bookshelf, 2018, chapter 5, "Stability Patterns" — Timeouts.
- Marc Brooker, Timeouts, retries, and backoff with jitter, the Amazon Builders' Library.
14. See also
- Service architecture patterns — the structure note that orders this pattern language, and where to read this page in it.