Message Queue
Table of Contents
1. Summary
Components in ORE Studio are separate processes with their own databases and no shared memory, so the only way one acts on another's data is by sending it a message. A service's public surface is exactly the set of subjects it answers, and a message is the entire interface: what a component keeps private is reachable by no other means.
The cluster covers that interface in the four concerns it separates. The bus supplies the patterns a message may use – request and reply, publish and subscribe, queue groups, streams. The namespace decides which component owns an address, and therefore which one answers. The contract fixes what a message may contain: the grammar of a subject, the verbs an entity supports, what a write may assert, and how a failure is reported. The binding turns subjects into handlers inside a service.
Read in that order. Each concern rests on the one before it: a contract cannot be tested without a namespace to address it in, and a binding is mechanical once the contract is fixed.
2. Reading order
2.1. What the bus is
NATS — start here. The substrate: subjects and wildcards, pub/sub against request/reply, queue groups, and streams. Every name below is a NATS name, and the patterns below are ones this substrate already provides.
ores.nats — the library that wraps it: what a call site looks like, the shared configuration knobs, and the request/reply helpers. Read second, because it is the concrete form the substrate takes in this codebase.
2.2. Where a subject may live
ORE Studio Messaging Reference — the namespace map: which component owns which subjects, and the rule that no subject is ownerless. Read this before writing a subject, because a subject in the wrong namespace is served by the wrong service and nothing reports it.
2.3. What a message must look like
NATS Entity Protocol Specification — the centre of the cluster. The subject grammar, the eight verbs, the request and response envelopes, the write model that keeps server-owned fields off the wire, the error model, the event contract, and the security rules. Read it before writing any message; it is the page the rest of the cluster defers to.
NATS Wire Format — the seam the specification assumes rather than restates: the envelope against the payload, and the process-wide encoding choice. Read straight after, because it is what says which bytes are the message and which are not.
2.4. How a payload may vary
Polymorphic types over NATS — one subject per concrete type, and why a variant is never carried as an untyped blob. Read when a message has to carry more than one shape, which is exactly when a shortcut is tempting.
2.5. What a write claims, and what the past means
Optimistic Concurrency Versioning — what the version on a write actually asserts, who owns it, and why an integer must not double as a mode. Read before implementing a write path; the failure it prevents is a silent overwrite.
History Diff Architecture — presenting one version against its predecessor. Read after the version page, because it presumes the version is the database's to assign and not the caller's to claim.
2.6. How a service binds to the bus
Anatomy of a Service — the layers, where the registrar sits, and the error contract, with the stand-up checklist. Read when you are building or repairing a service rather than a message.
Entity-composed registrars — how subjects become handlers, one entity at a time. Read straight after the anatomy page: it is the mechanism that page names.
Service Bootstrap Phases — when a subject becomes callable, and the ordering that follows for anything that calls back into its own service. Read last of the three, because it only bites once subscriptions exist.
2.7. State kept from the bus
Where should an entity-mirror cache live? — the placement of a cache fed by events rather than by reads. Read when a consumer keeps state instead of answering requests, which is a different problem from every page above it.
2.8. Alongside, when the question is running it
NATS certificates — generation, TLS layout, and container deployment. Not part of the message story; reach for it when the question is securing or deploying the bus rather than speaking on it.
3. What this cluster does not cover
Identity and Access is a separate cluster with a structure note of its own. Service-to-service authentication and impersonation travel in message headers, so the headers are named here, but the question of who an actor is and where each layer enforces it belongs there.
PostgreSQL Architecture owns the schema, and with it the database half of the eventing relay — the notification that becomes a published event. This cluster starts on the bus side of that seam.
Market Data Architecture is a domain cluster whose interface happens to be this protocol. Read the protocol here and what market data does with it there.
Data-Oriented Design supplies the criteria a message's shape is reviewed against. It judges the shape; it does not define it.
4. See also
- Knowledge — the index this structure note hangs from.
- Zettelkasten — what a structure note is, and why its order is authored rather than computed.