Story: Define and adopt the canonical NATS entity protocol
Table of Contents
This page documents a story in Sprint 25. It captures the goal, current status, acceptance criteria, and the tasks that compose it.
1. Goal
One target-state specification defines every common, entity-related message a component speaks. It states the subject grammar, the canonical verb set for the operations an entity supports, the request and response envelopes, the pagination and filtering contract, the concurrency contract, the error model, the event envelope and delivery semantics, and the security rules for writes. It names no component and describes no legacy state. Codegen is its implementation, and the documents that carry fragments of it today are reconciled to one responsibility each. ores.iam is the first component converted.
2. Status
| Field | Value |
|---|---|
| State | ABANDONED |
| Parent sprint | Sprint 25 |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-09-23 |
3. Acceptance
- A target-state specification for common entity NATS messages exists in the architecture knowledge, names no component, and describes no current or legacy state.
- The specification defines the subject grammar, the canonical verb set, the request and response envelopes, pagination and filtering, the concurrency contract, the error model, the event envelope and delivery semantics, and the security rules for writes.
- The specification states that a write request carries only user-owned fields: every server-owned field is absent from the wire and derived server-side from the authenticated context.
- Every document that describes NATS messaging states exactly one responsibility, the conflicts between them are resolved, and superseded documents are marked historical rather than deleted.
- ores.iam speaks the specification across generated C++ messages, generated shell commands and their tests, the generated BFF route descriptor and the web screens.
- A conformance check fails when a generated message violates the specification's grammar.
4. Tasks
| Task | State | Start | End | Description |
|---|---|---|---|---|
| Scaffold story: Define and adopt the canonical NATS entity protocol | DONE | 2026-09-21 | 2026-09-21 | Story scaffolding rides this task: documents, sprint wiring, and the scaffold PR. Close it before merging that PR. |
| Write the NATS entity protocol specification | DONE | 2026-09-21 | 2026-09-21 | Author the target-state specification for common entity messages, and absorb the unowned backlog captures that are fragments of it. |
| Reconcile every NATS document to one responsibility | ABANDONED | 2026-09-23 | Around thirty documents describe ORE Studio's NATS messaging, and they disagree: three competing history subjects, two event subject prefixes, three different claims about the payload format, and two unreconciled error channels. This task gives every one of them exactly one responsibility and resolves the conflicts. | |
| Adopt the canonical protocol in ores.iam | DONE | 2026-09-21 | 2026-09-21 | The first component converted to the specification. Codegen emits the canonical messages, generated ores.shell commands and their tests exercise every verb, and the BFF route descriptor and the web screens consume the new shapes. The shell surface is proved against the running fleet, and the recipes that exercise it are generated. |
| Enforce the protocol with a conformance gate | ABANDONED | 2026-09-23 | A specification without a gate drifts, which is exactly what the protocol twin coverage check proved when its written rule and its implementation disagreed for a whole component. This task lands the check that keeps generated messages honest. | |
| Verify every IAM surface against the specification | DONE | 2026-09-22 | 2026-09-23 | The second PR of the IAM pilot, and now the web half of it. The shell half landed with the conversion: 124 generated commands, 123 of them run against the live fleet. This task exercises every IAM entity through ores.web and fixes what the sweep finds. |
| Adopt the canonical protocol in ores.refdata | STARTED | 2026-09-22 | The second component converted to the specification. Codegen emits the canonical messages for every refdata entity, junction and operation; generated ores.shell commands and their tests exercise every verb; the BFF route descriptor and the web screens consume the new shapes; and refdata joins COMPONENTS_UNDER_TEST. | |
| Adopt the canonical protocol in the refdata web layer | DONE | 2026-09-22 | 2026-09-22 | The refdata conversion's web half. The wire-protocol package carries a hand-written duplicate of the country protocol, and the BFF and the country screens name the retired save, delete and history subjects, so the schema and the subjects must come from the generated protocol or be generated themselves. |
| Address the web-layer review findings | DONE | 2026-09-22 | 2026-09-22 | The review of PR #2127 left three observations and the PR itself left two limitations. They belong to the shared web layer, so they land in the next PR that touches it rather than a reopened #2127: the save route's implicit create/amend contract, the write-default sentinel Python and TypeScript share, the image picker's hardcoded narrowing, a route-level test for the factory, and the as-of control the BFF forwards but the shared list screen does not offer. |
| Add screens for every refdata entity | DONE | 2026-09-22 | 2026-09-22 | Sixty-four refdata entities have generated protocol, domain and UI metadata, and one of them (country) has screens. This wires the rest onto the shared machinery: the model opts into ores.ts.web, codegen emits the declaration, the route descriptor and the catalogue words it already holds, and the navigation and the router read a generated list rather than one hand-written line per entity. |
5. Decisions
Agreed before any work started, and the basis the specification is written against. Each one answers a shortcoming found in the existing messages; the evidence for each is in the specification task's notes.
- The specification is one generic document, not a per-component one. It defines the shape of every common entity message and names no component. A component's own operations are additions that must still obey the grammar.
- The subject grammar is
{component}.v1.{resource_plural}.{verb}. Exactly one subject per resource and verb. No three-segment subjects, no kebab-case, no duplicate definitions. - The canonical verb set is eight, and orthogonal:
get,get_many,list,list_by_<relation>,put,put_many,delete,delete_many– read one, read many, read a scoped page, write one, write many, remove one, remove many. The paged read's message is renamed fromget_<plural>_requesttolist_<plural>_request; the current name contradicts its own.listsubject. - A read that returns something other than a page of the resource is a read of
that something. An entity's versions and the entities below a node in a
hierarchy are collections in their own right, read with the verbs above rather
than with a verb of their own. Versions become a read-only
<resource>_versionssub-resource addressed by a composite key, so the version axis is a filter rather than a second selection mechanism; a subtree becomes the relation-scoped list with asubtreescope, so the reply is entities and the caller assembles the tree. This is what removes thehistoryandlist_treeverbs. - Single and batch are paired, element-wise in their claims and set-wise in their atomicity. Every operation on one entity has a set form and a resource declares both or neither, so no entity is batch-less. Each element states its own precondition, because a batch of updates is a set of rows each at its own version and one version claim could not describe it; the batch then lands or fails as one transaction. Batch and page bounds are part of the protocol.
- No verb is derived that the specification does not name. Whether an entity has a write verb is what the model declares, not something inferred from a version column or a repository flag. That inference is what leaves 22 resources with a list and no write today.
- A write request carries only user-owned fields.
tenant_id,party_id,version,modified_by,performed_by,recorded_atand the validity window never appear on the wire; the server derives them from the authenticated context. This followsservice-to-service-delegation, which already states that delegation context travels in headers and that message structs carry only business data. - Change intent travels explicitly. The user-supplied reason and commentary are part of the write rather than server-owned, and their shape is stated once rather than restated per protocol.
- Concurrency is an explicit precondition, not a sentinel.
version = 0currently conflates create, upsert, regeneration and revert, which is why a duplicate create can silently overwrite. The specification replaces it with a stated precondition. - One error model. Every response carries a single result with a stable
error code. The transport
X-Errorheader is reserved for failures that cannot produce a typed body, and the specification names which those are. - Pagination stays unconditional; filtering is additive.
offset,limitand the total count remain unconditional members of every list, because thehas_paginationfork was deliberately removed to stop clients disagreeing about whether they page. Filtering, sorting and the version axis are added without reintroducing it. A separatecountverb is therefore not in the set. - Events get one prefix and a stated delivery contract.
ores.{component}.{entity}_changedand{component}.v1.<resource>.<verb>currently coexist. The specification picks one, and states idempotency, ordering and the queue-group rule, which today follow per-registrar habit. - The specification is enforced by a gate, not by review. The twin coverage check proved that a written rule and its implementation drift silently, so the specification lands with a conformance check rather than as prose alone.
- Adoption is one component at a time, starting with ores.iam.
COMPONENTS_UNDER_TESTincomponent_registry.pyis the single list of components under test, and a component joins it when its protocol is converted.
6. Out of scope
- Components other than
ores.iam. Their adoption follows once the specification and its conformance gate exist, and is tracked by the shared component list rather than by tasks in this story. - The wire encoding. JSON versus MessagePack is a process-wide choice already
documented in
nats_wire_format, and this story does not change it. - Transport, certificates and deployment.
nats_certificatesand the service hosting runbooks keep their own responsibilities.
7. Result
Abandoned at sprint 25 close. Sprint 26 replaces the codegen approaches tried so far with one method: for each module, sync codegen and clear its drift. Sprint 26 opens new, clean stories for each module. This story is not carried forward.