Story: Define and adopt the canonical NATS entity protocol

Table of Contents

This page documents a story in Sprint 25. It captures the goal, current status, acceptance criteria, and the tasks that compose it.

1. Goal

One target-state specification defines every common, entity-related message a component speaks. It states the subject grammar, the canonical verb set for the operations an entity supports, the request and response envelopes, the pagination and filtering contract, the concurrency contract, the error model, the event envelope and delivery semantics, and the security rules for writes. It names no component and describes no legacy state. Codegen is its implementation, and the documents that carry fragments of it today are reconciled to one responsibility each. ores.iam is the first component converted.

2. Status

Field Value
State ABANDONED
Parent sprint Sprint 25
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-09-23

3. Acceptance

  • A target-state specification for common entity NATS messages exists in the architecture knowledge, names no component, and describes no current or legacy state.
  • The specification defines the subject grammar, the canonical verb set, the request and response envelopes, pagination and filtering, the concurrency contract, the error model, the event envelope and delivery semantics, and the security rules for writes.
  • The specification states that a write request carries only user-owned fields: every server-owned field is absent from the wire and derived server-side from the authenticated context.
  • Every document that describes NATS messaging states exactly one responsibility, the conflicts between them are resolved, and superseded documents are marked historical rather than deleted.
  • ores.iam speaks the specification across generated C++ messages, generated shell commands and their tests, the generated BFF route descriptor and the web screens.
  • A conformance check fails when a generated message violates the specification's grammar.

4. Tasks

Task State Start End Description
Scaffold story: Define and adopt the canonical NATS entity protocol DONE 2026-09-21 2026-09-21 Story scaffolding rides this task: documents, sprint wiring, and the scaffold PR. Close it before merging that PR.
Write the NATS entity protocol specification DONE 2026-09-21 2026-09-21 Author the target-state specification for common entity messages, and absorb the unowned backlog captures that are fragments of it.
Reconcile every NATS document to one responsibility ABANDONED   2026-09-23 Around thirty documents describe ORE Studio's NATS messaging, and they disagree: three competing history subjects, two event subject prefixes, three different claims about the payload format, and two unreconciled error channels. This task gives every one of them exactly one responsibility and resolves the conflicts.
Adopt the canonical protocol in ores.iam DONE 2026-09-21 2026-09-21 The first component converted to the specification. Codegen emits the canonical messages, generated ores.shell commands and their tests exercise every verb, and the BFF route descriptor and the web screens consume the new shapes. The shell surface is proved against the running fleet, and the recipes that exercise it are generated.
Enforce the protocol with a conformance gate ABANDONED   2026-09-23 A specification without a gate drifts, which is exactly what the protocol twin coverage check proved when its written rule and its implementation disagreed for a whole component. This task lands the check that keeps generated messages honest.
Verify every IAM surface against the specification DONE 2026-09-22 2026-09-23 The second PR of the IAM pilot, and now the web half of it. The shell half landed with the conversion: 124 generated commands, 123 of them run against the live fleet. This task exercises every IAM entity through ores.web and fixes what the sweep finds.
Adopt the canonical protocol in ores.refdata STARTED 2026-09-22   The second component converted to the specification. Codegen emits the canonical messages for every refdata entity, junction and operation; generated ores.shell commands and their tests exercise every verb; the BFF route descriptor and the web screens consume the new shapes; and refdata joins COMPONENTS_UNDER_TEST.
Adopt the canonical protocol in the refdata web layer DONE 2026-09-22 2026-09-22 The refdata conversion's web half. The wire-protocol package carries a hand-written duplicate of the country protocol, and the BFF and the country screens name the retired save, delete and history subjects, so the schema and the subjects must come from the generated protocol or be generated themselves.
Address the web-layer review findings DONE 2026-09-22 2026-09-22 The review of PR #2127 left three observations and the PR itself left two limitations. They belong to the shared web layer, so they land in the next PR that touches it rather than a reopened #2127: the save route's implicit create/amend contract, the write-default sentinel Python and TypeScript share, the image picker's hardcoded narrowing, a route-level test for the factory, and the as-of control the BFF forwards but the shared list screen does not offer.
Add screens for every refdata entity DONE 2026-09-22 2026-09-22 Sixty-four refdata entities have generated protocol, domain and UI metadata, and one of them (country) has screens. This wires the rest onto the shared machinery: the model opts into ores.ts.web, codegen emits the declaration, the route descriptor and the catalogue words it already holds, and the navigation and the router read a generated list rather than one hand-written line per entity.

5. Decisions

Agreed before any work started, and the basis the specification is written against. Each one answers a shortcoming found in the existing messages; the evidence for each is in the specification task's notes.

  • The specification is one generic document, not a per-component one. It defines the shape of every common entity message and names no component. A component's own operations are additions that must still obey the grammar.
  • The subject grammar is {component}.v1.{resource_plural}.{verb}. Exactly one subject per resource and verb. No three-segment subjects, no kebab-case, no duplicate definitions.
  • The canonical verb set is eight, and orthogonal: get, get_many, list, list_by_<relation>, put, put_many, delete, delete_many – read one, read many, read a scoped page, write one, write many, remove one, remove many. The paged read's message is renamed from get_<plural>_request to list_<plural>_request; the current name contradicts its own .list subject.
  • A read that returns something other than a page of the resource is a read of that something. An entity's versions and the entities below a node in a hierarchy are collections in their own right, read with the verbs above rather than with a verb of their own. Versions become a read-only <resource>_versions sub-resource addressed by a composite key, so the version axis is a filter rather than a second selection mechanism; a subtree becomes the relation-scoped list with a subtree scope, so the reply is entities and the caller assembles the tree. This is what removes the history and list_tree verbs.
  • Single and batch are paired, element-wise in their claims and set-wise in their atomicity. Every operation on one entity has a set form and a resource declares both or neither, so no entity is batch-less. Each element states its own precondition, because a batch of updates is a set of rows each at its own version and one version claim could not describe it; the batch then lands or fails as one transaction. Batch and page bounds are part of the protocol.
  • No verb is derived that the specification does not name. Whether an entity has a write verb is what the model declares, not something inferred from a version column or a repository flag. That inference is what leaves 22 resources with a list and no write today.
  • A write request carries only user-owned fields. tenant_id, party_id, version, modified_by, performed_by, recorded_at and the validity window never appear on the wire; the server derives them from the authenticated context. This follows service-to-service-delegation, which already states that delegation context travels in headers and that message structs carry only business data.
  • Change intent travels explicitly. The user-supplied reason and commentary are part of the write rather than server-owned, and their shape is stated once rather than restated per protocol.
  • Concurrency is an explicit precondition, not a sentinel. version = 0 currently conflates create, upsert, regeneration and revert, which is why a duplicate create can silently overwrite. The specification replaces it with a stated precondition.
  • One error model. Every response carries a single result with a stable error code. The transport X-Error header is reserved for failures that cannot produce a typed body, and the specification names which those are.
  • Pagination stays unconditional; filtering is additive. offset, limit and the total count remain unconditional members of every list, because the has_pagination fork was deliberately removed to stop clients disagreeing about whether they page. Filtering, sorting and the version axis are added without reintroducing it. A separate count verb is therefore not in the set.
  • Events get one prefix and a stated delivery contract. ores.{component}.{entity}_changed and {component}.v1.<resource>.<verb> currently coexist. The specification picks one, and states idempotency, ordering and the queue-group rule, which today follow per-registrar habit.
  • The specification is enforced by a gate, not by review. The twin coverage check proved that a written rule and its implementation drift silently, so the specification lands with a conformance check rather than as prose alone.
  • Adoption is one component at a time, starting with ores.iam. COMPONENTS_UNDER_TEST in component_registry.py is the single list of components under test, and a component joins it when its protocol is converted.

6. Out of scope

  • Components other than ores.iam. Their adoption follows once the specification and its conformance gate exist, and is tracked by the shared component list rather than by tasks in this story.
  • The wire encoding. JSON versus MessagePack is a process-wide choice already documented in nats_wire_format, and this story does not change it.
  • Transport, certificates and deployment. nats_certificates and the service hosting runbooks keep their own responsibilities.

7. Result

Abandoned at sprint 25 close. Sprint 26 replaces the codegen approaches tried so far with one method: for each module, sync codegen and clear its drift. Sprint 26 opens new, clean stories for each module. This story is not carried forward.

Emacs 29.3 (Org mode 9.6.15)