NATS entity protocol specification
Table of Contents
1. Summary
Every common, entity-related message that crosses the bus has the same shape. A
subject has four segments – component, version, resource, verb – or five when
it announces a change, and its resource is a plural noun. A request carries the operation's parameters and its
keys; a response carries one result, and the data the operation produced when
the outcome is ok.
Two rules govern what a message may assert. A write carries a write record of user-owned fields alone: tenancy, version and audit provenance are derived server-side from the authenticated context. And the verbs a resource supports are the ones its model declares, so what an entity can be asked for is a property of the model.
2. Detail
2.1. Scope
This specification defines the shape of the common, entity-related messages: the operations that read an entity, write an entity, and read an entity's versions, together with the events that announce that an entity changed. It applies to every component, and it is deliberately silent about which entities exist.
Four things are outside it and keep their own homes:
- The encoding. Whether the payload is JSON or MessagePack is a process-wide choice made once at startup, not a per-message negotiation. See NATS Wire Format.
- The transport, certificates and deployment. Those are operational concerns and are not part of a message's contract.
- Which components exist and which subjects they own. That is the namespace index in ORE Studio Messaging Reference.
- A component's own domain operations, beyond the grammar every subject must obey. See Domain operations below.
2.2. Vocabulary
| Term | Meaning |
|---|---|
| Component | The owner of a namespace. Every subject lives inside exactly one. |
| Resource | A plural noun naming the kind of entity a subject acts on, in snake_case. |
| Verb | The operation a subject performs, from the closed set below. |
| Subject | The routable address, {component}.v1.{resource}.{verb}. |
| Envelope | The transport metadata: subject, reply subject and headers. |
| Payload | The encoded request or response. |
| Key | The fields that identify one entity. It may be composite. |
| Write record | The user-owned fields of an entity, and the only part of an entity a client sends. |
| Server-owned | A field the service derives from the authenticated context and the database, and which never appears on the wire. |
2.3. Subject grammar
Every subject has four dot-separated segments, or five when it announces a
change, and the second is always the literal v1 rather than a variable. The
first and the last differ according to whether the subject performs an operation
or announces one:
{component}.v1.{resource}.{verb} -- a request
{component}.v1.events.{collection}.{action} -- an event
Each segment is lower snake_case and every segment is mandatory: a request has
four, an event five. The resource segment is a plural noun. An event carries the
events namespace in the resource's place, so the collection is a token of its
own, and the collection is a plural noun too.
2.3.1. The component namespace
A subject's first segment is the component that owns it, and the component that owns it is the one that serves it. A subject that no single component owns still belongs to one: a concept shared by every entity is owned by the component whose service answers for it, and there is no bare, ownerless top-level namespace.
This is what keeps request/reply correct. A request subject is subscribed by one component's queue group, so the reply comes from the service that owns the resource. A generic subject that every component subscribes to is answered by whichever service wins the race, which is not a property any caller can rely on.
2.3.2. The verb suffix
On a request subject, the last segment is one verb from the closed set. The
resource segment is a plural noun in snake_case. list_by_<relation> carries
one further word so that
a scoped read is still one subject per operation, and a sub-resource's name
carries the collection it holds after the resource it belongs to.
2.3.3. The subject prefix
A deployment may prepend a prefix to every subject, so that several deployments can share one bus without colliding. The prefix is configuration rather than protocol: subjects are written relative and the prefix is applied when a client connects. A message's contract is stated on the relative form.
2.3.4. One subject per operation
Each (resource, verb) pair has exactly one subject. Two components never
publish the same subject, and one component never declares the same subject
twice. Where a resource is reachable from two directions – a junction whose
rows are addressable by either side – the subject is still one, and the
direction is a field of the request rather than a second subject.
2.3.5. Sub-resources
A collection that belongs to one entity is a sub-resource, and it is a resource like any other: it has a name, a key, its own subjects, and it is read with the standard verbs. It is named for the entity it belongs to, followed by the collection:
{component}.v1.{resource}_{collection}.{verb}
A sub-resource exists because a read that returns something other than a page of the resource is not a read of the resource. The versions of an entity are the case in point: a read that returns version records is not a read of the entity, and giving it a verb of its own would put a second kind of thing behind the verb slot. Naming the collection instead keeps a request at four segments, keeps one subject per operation, and lets the collection inherit the list contract rather than restate it.
A sub-resource's key is composite: it is the parent's key together with the member's own identifier. The composite-key rule below already covers it, so a sub-resource needs no machinery that an ordinary composite-key entity does not already need.
A sub-resource may be read-only. Versions are written by the database and not
by a caller, so the versions collection declares list and get and no write
verb – for the same reason a read-only entity declares none.
2.4. The verb set
Eight verbs cover every operation on an entity. A component's resource may support any subset, but it may not invent a verb outside this set.
| Verb | Subject suffix | The operation it expresses |
|---|---|---|
get |
.get |
Read one entity, named by its full key. |
get_many |
.get_many |
Read the entities named by a set of keys, in one reply. |
list |
.list |
Read a page of entities matching a filter, in a stated order. |
list_by_<relation> |
.list_by_<relation> |
Read a page of entities scoped by a related entity's key, directly or through its whole subtree. |
put |
.put |
Create or replace one entity. |
put_many |
.put_many |
Create or replace a set of entities as one transaction. |
delete |
.delete |
Remove one entity, named by its full key. |
delete_many |
.delete_many |
Remove a set of entities as one transaction. |
The set is orthogonal: read one, read many, read a scoped page, write one, write many, remove one, remove many. Five consequences of it are deliberate.
- Single and batch are paired. Every operation on one entity has a set form,
and a resource declares both or neither:
getwithget_many,putwithput_many,deletewithdelete_many. The single form is the batch of one, so there is no per-model batching flag and no entity that can be written one at a time but not as a set. - A batch is one transaction. Several writes land together or not at all. Atomicity is a property of the set; the precondition is a property of the element (see Batching).
- A read that returns something other than a page of the resource is a read of that something. The versions of an entity and the entities beneath a node in a hierarchy are collections in their own right, read with the verbs above rather than with a verb of their own (see Sub-resources).
- There is no
countorcount_by_<relation>. Counting is a property of a list rather than an operation on an entity, and every list reports its total (see Reads). - There is no verb for reading everything. A full read is a list paged to its end, and a cache is fed by events rather than by a bulk read (see Events).
A model declares which verbs a resource supports. A read-only resource simply declares no write verb. A verb is never derived from a version column, a repository flag, or the presence of a route; if the model does not declare it, it does not exist, and if the model declares it, it is served.
2.5. Keys
A key identifies one entity, and it is the same key everywhere: in get, in
delete, in a versions read, and in the key fields of a write record.
A key is a record with one field per identifying column, each carrying the column's own type rather than a string. An entity whose identity is one column has a one-field key; an entity whose identity is composite has a record with all of its columns, and every operation that addresses it carries all of them. A composite key is never flattened into a single string and never partially sent, because a partial key addresses a row that need not exist.
A sub-resource's key is the parent's key with the member's own identifier alongside it – a version's key is the entity's key plus the version number – so a member of a sub-resource is addressed by a composite key and nothing else is needed to reach it.
Holding a natural key and a surrogate primary key is permitted, and the two are not interchangeable. Which one identifies the entity on the wire is a declaration of the model, made once. It is the same key that appears in the subject's resource's operations and in the HTTP projection's path parameter, so that a caller never has to translate between an address and a request.
2.6. Requests
A request is a record whose members are exactly the parameters of the operation. It contains:
- the operation's addressing: the key, the keys, or the filter;
- the operation's modifiers: a page, an order, a version range, a precondition;
- for a write, the write record and the change intent.
A request never contains:
- Identity or tenancy. Who is calling and which tenant and party they act
for are established once, by the connection and the token it carries, and
are read from the authenticated context. A request that carries
tenant_id,party_id, or a caller identity is malformed, because those values are not the caller's to state. See Security. - Server-owned fields. See Writes.
- Provenance. Audit fields are derived, never supplied.
- Correlation or idempotency. Those travel in the envelope's headers, so that a payload is entirely business data.
Every request names its reply in the type system, so a request's response type is not something a caller has to look up.
Message names follow the operation: <verb>_<resource>_request and
<verb>_<resource>_response, with the resource in the singular for operations
on one entity and the plural for operations on a set.
2.7. Responses and outcomes
Every response carries a result, and a result is the only place a caller looks to learn whether the operation happened.
result {
outcome : ok | invalid | denied | missing | conflict | unavailable | failed
code : string -- stable and machine-readable, empty when ok
message : string -- one sentence, safe to show to a person
fields : list of field_failure -- present only for invalid
}
The outcomes are a closed set, and each maps to exactly one status when the operation is projected onto HTTP:
| Outcome | Meaning | HTTP |
|---|---|---|
ok |
The operation happened. | 200 / 201 / 204 |
invalid |
The request is well-formed but violates the operation's contract. | 400 |
denied |
The caller may not perform this operation on this resource. | 403 |
missing |
The addressed entity does not exist. | 404 |
conflict |
The request's precondition does not hold, or the key already exists. | 409 |
unavailable |
A dependency the operation needs is not reachable. | 503 |
failed |
An invariant was broken or an unexpected error occurred. | 500 |
ok is the only outcome under which a payload's data is meaningful. A reply
whose outcome is anything else carries no data, and a caller that reads data
without checking the outcome is wrong.
A denied outcome is not an authentication failure. Authentication failures
happen before an operation is reached and are described by the envelope rather
than by a payload; see Security.
2.8. Reads
A read never modifies state, and it is safe to repeat. Every read is tenant-scoped by the authenticated context, and none of them accepts a tenant as a parameter.
2.8.1. list
list returns one page of entities and the total that the filter matches.
list_<resource>_request {
offset : uint32 -- default 0
limit : uint32 -- default 100
order : order -- the sort, stated explicitly
filter : <resource>_filter -- optional; absent means no filter
as_of : optional of string -- optional; the instant to resolve at
}
list_<resource>_response {
result : result
<resources> : list of <resource>
total : uint64 -- how many the filter matches, ignoring the page
}
Three rules make the contract unambiguous.
offset,limitandtotalare unconditional. Every list is paged, every list reports its total, and no caller has to discover whether this particular resource pages. The total is the count before the page is applied, so a caller can drive paging to the end without a second operation. That is what it costs: the count is taken on every list, whether or not the caller wants it, which makes every list two queries rather than one. Uniformity is bought with that and is worth it, because the alternative is a client that cannot know whether the number it holds is the total or the size of the page it happens to have.- Ordering is stated, never implied. A page of an unordered set is not reproducible, so the request names its sort and the service honours it. Where a caller asks for no particular order, the order is the model's default, and the key always breaks a tie, so that paging is stable across calls. See The stated order.
- Filtering is a record, not a query language. A filter is a record with optional, typed members. There is no expression to parse and no operator a caller chooses. A resource that supports no filtering simply has no filter record. See The filter record.
list replaces every read that returned a filtered set: an entity's versions
and the entities beneath a node in a hierarchy are scoped reads of collections
in their own right, and they differ from the plain case only in how the scope is
named.
2.8.2. The stated order
order {
field : string -- empty means the default order
descending : bool -- default false
}
The model states which fields a list can be ordered by. A column declares
:sortable: true, and the entity declares :default_order: in its flags: a
sortable column, optionally followed by desc. An entity that declares no
default order is ordered by its key.
- An empty field is the default order.
descendingwith an empty field reverses the default order. - A sortable field is honoured. The page is ordered by that field, in the direction stated.
- Any other field is refused. The outcome is
invalidwith the codeorder_not_supported, and the message names the field. A field that is not sortable is never ignored, because a caller that asked for an order and got another one pages through rows it cannot reproduce. - The key is the last tie-break. Rows with equal values in the ordered field are ordered by the key, ascending, whatever the stated direction. Two reads of the same set therefore return the same pages.
For example, an entity declares :default_order: name desc. A request with an
empty field and descending false orders by name descending. The same request
with descending true orders by name ascending. In both, rows with the same
name are in ascending key order.
An entity that keeps its history may state :sortable_recorded_at: true in its
flags. The list then accepts the field recorded_at, the time the current
version of the row was recorded, and the entity may name it in
:default_order:, as recorded_at desc. The model must index valid_from,
the column that holds it. A row that changes is recorded again, so it moves in
a list ordered by this field. The field gives the order of the latest change
and not the order of creation.
The allow-list is the model's and not the caller's, because an order by a column with no index is a full sort of the table on every page. A model makes a column sortable when it is prepared to index it.
2.8.3. The filter record
<resource>_filter has four kinds of member. Every member is optional. A
member that is absent sets no condition, and the members that are set must all
hold for a row to match.
<resource>_filter {
<column> : optional of <column type> -- equals
<column>_one_of : optional of list of <column type> -- is one of
<column>_from : optional of <column type> -- is within, lower bound
<column>_to : optional of <column type> -- is within, upper bound
search : optional of string -- contains
}
- Equals. Each filterable column has a member of the column's own type, and a
row matches when its column equals the value. The filterable columns are the
ones the model already reads by – the column the entity declares in
:list_filter_column:, and every relation column a scoped read is declared for – and every column that declares:filterable: true. A column of an enum type is filterable like any other. Its member has the enum's type, the wire carries the enum's text, and a row matches when the text it is stored as equals that text. The match is exact and case-sensitive, because an enum's text is a code, unlikesearch, which ignores case. The one-of member of an enum column is a list of that type. - Is one of. The primary key column, every relation column and every
filterable column have a member
that carries a list of the column's type. A row matches when its column
equals any value in the list. The primary key has a one-of member and no
equals member, because one key is a
get. This is how a screen reads the rows for the keys on its current page in one call – the parent of each party on a page, or the provisioning runs of the tenants on a page – with no fixed cap on the read. An empty list matches no row. A list carries at most1000values, the bound a key set carries (see get_many), and a longer list isinvalidwith the codefilter_too_large. The page still applies, so a caller setslimitto at least the length of the list when it wants every match. A page carries up to1000rows (see Batching), so one read returns every match of a full list. - Contains. A column declares
:searchable: true, and an entity with one or more searchable columns has asearchmember. A row matches when any searchable column contains the text, ignoring case. Case is folded as the database'slowerfolds it, so letters outside ASCII also match. The text is matched literally, so%and_are not wildcards. An empty string sets no condition. The text is at most 256 characters, because each searchable column is scanned whole; a longer text isinvalidwith the codefilter_too_large. - Is within. A column of a date or timestamp type that declares
:filterable: truealso has a_fromand a_tomember. A row matches when its column is not before_fromand not after_to: both bounds are inclusive. A member that is absent sets no bound, so a caller can state one end only. Both members carry text, because one member serves a date and a timestamp column. The text is a date,2026-03-05, or a UTC timestamp,2026-03-05T10:00:00Z, in the form theas_offield takes. A date on a timestamp column means a whole day: the start of that day for_fromand its last microsecond for_to, so that a person who asks for "up to the 5th" gets the whole of the 5th. The day is a day in UTC, because the store keeps every timestamp in UTC, and a screen that shows another zone converts the date before it sends it. A column of a date type takes a date alone, and the date is the bound itself. A text that is neither, or a date that is not on the calendar, isinvalidwith the codefilter_range_invalid, which names the member and the text. A_fromlater than the_tomatches no row, as an empty one-of list does. A row whose column is null never matches a range. Equals on a date or a timestamp stays available, but no screen should use it to find a period.
search is not a query language. It is one typed member with one fixed
meaning, and a caller cannot combine terms, choose columns or choose an
operator. It exists because a person looking for a row knows part of its name,
and a read by exact value cannot find it.
The filter applies before the page and before the count, so total is the
number of rows the filter matches.
2.8.4. Resolving a row at an instant
A row whose store keeps its validity window can be resolved at an instant rather
than at a version number. The list carries the instant in as_of: when it is
stated, the read returns the rows whose own window covers it – @c valid_from <=
as_of < valid_to – and when it is absent, the read is the ordinary page.
-- absent: one page of the current state
list_<resource>_request { offset, limit, order }
-- stated: every row that was current at that instant, in key order
list_<resource>_request { as_of }
Four things follow from it, and each is deliberate.
- It names an instant, not a version. A versions read names a version number and returns the row that version produced, validity window and all. This read names a point in time and returns what was current then. The two answer different questions and neither replaces the other.
- It is the entity's own window. A composite child's as-of read overlaps a parent version's window; this one resolves the entity's own row. The two are composed by a caller that wants the composite object as it stood.
- The instant is a condition, not a page bound. Every row current at that
instant is returned, so
offset,limitandtotaldo not narrow it. The reply'stotalis the size of the set it returned. - It is declared, not inferred. A resource offers it when the model says so. A model that does not declare it serves the page alone, and the field is absent from its request.
2.8.5. A read model
Some screens list rows that join several entities: a trade with its current state, its book and its identifiers. Each part is an entity of its own with its own key and its own timeline, and the filter record covers only the columns of one entity. A screen does not close that gap with a join in the client, because a join in the client breaks the total, the page and the order, and breaks every filter that combines columns of two parts. It also does not close it with a filter member that reaches into another entity. The gap is closed by an entity that is a read model.
A read model is an entity the model declares as derived from other entities. It has these rules.
- It has
listandgetand no write. It has noput, nodeleteand no version history of its own. A change goes to the part it belongs to. - Its filter record is the standard one. It has an equals member for each
column the model declares filterable, whichever part the column comes from, a
one-of member for the key and the relation columns, the range members, and
one
searchmember. A column of a part that has many rows for one row of the read model, such as the identifiers of a trade, is searched as one text that holds all of them, so a search matches any of them. The search is literal and ignores case, as every search does. - It answers one instant. The parts version on their own timelines. When a
read states
as_of, every part is resolved at that instant, so the state and the booking in a row come from the same instant. Whenas_ofis absent, every part is the current row. A read model that cannot honour this does not offeras_of. The model declares each part as required or optional. A required part with no row at the instant means the entity did not exist then, so the row is left out of the page and out oftotal: a trade booked afteras_ofis not listed. An optional part with no row leaves its columns null, and a collection part, such as the identifiers, is empty. - It has its own key and its own permission. The key is the key of its main part. The permission is a read permission of its own, because a person who may see a trade's state may not see everything the read model carries.
- It emits no change event. A screen of a read model listens to the events of its parts, and refreshes when one of them names a row that is on the page.
The first read model is the trade summary, which the trade list reads.
2.8.6. get
get returns exactly one entity, addressed by its full key.
get_<resource>_request {
key : <resource>_key
}
get_<resource>_response {
result : result
<resource> : optional of <resource>
}
There is no separate "found" flag: an entity that does not exist is the
missing outcome. ok always carries an entity, and missing never does.
get is the operation a screen uses to show one record, and it is the reason
a screen never has to read a page to find a row in it.
2.8.7. get_many
get_many returns the entities named by a set of keys, and answers in the
shape of its question.
get_many_<resources>_request {
keys : list of <resource>_key
}
get_many_<resources>_response {
result : result
entries : list of <resource>_lookup
}
<resource>_lookup {
key : <resource>_key
<resource> : optional of <resource>
}
The reply carries one entry per requested key, in the order the keys were asked for, so the correspondence between a request and its reply is positional and a caller never has to diff the two to find out what it did not get.
An entity that does not exist is an entry whose entity is empty, not a missing
entry and not a failure: asking for a set is not a claim that every member of
it exists, so the outcome stays ok and the caller reads absence from the
entry. Keys may repeat, and each occurrence gets its own entry, because a
positional reply is only unambiguous if it answers the question that was
asked.
A key set carries at most 1000 keys, the same bound a batch carries (see
Batching), so that one request cannot ask for an unbounded reply.
2.8.8. An entity's versions
An entity whose rows are versioned has a versions sub-resource, and its versions are read with the verbs every other collection uses.
list_<resource>_versions_request {
key : <resource>_key -- the entity whose versions these are
offset : uint32
limit : uint32
order : order
filter : <resource>_versions_filter
}
list_<resource>_versions_response {
result : result
versions : list of <resource>_version
total : uint64
}
get_<resource>_version_request {
key : <resource>_version_key
}
get_<resource>_version_response {
result : result
version : <resource>_version
}
<resource>_version_key {
<resource> : <resource>_key
version : uint32
}
A <resource>_version is a member of the versions collection: it carries the
version number, the audit provenance the service derived when it wrote that
version, and the entity's fields as they stood at that version. Because the
member type is the collection's own, reading versions is an ordinary read of a
resource and nothing about it is a special case.
The collection is read-only. A version is written by the database when the row
changes, never by a caller, so list and get are the only verbs it declares.
The version axis is a filter, not a second selection mechanism. A
<resource>_versions_filter record carries an exact version and the
from_version and to_version bounds, each optional and each typed, alongside
whatever else the entity's versions may be filtered by – who modified them, and
when they were recorded.
A point-in-time read then falls out of the same shape: a caller that wants the entity as it stood at version 7 lists the versions filtered to version 7, and a caller that wants the last month of changes lists the versions recorded in that window. There is no separate operation for either, and no second way to ask.
The parent's key is one field of the collection's key and also a field of the request, so a composite-key entity's versions are addressed exactly as the entity itself is.
2.8.9. list_by_<relation>
list_by_<relation> is list scoped to the entities related to one other
entity, addressed by that entity's key. It carries the relation's key in place
of part of the filter, and is otherwise list exactly: same paging, same
total, same stated order.
list_by_<relation>_<resources>_request {
<relation> : <relation>_key
scope : direct | subtree -- default direct
offset : uint32
limit : uint32
order : order
filter : <resource>_filter
}
list_by_<relation>_<resources>_response {
result : result
<resources> : list of <resource>
total : uint64
}
A relation is named in the subject suffix and its key is a field of the request, and the two always agree. Scoping by a related entity is not a filter, because an authorisation decision may depend on the relation, and a filter is not something a caller's access is evaluated against.
Where the relation is an entity's parent, subtree scopes the page to every
entity below the named one rather than to its immediate children. The reply is
still a page of entities, each carrying its own parent key as any entity does,
so a caller assembles the tree from the keys in the reply: it needs no second
read and no node type of its own. A page of a subtree need not be connected, so
a caller that renders a whole tree pages to the end of the subtree, which the
unconditional total lets it drive.
2.8.10. Declaring a scoped read
A model declares list_by_<relation> on the foreign key it scopes by, in the
Foreign keys section, with :list_by: true. The declaration goes on the
foreign key and not on the column, because the generator reads the relation from
the foreign key. It works for a column that is also part of the entity's primary
key: the trade identifiers are keyed by the trade and are read by it.
The subtree scope is available only where the relation is the entity's own
parent, a column that refers to the same entity, as a portfolio's parent
portfolio does. A foreign key to another entity serves direct only, and a
subtree request on it is refused with scope_not_supported.
A scoped read takes the entity's filter record, so a caller can narrow the rows
of one trade further. The filter members that name the relation column stay
valid, and they combine with the scope by AND, so a filter that names another
trade than the scope gives an empty page with a total of 0 and no error.
A scoped read is list exactly, so it carries as_of when the entity's list
does, and a record page that shows a past version of its parent reads the
children at that instant. The generator does not emit this yet: a generated
scoped read serves the current rows only. The task "Carry as_of on a scoped
read" of the story "Complete the trade list's filters" closes the gap.
:list_by_as_of: true does not declare it. That flag adds a window read to the
repository for the composite history of a parent, and it adds nothing to the
wire. A model does not declare it for a scoped read.
2.8.11. Finding an entity
A screen finds rows in one of these ways, in this order of preference. It never filters in the browser and never invents an operation.
| The need | The mechanism | Example |
|---|---|---|
| Rows of one entity by a column of its own | An equals or a one-of member on a filterable column | Trades of a counterparty, or of several trade types |
| Rows of one entity whose text contains what a person typed | The search member over the searchable columns |
A trade type typed in the search box |
| Rows of one entity within a period | The _from and _to members |
Trades agreed between two dates |
| Rows of one entity by an enum value | An equals or a one-of member of the enum's type | Trades with an inter-entity counterparty |
| The children of one row | A scoped read, list_by_<relation> |
The identifiers of a trade |
| The rows below a node of a hierarchy | A scoped read with scope subtree on the entity's own parent column |
The portfolios below a portfolio |
| Rows by a column of another entity | A read model that carries the column | Trades by state, by book and by identifier |
| The rows below a node of another entity's hierarchy | The screen reads the nodes first and sends their keys in a one-of member, at most 1000 | The trades of a portfolio and its subtree: the books below it, then book_id_one_of |
The server refuses a one-of list of more than 1000 keys with
filter_too_large, and the client limit is the same number. A screen that has
more than 1000 keys says so to the person and applies no scope. It never sends a part of the keys, because a
filter that silently drops some keys returns the wrong rows and the wrong total.
2.9. Writes
A write is a request to change one entity or a set of them. Every write states what the caller wants the entity to become, and what the caller believes about the entity it is replacing.
2.9.1. The write record
A client sends a write record: the entity's user-owned fields, and nothing else.
<resource>_write {
... the fields the user owns, and no others ...
}
The write record is a distinct type from the entity a read returns. A read returns the entity with its server-owned fields populated, because a caller needs to display and reason about them. A write carries the write record, because a caller has no business asserting them. The two are never the same type, so a client cannot accidentally round-trip a read into a write.
2.9.2. Server-owned fields
A field is server-owned when the service can derive it, and when a client asserting it would be a way to lie. Server-owned fields never appear in a write record:
| Field | Where it comes from |
|---|---|
| tenant | The authenticated context. |
| party | The authenticated context. |
| version | The database, which owns the version of every row. |
| modified by | The authenticated caller, from the validated token. |
| performed by | The service account that owns the running process, never a request. |
| recorded at | The database, as the instant the version was recorded. |
| valid from, valid to | The database, which owns the validity window. |
A service derives every one of these itself. It does not read them from a request, and it does not trust them if a request carries them anyway: a request that carries a server-owned field is invalid, so that a client is told it is wrong rather than silently ignored. This is the rule that makes tenancy and provenance trustworthy: no caller can write another tenant's data, or claim another person's authorship, because neither is a value the caller supplies.
2.9.3. Change intent
What a caller does supply about a write, beyond the entity's fields, is its intent: the reason the change is being made, and any commentary the operator recorded. Intent is user-owned data, so it travels in the write record's operation rather than being derived.
change_intent {
reason_code : string
commentary : string -- optional
}
Intention is required of every write, because a change with no stated reason
is not auditable. A service that rejects a write with no intent answers
invalid.
2.9.4. Preconditions
A write states what it believes about the entity it is about to change, and the service refuses the write when the belief is false. This is how two writers cannot silently overwrite each other, and how a create cannot silently become an overwrite.
precondition {
kind : any | must_not_exist | must_match_version
version : uint32 -- present only for must_match_version
}
| Precondition | The write lands when | Otherwise |
|---|---|---|
any |
Always. The write replaces whatever is there. | -- |
must_not_exist |
The key does not yet exist. | conflict |
must_match_version |
The row is at the stated version. | conflict |
The three are distinct on purpose. A create sends must_not_exist and fails
loudly when the key is taken, which is what makes a duplicate create
detectable. A caller updating a row it read sends must_match_version with the
version it read, and learns that someone else got there first. A caller
deliberately replacing a row, or republishing it, sends any. A revert is a
write like any other: it sends the fields of the version being restored and
states the version it believes is current, so it takes must_match_version and
lands as a new version rather than erasing the ones after it. No integer
doubles as a mode: there is no value of version that means "do not check",
because that is what conflated a create with an overwrite.
A write with no precondition is invalid. Stating the belief is not optional.
A precondition belongs to one change, never to a request that carries several.
A batch of updates is a set of rows each at its own version, so a single version
claim could not describe it, and a batch-level claim would either weaken the
check to any or reject a batch that is entirely valid.
2.9.5. Batching
Batching is how a caller changes a set of entities in one operation. It is not a separate family of messages: a batch is the same element repeated, and the single-entity verb is the batch of one.
<resource>_change {
write : <resource>_write
precondition : precondition
}
put_<resource>_request {
change : <resource>_change
intent : change_intent
}
put_<resource>_response {
result : result
<resource> : <resource>
}
put_many_<resources>_request {
changes : list of <resource>_change
intent : change_intent
}
put_many_<resources>_response {
result : result
<resources> : list of <resource>
}
A write answers with the entity as it now stands, so that the caller learns the version and the audit provenance the service assigned without a second read. A batch answers with the entities in the order the changes were sent.
Four rules make batching coherent across every entity.
- Every change states its own precondition. The element is where the belief lives, so a batch may create one entity and update another, and each is checked against what the caller actually believes about it. A verb's single form and its batch form share one element type, so they cannot drift apart in what they permit.
- The batch is atomic; the elements are independent in what they claim. The
set lands together or not at all. A precondition that fails anywhere makes
the whole operation
conflict, and nothing is written – which is what makes a batch safe to retry, because a half-applied batch is not a state a caller has to reason about. - Intent is stated once for the set. A batch is one act by one operator, so the reason for it is one reason. Per-element intent would let one operation tell several different stories about why it happened.
- A batch has a bound. A batch carries at most
1000elements and a page carries at most1000entities, with a list'slimitdefaulting to100. The bounds are part of the protocol rather than a runtime surprise, so a caller can size its chunks and a generated message can enforce them. A request that exceeds a bound isinvalid.
A batch is not a way to escape authorisation. Every element is authorised as if
it had been sent alone, and the operation is denied if any element is not
permitted, so a batch cannot reach a row its author would not have reached
singly.
2.9.6. delete and delete_many
A delete names the key of what it removes and carries its own precondition, on the same terms as a write: removing a row a caller has not read is a claim that it wants to remove whatever is there. The reason the deletion is recorded travels as change intent.
<resource>_removal {
key : <resource>_key
precondition : precondition
}
delete_<resource>_request {
removal : <resource>_removal
intent : change_intent
}
delete_many_<resources>_request {
removals : list of <resource>_removal
intent : change_intent
}
The element carries the precondition for the reason a write's element does: a batch of deletions is a set of rows at different versions, and a caller removing several rows it read is claiming something about each one. Both responses carry a result and nothing else, because a deletion has no data to return.
A removal states any or must_match_version; must_not_exist describes a
create and has no meaning for a removal, so a removal that states it is
invalid. A row that does not exist is missing and a row that exists at
another version is conflict, so a caller can tell "someone else removed it"
from "someone else changed it". Under any both are ok: a removal with
nothing left to remove has still achieved what it asked for, which is what
makes a retried delete safe.
The batch is one transaction, and it does not report per-entity success: the whole operation either happened or it did not.
2.10. Events
A change to an entity is announced as an event, and events are a separate namespace from request/reply.
{component}.v1.events.{collection}.{action}
An event subject takes one segment more than a request and stays inside its
component's namespace. Its namespace segment is the literal events, the
collection follows it, and its last segment is the action – created,
updated or deleted – which is an action and not a verb because an event reports what
happened and no caller asked for it.
Every event carries:
event {
event_id : uuid -- identifies this publication
key : <resource>_key
action : created | updated | deleted
version : uint32 -- the version the change produced
occurred_at : timestamp
correlation_id : string -- optional; the request that caused the change
}
2.10.1. Tenancy of an event
An event belongs to a tenant, and to a party when the row it announces belongs to one. Neither travels in the payload, because the payload is business data and a message never states its own tenancy. Neither travels in the subject, because the subject names the collection and keeps the grammar's segments. Both travel in the envelope, in two headers:
| Header | Value |
|---|---|
X-Tenant-Id |
The tenant that owns the row. Always present on an event. |
X-Party-Id |
The party that owns the row. Present only on an event of a party-owned table. |
The publisher sets them from the row the store reported, never from a caller. An
event without X-Party-Id concerns the whole tenant. An event of the system
tenant concerns every tenant, because system rows (change reasons,
classification lists) are shared.
A subscriber that serves people applies the headers before it passes news on.
The web BFF passes an event to a session only when its tenant is the session's
tenant or the system tenant, and, when the event names a party, only when that
party is one the session may see. A subscriber that serves a service, such as an
entity cache, keeps the tenants it holds and ignores the rest. A subscriber that
finds no X-Tenant-Id on an event passes nothing on, because an event whose
tenant is unknown cannot be shown to be the reader's.
The broker does not enforce this. Every client that holds a certificate can subscribe to every event subject, so the rule is kept by the subscriber, and a relay that serves browsers is where it matters. Moving the tenant into the subject, or giving each tenant its own broker account, would let the broker enforce it. That changes the grammar and the broker's provisioning, so it is recorded here as a later option and is not part of this version.
2.10.2. The record an event names
An event names the record it announces. The payload of every entity event holds
key, the entity's declared key as an object of its columns, such as
{"username": "diane.kay"} for an account. The key is business data about the
entity, so it is in the payload and not in a header. A header is for what a
subscriber must decide before it reads the payload, which the tenant and the
party are and the key is not.
A subscriber that cannot read the key from the notification publishes the event
with an empty key. An empty key names no record, and a reader treats the event
as a change to records it cannot name.
A relay that serves people may tell its readers which records changed, so that a screen showing one record can tell a change to that record from a change to another. It names the records only when there are few. The web BFF joins the events of one entity that arrive within 150 ms into one message, names up to 20 records, and names none when there are more or when any event had an empty key. A reader treats a message that names no records as a change to any record.
Events are published at least once and may arrive more than once, and they may
arrive out of order. A consumer is therefore idempotent and does not assume
ordering: it applies change by key and version, so that applying an event it has
already applied changes nothing, and ignoring an event older than the version it
holds changes nothing either. A consumer that needs the entity reads it with
get rather than assuming the event carried everything, so an event is a
notification and not a substitute for a read.
An event's version is the version of the row it announces, so the two views of
an entity's past agree: the versions sub-resource holds what each version was,
the events say when each was produced, and a consumer that has been away reads
the versions collection rather than replaying the event stream. The stream is
for reacting, and the collection is for catching up.
Request subjects and event subjects are served by different mechanisms and must not be mixed. A request subject is subscribed by one queue group, because a request must be answered exactly once by the service that owns the resource. Several queue groups on one request subject fan the request out and more than one reply races back, which is not something a request/reply caller can use. An event subject may have several independent consumers, because several subscribers each wanting every event is the point.
2.11. Security
2.11.1. Identity
A caller is authenticated before any operation is reached, and its identity is carried by the connection in the envelope, never in a payload. Every operation runs against that identity. A payload cannot assert one.
2.11.2. Tenancy
Tenant and party are derived from the authenticated token and applied by the service. They are not request parameters and not filter fields. A caller therefore cannot read or write outside its own tenancy by asking to, and the rule that enforces this is the same rule that keeps server-owned fields off the wire – there is nothing to override because there is no field to send.
2.11.3. Cross-tenant access
Operations that act outside the caller's own tenancy are not expressed by widening a request. They are addressed by a subject in a reserved administrative namespace, authorised separately and more narrowly than the ordinary subject, and audited as privileged. This keeps the common path free of an override and makes every privileged operation a visible address rather than an argument.
2.11.4. Authorisation
Authorisation is per operation and per resource. The service checks it before
performing the operation, and answers denied when it fails. A verb that
modifies state is never authorised by the same permission as a verb that reads,
so that read access cannot be widened into write access by a request that names
a different verb.
Authorisation is not the client's concern: a caller does not ask whether it may do something, it does it and is told. A capability a client displays is a hint, not a grant.
2.11.5. Headers
Envelope headers are the registry for everything that is not business data.
| Header | Purpose |
|---|---|
Authorization |
The caller's token, as Bearer <token>. |
X-Delegated-Authorization |
A token for acting on behalf of another principal. |
X-Correlation-Id |
Correlates every message caused by one originating request. |
X-Idempotency-Key |
Makes a command safe to repeat. |
X-Tenant-Id |
On an event: the tenant that owns the row. See Tenancy of an event. |
X-Party-Id |
On an event of a party-owned table: the party that owns the row. |
X-Error |
The transport failure code, on an empty reply. |
A service propagates the correlation id onto every message it publishes while serving a request, so that one cause is traceable across components.
A command that cannot be expressed idempotently at the data layer carries an idempotency key, and a service that sees a key it has already served returns the original result rather than performing the operation twice. This is what makes a retry safe on a bus that delivers at least once.
Authorization, X-Delegated-Authorization and X-Error are handled at the
transport boundary, before a payload is decoded, so that a failure to
authenticate is reported even though no operation was reached. X-Error
carries one of a small closed set of codes on a reply with an empty body, and
covers only those failures: everything an operation itself decides is reported
by the result inside the response.
2.12. Versioning
The v1 segment names the version of this protocol. It changes only when a
message's shape changes incompatibly.
Within a version, change is additive. A new field is optional, a new verb is a new subject, and an existing field keeps its meaning and its type. A caller that ignores a field it does not know about continues to work, so a service may add to a response without coordinating every client.
Where a shape must change incompatibly, the component serves the new version alongside the old for a deprecation window and then withdraws the old. Serving two versions is a transition, not a steady state: a component carries one version in the normal case.
2.13. Domain operations
Not every message is a common entity operation. A component may need an operation that is genuinely its own: an import that runs a job, a curve that is republished, a workflow that is started.
A domain operation lives in a reserved namespace inside its component, so that it is visibly not a common entity operation:
{component}.v1.ops.{operation}
A domain operation obeys everything above that applies to it: the envelope, the
result, the headers, the server-owned field rule, the security rules and the
version rule. What it does not have is a resource and a verb from the common
set, because it does not act on an entity in the common way.
The namespace is reserved so that domain operations are enumerable and distinguishable. An operation that acts on an entity the common way belongs in the common set, and an operation that merely has an unusual name does not qualify as a domain operation.
events is reserved on the same terms, for the announcements rather than for
the operations. It sits in the resource's place, so an event has one segment
more than a request, and no entity may be named events: a resource with that
name would be indistinguishable from the announcements, and the wildcard that
hears every event would start hearing its requests too.
2.14. HTTP projection
A client that speaks HTTP rather than the bus reaches the same operations through a projection, and the projection is mechanical: one operation, one route. It exists so that a browser client does not need a second, separately invented contract.
| Verb | Method and path |
|---|---|
get |
GET /{resource}/{key} |
get_many |
GET /{resource}?keys…= |
list |
GET /{resource}?offset=&limit=&sort=&descending=&<filter> |
list_by_<relation> |
GET /{resource}?<relation>…&scope=subtree= |
list (versions) |
GET /{resource}/{key}/versions?offset=&limit=&sort=&descending=&<filter> |
get (a version) |
GET /{resource}/{key}/versions/{version} |
put |
PUT /{resource}/{key}, or POST /{resource} to create |
put_many |
PUT /{resource} |
delete |
DELETE /{resource}/{key} |
delete_many |
DELETE /{resource} |
A sub-resource nests under its parent here, which is the one place the projection and the bus differ in shape rather than in contract: HTTP has a path to nest in and a subject does not, so the sub-resource is named in the resource segment instead. The verbs, the keys, the filters and the outcomes are the same on both sides.
Order and filter become query parameters. order names the field and
descending=true reverses it. An equals member is <column>=<value>, a one-of
member repeats <column>_one_of once per value, and search carries the text.
offset, limit, order, descending, search, scope, as_of and
keys are reserved. The generator refuses a model whose filter member has one
of these names, so a parameter always has one meaning.
The projection does not restate the contract. The key in the path is the key in
the request, the filter is the filter record, and the result is the HTTP
status: a caller reading the status learns exactly what a caller reading the
outcome learns. A route exists for an operation when the model declares the
operation, so a route never advertises an operation the service does not
perform.
Identity and tenancy follow the same rule as on the bus. They are carried by the session and never by the request body.
2.15. Conformance
Every clause above that can be checked mechanically is checked mechanically, and a generated message that violates one fails the check rather than passing review.
A conforming message has a subject of four segments, or five when it announces
a change, whose component owns it and whose last segment is in the verb set or
in the event actions; a key that carries every identifying
column with its own type; a response that carries a result and no data
outside ok; a list whose offset, limit and total are unconditional, and
whose order is from the model's sortable fields with the key as the last
tie-break, and whose filter members are the ones the model declares; a write
record with no server-owned field and no tenancy; a single-entity verb that comes with its batch form; a batch whose
every element carries its own precondition; a batch or page within its bound; a
collection subordinated to an entity that is named as a sub-resource rather than
given a verb of its own; and a name that follows <verb>_<resource>_request and
<verb>_<resource>_response.
Conformance is enforced for the components that have adopted this specification. A component adopts it whole – there is no partial conformance, because a component half on this protocol is a component whose messages cannot be reasoned about – and the components that have adopted it are named by one list, so that the set of components under the gate is a single fact rather than a per-check opinion.
3. See also
- Message Queue — the structure note that orders this cluster, and where to read this page in it.
- NATS Wire Format — the envelope and the payload encoding this specification assumes.
- ORE Studio Messaging Reference — which component owns which namespace.
- Optimistic concurrency versioning — why the version is the database's to assign.
- Polymorphic types over NATS — one subject per concrete type, and why a variant payload field is not used.
- History diff architecture — how a version's difference from its predecessor is presented.
- Entity mirror cache placement — where an event-fed mirror lives.
- Service-to-service delegation — the header registry, and why message structs carry only business data.
- NATS multitenancy design — the token claims tenancy is derived from.
- Service RBAC design — the permission scheme authorisation uses.
- Data-oriented design — the wire criteria a message is reviewed against.
- Code review checklist — the protocol rules a reviewer checks.