NATS entity protocol specification

Table of Contents

1. Summary

Every common, entity-related message that crosses the bus has the same shape. A subject has four segments – component, version, resource, verb – or five when it announces a change, and its resource is a plural noun. A request carries the operation's parameters and its keys; a response carries one result, and the data the operation produced when the outcome is ok.

Two rules govern what a message may assert. A write carries a write record of user-owned fields alone: tenancy, version and audit provenance are derived server-side from the authenticated context. And the verbs a resource supports are the ones its model declares, so what an entity can be asked for is a property of the model.

2. Detail

2.1. Scope

This specification defines the shape of the common, entity-related messages: the operations that read an entity, write an entity, and read an entity's versions, together with the events that announce that an entity changed. It applies to every component, and it is deliberately silent about which entities exist.

Four things are outside it and keep their own homes:

  • The encoding. Whether the payload is JSON or MessagePack is a process-wide choice made once at startup, not a per-message negotiation. See NATS Wire Format.
  • The transport, certificates and deployment. Those are operational concerns and are not part of a message's contract.
  • Which components exist and which subjects they own. That is the namespace index in ORE Studio Messaging Reference.
  • A component's own domain operations, beyond the grammar every subject must obey. See Domain operations below.

2.2. Vocabulary

Term Meaning
Component The owner of a namespace. Every subject lives inside exactly one.
Resource A plural noun naming the kind of entity a subject acts on, in snake_case.
Verb The operation a subject performs, from the closed set below.
Subject The routable address, {component}.v1.{resource}.{verb}.
Envelope The transport metadata: subject, reply subject and headers.
Payload The encoded request or response.
Key The fields that identify one entity. It may be composite.
Write record The user-owned fields of an entity, and the only part of an entity a client sends.
Server-owned A field the service derives from the authenticated context and the database, and which never appears on the wire.

2.3. Subject grammar

Every subject has four dot-separated segments, or five when it announces a change, and the second is always the literal v1 rather than a variable. The first and the last differ according to whether the subject performs an operation or announces one:

{component}.v1.{resource}.{verb}            -- a request
{component}.v1.events.{collection}.{action} -- an event

Each segment is lower snake_case and every segment is mandatory: a request has four, an event five. The resource segment is a plural noun. An event carries the events namespace in the resource's place, so the collection is a token of its own, and the collection is a plural noun too.

2.3.1. The component namespace

A subject's first segment is the component that owns it, and the component that owns it is the one that serves it. A subject that no single component owns still belongs to one: a concept shared by every entity is owned by the component whose service answers for it, and there is no bare, ownerless top-level namespace.

This is what keeps request/reply correct. A request subject is subscribed by one component's queue group, so the reply comes from the service that owns the resource. A generic subject that every component subscribes to is answered by whichever service wins the race, which is not a property any caller can rely on.

2.3.2. The verb suffix

On a request subject, the last segment is one verb from the closed set. The resource segment is a plural noun in snake_case. list_by_<relation> carries one further word so that a scoped read is still one subject per operation, and a sub-resource's name carries the collection it holds after the resource it belongs to.

2.3.3. The subject prefix

A deployment may prepend a prefix to every subject, so that several deployments can share one bus without colliding. The prefix is configuration rather than protocol: subjects are written relative and the prefix is applied when a client connects. A message's contract is stated on the relative form.

2.3.4. One subject per operation

Each (resource, verb) pair has exactly one subject. Two components never publish the same subject, and one component never declares the same subject twice. Where a resource is reachable from two directions – a junction whose rows are addressable by either side – the subject is still one, and the direction is a field of the request rather than a second subject.

2.3.5. Sub-resources

A collection that belongs to one entity is a sub-resource, and it is a resource like any other: it has a name, a key, its own subjects, and it is read with the standard verbs. It is named for the entity it belongs to, followed by the collection:

{component}.v1.{resource}_{collection}.{verb}

A sub-resource exists because a read that returns something other than a page of the resource is not a read of the resource. The versions of an entity are the case in point: a read that returns version records is not a read of the entity, and giving it a verb of its own would put a second kind of thing behind the verb slot. Naming the collection instead keeps a request at four segments, keeps one subject per operation, and lets the collection inherit the list contract rather than restate it.

A sub-resource's key is composite: it is the parent's key together with the member's own identifier. The composite-key rule below already covers it, so a sub-resource needs no machinery that an ordinary composite-key entity does not already need.

A sub-resource may be read-only. Versions are written by the database and not by a caller, so the versions collection declares list and get and no write verb – for the same reason a read-only entity declares none.

2.4. The verb set

Eight verbs cover every operation on an entity. A component's resource may support any subset, but it may not invent a verb outside this set.

Verb Subject suffix The operation it expresses
get .get Read one entity, named by its full key.
get_many .get_many Read the entities named by a set of keys, in one reply.
list .list Read a page of entities matching a filter, in a stated order.
list_by_<relation> .list_by_<relation> Read a page of entities scoped by a related entity's key, directly or through its whole subtree.
put .put Create or replace one entity.
put_many .put_many Create or replace a set of entities as one transaction.
delete .delete Remove one entity, named by its full key.
delete_many .delete_many Remove a set of entities as one transaction.

The set is orthogonal: read one, read many, read a scoped page, write one, write many, remove one, remove many. Five consequences of it are deliberate.

  • Single and batch are paired. Every operation on one entity has a set form, and a resource declares both or neither: get with get_many, put with put_many, delete with delete_many. The single form is the batch of one, so there is no per-model batching flag and no entity that can be written one at a time but not as a set.
  • A batch is one transaction. Several writes land together or not at all. Atomicity is a property of the set; the precondition is a property of the element (see Batching).
  • A read that returns something other than a page of the resource is a read of that something. The versions of an entity and the entities beneath a node in a hierarchy are collections in their own right, read with the verbs above rather than with a verb of their own (see Sub-resources).
  • There is no count or count_by_<relation>. Counting is a property of a list rather than an operation on an entity, and every list reports its total (see Reads).
  • There is no verb for reading everything. A full read is a list paged to its end, and a cache is fed by events rather than by a bulk read (see Events).

A model declares which verbs a resource supports. A read-only resource simply declares no write verb. A verb is never derived from a version column, a repository flag, or the presence of a route; if the model does not declare it, it does not exist, and if the model declares it, it is served.

2.5. Keys

A key identifies one entity, and it is the same key everywhere: in get, in delete, in a versions read, and in the key fields of a write record.

A key is a record with one field per identifying column, each carrying the column's own type rather than a string. An entity whose identity is one column has a one-field key; an entity whose identity is composite has a record with all of its columns, and every operation that addresses it carries all of them. A composite key is never flattened into a single string and never partially sent, because a partial key addresses a row that need not exist.

A sub-resource's key is the parent's key with the member's own identifier alongside it – a version's key is the entity's key plus the version number – so a member of a sub-resource is addressed by a composite key and nothing else is needed to reach it.

Holding a natural key and a surrogate primary key is permitted, and the two are not interchangeable. Which one identifies the entity on the wire is a declaration of the model, made once. It is the same key that appears in the subject's resource's operations and in the HTTP projection's path parameter, so that a caller never has to translate between an address and a request.

2.6. Requests

A request is a record whose members are exactly the parameters of the operation. It contains:

  • the operation's addressing: the key, the keys, or the filter;
  • the operation's modifiers: a page, an order, a version range, a precondition;
  • for a write, the write record and the change intent.

A request never contains:

  • Identity or tenancy. Who is calling and which tenant and party they act for are established once, by the connection and the token it carries, and are read from the authenticated context. A request that carries tenant_id, party_id, or a caller identity is malformed, because those values are not the caller's to state. See Security.
  • Server-owned fields. See Writes.
  • Provenance. Audit fields are derived, never supplied.
  • Correlation or idempotency. Those travel in the envelope's headers, so that a payload is entirely business data.

Every request names its reply in the type system, so a request's response type is not something a caller has to look up.

Message names follow the operation: <verb>_<resource>_request and <verb>_<resource>_response, with the resource in the singular for operations on one entity and the plural for operations on a set.

2.7. Responses and outcomes

Every response carries a result, and a result is the only place a caller looks to learn whether the operation happened.

result {
    outcome : ok | invalid | denied | missing | conflict | unavailable | failed
    code    : string   -- stable and machine-readable, empty when ok
    message : string   -- one sentence, safe to show to a person
    fields  : list of field_failure   -- present only for invalid
}

The outcomes are a closed set, and each maps to exactly one status when the operation is projected onto HTTP:

Outcome Meaning HTTP
ok The operation happened. 200 / 201 / 204
invalid The request is well-formed but violates the operation's contract. 400
denied The caller may not perform this operation on this resource. 403
missing The addressed entity does not exist. 404
conflict The request's precondition does not hold, or the key already exists. 409
unavailable A dependency the operation needs is not reachable. 503
failed An invariant was broken or an unexpected error occurred. 500

ok is the only outcome under which a payload's data is meaningful. A reply whose outcome is anything else carries no data, and a caller that reads data without checking the outcome is wrong.

A denied outcome is not an authentication failure. Authentication failures happen before an operation is reached and are described by the envelope rather than by a payload; see Security.

2.8. Reads

A read never modifies state, and it is safe to repeat. Every read is tenant-scoped by the authenticated context, and none of them accepts a tenant as a parameter.

2.8.1. list

list returns one page of entities and the total that the filter matches.

list_<resource>_request {
    offset : uint32          -- default 0
    limit  : uint32          -- default 100
    order  : order           -- the sort, stated explicitly
    filter : <resource>_filter   -- optional; absent means no filter
    as_of  : optional of string  -- optional; the instant to resolve at
}
list_<resource>_response {
    result : result
    <resources> : list of <resource>
    total  : uint64          -- how many the filter matches, ignoring the page
}

Three rules make the contract unambiguous.

  • offset, limit and total are unconditional. Every list is paged, every list reports its total, and no caller has to discover whether this particular resource pages. The total is the count before the page is applied, so a caller can drive paging to the end without a second operation. That is what it costs: the count is taken on every list, whether or not the caller wants it, which makes every list two queries rather than one. Uniformity is bought with that and is worth it, because the alternative is a client that cannot know whether the number it holds is the total or the size of the page it happens to have.
  • Ordering is stated, never implied. A page of an unordered set is not reproducible, so the request names its sort and the service honours it. Where a caller asks for no particular order, the order is the model's default, and the key always breaks a tie, so that paging is stable across calls. See The stated order.
  • Filtering is a record, not a query language. A filter is a record with optional, typed members. There is no expression to parse and no operator a caller chooses. A resource that supports no filtering simply has no filter record. See The filter record.

list replaces every read that returned a filtered set: an entity's versions and the entities beneath a node in a hierarchy are scoped reads of collections in their own right, and they differ from the plain case only in how the scope is named.

2.8.2. The stated order

order {
    field      : string   -- empty means the default order
    descending : bool     -- default false
}

The model states which fields a list can be ordered by. A column declares :sortable: true, and the entity declares :default_order: in its flags: a sortable column, optionally followed by desc. An entity that declares no default order is ordered by its key.

  • An empty field is the default order. descending with an empty field reverses the default order.
  • A sortable field is honoured. The page is ordered by that field, in the direction stated.
  • Any other field is refused. The outcome is invalid with the code order_not_supported, and the message names the field. A field that is not sortable is never ignored, because a caller that asked for an order and got another one pages through rows it cannot reproduce.
  • The key is the last tie-break. Rows with equal values in the ordered field are ordered by the key, ascending, whatever the stated direction. Two reads of the same set therefore return the same pages.

For example, an entity declares :default_order: name desc. A request with an empty field and descending false orders by name descending. The same request with descending true orders by name ascending. In both, rows with the same name are in ascending key order.

An entity that keeps its history may state :sortable_recorded_at: true in its flags. The list then accepts the field recorded_at, the time the current version of the row was recorded, and the entity may name it in :default_order:, as recorded_at desc. The model must index valid_from, the column that holds it. A row that changes is recorded again, so it moves in a list ordered by this field. The field gives the order of the latest change and not the order of creation.

The allow-list is the model's and not the caller's, because an order by a column with no index is a full sort of the table on every page. A model makes a column sortable when it is prepared to index it.

2.8.3. The filter record

<resource>_filter has four kinds of member. Every member is optional. A member that is absent sets no condition, and the members that are set must all hold for a row to match.

<resource>_filter {
    <column>        : optional of <column type>          -- equals
    <column>_one_of : optional of list of <column type>  -- is one of
    <column>_from   : optional of <column type>          -- is within, lower bound
    <column>_to     : optional of <column type>          -- is within, upper bound
    search          : optional of string                 -- contains
}
  • Equals. Each filterable column has a member of the column's own type, and a row matches when its column equals the value. The filterable columns are the ones the model already reads by – the column the entity declares in :list_filter_column:, and every relation column a scoped read is declared for – and every column that declares :filterable: true. A column of an enum type is filterable like any other. Its member has the enum's type, the wire carries the enum's text, and a row matches when the text it is stored as equals that text. The match is exact and case-sensitive, because an enum's text is a code, unlike search, which ignores case. The one-of member of an enum column is a list of that type.
  • Is one of. The primary key column, every relation column and every filterable column have a member that carries a list of the column's type. A row matches when its column equals any value in the list. The primary key has a one-of member and no equals member, because one key is a get. This is how a screen reads the rows for the keys on its current page in one call – the parent of each party on a page, or the provisioning runs of the tenants on a page – with no fixed cap on the read. An empty list matches no row. A list carries at most 1000 values, the bound a key set carries (see get_many), and a longer list is invalid with the code filter_too_large. The page still applies, so a caller sets limit to at least the length of the list when it wants every match. A page carries up to 1000 rows (see Batching), so one read returns every match of a full list.
  • Contains. A column declares :searchable: true, and an entity with one or more searchable columns has a search member. A row matches when any searchable column contains the text, ignoring case. Case is folded as the database's lower folds it, so letters outside ASCII also match. The text is matched literally, so % and _ are not wildcards. An empty string sets no condition. The text is at most 256 characters, because each searchable column is scanned whole; a longer text is invalid with the code filter_too_large.
  • Is within. A column of a date or timestamp type that declares :filterable: true also has a _from and a _to member. A row matches when its column is not before _from and not after _to: both bounds are inclusive. A member that is absent sets no bound, so a caller can state one end only. Both members carry text, because one member serves a date and a timestamp column. The text is a date, 2026-03-05, or a UTC timestamp, 2026-03-05T10:00:00Z, in the form the as_of field takes. A date on a timestamp column means a whole day: the start of that day for _from and its last microsecond for _to, so that a person who asks for "up to the 5th" gets the whole of the 5th. The day is a day in UTC, because the store keeps every timestamp in UTC, and a screen that shows another zone converts the date before it sends it. A column of a date type takes a date alone, and the date is the bound itself. A text that is neither, or a date that is not on the calendar, is invalid with the code filter_range_invalid, which names the member and the text. A _from later than the _to matches no row, as an empty one-of list does. A row whose column is null never matches a range. Equals on a date or a timestamp stays available, but no screen should use it to find a period.

search is not a query language. It is one typed member with one fixed meaning, and a caller cannot combine terms, choose columns or choose an operator. It exists because a person looking for a row knows part of its name, and a read by exact value cannot find it.

The filter applies before the page and before the count, so total is the number of rows the filter matches.

2.8.4. Resolving a row at an instant

A row whose store keeps its validity window can be resolved at an instant rather than at a version number. The list carries the instant in as_of: when it is stated, the read returns the rows whose own window covers it – @c valid_from <= as_of < valid_to – and when it is absent, the read is the ordinary page.

-- absent: one page of the current state
list_<resource>_request { offset, limit, order }
-- stated: every row that was current at that instant, in key order
list_<resource>_request { as_of }

Four things follow from it, and each is deliberate.

  • It names an instant, not a version. A versions read names a version number and returns the row that version produced, validity window and all. This read names a point in time and returns what was current then. The two answer different questions and neither replaces the other.
  • It is the entity's own window. A composite child's as-of read overlaps a parent version's window; this one resolves the entity's own row. The two are composed by a caller that wants the composite object as it stood.
  • The instant is a condition, not a page bound. Every row current at that instant is returned, so offset, limit and total do not narrow it. The reply's total is the size of the set it returned.
  • It is declared, not inferred. A resource offers it when the model says so. A model that does not declare it serves the page alone, and the field is absent from its request.

2.8.5. A read model

Some screens list rows that join several entities: a trade with its current state, its book and its identifiers. Each part is an entity of its own with its own key and its own timeline, and the filter record covers only the columns of one entity. A screen does not close that gap with a join in the client, because a join in the client breaks the total, the page and the order, and breaks every filter that combines columns of two parts. It also does not close it with a filter member that reaches into another entity. The gap is closed by an entity that is a read model.

A read model is an entity the model declares as derived from other entities. It has these rules.

  • It has list and get and no write. It has no put, no delete and no version history of its own. A change goes to the part it belongs to.
  • Its filter record is the standard one. It has an equals member for each column the model declares filterable, whichever part the column comes from, a one-of member for the key and the relation columns, the range members, and one search member. A column of a part that has many rows for one row of the read model, such as the identifiers of a trade, is searched as one text that holds all of them, so a search matches any of them. The search is literal and ignores case, as every search does.
  • It answers one instant. The parts version on their own timelines. When a read states as_of, every part is resolved at that instant, so the state and the booking in a row come from the same instant. When as_of is absent, every part is the current row. A read model that cannot honour this does not offer as_of. The model declares each part as required or optional. A required part with no row at the instant means the entity did not exist then, so the row is left out of the page and out of total: a trade booked after as_of is not listed. An optional part with no row leaves its columns null, and a collection part, such as the identifiers, is empty.
  • It has its own key and its own permission. The key is the key of its main part. The permission is a read permission of its own, because a person who may see a trade's state may not see everything the read model carries.
  • It emits no change event. A screen of a read model listens to the events of its parts, and refreshes when one of them names a row that is on the page.

The first read model is the trade summary, which the trade list reads.

2.8.6. get

get returns exactly one entity, addressed by its full key.

get_<resource>_request {
    key : <resource>_key
}
get_<resource>_response {
    result : result
    <resource> : optional of <resource>
}

There is no separate "found" flag: an entity that does not exist is the missing outcome. ok always carries an entity, and missing never does.

get is the operation a screen uses to show one record, and it is the reason a screen never has to read a page to find a row in it.

2.8.7. get_many

get_many returns the entities named by a set of keys, and answers in the shape of its question.

get_many_<resources>_request {
    keys : list of <resource>_key
}
get_many_<resources>_response {
    result  : result
    entries : list of <resource>_lookup
}
<resource>_lookup {
    key        : <resource>_key
    <resource> : optional of <resource>
}

The reply carries one entry per requested key, in the order the keys were asked for, so the correspondence between a request and its reply is positional and a caller never has to diff the two to find out what it did not get.

An entity that does not exist is an entry whose entity is empty, not a missing entry and not a failure: asking for a set is not a claim that every member of it exists, so the outcome stays ok and the caller reads absence from the entry. Keys may repeat, and each occurrence gets its own entry, because a positional reply is only unambiguous if it answers the question that was asked.

A key set carries at most 1000 keys, the same bound a batch carries (see Batching), so that one request cannot ask for an unbounded reply.

2.8.8. An entity's versions

An entity whose rows are versioned has a versions sub-resource, and its versions are read with the verbs every other collection uses.

list_<resource>_versions_request {
    key    : <resource>_key   -- the entity whose versions these are
    offset : uint32
    limit  : uint32
    order  : order
    filter : <resource>_versions_filter
}
list_<resource>_versions_response {
    result   : result
    versions : list of <resource>_version
    total    : uint64
}
get_<resource>_version_request {
    key : <resource>_version_key
}
get_<resource>_version_response {
    result  : result
    version : <resource>_version
}
<resource>_version_key {
    <resource> : <resource>_key
    version    : uint32
}

A <resource>_version is a member of the versions collection: it carries the version number, the audit provenance the service derived when it wrote that version, and the entity's fields as they stood at that version. Because the member type is the collection's own, reading versions is an ordinary read of a resource and nothing about it is a special case.

The collection is read-only. A version is written by the database when the row changes, never by a caller, so list and get are the only verbs it declares.

The version axis is a filter, not a second selection mechanism. A <resource>_versions_filter record carries an exact version and the from_version and to_version bounds, each optional and each typed, alongside whatever else the entity's versions may be filtered by – who modified them, and when they were recorded.

A point-in-time read then falls out of the same shape: a caller that wants the entity as it stood at version 7 lists the versions filtered to version 7, and a caller that wants the last month of changes lists the versions recorded in that window. There is no separate operation for either, and no second way to ask.

The parent's key is one field of the collection's key and also a field of the request, so a composite-key entity's versions are addressed exactly as the entity itself is.

2.8.9. list_by_<relation>

list_by_<relation> is list scoped to the entities related to one other entity, addressed by that entity's key. It carries the relation's key in place of part of the filter, and is otherwise list exactly: same paging, same total, same stated order.

list_by_<relation>_<resources>_request {
    <relation> : <relation>_key
    scope      : direct | subtree   -- default direct
    offset     : uint32
    limit      : uint32
    order      : order
    filter     : <resource>_filter
}
list_by_<relation>_<resources>_response {
    result      : result
    <resources> : list of <resource>
    total       : uint64
}

A relation is named in the subject suffix and its key is a field of the request, and the two always agree. Scoping by a related entity is not a filter, because an authorisation decision may depend on the relation, and a filter is not something a caller's access is evaluated against.

Where the relation is an entity's parent, subtree scopes the page to every entity below the named one rather than to its immediate children. The reply is still a page of entities, each carrying its own parent key as any entity does, so a caller assembles the tree from the keys in the reply: it needs no second read and no node type of its own. A page of a subtree need not be connected, so a caller that renders a whole tree pages to the end of the subtree, which the unconditional total lets it drive.

2.8.10. Declaring a scoped read

A model declares list_by_<relation> on the foreign key it scopes by, in the Foreign keys section, with :list_by: true. The declaration goes on the foreign key and not on the column, because the generator reads the relation from the foreign key. It works for a column that is also part of the entity's primary key: the trade identifiers are keyed by the trade and are read by it.

The subtree scope is available only where the relation is the entity's own parent, a column that refers to the same entity, as a portfolio's parent portfolio does. A foreign key to another entity serves direct only, and a subtree request on it is refused with scope_not_supported.

A scoped read takes the entity's filter record, so a caller can narrow the rows of one trade further. The filter members that name the relation column stay valid, and they combine with the scope by AND, so a filter that names another trade than the scope gives an empty page with a total of 0 and no error.

A scoped read is list exactly, so it carries as_of when the entity's list does, and a record page that shows a past version of its parent reads the children at that instant. The generator does not emit this yet: a generated scoped read serves the current rows only. The task "Carry as_of on a scoped read" of the story "Complete the trade list's filters" closes the gap.

:list_by_as_of: true does not declare it. That flag adds a window read to the repository for the composite history of a parent, and it adds nothing to the wire. A model does not declare it for a scoped read.

2.8.11. Finding an entity

A screen finds rows in one of these ways, in this order of preference. It never filters in the browser and never invents an operation.

The need The mechanism Example
Rows of one entity by a column of its own An equals or a one-of member on a filterable column Trades of a counterparty, or of several trade types
Rows of one entity whose text contains what a person typed The search member over the searchable columns A trade type typed in the search box
Rows of one entity within a period The _from and _to members Trades agreed between two dates
Rows of one entity by an enum value An equals or a one-of member of the enum's type Trades with an inter-entity counterparty
The children of one row A scoped read, list_by_<relation> The identifiers of a trade
The rows below a node of a hierarchy A scoped read with scope subtree on the entity's own parent column The portfolios below a portfolio
Rows by a column of another entity A read model that carries the column Trades by state, by book and by identifier
The rows below a node of another entity's hierarchy The screen reads the nodes first and sends their keys in a one-of member, at most 1000 The trades of a portfolio and its subtree: the books below it, then book_id_one_of

The server refuses a one-of list of more than 1000 keys with filter_too_large, and the client limit is the same number. A screen that has more than 1000 keys says so to the person and applies no scope. It never sends a part of the keys, because a filter that silently drops some keys returns the wrong rows and the wrong total.

2.9. Writes

A write is a request to change one entity or a set of them. Every write states what the caller wants the entity to become, and what the caller believes about the entity it is replacing.

2.9.1. The write record

A client sends a write record: the entity's user-owned fields, and nothing else.

<resource>_write {
    ... the fields the user owns, and no others ...
}

The write record is a distinct type from the entity a read returns. A read returns the entity with its server-owned fields populated, because a caller needs to display and reason about them. A write carries the write record, because a caller has no business asserting them. The two are never the same type, so a client cannot accidentally round-trip a read into a write.

2.9.2. Server-owned fields

A field is server-owned when the service can derive it, and when a client asserting it would be a way to lie. Server-owned fields never appear in a write record:

Field Where it comes from
tenant The authenticated context.
party The authenticated context.
version The database, which owns the version of every row.
modified by The authenticated caller, from the validated token.
performed by The service account that owns the running process, never a request.
recorded at The database, as the instant the version was recorded.
valid from, valid to The database, which owns the validity window.

A service derives every one of these itself. It does not read them from a request, and it does not trust them if a request carries them anyway: a request that carries a server-owned field is invalid, so that a client is told it is wrong rather than silently ignored. This is the rule that makes tenancy and provenance trustworthy: no caller can write another tenant's data, or claim another person's authorship, because neither is a value the caller supplies.

2.9.3. Change intent

What a caller does supply about a write, beyond the entity's fields, is its intent: the reason the change is being made, and any commentary the operator recorded. Intent is user-owned data, so it travels in the write record's operation rather than being derived.

change_intent {
    reason_code : string
    commentary  : string   -- optional
}

Intention is required of every write, because a change with no stated reason is not auditable. A service that rejects a write with no intent answers invalid.

2.9.4. Preconditions

A write states what it believes about the entity it is about to change, and the service refuses the write when the belief is false. This is how two writers cannot silently overwrite each other, and how a create cannot silently become an overwrite.

precondition {
    kind    : any | must_not_exist | must_match_version
    version : uint32   -- present only for must_match_version
}
Precondition The write lands when Otherwise
any Always. The write replaces whatever is there. --
must_not_exist The key does not yet exist. conflict
must_match_version The row is at the stated version. conflict

The three are distinct on purpose. A create sends must_not_exist and fails loudly when the key is taken, which is what makes a duplicate create detectable. A caller updating a row it read sends must_match_version with the version it read, and learns that someone else got there first. A caller deliberately replacing a row, or republishing it, sends any. A revert is a write like any other: it sends the fields of the version being restored and states the version it believes is current, so it takes must_match_version and lands as a new version rather than erasing the ones after it. No integer doubles as a mode: there is no value of version that means "do not check", because that is what conflated a create with an overwrite.

A write with no precondition is invalid. Stating the belief is not optional.

A precondition belongs to one change, never to a request that carries several. A batch of updates is a set of rows each at its own version, so a single version claim could not describe it, and a batch-level claim would either weaken the check to any or reject a batch that is entirely valid.

2.9.5. Batching

Batching is how a caller changes a set of entities in one operation. It is not a separate family of messages: a batch is the same element repeated, and the single-entity verb is the batch of one.

<resource>_change {
    write        : <resource>_write
    precondition : precondition
}
put_<resource>_request {
    change : <resource>_change
    intent : change_intent
}
put_<resource>_response {
    result     : result
    <resource> : <resource>
}
put_many_<resources>_request {
    changes : list of <resource>_change
    intent  : change_intent
}
put_many_<resources>_response {
    result      : result
    <resources> : list of <resource>
}

A write answers with the entity as it now stands, so that the caller learns the version and the audit provenance the service assigned without a second read. A batch answers with the entities in the order the changes were sent.

Four rules make batching coherent across every entity.

  • Every change states its own precondition. The element is where the belief lives, so a batch may create one entity and update another, and each is checked against what the caller actually believes about it. A verb's single form and its batch form share one element type, so they cannot drift apart in what they permit.
  • The batch is atomic; the elements are independent in what they claim. The set lands together or not at all. A precondition that fails anywhere makes the whole operation conflict, and nothing is written – which is what makes a batch safe to retry, because a half-applied batch is not a state a caller has to reason about.
  • Intent is stated once for the set. A batch is one act by one operator, so the reason for it is one reason. Per-element intent would let one operation tell several different stories about why it happened.
  • A batch has a bound. A batch carries at most 1000 elements and a page carries at most 1000 entities, with a list's limit defaulting to 100. The bounds are part of the protocol rather than a runtime surprise, so a caller can size its chunks and a generated message can enforce them. A request that exceeds a bound is invalid.

A batch is not a way to escape authorisation. Every element is authorised as if it had been sent alone, and the operation is denied if any element is not permitted, so a batch cannot reach a row its author would not have reached singly.

2.9.6. delete and delete_many

A delete names the key of what it removes and carries its own precondition, on the same terms as a write: removing a row a caller has not read is a claim that it wants to remove whatever is there. The reason the deletion is recorded travels as change intent.

<resource>_removal {
    key          : <resource>_key
    precondition : precondition
}
delete_<resource>_request {
    removal : <resource>_removal
    intent  : change_intent
}
delete_many_<resources>_request {
    removals : list of <resource>_removal
    intent   : change_intent
}

The element carries the precondition for the reason a write's element does: a batch of deletions is a set of rows at different versions, and a caller removing several rows it read is claiming something about each one. Both responses carry a result and nothing else, because a deletion has no data to return.

A removal states any or must_match_version; must_not_exist describes a create and has no meaning for a removal, so a removal that states it is invalid. A row that does not exist is missing and a row that exists at another version is conflict, so a caller can tell "someone else removed it" from "someone else changed it". Under any both are ok: a removal with nothing left to remove has still achieved what it asked for, which is what makes a retried delete safe.

The batch is one transaction, and it does not report per-entity success: the whole operation either happened or it did not.

2.10. Events

A change to an entity is announced as an event, and events are a separate namespace from request/reply.

{component}.v1.events.{collection}.{action}

An event subject takes one segment more than a request and stays inside its component's namespace. Its namespace segment is the literal events, the collection follows it, and its last segment is the action – created, updated or deleted – which is an action and not a verb because an event reports what happened and no caller asked for it.

Every event carries:

event {
    event_id     : uuid     -- identifies this publication
    key          : <resource>_key
    action       : created | updated | deleted
    version      : uint32   -- the version the change produced
    occurred_at  : timestamp
    correlation_id : string -- optional; the request that caused the change
}

2.10.1. Tenancy of an event

An event belongs to a tenant, and to a party when the row it announces belongs to one. Neither travels in the payload, because the payload is business data and a message never states its own tenancy. Neither travels in the subject, because the subject names the collection and keeps the grammar's segments. Both travel in the envelope, in two headers:

Header Value
X-Tenant-Id The tenant that owns the row. Always present on an event.
X-Party-Id The party that owns the row. Present only on an event of a party-owned table.

The publisher sets them from the row the store reported, never from a caller. An event without X-Party-Id concerns the whole tenant. An event of the system tenant concerns every tenant, because system rows (change reasons, classification lists) are shared.

A subscriber that serves people applies the headers before it passes news on. The web BFF passes an event to a session only when its tenant is the session's tenant or the system tenant, and, when the event names a party, only when that party is one the session may see. A subscriber that serves a service, such as an entity cache, keeps the tenants it holds and ignores the rest. A subscriber that finds no X-Tenant-Id on an event passes nothing on, because an event whose tenant is unknown cannot be shown to be the reader's.

The broker does not enforce this. Every client that holds a certificate can subscribe to every event subject, so the rule is kept by the subscriber, and a relay that serves browsers is where it matters. Moving the tenant into the subject, or giving each tenant its own broker account, would let the broker enforce it. That changes the grammar and the broker's provisioning, so it is recorded here as a later option and is not part of this version.

2.10.2. The record an event names

An event names the record it announces. The payload of every entity event holds key, the entity's declared key as an object of its columns, such as {"username": "diane.kay"} for an account. The key is business data about the entity, so it is in the payload and not in a header. A header is for what a subscriber must decide before it reads the payload, which the tenant and the party are and the key is not.

A subscriber that cannot read the key from the notification publishes the event with an empty key. An empty key names no record, and a reader treats the event as a change to records it cannot name.

A relay that serves people may tell its readers which records changed, so that a screen showing one record can tell a change to that record from a change to another. It names the records only when there are few. The web BFF joins the events of one entity that arrive within 150 ms into one message, names up to 20 records, and names none when there are more or when any event had an empty key. A reader treats a message that names no records as a change to any record.

Events are published at least once and may arrive more than once, and they may arrive out of order. A consumer is therefore idempotent and does not assume ordering: it applies change by key and version, so that applying an event it has already applied changes nothing, and ignoring an event older than the version it holds changes nothing either. A consumer that needs the entity reads it with get rather than assuming the event carried everything, so an event is a notification and not a substitute for a read.

An event's version is the version of the row it announces, so the two views of an entity's past agree: the versions sub-resource holds what each version was, the events say when each was produced, and a consumer that has been away reads the versions collection rather than replaying the event stream. The stream is for reacting, and the collection is for catching up.

Request subjects and event subjects are served by different mechanisms and must not be mixed. A request subject is subscribed by one queue group, because a request must be answered exactly once by the service that owns the resource. Several queue groups on one request subject fan the request out and more than one reply races back, which is not something a request/reply caller can use. An event subject may have several independent consumers, because several subscribers each wanting every event is the point.

2.11. Security

2.11.1. Identity

A caller is authenticated before any operation is reached, and its identity is carried by the connection in the envelope, never in a payload. Every operation runs against that identity. A payload cannot assert one.

2.11.2. Tenancy

Tenant and party are derived from the authenticated token and applied by the service. They are not request parameters and not filter fields. A caller therefore cannot read or write outside its own tenancy by asking to, and the rule that enforces this is the same rule that keeps server-owned fields off the wire – there is nothing to override because there is no field to send.

2.11.3. Cross-tenant access

Operations that act outside the caller's own tenancy are not expressed by widening a request. They are addressed by a subject in a reserved administrative namespace, authorised separately and more narrowly than the ordinary subject, and audited as privileged. This keeps the common path free of an override and makes every privileged operation a visible address rather than an argument.

2.11.4. Authorisation

Authorisation is per operation and per resource. The service checks it before performing the operation, and answers denied when it fails. A verb that modifies state is never authorised by the same permission as a verb that reads, so that read access cannot be widened into write access by a request that names a different verb.

Authorisation is not the client's concern: a caller does not ask whether it may do something, it does it and is told. A capability a client displays is a hint, not a grant.

2.11.5. Headers

Envelope headers are the registry for everything that is not business data.

Header Purpose
Authorization The caller's token, as Bearer <token>.
X-Delegated-Authorization A token for acting on behalf of another principal.
X-Correlation-Id Correlates every message caused by one originating request.
X-Idempotency-Key Makes a command safe to repeat.
X-Tenant-Id On an event: the tenant that owns the row. See Tenancy of an event.
X-Party-Id On an event of a party-owned table: the party that owns the row.
X-Error The transport failure code, on an empty reply.

A service propagates the correlation id onto every message it publishes while serving a request, so that one cause is traceable across components.

A command that cannot be expressed idempotently at the data layer carries an idempotency key, and a service that sees a key it has already served returns the original result rather than performing the operation twice. This is what makes a retry safe on a bus that delivers at least once.

Authorization, X-Delegated-Authorization and X-Error are handled at the transport boundary, before a payload is decoded, so that a failure to authenticate is reported even though no operation was reached. X-Error carries one of a small closed set of codes on a reply with an empty body, and covers only those failures: everything an operation itself decides is reported by the result inside the response.

2.12. Versioning

The v1 segment names the version of this protocol. It changes only when a message's shape changes incompatibly.

Within a version, change is additive. A new field is optional, a new verb is a new subject, and an existing field keeps its meaning and its type. A caller that ignores a field it does not know about continues to work, so a service may add to a response without coordinating every client.

Where a shape must change incompatibly, the component serves the new version alongside the old for a deprecation window and then withdraws the old. Serving two versions is a transition, not a steady state: a component carries one version in the normal case.

2.13. Domain operations

Not every message is a common entity operation. A component may need an operation that is genuinely its own: an import that runs a job, a curve that is republished, a workflow that is started.

A domain operation lives in a reserved namespace inside its component, so that it is visibly not a common entity operation:

{component}.v1.ops.{operation}

A domain operation obeys everything above that applies to it: the envelope, the result, the headers, the server-owned field rule, the security rules and the version rule. What it does not have is a resource and a verb from the common set, because it does not act on an entity in the common way.

The namespace is reserved so that domain operations are enumerable and distinguishable. An operation that acts on an entity the common way belongs in the common set, and an operation that merely has an unusual name does not qualify as a domain operation.

events is reserved on the same terms, for the announcements rather than for the operations. It sits in the resource's place, so an event has one segment more than a request, and no entity may be named events: a resource with that name would be indistinguishable from the announcements, and the wildcard that hears every event would start hearing its requests too.

2.14. HTTP projection

A client that speaks HTTP rather than the bus reaches the same operations through a projection, and the projection is mechanical: one operation, one route. It exists so that a browser client does not need a second, separately invented contract.

Verb Method and path
get GET /{resource}/{key}
get_many GET /{resource}?keys…=
list GET /{resource}?offset=&limit=&sort=&descending=&<filter>
list_by_<relation> GET /{resource}?<relation>…&scope=subtree=
list (versions) GET /{resource}/{key}/versions?offset=&limit=&sort=&descending=&<filter>
get (a version) GET /{resource}/{key}/versions/{version}
put PUT /{resource}/{key}, or POST /{resource} to create
put_many PUT /{resource}
delete DELETE /{resource}/{key}
delete_many DELETE /{resource}

A sub-resource nests under its parent here, which is the one place the projection and the bus differ in shape rather than in contract: HTTP has a path to nest in and a subject does not, so the sub-resource is named in the resource segment instead. The verbs, the keys, the filters and the outcomes are the same on both sides.

Order and filter become query parameters. order names the field and descending=true reverses it. An equals member is <column>=<value>, a one-of member repeats <column>_one_of once per value, and search carries the text. offset, limit, order, descending, search, scope, as_of and keys are reserved. The generator refuses a model whose filter member has one of these names, so a parameter always has one meaning.

The projection does not restate the contract. The key in the path is the key in the request, the filter is the filter record, and the result is the HTTP status: a caller reading the status learns exactly what a caller reading the outcome learns. A route exists for an operation when the model declares the operation, so a route never advertises an operation the service does not perform.

Identity and tenancy follow the same rule as on the bus. They are carried by the session and never by the request body.

2.15. Conformance

Every clause above that can be checked mechanically is checked mechanically, and a generated message that violates one fails the check rather than passing review.

A conforming message has a subject of four segments, or five when it announces a change, whose component owns it and whose last segment is in the verb set or in the event actions; a key that carries every identifying column with its own type; a response that carries a result and no data outside ok; a list whose offset, limit and total are unconditional, and whose order is from the model's sortable fields with the key as the last tie-break, and whose filter members are the ones the model declares; a write record with no server-owned field and no tenancy; a single-entity verb that comes with its batch form; a batch whose every element carries its own precondition; a batch or page within its bound; a collection subordinated to an entity that is named as a sub-resource rather than given a verb of its own; and a name that follows <verb>_<resource>_request and <verb>_<resource>_response.

Conformance is enforced for the components that have adopted this specification. A component adopts it whole – there is no partial conformance, because a component half on this protocol is a component whose messages cannot be reasoned about – and the components that have adopted it are named by one list, so that the set of components under the gate is a single fact rather than a per-check opinion.

3. See also

Emacs 29.3 (Org mode 9.6.15)