Task: Hear the events that move the rows
Table of Contents
This page documents a task in the Stream the changes as they happen story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
The change stream carries what the stores announce. One subscription hears every entity event, each message becomes the coarsest change that is true of it (this collection moved for this tenant), and the pump has something to drain on every tick.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Stream the changes as they happen |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-31 |
3. Acceptance
- One subscription hears every entity event, whatever its component or collection.
- A received event records one change naming the tenant, the party when the event has one, and the collection.
- An event that names no tenant records nothing, because no reader could be shown it.
- The coalescer and the frame name the collection rather than a row key, because a key is the entity's own record and no two entities share one.
- The feed runs beside the ticker in the server and stops when the server drains.
- A unit test proves each mapping without a broker, and the live stream test proves an event arrives as a frame.
4. Plan
The first real producer of a change decides the shape of a change, so the shape changes here rather than being carried into the feed.
- A subject names a collection and an action; it never names a row. The row key is the entity's own record, encoded by that entity's codec, so one subscription cannot decode the key of every entity. The change therefore names the component and the collection and nothing finer, and the coalescer drops the identifier it was scaffolded with. A client that hears a frame refetches the collection it draws; naming a row was an optimisation that no producer can supply.
- Parsing runs from the end of the subject, because the delivered subject
carries the deployment prefix: action, collection, version, component.
The collection token drops its
_eventssuffix, so it is the same name the route path uses. - The tenancy travels in the
X-Tenant-IdandX-Party-Idheaders, which the publisher sets. A message without a tenant is dropped: no reader could be shown it. change_feedtakes anores::nats::messageand records it, so a case builds a message and asserts the frame that a drain produces, with no broker in the test.- The server subscribes through the same registrar that already returns the subscriptions the service runner keeps alive, and runs the ticker from the runner's start hook. Both ends now exist, so the stream is no longer empty.
- The pattern is
*.v1.>. A NATS wildcard is a whole token, so the collection's_eventssuffix cannot be matched in the subscription and is filtered in the parse instead.
5. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit | |
|---|---|---|---|---|
| Core suite cases | 45 | ores.http.core.tests |
f32ce6b39f |
|
| Core suite assertions | 187 | ores.http.core.tests |
f32ce6b39f |
|
| Core suite cases before this task | 36 | ores.http.core.tests on 5be7a9469f |
5be7a9469f |
|
| Live subscription line | Hearing entity events on *.v1.> |
compass services logs ores.http.server |
f32ce6b39f |
|
| Live frames in one stream, 2 seconds | 8, from 3 components | curl -N -b <cookie> /api/v1/changes |
f32ce6b39f |
|
| Files main lost when #4293 merged | 174 | =git diff –diff-filter=D –name-only 9e500a223f^1 9e500a223f \ | wc -l= | 9e500a223f |
6. Notes
The live proof found what the cases could not. The check is: sign a browser in,
open /api/v1/changes, write something, read a frame. Three things blocked it in
turn, and each was a real defect rather than a test gap.
First, nobody could sign in. Signing in answered Authentication is not
configured although the log showed Fetched JWKS public key from IAM at startup:
the door was built with server.verifier() before the runner installed the
verifier. Fixed by Let the door reach the verifier.
$ curl -s -X POST http://127.0.0.1:21601/api/v1/auth/sessions \
-H 'Content-Type: application/json' \
-d '{"principal":"tenant_admin@acme_corporation","password":"…"}'
{"error":"Authentication is not configured"}
Second, the pattern heard nothing. With the door fixed, the stream opened and
delivered no frame at all, and the pump logged nothing, so nothing arrived from the
broker. The subscription was *.v1.*_events.>: a NATS wildcard is a whole token,
so *_events is a literal, and the server accepts the subscription and delivers
nothing to it. The pattern is now *.v1.> with the collection filtered in the
parse, and a case walks the pattern token by token so the spelling cannot regress
quietly. Captured as
An events collection cannot be wildcarded.
Third, main had lost the work twice over. The build could not link because
ores.iam.core called ores::synthetic::service::routed_code_count, which did
not exist; main was fixed while I waited. Then the merged increments turned out to
be gone from main entirely: merging #4293 deleted 174 files, including
change_feed, the collection-shaped change and the door's verifier. Captured as
A merge took an old tree over two merged increments,
and restored by hotfix/restore-the-feed-and-the-door.
What the frames prove. A session opened as tenant_admin@acme_corporation, then
a second sign-in. One stream carried three components' stores without the client
asking anything: iam.login_info and iam.sessions from the sign-in, and
marketdata.delivery_counts from the feed path. Every frame named its tenant and
its component, and the party was empty for the tenant-wide ones.
What is still not proven: nothing in this task. The unit cases cover the mapping and the frame, and the deployment covers the composition.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
8. PRs
9. Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | Local review of the restore-the-rest docs: no findings; misspell clean | 4 task docs | accept | Each restored line matches a replay of every PR since 6b28050821 without the bad commit |
10. Result
The stream now has a producer. change_feed turns one NATS message into one
change: the subject gives the component, the collection and the action, read
from the right so that the deployment's prefix does not matter, and the headers
give the tenant and the party. A message with no tenant records nothing.
The first real producer also decided what a change is. A change names the
collection and not the row, so change_event and coalesced_frame lost their
identifier, the coalescer keeps a set of dirty collections instead of a map to
identifiers, and a frame reads:
event: iam.tenant_types
data: {"tenant":"acme","party":"ops","component":"iam","entity":"tenant_types"}
The server subscribes once, in the same registrar that already returns the handles the runner keeps alive, and runs the ticker from the runner's start hook, so the hub is no longer empty. The window is 150 ms, the constant the design states.
The pattern the subscription uses is *.v1.>, not *.v1.*_events.>. A NATS
wildcard is a whole token, so the second spelling is a literal that matches
nothing; the server accepts it and delivers nothing, and the stream stayed silent
with no error to read. The live run below is what found it, after the unit cases
had passed the parser and the frames a hundred times. The feed rejects what is not
an event in the parse instead, and the design states the cost of that.
Proven on a running deployment, once the session door could sign a browser in and
the pattern could match: a signed-in caller opens /api/v1/changes, signs in a
second time, and reads
event: iam.login_info
data: {"tenant":"55ae77e3-e255-4bde-8dc6-22ee2250fabb","party":"","component":"iam","entity":"login_info"}
event: iam.sessions
data: {"tenant":"55ae77e3-e255-4bde-8dc6-22ee2250fabb","party":"","component":"iam","entity":"sessions"}
event: marketdata.delivery_counts
data: {"tenant":"55ae77e3-e255-4bde-8dc6-22ee2250fabb","party":"3956198f-2b62-4bad-9047-5bfd2bfd0793","component":"marketdata","entity":"delivery_counts"}
followed by the keep-alive comment. Three components' stores reached one open stream without the client asking for anything.