Story: Stream the changes as they happen
Table of Contents
This page documents a story in Sprint 29. It captures the goal, current status, acceptance criteria, and the tasks that compose it.
1. Goal
A client can follow the changes over one long response, and a quiet market does not look like a dead connection.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent sprint | Sprint 29 |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-11-04 |
3. Acceptance
- A change published to the hub reaches every open stream.
- A stream that is not read drops its oldest frames instead of growing without limit.
- A quiet stream writes a keep-alive comment and does not spin on the queue.
- A closed subscription ends the stream.
4. Tasks
| Task | State | Start | End | Description |
|---|---|---|---|---|
| Scaffold story: Stream the changes as they happen | DONE | 2026-10-31 | 2026-10-31 | Story scaffolding rides this task: documents, sprint wiring, and the scaffold PR. Close it before merging that PR. |
| Open the change stream | DONE | 2026-10-31 | 2026-10-31 | Initial task for: Stream the changes as they happen |
| Coalesce a burst into one frame | DONE | 2026-10-31 | 2026-10-31 | A bulk import writes thousands of rows and a stream that sends one frame per row is useless to a screen that redraws a list. Accumulate the changes into a dirty set per tenant, party and entity, and emit at most one frame per entity per tick, listing what changed. |
| Put the coalesced frames on the stream | DONE | 2026-10-31 | 2026-10-31 | The coalescer gathers a burst and the hub fans text to the streams, and nothing joins them. Format a coalesced frame as one event-stream message and pump what a window gathered into every open stream, so an event recorded anywhere reaches a client as one frame per entity. |
| Tick the window | DONE | 2026-10-31 | 2026-10-31 | The pump gives a window to every stream when something calls it, and nothing does. Run a timer that ticks it on an interval, stop cleanly when the server drains, and prove that a change recorded between ticks reaches a subscriber on the next one. |
| Hear the events that move the rows | DONE | 2026-10-31 | 2026-10-31 | Every store writes an entity event on a subject that names its collection, action and tenancy, and the change stream hears none of them. Subscribe once to every entity event, turn each into a change for the coalescer, and run the feed beside the ticker so a written row reaches an open stream. |
| Keep each stream to its own tenant, and let a dead one go | DONE | 2026-11-01 | 2026-11-01 | The stream fans every frame to every open stream, so one tenant reads another's changes, and a stream whose client is gone stays in the hub for the life of the process. Filter frames by the caller's tenant, close the subscription when its source dies, and hold the waker's timer safely. |
| Give the events a subject of their own | DONE | 2026-11-03 | The feed subscribes to *.v1.> because a NATS wildcard is a whole token, so *_events cannot be matched. That delivers every versioned request in the fleet to the gateway and throws it away, and it couples the stream to the request protocol. Move the events to <component>.v1.events.<collection>.<action> so one subscription hears events and nothing else. | |
| Give every frame an identity, so a client can resume | DONE | 2026-11-04 | 2026-11-04 | A frame carries no id: a client that reconnects cannot say what it last saw, so the stream cannot replay and cannot tell a reconnecting screen to resync. Number the frames, honour Last-Event-ID, and state the gap rule. |
| Keep a member to the parties it works in | DONE | 2026-11-04 | A stream hears its whole tenant, so a member reads the changes of parties they never look at. Filter the frames by party: a member hears their own party and the tenant-wide changes, and an administrator hears every party of the tenant. | |
| Conflate a burst into one window and one frame | DONE | 2026-11-04 | 2026-11-04 | The window is a fixed tick, so a burst that starts just after one is split in two, and a stream that is behind queues a frame per window. Arm the window with the first change, and let a newer frame replace the pending one for its collection. |
5. Decisions
A party is the second axis of a stream, and the tenant is still the first. A change that names no party concerns the whole tenant and reaches every party; one that names a party reaches that party and the administrator. The hub applies the rule on replay as well as on delivery, so a resume cannot widen a client's view.
A window opens with its first change and the rest do not extend it. A clock re-armed on a fixed interval cuts a burst that begins just after a tick in two, and the screen redraws twice for one import. The coalescer calls the ticker only when its set goes from empty to non-empty, so the burst's own first change starts its window and a quiet system arms nothing. The call is a post, because the change arrives on the publishing thread and the timer belongs to the server's.
A stream holds one frame per collection, and the tenancy is part of the key. A
frame says that a collection moved and the client refetches it, so the frames the
client never read carried nothing it could use: a newer frame replaces the pending
one rather than queueing behind it. The replaced frame moves to the back, so the
ids a client reads keep ascending and its Last-Event-ID stays a statement it can
rely on. The key carries the tenant and the party as well as the collection,
because the collection alone would let one party's change replace another's.
A frame that carries no id cannot be resumed, and a gap it cannot see is worse than a resync. The hub numbers every frame and keeps a short window of what it published, so a client inside the window is caught up and a client outside it is told to reload. Core NATS has no replay, so "tell it to resync" is the only honest answer for the rest.
The events namespace is a token of its own. One subscription hears every event
and no request: *.v1.events.>. The collection could not be wildcarded while it
was a suffix of another token, and the pattern that tried heard nothing at all.
A stream is kept to its tenant, not to whatever the hub publishes. The first version gave every frame to every stream, which made the tenant in the frame a label rather than a rule. The hub now filters on the tenant a subscription was opened for, and the system tenant reaches everyone because it is the catalogue they share. A caller with no tenant in its claims hears the system tenant alone.
A stream whose client is gone must leave. The hub holds its own reference to a subscription, so a source that closed nothing left the queue filling with every frame for the life of the process. The source closes it in its destructor.
A change names a collection, not a row. The first real producer settled it. An entity event's subject names the collection and the action, its headers name the tenant and the party, and its key is that entity's own key record, encoded by that entity's codec. One wildcard subscription cannot decode the key of every entity, so a frame says "this collection moved for this tenant" and the client refetches the collection it draws. Keys can return as an optimisation when a per-entity decoder exists; the frame's event name is then unchanged.
The stream hears the broker, not the database. The server subscribes to
*.v1.events.> and turns what it hears into changes. It does not read the
stores' notify channels itself, so a change published by another service process
reaches a client of this one.
6. Result
A burst costs one frame per collection however it falls across the clock, and a stream that was too slow to read holds one frame per collection rather than one per window, with the ids still ascending.
One slice, and the path it proves: a publish reaches every open stream, a slow reader loses its oldest frames rather than the server's memory, a quiet stream is kept alive without spinning, a closed subscription ends the body, and an entity event from any component becomes one frame per collection per window.
Proven end to end on the deployment: a signed-in caller opened /api/v1/changes
and read frames from iam.login_info, iam.sessions and
marketdata.delivery_counts, each naming its tenant and component, with the
keep-alive on the quiet stretch.
The live run also found two things the cases could not: the session door held no verifier, so no one could sign in, and the subscription's wildcard could never match, so the stream heard nothing. Both are fixed and both are captured.