Story: Rework the market data feeds into a market data service with adapters — the remaining work
Table of Contents
This page documents a story in Sprint 29, carried unfinished from Sprint 28 at close. It captures the goal, current status, acceptance criteria, and the tasks that compose it.
1. Goal
ores.marketdata owns the catalogue, the mapping, the ingest path, routing and metering. Adapters, including the simulator, publish vendor-id ticks over NATS and answer a demand set. Every simulated feed belongs to the system tenant and the system party, a party takes a provider by subscription, and no part of the system behaves differently because a source is simulated.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent sprint | Sprint 29 |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-11-08 |
3. Acceptance
The story closes on the tasks it finished:
- Add the demo approvers and document the market data setup.
- Add a provisioning to ticks smoke check.
- Address the blueprint review.
- Audit the market data documents for the service.
- Bind a report to the published curve.
- Build the cut screens.
- Build the cut take and its authorisation.
- Build the cut data model.
- Build the published curve screens.
- Build the market data sources board and the subscriptions screen.
- Close the approval model gaps.
- Complete the publish policies.
- Give the cut families and values system labels, and fix the history.
- Decide a cut's authorisation and apply it.
- Decide the binding's place in the route architecture.
- Decide whether ticks are stored once for each source.
- Design the published curve screens and decide the publish model gaps.
- Design the bootstrapping screens.
- Design the cut's journeys and screens.
- Document the market data path.
- Fix the post-merge checks.
- Fix the system lookups and the approval read.
- Light the intraday reports on the LIVE cut.
- Make the demo tenant exercise the system.
- Make the existing market data screens correct.
- Make the demand self-correcting and visible.
- Name a curve pillar's market series.
- Name the publish subject and record who published it.
- Notify a person that a scenario is ready.
- Parse the engine log when a run fails.
- Port the market data cuts reference.
- Reconcile the binding stories and the market data docs.
- Record the trades per book on the provisoned config.
- Remove the committed session cookie.
- Repair the web after the feed rename.
- Retire the feed binding.
- Route the synthetic feeds into the ingest.
- Schedule a report from its own page.
- Schedule the catalogue when Acme is provisioned.
- Seed the ladder family and let it fail like the rest.
- Set the intraday frequencies a desk expects.
- Test the market data service in a non-system tenant.
- Use oresmd on the market data generation path.
4. Tasks
| Task | State | Start | End | Description |
|---|---|---|---|---|
| Load published quotes from the public sources | ABANDONED | 2026-10-28 | Superseded by the story Publish public fixings through an adapter. | |
| Add a provisioning to ticks smoke check | BACKLOG | One script that provisions a tenant on a fresh database and asserts that ticks arrive. | ||
| Make the demand self-correcting and visible | DONE | 2026-10-25 | 2026-10-25 | Refresh the supervisor snapshot on a timer as well as on events, and show demand and ingest health in one status. |
| Test the market data service in a non-system tenant | BACKLOG | Run the demand, the ingest and the vintage tests with routes and subscriptions in a tenant that is not the system tenant. | ||
| Decide whether ticks are stored once for each source | DONE | 2026-10-25 | 2026-10-25 | Today the ingest writes one copy of a tick for each party that routes to the source. Decide if one shared copy with entitlement at read time is better. |
| Decide the binding's place in the route architecture | DONE | 2026-10-24 | 2026-10-24 | Settle whether a feed binding stays as the persisted consumer record or retires in favour of the route, before the ingest, the screen and the policy audit build on it. |
| Retire the feed binding | BACKLOG | Delete the feed binding and read the consumers from the routes, so one record says which party consumes which source. | ||
| Reconcile the binding stories and the market data docs | DONE | 2026-10-24 | Make the agile record and the market data docs say that the route is the consumer record and the binding is gone. | |
| Audit the market data documents for the service | DONE | 2026-10-25 | 2026-10-25 | The standards, journeys and prototype notes still describe the old feeds world: bindings, auto-start, feeds that should run, and real market data yet to arrive. |
| Make the existing market data screens correct | DONE | 2026-10-26 | The screens still read the old run model and some names no longer match the service. Make the names and the behaviour correct before adding anything. | |
| Build the market data sources board and the subscriptions screen | DONE | 2026-10-26 | 2026-10-26 | The two screens are signed off as prototypes and not built. Build them in the app. |
| Make the demo tenant exercise the system | DONE | 2026-10-26 | 2026-10-26 | Every screen must be reachable by the roles whose job it is, and Acme must hold the data to exercise them. |
| Add the demo approvers and document the market data setup | DONE | 2026-10-26 | 2026-10-26 | Every approval function needs a persona that can decide, and the Acme market data setup needs Zettelkasten documentation with UML flows. |
| Close the approval model gaps | DONE | 2026-10-27 | 2026-10-27 | The controller part is never asked for, a market data publish has no kind, a tenant-wide change names no decider, and a hold has no reason lookup. |
| Name the publish subject and record who published it | DONE | 2026-10-27 | 2026-10-27 | A publish request authorises any subject, and a curve has no published marker or publisher. |
| Design the published curve screens and decide the publish model gaps | DONE | 2026-10-27 | 2026-10-27 | The published-curves list, the delta and the queue subject need a prototype; the withdrawal, zero-approval, notification and maker-checker need decisions. |
| Build the published curve screens | STARTED | 2026-10-27 | Implement the signed-off prototype: the list, the record page with its four tabs, the last-N overlay, the comparison, the history and the queue subject. | |
| Design the bootstrapping screens | DONE | 2026-10-28 | 2026-10-28 | The Qt app has the IR curve bootstrapping screens; the web has none. The journeys and the prototype come before any code. |
| Complete the publish policies | DONE | 2026-10-28 | 2026-10-28 | The zero-approval path, the maker/checker pair and the publish notice are decided and not built. |
| Address the blueprint review | DONE | 2026-10-28 | 2026-10-28 | The blueprint needs a name in the database, a null discount curve instead of the nil sentinel, and the publisher drawn as a person. |
| Name a curve pillar's market series | DONE | 2026-10-29 | 2026-10-29 | Implement the decision that a pillar names its market series: model, reader, picker, seed and tests. |
| Document the market data path | DONE | 2026-10-28 | 2026-10-29 | One path from a public provider's fixing to a report, clean via oresmd, with the in-flight Frankfurter work covered. |
| Use oresmd on the market data generation path | DONE | 2026-10-29 | 2026-10-29 | The synthetic IR feed, the series classifier and the oresmd codec still spell data the ORE way at runtime. |
| Route the synthetic feeds into the ingest | STARTED | 2026-10-29 | The synthetic IR feeds publish, but no synthetic-provider tick reaches the store, so the bootstrap cannot be exercised live. | |
| Notify a person that a scenario is ready | DONE | 2026-10-30 | 2026-10-30 | A scenario is created but the person who should run it is never told; the Development area already renders the scenario. |
| Bind a report to the published curve | DONE | 2026-10-30 | 2026-10-30 | The signed-off target says a report references the published set; today it reads a curve configuration document. |
| Port the market data cuts reference | DONE | 2026-10-30 | 2026-10-30 | The external cuts reference defines the concept the report binding depends on; our docs have only fragments. |
| Design the cut's journeys and screens | DONE | 2026-10-30 | 2026-10-31 | The cut is the report binding's foundation: the fixed body at a timepoint for a reproducible valuation. Design its journeys and screens before the data model. |
| Build the cut data model | DONE | 2026-10-31 | 2026-10-31 | The cut design is signed off; build the data model: the cut type lookup, the cut entity, the report's selection, the resolution, and the retention. |
| Repair the web after the feed rename | DONE | 2026-10-31 | 2026-10-31 | The source entity became the feed; the generated source protocol is gone and the web still imports and names it. |
| Build the cut screens | DONE | 2026-10-31 | 2026-10-31 | The cut model exists; build its screens: the cut types in Reference data, and the cuts list and record in Reporting. |
| Fix the system lookups and the approval read | DONE | 2026-10-31 | 2026-10-31 | Live testing found the cut families and rules read as an empty list for a tenant, and the types screen refused for a normal role. |
| Give the cut families and values system labels, and fix the history | DONE | 2026-10-31 | 2026-10-31 | The cut types screen shows raw family names and ad-hoc value words, and the History tab fails because the cut entities' history providers are not registered. |
| Remove the committed session cookie | DONE | 2026-10-31 | 2026-10-31 | build/cj.txt holds a live curl session cookie and is tracked in git. |
| Build the cut take and its authorisation | DONE | 2026-10-31 | 2026-11-01 | The cut screens read only; the take flow, the body assembly and the approval request it raises are the next unit. |
| Decide a cut's authorisation and apply it | DONE | 2026-11-01 | 2026-11-01 | The take raises the request; the inbox decides and the reporting side must stamp the authorisation. |
| Light the intraday reports on the LIVE cut | DONE | 2026-11-03 | 2026-11-03 | The intraday reports exist unconfigured or unselected; every definition has a NULL cut and the configs say eod. |
| Schedule the catalogue when Acme is provisioned | DONE | 2026-11-03 | 2026-11-03 | A provisioned Acme ticks nothing: the catalogue states schedules but no one schedules the reports. The provisioner should, as the run's administrator, with its own credentials. |
| Fix the post-merge checks | DONE | 2026-11-03 | 2026-11-03 | The merged reports pages do not link their structure notes, and four web files are not formatted. |
| Parse the engine log when a run fails | DONE | 2026-11-03 | 2026-11-04 | A failed compute records the engine's whole output as one error; the parsed trade failures exist in the log but never reach the run's errors. |
| Seed the ladder family and let it fail like the rest | STARTED | 2026-11-04 | The thirteen ladder reports are documented but no definition states them; they can be created and fail at the engine like every other report until the inputs and the vol data arrive. | |
| Set the intraday frequencies a desk expects | DONE | 2026-11-04 | 2026-11-04 | The expensive triple runs every fifteen minutes, the surface every thirty, and the spot ladder and trades-since are on demand; a desk expects them to tick at a cadence that matches their cost. |
| Schedule a report from its own page | DONE | 2026-11-04 | 2026-11-04 | Only the shell and the provisioner can schedule a report; the app has no Schedule action and no way to activate an on-demand report. |
| Store one tick for each source | DONE | 2026-10-25 | 2026-11-09 | Write each tick once under the system tenant, point the per-party series at it, and remove the ingest's routes read. |
5. Decisions
- The intraday reports run at the cadences a desk expects, decided by the product owner on 2026-11-04 and stated in the catalogue, the pages and the live scheduler. The triple vol spot ladder and the surface vega ladder run hourly at every hour rather than inside the London day, because each costs a full sensitivity run and a desk reads the shape between the closes. The spot ladder runs every half hour and trades since last in every fifteen minutes, so neither is a definition that can never fire. The spike, strike and option maps keep the on-demand placeholder, and every other report keeps its cadence. The close twins do not move. See Set the intraday frequencies a desk expects.
- The ladder family is stated in the catalogue at every party and fails at the engine (2026-11-04). The product owner overruled the design's decision to hold it back until the report inputs and the volatility data arrive: a definition that produces no figure costs nothing to hold, and the list should show what the platform runs. Ten of the design's fourteen rows are seeded as fifteen definitions per party, because five of them run on both cycles, and they bind the sensitivity run settings a party already holds. Rows 11 to 14 stay unseeded on their market data. See Seed the ladder family and let it fail like the rest.
- A schedule is made by the one operation the shell calls, and a screen adds no
schedule of its own (2026-11-04). The BFF relays
schedule_report_definitionsandunschedule_report_definitionson the person's own session and answers their count, their failed ids and their words, gated on the permission each operation checks — the definition's write to schedule it, its delete to take the schedule away. Activating an on-demand report writes the expression the job will fire on and schedules it in one call, because the scheduler reads the expression from the definition. A definition with no scheduler job is on demand, and so is one whose expression never fires, so the list and the page read one word. See Schedule a report from its own page. - The tick store holds one tick for each source, owned by the system tenant, and the history read checks the subscription (2026-10-25). The per-party copy added 80% of the store on the measured tenant, and neither the usage recorder nor the delivery meter reads the store. See A source tick is stored once and entitled when it is read.
- Split from Rework the market data feeds into a market data service with adapters on 2026-10-24 at sprint 28 close. The acceptance above is copied from that story; its done tasks are recorded there.
- The route is the one consumer record (2026-10-24). The feed binding retires, with its two tables, its screen and its policies. A projection of the routes would keep two records for one fact and need a writer to hold them in step. This reverses the sprint 27 decision that consumption is a binding. Evidence and consequences are in Decide the binding's place in the route architecture.
- Closed on 2026-11-08 at sprint 29 close with its done tasks. The open tasks and the original acceptance moved to Rework the market data feeds into a market data service with adapters — the remaining work.