Sprint 30
Table of Contents
This page documents a sprint (Sprint 30) of ORE Studio. It captures the sprint's mission, current status, and the stories that compose it. For the surrounding context — version goals, sprint order, and product identity — see Version 0.
1. Mission
Finish moving off sqlgen and reflect-cpp, with a limit on the stories in flight.
2. Status
3. Stories
For the definitions of the themes see Themes.
3.1. Product
| Story | State | Start | End | Description |
|---|---|---|---|---|
| Open sprint 30 | DONE | 2026-11-08 | 2026-11-08 | Scaffold sprint 30, wire the version manifest and the agile index, bump the version and update vcpkg. |
| Build the sensitivity ladders | BACKLOG | A desk reads its sensitivities as ladders: rates delta and gamma, par-rate DV01, theta, vega ladders, risk reversal and strangle, and cross gamma, from sensitivity runs bound to a report. | ||
| Let a busy market move prices, not only the clock | BACKLOG | The market-wide arrival factor speeds up ticks today and leaves the price law alone: a panic arrives as more observations rather than as more variance, fatter tails or a gap. Carry the same shared factor into the price processes, with a volatility loading per feed, and settle whether a rate change moves time at all. | ||
| Add entity groups and finish the remaining codegen work — the remaining work | BACKLOG | Add an entity group annotation to the model and a group flag to codegen, and finish the remaining codegen work: the model drift fixes, the counterparty logo picker and the tenor convention messaging layer under codegen, and the literate template split. | ||
| Approve a reporting line change by the proposed manager | BACKLOG | A change to someone's reporting line becomes a four-eyes decision: it is proposed, and the person it would make their manager approves or declines it, like a role request. | ||
| Fix the defects the bond family review found | BACKLOG | The review of the bond family is done and found defects in its money columns and schedules. Fix each one. | ||
| Audit the policies that let the system tenant read every tenant | BACKLOG | Each policy that lets the system tenant read every tenant is either replaced by acting inside the tenant or listed as a reviewed exception with its reason, and a check fails the build on an unlisted one. | ||
| Author the report configurations on guided screens | BACKLOG | A person creates and edits a report configuration on screens that guide each step. A configuration holds many related entities, so each entity gets a screen that helps the person choose, check and connect its parts. The configuration screens link later to the report definition screens. | ||
| Bring the FX spot conventions onto the market standard | BACKLOG | Correct the FX spot conventions where they disagree with the market: the pair's spot lag, the quote precision, the spot tick size and the end-of-month roll, and record the evidence behind each. | ||
| Bring the role screens to the standard | DONE | 2026-11-08 | Move the roles list and role page onto the shared record parts, with paged server reads for roles and for the people who hold a role. | |
| Bring the service code onto the NATS patterns — the remaining work | BACKLOG | Audit every NATS call site against the NATS service pattern library, and fix each deviation from it. | ||
| Browse and follow the grid's jobs — the remaining work | BACKLOG | The grid screen counts batches, workunits and results and shows no row of any of them. The journeys Browse the grid's jobs and Follow a job are written and prototyped. This story builds the screens: the paged tree of batches, workunits and results, the details of each, the job page, and the file download, for the operator, the tenant administrator and the party member. | ||
| Build the retire or reset tenant journey — the remaining work | BACKLOG | The Retire or reset a tenant journey has a document and no implementation: suspend has no operation, terminate, deprovision and purge have SQL functions but no subjects, and a party-level reset has no subject. The roster's row menu shows Retire or reset as not built until these exist. | ||
| Build the synthetic collection and model catalogue journeys | BACKLOG | A person runs a synthetic collection and keeps the yield curve model catalogue in ores.web, on the subjects the journeys name. | ||
| Clean ores.synthetic to the component clean standard | DONE | 2026-11-08 | Bring ores.synthetic to the Component Clean Standard (Entity items): tier 3, domain component. | |
| Close the tenor journey and the remaining refdata gaps — the remaining work | BACKLOG | The tenor resolution journey and the remaining refdata gaps are built: the tenor resolve operation and its screen, party membership versions, the convention family mapping and term rules, the portfolio right key, the Acme book-control actors, and the party list and history reads. | ||
| Complete the ACME report inputs and prove they run | BACKLOG | Add the analytic and exposure input documents the reports read through DQ, add a readiness check, give the platform reports their own report type, split an ORE file by a key, resolve the curve quote names against the feeds, and prove the exported packages run in ORE. | ||
| Import and export conventions without a document aggregate | BACKLOG | Replace the whole-document conventions request with per-family messages staged into one atomic apply, so a new convention adds a family rather than growing a god type. | ||
| Design and seed the reports, configurations and portfolio trees each ACME party runs | DONE | 2026-11-08 | Each ACME party gets the reports it would really run, with the configuration those reports need and a realistic portfolio tree, root portfolio and currencies. | |
| Draw every coded value as its badge — the remaining work | BACKLOG | A value that is a catalogue code is drawn through one shared renderer, so a screen shows the badge the code domain paints and never a bare code. Census the screens that still print codes as text, give the domains that have no badge their definitions, and hold the rule with a gate. | ||
| Draw the node heatmap and carry the engine run statistics | BACKLOG | The grid screen draws the node heatmap and state ribbon, a node sample states the engine's run time and peak memory, and the run statistics are measured, recorded and aggregated for the grid. | ||
| Edit a party on a standard entity page | BACKLOG | Replace the party wizards with a standard entity page for a party: a list, a page with cards and tabs, and dialogs, built from the shared record parts. A save writes directly for now. The four-eyes approval of party changes belongs to the story Build the four-eyes framework. | ||
| End a session when its window closes, not when it is unattended | BACKLOG | A session ends when the browser window that held it is gone, and never because a screen sat unattended for a while, so traders can leave their screens open. Presence, not activity, decides. | ||
| Find a trade by its identifiers, state, book and date from one list — the remaining work | BACKLOG | A person finds a trade by its identifier, state, book, date, counterparty, trade type or enum column from one list. | ||
| Finish the tick ingest pipeline | BACKLOG | Why ticks lagged led to a new ingest pipeline. Finish its remaining steps: lanes, the recent window, scale out, the policy table and the clean-up. | ||
| Finish moving the hand-written SQL onto models | BACKLOG | Two groups of hand-written SQL still sit outside the model: four populate pipelines that predate the generic dataset archetypes, and the tables no model covers at all. Both are the defect the DQ artefact migration fixed for staging tables — schema and its loader written by hand, and checked by nothing. | ||
| Follow the child run a workflow hands off | DONE | 2026-11-08 | bootstrap setup and provision tenant report 'All 2 step(s) completed' and 'the tenant is ready to work in' as soon as the provision_tenant_workflow finishes, while the tenant is still being populated by the 9-step tenant_setup_workflow that start_tenant_setup creates. The child run's id is already in the final step's response as child_instance_id, so the shell can follow it; it simply never reads it. The follow loop has no notion of a child run. | |
| Gate every ER diagram | BACKLOG | Make the er-diagram check compare every generated group diagram and the index, discovered from the model rather than a hard-coded list. | ||
| Give the other prototype groups their design notes | BACKLOG | Give the eighteen remaining prototypes across eight groups their design notes, make every group page state the rule, and make the check refuse a prototype without a note. | ||
| Give each tenant its own buckets and scope the reaper | BACKLOG | A tenant owns a bucket per data class, the storage service derives the tenant from the bucket, a step service reaches its tenant's buckets with its run token, a run's inputs and outputs are removed at run end with a 24-hour backstop, and the compute reaper acts inside each tenant. | ||
| Keep the record screens up to date — the remaining work | BACKLOG | The web screens read once and never learn that a record changed elsewhere. Add live updates from the change events to the browser, a Refresh button on the record page and the list, and a stated behaviour when the record under edit changed: tell the person, show what changed, and let them keep their edit or take the new values. Complete the 'Updates and refresh' section of the entity screen specification and the Change notification page. | ||
| Manage reports from the list a party runs — the remaining work | BACKLOG | Give report management its own UX, whose front door is the list of reports a party runs, and keep the analytic authoring surfaces out of it. | ||
| Validate that every trade type is modelled as ORE reads it | BACKLOG | We model every trade type ORE knows, yet a run of generated trades drops most of them for missing or wrong properties. The XSD is laxer than ORE, and the sample roundtrip checks only that output re-reads, not that it says what the input said. Find a measure of how well each trade type is modelled, and make it a check. | ||
| Name the record in a change event | BACKLOG | A change event says that an entity changed and names no record, so a record screen cannot tell whether the change concerns the record it shows. Carry the key of the changed row on every entity event, pass it to the browser, and let a record screen go stale only for its own record. | ||
| Preview a convention's term structure on a calendar — the remaining work | BACKLOG | A person picks a tenor convention and a calendar in ores.web and sees every tenor of the convention resolved to its date at once. | ||
| Codegen facet: generate the DQ publish-from-dq pattern | BACKLOG | Add an ores.codegen facet that generates the SECURITY DEFINER publish-from-dq SQL function for the homogeneous family — about 29 of the 57 hand-written copies — from a declaration on the entity's codegen model, and unify the eight per-service publish_from_dq handlers rather than generating 57 of them. | ||
| Raise a notification for a scenario waiting for a tester — the remaining work | BACKLOG | Story 3 of the QA runner arc. A scenario that waits for a tester raises an inbox notification and the bell shows it. Add a tester role and an assignment, and a permission for the Development module. Today any signed-in user may call the QA routes. Also settle whether the module runs in production, since a deployed BFF has no work tree. | ||
| Raise approval requests from the remaining gated write paths | BACKLOG | The party, convention and tenor screens, the ORE import and the shell counterparty commands raise approval requests instead of writing gated data directly. | ||
| Record service lifecycle events and surface them in system health | BACKLOG | Services announce their starts and graceful stops, telemetry records the events and raises restart notices, a service logs out on shutdown, and the system-health home shows the services. | ||
| Record system events in a central event log | BACKLOG | Extend ores.telemetry with a central log of key system events, apart from the logs: a catalogue of event types, a publish API every service calls, a collector that stores them with their subject, an Operations Events screen, and events in each record's History. Feed runs are the first client. | ||
| Refactor conventions onto families and bundles — the remaining work | BACKLOG | Conventions fail to build on MSVC because one aggregate carries every family. Refactor them onto the classification settled in the data model: one family per convention, a bundle per import applied in one unit of work, and the ORE file demoted to a package at the boundary. | ||
| Restore the trading guards the versioned model dropped — the remaining work | BACKLOG | Making every trading entity versioned removed the immutable profiles, and two guarantees went with them without a replacement: a trade's reported portfolio no longer has to belong to the trade's party, and a raw update rewrites every version of a row instead of opening a new one. | ||
| Run a report end to end on a synthetic bond population — the remaining work | BACKLOG | Trigger the NPV report of an ACME party over its synthetic bonds and follow the run to a completed instance, recording every issue met on the way and fixing what can be fixed. | ||
| Scope a report by regulatory book type | BACKLOG | A report states the regulatory book type of the books it may read, from the same vocabulary a book uses, so a trading book report never prices the banking book. | ||
| See and change everything a report runs — the remaining work | BACKLOG | A report page shows its definition, its configurations and its conventions on tabs, and a person can choose the conventions a report reads. | ||
| Seed a relative-time trade population with the market data it needs | BACKLOG | A new ACME system has no time horizon and no deal population, so nothing says which historical fixings or market data it needs. Hold a sample ACME trade corpus in DQ whose dates are relative to the install date, then derive the market data that corpus requires and generate it, including historical fixings, in ores.synthetic. | ||
| Sort every list by when it was updated | BACKLOG | Every list can be ordered by the time of the latest save, so the newest changes come first. | ||
| Subscribe tenants to provider feeds and prove the browser at scale | BACKLOG | A tenant or party subscribes to a provider's feeds through a subscription and a route, the feed screen standard states the browser, the two scopes and the subscription, and the feed browser is proved with tens of thousands of feeds. | ||
| Draw a trade's status as its badge | BACKLOG | The trade screens state a deal's status as words, and the badge catalogue has no trade-status domain, so nothing paints it. Add the domain and its badges for the codes a trade holds, and draw the status through it wherever a trade's status appears. | ||
| Trace a process run and the trade groups it creates — the remaining work | BACKLOG | A process that acts on a set of trades must be traceable, so it can be undone or repeated. Record each run with an identity and a state machine, name the process it is a run of, and let the trade groups its inputs and outputs carry the run's key. Waits on one process with a real client. | ||
| Validate a curve pillar's tenor codes | BACKLOG | ir_curve_bootstrap_pillar.start_tenor_code and end_tenor_code are undeclared soft references to refdata_tenors. The only database check validates curve_role_code, so a typo in a tenor code saves and then throws at bootstrap time inside curve_republish_resolver, where the end tenor doubles as the quote point id. | ||
| Replace sqlgen and reflect-cpp with generated code | DONE | 2026-11-09 | Generated statements on a thin libpq layer replace sqlgen, and generated JSON and MessagePack codecs replace reflect-cpp, in every component, until neither dependency is left. | |
| Eight columns in the trading models hold JSON blobs where th… | BACKLOG | Eight columns in the trading models hold JSON blobs where the domain should hold rows, and each is a place where the store cannot answer a question about the trade without parsing text. | ||
| Configure every seeded report from the ORE examples | BACKLOG | Every report ACME is seeded with needs a risk report configuration that ORE can actually run: the right engine settings, market data bindings and portfolio scope, built from the ORE examples. This was a task inside Prove the report grid run end to end, and it is its own story because it is authoring work over dozens of reports, not chain work, and the chain can be proven without it. | ||
| Read a batch of rows in one query in the generated services | BACKLOG | The generated get-many, put-many and delete-many paths read each row by key, one query per row, so a 1,000-row put makes about 2,000 reads and stores about 104 rows a second; each batch path reads its rows in one query instead. | ||
| Run the adapter per session | BACKLOG | The session is the runtime identity: one announce, demand and heartbeat per session, the transport read from the catalogue, the simulator as one session, and health reported per session. | ||
| Read the public interest rate sources | STARTED | 2026-11-09 | The public overnight rate of each currency is stored as fixings in the system tenant, read through one specialised adapter for each publisher, the way the FX rates are read through Frankfurter, so the synthetic IR feeds can calibrate from it. | |
| Own the refresh in the scheduler | BACKLOG | One refresh request serves the scheduler and the Fetch action; a job per feed carries the rhythm; the adapter no longer polls on its own flags. | ||
| Publish the market data catalogue from DQ | BACKLOG | The ACME bundle publishes publishers, suppliers, sessions, feeds, scopes, routes and schedules, references adapters by code, and carries no secret. | ||
| Govern the public feeds | BACKLOG | Terms acceptance in Connectivity, a defined pause, and a route switch with history, with the old acts removed from Public rates. | ||
| Seed the public fixings history from a DQ snapshot | BACKLOG | Every environment holds years of daily public fixings from a static DQ snapshot refreshed by hand, and the public sources adapter fetches only the gap to today, so history-hungry calibrations have depth and the publishers' API carries little load. | ||
| Account for the grid's usage — the remaining work | BACKLOG | A super administrator needs to know which tenant is using the grid, and the work that answers it is tenant-owned, so reading it across tenants is the widening the guard forbids. The installation records what it did instead: the compute service, already inside each tenant as it dispatches that tenant's work and takes the result back, writes a usage row. This story builds that ledger and the global view it feeds. | ||
| Atomise every convention onto the central entity — the remaining work | BACKLOG | Pilot the atomised entity shape with tenor basis swap, then convert every remaining convention onto one central convention row plus its kind-specific parts, one convention per PR. | ||
| Build the Connectivity screens — the remaining work | BACKLOG | The delivery tree, the supplier dashboard, the adapter catalogue and the session and feed records, with the two setup flows and the writes behind them. | ||
| Build the Public rates screens — the remaining work | BACKLOG | Publisher and class rows, the routed feed and supplier, freshness derived from the feed schedule, availability and compare, and a service-aware fetch. | ||
| Carry one hundred thousand feeds on the data path — the remaining work | BACKLOG | Rebuild the data path so the system carries 100,000 feeds on one machine shared by ten environments: pooled long-lived connections, a transaction API, bounded concurrency, ticks stored once and written in bulk, a bounded tick store, work pushed to the grid, and built-in measurement. | ||
| Fix the tenor basis swap convention model and the document TU — the remaining work | BACKLOG | Split the tenor basis swap convention onto the data-oriented field groups its access patterns need, and find out why the configuration document protocol builds such a large translation unit, so the Windows MSVC build stops failing with C1202. | ||
| Gather each trade set and its market data — the remaining work | BACKLOG | Partition a report's trade population into trade sets and record them against the run, export one trade set as an archive and map it into the ORE portfolio, resolve and export each set's market data at its as-of, hand each configuration to its owner by id at an as-of, and assemble the package only w | ||
| Generate roundtrip tests for the wire codecs — the remaining work | BACKLOG | Codegen emits a JSON and MessagePack roundtrip test for every model that opts in to the wire codec, so no codec depends on a hand-written test. | ||
| Let the blotter read a page, not the book — the remaining work | BACKLOG | The blotter reads every summary to count its rail, so at ten thousand trades it takes fifty-four seconds and then the service refuses a filter with ten thousand values. The screen needs a read model that answers one page and one count, owned below the BFF. | ||
| Normalise every ORE parameter representation — the remaining work | BACKLOG | ORE accepts more than one spelling for many of its values: 27 spellings for seven business day conventions, four vocabularies for a boolean, alias rows for a day counter, and any case for the enumerations its parser upper-cases. The store must hold one representation of each fact, and the knowledge of which spellings mean what must live in data, not in code that drifts from the engine. | ||
| Make every configuration screen follow one standard — the remaining work | BACKLOG | The engine settings work set the pattern for every configuration that comes off ORE XML. Generalise it into one UX standard, apply it to the pricing engine configuration, and make the two similarly named configurations impossible to confuse. | ||
| Open sprint 29 — the remaining work | ABANDONED | 2026-11-08 | Scaffold sprint 29, wire the version manifest and the agile index, and move the postponed stories. | |
| Populate the ACME books with a realistic trade population — the remaining work | BACKLOG | ACME's books hold no trades, so a report package is structurally valid and substantively empty. Populate them from a synthetic generator in present time, referencing ACME's own books, counterparties and staff, starting with bonds and extending to every instrument. | ||
| Prove the report grid run end to end — the remaining work | BACKLOG | The ACME report definition runs to completion against the compute grid from an input bundle that already exists, the compute results are retrieved, and the run is reproducible from a freshly recreated database. | ||
| Rework the market data feeds into a market data service with adapters — the remaining work | BACKLOG | Move the simulated feeds behind a market data service: vendor ids and a mapping, sources with a kind, subscriptions and routes, adapters on NATS, metering and cost, and the simulator as one adapter, so the system treats live and simulated data in the same way. | ||
| Run only the checks a change needs — the remaining work | BACKLOG | The local check gate runs every heavy check for any C++ change, so a test-only header change pays for the database rebuild, the tick smoke and all 98 suites. | ||
| Stop the workflow from losing steps to a transient storm — the remaining work | DONE | 2026-11-08 | A provisioning run lost its first step to a transient database connection storm. The engine published the step command, the iam service never handled it, and twenty minutes later the run failed with a message that names no cause. The command was dropped rather than retried, the pool storm left no explanation, and the handler has a path that discards a command in silence. | |
| Support volatility surfaces — the remaining work | BACKLOG | Add volatility surfaces to the system: their structure, quotes, construction and storage, and how they flow to pricing. A long arc that starts from a documented architecture. | ||
| Take the currencies topic through the screen pipeline — the remaining work | BACKLOG | Run the currencies and pairs topic through the screen pipeline runbook: journeys judged, prototype signed off, screens built, scenario, harness and sheet, the owner's sign off. | ||
| Build the IR curves of every supported currency from public data — the remaining work | BACKLOG | Each supported currency has an overnight-index curve bootstrapped from its synthetic quotes and published in the system tenant, and the synthetic IR feeds start from public rates; today only USD has a bootstrap config, it was never published, and no currency's synthetic feed is calibrated. | ||
| Describe every product type on one page — the remaining work | BACKLOG | The catalogue holds the product types a tenant can book, and the screens name them one trade at a time. There is no page a person can read to see the whole set, what each one is, where its documentation lives, and which books and permissions allow it. Add the catalogue's own page. | ||
| Make the generated trades value correctly in ORE | BACKLOG | Generated trades are correct according to ORE: it builds them, and a report values them to ORE's own NPVs. | ||
| Serve what the BFF serves, then retire it — the remaining work | STARTED | 2026-11-09 | The C++ server answers a fraction of what the BFF answers, and the browser calls none of it. Port the layer to parity one functionality at a time, merge each as it lands, and leave the BFF serving the browser until every route it answers has an equivalent. Retiring the BFF is a later set of pull requests, once the coverage is complete. | |
| Split the types that carry too many columns — the remaining work | BACKLOG | The persisted types hold 30 to 60 flat columns, which the data-oriented model forbids. Inventory them, decide which are one aggregate over several components, and split those into components of their own. |
3.2. Hotfixes
| Story | State | Start | End | Description |
|---|---|---|---|---|
| Hotfix: nightly CI failures | STARTED | 2026-11-09 | The Linux, macOS, Windows and valgrind nightlies fail on shell completion tests, compile errors, test defects and memcheck reports. | |
| Hotfix: two changes on main fail four local checks | DONE | 2026-11-09 | 2026-11-09 | The holiday calendar and auto-approval changes on main fail four local checks: entity table writers, step translations, catalogue formatting and a generated shell recipe. |
4. Achievements
Milestones worth capturing as they happen, so the release notes at
sprint close have the evidence to hand. Add a dated ** subheading
per milestone, with any screenshot via [[proj:assets/images/<file>.png]].
5. Captures
Real captures filed during this sprint that have not been pulled into a
story as tasks — listed here purely for visibility into what the sprint
surfaced. Each is an inbox capture (compass capture); triage them at the
health review (promote to a story, move to next, or discard).
| Capture | Tags | Description |
|---|---|---|
6. Health Review
(Run sprint-reviewer to generate this section.)
7. Charts
Charts generated via compass sprint charts. See Sprint health charts for what each chart measures, its purpose, and how to read it.
7.1. PRs & Commits per Day
Dual-axis bar chart. PRs (left axis) and commits (right axis) per day. A high commits-to-PR ratio may indicate scope creep.
7.2. Daily Line Churn
Lines added (green) and deleted (red) per day. Building work produces mostly additions; refactoring produces a mix. Days with no churn may indicate blockers.
7.3. PR Cycle Time
Hours from PR open to merge, one dot per PR for every PR merged in the sprint, on a log scale, with the median and the upper quartile as lines. High dots indicate review bottlenecks. Generated only when PR data is available.
7.4. Cumulative Stories Done
Line chart tracking stories marked DONE during the sprint. Steady upward slope is healthy; plateauing signals a stall.
8. Retrospective
(Filled at sprint close.)