Sprint 25
Table of Contents
This page documents a sprint (Sprint 25) of ORE Studio. It captures the sprint's mission, current status, and the stories that compose it. For the surrounding context — version goals, sprint order, and product identity — see Version 0.
Mission
Resolve sprint drift on as many components as possible and progress IR curves work.
Status
Stories
For the definitions of the themes see Themes.
Product
| Story | State | Start | End | Description |
|---|---|---|---|---|
| Open sprint 25 | DONE | 2026-08-03 | 2026-08-04 | Scaffold sprint 25, wire version manifest, move postponed stories. |
| Acme Corporation follow-ups: manual chapter, logo fixes, synthetic feeds | STARTED | 2026-08-03 | Extension work beyond Commission Acme Corporation's own acceptance, split off at sprint 24 close. Reopened 2026-08-09 for the fit-for-purpose review task. | |
| Find root cause of FX Spot Monitor wire-codec decode mismatch | DONE | 2026-08-03 | 2026-08-04 | Deferred from sprint 24: fx_spot_subscription silently fails to decode ticks under msgpack despite both sides resolving msgpack at startup. Root cause found and fixed; live investigation also fixed two CRM bugs (stale JWT, party scoping). |
| Holding company treasury functionality: FX/CRM visibility, Group Treasury book, consolidated reporting | STARTED | 2026-08-05 | Acme's holding company party has no CRM/treasury data at all – surfaced while investigating the wire-codec story above. FX visibility, Group Treasury book, and a consolidated-reporting investigation. | |
| Refactor ores.codegen C++ generation | STARTED | 2026-05-30 | Pulled from product backlog: apply safe drift to the 11 remaining components (analytics, compute, controller, database, dq, iam, marketdata, reporting, scheduler, trading, workflow, workspace) following the refdata-cpp pilot pattern. | |
| Entity classification and drift baseline across all components | BACKLOG | Narrowed 2026-08-04: the 14 ad hoc drift tasks are re-scoped into the 12 stories below; remaining scope is retiring the 3 stale docs. | ||
| Systemd resource management and per-environment isolation | DONE | 2026-07-31 | 2026-08-05 | Pulled from product backlog: converge compass onto systemd for process lifecycle across the fleet, root-causing the 2026-07-31 reboot's cgroup-unaware process management. |
| Entity classification and drift baseline: ores.dq | DONE | 2026-08-04 | 2026-08-08 | Per-component rollout of the ores.refdata pilot pattern: bind dq/dq-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. |
| Entity classification and drift baseline: ores.iam | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind iam/iam-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.trading | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind trade/trading-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.synthetic | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind synthetic entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.marketdata | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind marketdata entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.analytics | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind analytics-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.compute | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind compute-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.database | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind database-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.reporting | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind reporting-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.scheduler | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind scheduler-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.workflow | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind workflow-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| Entity classification and drift baseline: ores.workspace | BACKLOG | Per-component rollout of the ores.refdata pilot pattern: bind workspace-cpp entities to profiles, verify zero-diff regen, extend the CI drift check. | ||
| IR curve bootstrapping + official curve republish | STARTED | Pulled from product backlog: bootstrap the raw instrument grid into a desk-consumable zero/discount curve and republish it as an official curve stream. | ||
| FOMC-dated OIS short end: calendar-event model and flat-forward pillars | DONE | 2026-08-09 | 2026-08-11 | Decide and implement how FOMC meeting dates enter the model: FOMC dates as typed events on a currency calendar, a unified FOMC/IMM/tenor resolution model, then bootstrap the SOFR OIS short end off FOMC-dated pillars with flat-forward interpolation. |
| Feed lifecycle harmonization: asset-class-agnostic feed control-plane | BACKLOG | The feed control-plane (folder start/stop cascade, per-config start/stop/list, auto-start) is implemented per asset class; harmonise it behind one common feed interface and a per-kind factory so every control-plane verb works for any asset class, ores.shell covers all asset classes, and a new asset class plugs in via one factory registration. | ||
| IR Rates synthetic data: dataset seeding, index cleanup, dual-curve, quoting conventions | STARTED | Pulled from product backlog: follow-on from Sprint 23's IR Rates synthetic data generation – seed a realistic vintage dataset, normalize duplicate floating-index catalogs, extend to dual discount/projection curve, add IR quoting-convention display. | ||
| Documentation improvements | STARTED | 2026-08-04 | Ongoing improvements to ORE Studio's documentation and public-facing site. Tasks are added incrementally as ideas come up. | |
| Collapse NATS config passthrough to a single environment-mapper fallback tier | STARTED | 2026-08-04 | Promoted from inbox: a new shared NATS knob currently needs manual fixes in five places (process_supervisor, SQL args_template, systemd_generate.py, compass_services.py, ores.qt); add a shared ORES_NATS_* fallback tier to environment_mapper_factory so services read it straight from .env. | |
| Synthetic data scope and binding: system/tenant/party levels, bound vs sandboxed | DONE | 2026-08-04 | 2026-08-08 | Add orthogonal scope (system/tenant/party) and binding-mode (bound/sandboxed) axes to synthetic configs so ACME's parties share one reality and testing can consume synthetic data without affecting real feeds. |
| Injectable environment provider in ores.platform for test-safe, parallel-safe env var access | DONE | 2026-08-07 | 2026-08-07 | Follow-on from the NATS config passthrough story: environment_provider abstraction with real_environment_provider and fake_environment_provider implementations; scoped_environment_override RAII guard; 11 config-parser tests migrated from scoped_env_unset to the new mechanism. |
| Reduce and configure per-service DB connection pool size | DONE | 2026-08-04 | 2026-08-05 | Bundle the pool-size reduction and runtime-configurability captures, plus an audit of compass services start's systemd integration, to lower steady-state Postgres connection load. |
| Review whether ores.cli/ores.shell's host.cpp divergence is load-bearing or historical | DONE | 2026-08-05 | 2026-08-05 | ores.cli and ores.shell are the only two tools left out of the shared run_host_async helper – review in detail whether their differences are genuinely necessary or just how the tools happened to be written. |
| Compass quality-of-life improvements: remaining follow-ups | DONE | 2026-08-06 | 2026-08-06 | Pulled from product backlog: mop up seven self-contained compass DX improvements – branch pruning, systemd scope info, named .env files, compass show full-content flag, blocked dependency tree in compass where, per-environment fleet status, compass help command. |
| Extract a sync sibling of run_host_async for ores.cli/ores.shell | DONE | 2026-08-05 | 2026-08-05 | Follow-on from the host.cpp divergence review: ores.cli and ores.shell's host.cpp bodies share ~70% structure with each other despite being correctly excluded from the async run_host_async helper; extract a sync sibling parameterised by a construction callable. |
| Backlog deep refinement | DONE | 2026-08-05 | 2026-08-07 | Deep audit of all product-backlog captures: inbox/next/deferred verified against the codebase, duplicates merged, buckets re-scored against sprint 25 mission. First run. |
| Fix codegen template drift | BACKLOG | Fix four codegen template bugs: history mapper provenance fields, compound-key is_uuid gating, pagination footgun, junction template follow-ups. | ||
| Close systemic codegen gaps | BACKLOG | Close two systemic codegen gaps: RLS policy facet (no template emits RLS policies) and history provider registrar codegen (918-line hand-maintained file). | ||
| IR curve follow-ups | BACKLOG | Three IR curve follow-ups: expand stochastic process types, fix seed data/combo gaps, investigate tenant_type drift. | ||
| Add new stochastic processes | STARTED | 2026-08-11 | Add new stochastic process types for FX and IR: finish the unwired IR engines (Black-Karasinski, BDT, affine, HJM, LMM, QG) by wiring them into the system, add further IR models (two-factor Hull-White, shifted log-normal, normal/Bachelier), and add an FX jump/Poisson process engine. SABR and vol-surface models are out of scope. | |
| Investigate hand-added cross-service grants in generated iam_service_db_grants_create.sql | DONE | 2026-08-06 | 2026-08-06 | Found while removing controller from the db recreate pipeline (PR #1867): regenerating iam_service_db_grants_create.sql drops hand-added cross-service grants the generator does not emit. Investigate whether the registry model/template should express per-service cross-domain grants so generation is lossless, and close the drift so regeneration never silently drops grants. |
| Normalise NATS protocol message shapes and inventory hand-written drift | DONE | 2026-08-07 | 2026-08-07 | Audit request/response structs for consistent shape (success/message fields per codegen convention) and inventory protocol messages hand-written instead of generated via ores.codegen's cpp_protocol template. |
| Migrate to oresmd, delete market_series qualifier and ore_key | DONE | 2026-07-28 | 2026-08-07 | Carried from sprint 24: migrate ir_curve_generation_config, market_series, and fx_spot_generation_config onto oresmd, then delete the legacy free-text qualifier/ore_key fields — single scheme, no permanent coexistence. XML round-trip descoped to Extend oresmd story. |
| Extend oresmd to full ORE quote-type coverage | DONE | 2026-08-07 | 2026-08-10 | Carried from sprint 24: extend oresmd from ~7 of 49 ORE quote types to full coverage across all asset classes — new families (inflation, correlation), new metric/dimension fields, new volatility-surface sub-schema. 9 tasks, all done. |
| Remote WSL offload and compute nodes | DONE | 2026-08-10 | 2026-08-10 | The WSL remote-host arc consolidated into sprint 25: containerize + deploy to Newton (sprint 24), the compass env deploy command family (runtime and compute roles), and a remote compute node joining the serving environment end-to-end. |
| Port trading instruments to codegen | STARTED | 2026-08-05 | Umbrella for the trading-instrument-to-codegen port: the decision/pilot (PR #1857), the remaining ~19 conversions (PR #1874 and ongoing), and the NATS eventing facet retrofit onto converted instruments (PR #1896). 15/19 converted. | |
| Absorb PR checks locally: targeted checks, local review, CDash submissions | BACKLOG | We spend too long waiting for GitHub PR checks. Absorb their cost locally: classify each change and run only the checks it needs; review code locally as a skill with findings registered on the task; submit builds to CDash as experimental. GitHub keeps the continuous builds and nightlies. | ||
| Adopt oresmd as the native market data identity across simulator, UIs, and shell | STARTED | 2026-08-11 | Make oresmd the single native market data identifier across the market simulator, all Qt UIs, and ores.shell, abandoning ORE-shaped representations on persisted entities. First task: the analysis that decides fields-vs-URI storage. | |
| party/counterparty_identifier_generator tests assert stale version/change_reason_code | DONE | 2026-08-11 | 2026-08-11 | generators_tests.cpp still expects version==1/system.new for party_identifier and counterparty_identifier generators, but the generators produce version==0/system.test — same stale-expectation pattern already fixed for the contact_information generators. Investigation confirmed the fix already landed on main (90ad8693be, c113b851e, 123ca2bfa, d0458e500); closed as done elsewhere. |
Hotfixes
| Story | State | Start | End | Description |
|---|---|---|---|---|
| Hotfix: mint_token_respects_the_requested_ttl flaky on nightly (1s JWT TTL) | DONE | 2026-08-04 | 2026-08-04 | Test mints a JWT with a 1-second TTL and validates it immediately with zero verifier leeway; on slower/loaded nightly runners the token expires before validation, causing a spurious failure. |
| Hotfix: shared NATS fallback tier leaks .env into parse-defaults tests | DONE | 2026-08-04 | 2026-08-04 | PR #1819's generic ORES_NATS_* shared-domain fallback tier makes 11 service config_parser test suites fail their parse_defaults_returns_expected_values case, because the checkout's real .env NATS URL/subject-prefix now leaks in as a fallback where the test expects the coded default with no overrides. |
| Hotfix: MSVC rejects ScopeLabel.hpp due to ambiguous scope identifier | DONE | 2026-08-05 | Windows (MSVC) build breaks on ScopeLabel.hpp:34 — the using-declaration and parameter named 'scope' collide; MSVC resolves 'scope' in 'switch (scope)' as the type rather than the variable. | |
| Hotfix: Replace setenv/unsetenv with ores.platform::environment in tests | DONE | 2026-08-06 | Windows (Clang/MSVC) builds break because setenv/unsetenv are POSIX-only; 12 test files use them and must switch to the portable ores.platform::environment wrapper. | |
| Hotfix: macOS CI fails to start NATS server, nats-server not installed | DONE | Continuous MacOS CI aborts at the Start NATS server step: the nats-server binary is only installed inside the Run CTest workflow step (brew install autoconf automake libtool autoconf-archive nats-server), which runs after the step that launches it, so the launch emits command not found and the 30s readiness poll fails. Linux CI installs it via compass env install-packages and Windows CI downloads a GitHub release (v2.10.24) – three platforms, three sources, three versions. Introduced by commit 41a07950e which added the Start NATS server step to all four workflows. Fix: every CI workflow gets an identical Setup NATS server step running the shared build/scripts/setup_nats_server.sh (PowerShell equivalent setup_nats_server.ps1 on Windows), which unpacks the vendored nats-server v2.14.3 release (external/nats/packages) and puts it on PATH – one pinned binary on every platform instead of apt/brew/GitHub-download. | ||
| Manage the compute grid from the shell | STARTED | Comprehensive ores.shell compute verbs so compute-grid tests run without a UI: add and list apps, app versions, platforms, hosts and batches; add batches and dispatch jobs; download job inputs and outputs; monitor grid health per node; plus a smoke-test script that dispatches 10-20 consecutive jobs and asserts every online node was exercised. | ||
| Hotfix: Valgrind flags NATS one-time init blocks as leaks on eventing tests | DONE | 2026-08-10 | 2026-08-10 | CDash dynamic analysis flags 21 still-reachable leak records on three components (ores.trading.core.tests, ores.reporting.core.tests, ores.refdata.core.tests), all from the NATS C library one-time global init (nats_openLib) reached via the eventing client connect. Add narrow suppressions to build/valgrind/custom.supp and create a valgrind recipe. |
| Hotfix: NATS cert generation fails on macOS/Windows (missing `ip` command) | DONE | 2026-08-11 | 2026-08-11 | NATS certificate generation fails on macOS and Windows because `_local_ipv4_addresses` shells out to the Linux-only `ip` binary and never reaches its gethostname fallback. |
| Hotfix: rate_tree fails to compile on Windows | DONE | 2026-08-11 | 2026-08-11 | rate_tree.cpp calls std::to_string without including the string header, breaking the MSVC build. |
| Hotfix: stochastic process statistical tests fail on macOS CI | DONE | 2026-08-12 | 2026-08-12 | Sample-variance and drift-mean Monte Carlo tests in ores.analytics.quant fail on macOS because their tolerances are tighter than the tests' own standard error. |
Achievements
Milestones worth capturing as they happen, so the release notes at
sprint close have the evidence to hand. Add a dated ** subheading
per milestone, with any screenshot via [[proj:assets/images/<file>.png]].
Captures
Real captures filed during this sprint that have not been pulled into a
story as tasks — listed here purely for visibility into what the sprint
surfaced. Each is an inbox capture (compass capture); triage them at the
health review (promote to a story, move to next, or discard).
| Capture | Tags | Description |
|---|---|---|
Health Review
| # | Date | Day | Overall | Task |
|---|---|---|---|---|
| 1 | 2026-08-07 | day 5 | AMBER | Health review 1 — sprint 25 mid-sprint analysis |
(Run sprint-reviewer to add more rows.)
Charts
Charts generated via compass sprint charts. See Sprint health charts for what each chart measures, its purpose, and how to read it.
PRs & Commits per Day
Dual-axis bar chart. PRs (left axis) and commits (right axis) per day. A high commits-to-PR ratio may indicate scope creep.
Daily Line Churn
Lines added (green) and deleted (red) per day. Building work produces mostly additions; refactoring produces a mix. Days with no churn may indicate blockers.
PR Cycle Time
Hours from PR open to merge, one bar per PR. Long bars indicate review bottlenecks. Generated only when PR data is available.
Cumulative Stories Done
Line chart tracking stories marked DONE during the sprint. Steady upward slope is healthy; plateauing signals a stall.
Retrospective
(Filled at sprint close.)