Sprint 19

Table of Contents

This page documents a sprint (Sprint 19) of ORE Studio. It captures the sprint's mission, current status, and the stories that compose it. For the surrounding context — version goals, sprint order, and product identity — see Version 0.

Mission

Verify all entity functionality post-NATS migration, document entities in the manual, create entity evaluation tooling, and fix Windows/macOS CI builds.

Status

Field Value
State DONE
Parent version Version 0
Previous Sprint 18
Start 2026-05-29
End 2026-06-06
Now Nothing.
Waiting on Nothing.
Next Sprint 20
Release Notes Sprint 19 release notes
Last touched 2026-06-06

Achievements

  • compass became the project's operational CLI: db/services/client/ sql pillars (eleven shell scripts deleted), environment status in bearings with DB-drift signalling, sprint audit and sprint charts, and the full PR lifecycle — create, checks, sync, and a guarded merge that closes the task and stamps the journal itself.
  • The whole C++ codebase was clang-formatted (16 group passes, CMake targets, nightly workflow) and regrouped under per-group parents; the rfl complexity decomposition finished 16/16, clearing the MSVC C1202 nested-struct limit at the root of the Windows CI break.
  • Codegen modernisation: entity models migrated from JSON to org-mode (28/32), C++ and SQL template refactors landed, legacy binary-protocol generator deleted.
  • API/core split investigated across all 17 components; the one systemic offender fixed and ores.eventing split into api/core.
  • Commission: currency reached 6/8 — codegen, SQL, CLI, shell verified post-NATS; manual chapters and NATS reference in flight.
  • The agile system hardened itself: per-concept process pages, five System 2 health reviews, literate sprint charts, and a sprint that closed with zero state-sync violations on its own audit.

Stories

For the definitions of the themes see Themes.

Product

Epic: ores.refdata Commissioning

Commission all ores.refdata entities across every access layer: verify Qt UI end-to-end post-NATS migration (list window, detail window, history, delete, eventing), implement or verify shell and CLI commands (list, add, remove), document each entity in the user manual, and file backlog captures for Wt and HTTP gaps. One story per entity.

Story State Start End Description
Commission: currency ABANDONED   2026-06-06 Verify Qt + shell + CLI (existing); manual; Wt/HTTP captures. Codegen task done; shell/NATS/Qt next.
Commission: country ABANDONED   2026-06-06 Verify Qt + shell + CLI (existing); manual; Wt/HTTP captures.
Commission: party ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: party_type ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: party_status ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: party_id_scheme ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: contact_type ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: counterparty ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: book ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: book_status ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: business_unit ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: business_centre ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: portfolio ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.
Commission: purpose_type ABANDONED   2026-06-06 Verify Qt; implement shell + CLI; manual; Wt/HTTP captures.

Epic: User Experience Improvements

The UI must communicate state truthfully and immediately: instant feedback under latency (no blocking dialogs, loading overlays, responsive combos), a consistent visual vocabulary (DB-driven badges, colour semantics, icons — gray always reads "inactive", orange always reads "definition missing"), and honest data presentation (timeframe-scoped PnL). Stories here come from friction observed while dogfooding the product.

Story State Start End Description
Make UI more responsive DONE 2026-06-01 2026-06-01 Fix waitForFinished() in destructors; add spinners; pre-populate combos; dedicated I/O thread pool.
PnL tracking improvements DONE 2026-06-03 2026-06-03 Capture: FX locking, timeframe-scoped PnL, dual charts, two-phase sell P&L, FIFO/WAVG. See Lightyear knowledge doc.
Account type badge in Qt DONE 2026-06-04 2026-06-04 Wire account_type as DB-driven badge in AccountItemDelegate; SQL population; badge setup recipe. PR #1038.
Badge colour semantics DONE 2026-06-04 2026-06-04 Gray = inactive only; recolour service/unlocked/done; orange missing-definition fallback; detail dialog badges; UX language doc. PR #1044.
Consolidate history dialogs ABANDONED 2026-06-05 2026-06-06 Server-side diffs via ores.diff + per-entity mappers; HistoryDialogBase everywhere; shell unified diff. Design done; 6 tasks raised.
Clear component documentation debt ABANDONED   2026-06-06 Seventeen validate_docs violations: six missing group overviews, five skeletal, six missing pumls; reconcile validator vs runbook.
Symmetric onboarding flags ABANDONED   2026-06-06 party_id column in variability settings; symmetric onboarding.* flags across the three provisioner wizards. Never wired into this table during the sprint; carried to inbox at close.

Tooling

Epic: Compass

Developer toolkit consolidation: fold operational scripts into compass, extend Orient with staleness and bearings, redesign CLI taxonomy.

Story State Start End Description
Compass session journal DONE 2026-05-31 2026-05-31 Per-machine .journal.org (gitignored) recording each Claude environment's story/task journey; compass journal commands; compass fleet refactor.
Compass CLI UX redesign DONE 2026-05-31 2026-06-01 Noun-based command taxonomy; fix tool description; –help by pillar; compass sprint/story status commands; audit recipes and skills.
Consolidate scripts into compass ABANDONED 2026-06-02 2026-06-06 Inventory the ~61 operational scripts; design operational compass pillars; fold scripts into documented compass subcommands. Absorbs the service-registry consolidation.
Compass bearings command DONE 2026-06-02 2026-06-05 Single-command LLM orientation (compass bearings): where, last session, memory pointer, and command cheat-sheet.
Add compass test run ABANDONED   2026-06-06 Run all / composite-component / single-component tests or Catch2 name/tag filters; –build delegates to cmake; .env loaded for direct binary runs.
Add PR management support to compass DONE 2026-06-05 2026-06-05 Review pillar (list/reply/resolve/pending) + PR pillar (sync/checks/create/record/merge); runbook and recipes compass end-to-end. Promotes the sprint-18 capture.
Compass quality-of-life improvements DONE 2026-06-06 2026-06-06 Grab-bag of small improvements: bearings PR-state detection, recipe body arguments in compass add, init-session skill. PR #1106.

Epic: Codegen

Unify the code generation model: migrate entity models from JSON to literate org-mode, refactor SQL and C++ generators, add safety guardrails and CI invariant.

Story State Start End Description
Refactor ores.codegen SQL generation DONE 2026-05-29 2026-05-30 Unified SQL template + model format + codegen.py CLI replacing all generate_*_schema.sh scripts.
Document MASD principles DONE 2026-06-01 2026-06-01 Technical knowledge document weaving MDE/MASD theory with ores.codegen practice and the reference-implementation backout strategy.
Codegen unified model: org migration ABANDONED 2026-06-01 2026-06-06 Migrate every entity model from JSON to a literate org-mode file per entity, co-located with the component. Supersedes the unified-JSON Phase 2 story.
Introduce ores.seeder component DONE 2026-06-02 2026-06-05 New tooling component owning SQL INSERT populate scripts. Slovaris is its first dataset; closes out the codegen org-mode migration story.
Refactor ores.codegen C++ generation ABANDONED   2026-06-06 Audit and resolve all drift between C++ templates and production files. Blocked on org-mode migration story — the pilot exposed that the JSON model format cannot express custom methods cleanly.
Decommission codegen bash scripts ABANDONED   2026-06-06 Replace all direct bash script invocations of ores.codegen with ores.compass or codegen.py; remove legacy scripts starting with generate_doc.sh.
Codegen model safety guardrails ABANDONED   2026-06-06 Runtime guards for missing component fields; block dual-template SQL; document all-cpp limitations. Prerequisite for C++ audit.
Codegen unified model: Phase 1 ABANDONED   2026-06-06 Derive Qt protocol class names from naming conventions; remove transcribed fields from all 75 models.
Codegen unified model: Phase 2 ABANDONED     Superseded by org-mode migration story which goes further than unified-JSON: literate org-mode with custom methods inline.
Codegen unified model: Phase 3 ABANDONED   2026-06-06 Merge parallel temporal and non-temporal template families via is_temporal flag.
Codegen CI zero-diff invariant ABANDONED   2026-06-06 CI job that regenerates all components and fails if any output differs from HEAD.

Epic: C++ Components

Regroup and refactor the C++ component layout to align with the product-group model.

Story State Start End Description
Regroup C++ components DONE 2026-06-01 2026-06-02 Move each projects/ores.<group>.<role>/ (api, core, service) and projects/ores.qt.<group>/ under a single projects/ores.<group>/ parent so the group owns its modeling/ dir, root CMakeLists, and child layout. Blocks per-group org-migration tasks beyond refdata.
Move trade-instrument parsing ABANDONED   2026-06-06 Extract the Qt-free parse logic into ores.trading.api so the dispatch test links a real target instead of hand-listing source files. Removes the stop-gap from the regroup story.
Investigate the API/core split DONE 2026-06-04 2026-06-05 Inventory api vs core across all 17 split components; flag heavy apis (ores.dq.api links ores.database.lib); decide refactor so apis are standard types or other apis only.
Unify entity timestamp handling ABANDONED 2026-06-05 2026-06-06 Throw on timestamp conversion failure; fix optional<db_timestamp> mappers. 2/5 tasks done. Never wired into this table during the sprint; carried to inbox at close.

Epic: Build Infrastructure

CI and build tooling improvements: code formatting, static analysis, and build hygiene.

Story State Start End Description
Set up clang-format DONE 2026-06-04 2026-06-05 Fix .clang-format (c++23, ORE include ordering); CMake format/check-format targets; nightly GH Actions format PR; initial format pass all 13 groups (~4700 files).

Agile

Story State Start End Description
Open sprint 19 DONE 2026-05-29 2026-06-05 Scaffold sprint 19, wire version manifest, bump version, PR.
Clarify versioning strategy DONE 2026-06-02 2026-06-02 Introduce milestone concept; separate version identity from release series; scaffold v1.0 and v2.0 milestone docs.
Sprint planning DONE 2026-05-29 2026-05-29 Select stories from backlog, size tasks, confirm mission.
System 2 health review DONE 2026-05-31 2026-06-06 Periodic System 2 health checks: goal alignment, load, PR velocity, story/task balance, focus, verdict.

LLMs

Story State Start End Description
Entity evaluation skill and runbook ABANDONED   2026-06-06 Reusable skill + runbook: coverage lookup, functional verification steps, gap report.

Documentation

Story State Start End Description
Entity manual: shell documentation ABANDONED   2026-06-06 Each entity chapter in the manual covers Qt UI and shell usage side by side.
Create a user guide for ores.qt DONE 2026-06-05 2026-06-05 Single home for manual-content work; next: accounts/roles/tenants administration chapter from the 2026-06-04 screenshots.
Fix knowledge graph issues DONE 2026-05-30 2026-05-31 Fix knowledge graph: images not displaying and left-panel controls missing.
Add tooling layer to system model DONE 2026-06-02 2026-06-03 Add system_model_tooling.org; tooling blurb in system_model.org; foundation layer back-links.
Document the sprint health charts DONE 2026-06-04 2026-06-05 Knowledge page for the four sprint charts; literate gnuplot sources tangled to scripts/; links from sprint template, process page, recipe, runbook.
Documentation review follow-ups DONE 2026-06-05 2026-06-05 Per-concept agile pages (story, task, sprint phases); split document types; DoD doc; outreach + demo steps; product identity → manual link.

Infrastructure

Story State Start End Description
Fix rfl complexity failure DONE 2026-05-30 2026-06-05 Fix ClientManagerTradeDetail.cpp rfl::Literal 502-field compile failure; restore CI builds.

Hotfixes

Health Review

# Date Day Overall Task
1 2026-05-31 day 3 AMBER Health review 1
2 2026-06-03 day 5 RED Health review 2
3 2026-06-05 day 8 Health review 3 — sync finished stories
4 2026-06-06 day 9 Health review 4 — post-merge paperwork sync
5 2026-06-06 day 9 RED Health review 5

Charts

Charts generated via compass sprint charts. See Sprint health charts for what each chart measures, its purpose, and how to read it.

PRs & Commits per Day

Dual-axis bar chart. PRs (left axis) and commits (right axis) per day. A high commits-to-PR ratio may indicate scope creep.

prs_commits.png

Daily Line Churn

Lines added (green) and deleted (red) per day. Building work produces mostly additions; refactoring produces a mix. Days with no churn may indicate blockers.

line_churn.png

PR Cycle Time

Hours from PR open to merge, one bar per PR. Long bars indicate review bottlenecks. Generated only when PR data is available.

pr_cycle.png

Cumulative Stories Done

Line chart tracking stories marked DONE during the sprint. Steady upward slope is healthy; plateauing signals a stall.

stories_done.png

Retrospective

What went well

  • Throughput and WIP discipline: 186 PRs in nine days, zero open PRs at close, no stale branches — the compass PR lifecycle (guarded merge, automatic task close-out) visibly tightened the loop.
  • The sprint audit caught state drift repeatedly and cheaply; the final audit ran clean. Paperwork stopped being archaeology.
  • Health reviews worked as designed: HR2 and HR5 both said RED honestly, and HR5's close recommendation is why this sprint is ending deliberately instead of drifting into week three.

What hurt

  • The mission lost to the tooling gravity well — again. The CI goal did get real groundwork (the rfl story cleared the MSVC C1202 root cause) but without a story naming the goal, the health reviews could not see it — and the builds were never re-verified. Goals with no story on day one never got one.
  • 30 stories left the sprint unfinished at close. Most were never decomposed — they were planning inventory, not sprint work, and their presence diluted the Stories table's signal all sprint.
  • Sprint closure itself was manual and error-prone: the carry mechanics (move vs copy, ID preservation) had to be designed mid-close. Now captured for compass automation.

What changed

  • Sprint closure now has explicit mechanics: unfinished stories move to the product backlog inbox preserving their immutable IDs (flat story.org for undecomposed stories, whole folders when task history exists), and are marked ABANDONED in the sprint table. Closing and opening sprints are separate PRs.
  • Sprint 20 opens with a deliberately narrow mission — the commissioning epic plus codegen tidy-up — and the tooling excuse is spent.

Release Notes

See release_notes.org for the full narrative.

Emacs 29.1 (Org mode 9.6.6)