Story: Retire legacy codegen profile system; add junction support to physical-space codegen

Table of Contents

This page documents a story in Sprint 23. It captures the goal, current status, acceptance criteria, and the tasks that compose it.

Goal

Two parallel codegen systems currently coexist, and one of them is dead code that nobody notices is dead because it silently no-ops instead of erroring. This story deletes the dead one (forcing anything that secretly depended on it to break loudly) and then adds real C++ generation support for junction models to the live system, so a junction entity like tenor_convention_resolution gets a generated domain/repository layer instead of hand-authored code standing in for a codegen gap.

Status

Field Value
State DONE
Parent sprint Sprint 23
Now Closed DONE 2026-07-20 (health review 1): own Acceptance met by tasks 1-4; remaining BACKLOG tasks split into a follow-on story.
Waiting on Nothing.
Next Nothing.
Last touched 2026-07-20

Analysis: two parallel codegen systems

Investigated while trying to get compass codegen entity generate tenor_convention_resolution to produce a C++ repository (not just SQL) for a junction model. Two independent systems exist side by side:

System 1 (legacy, dead for entity generate): facet_catalogue.org

  • File: projects/ores.codegen/library/facet_catalogue.org (id A8145FD7-0651-40FD-857E-BC37C9320020). Its own header claims to be "the authoritative source … consumed directly by generator.py via _load_profiles_from_org()"=.
  • Parsed by _load_profiles_from_org() in projects/ores.codegen/src/codegen/core.py (around line 520), used only by the --profile flag path (compass codegen generate --model ... --profile all-cpp, and the deprecated --profile flag on compass codegen entity generate).
  • Confirmed dead for the default/=–address= path: edited this file's domain facet to add junction to its :model_types:, re-ran compass codegen entity generate tenor_convention_resolution --dry-run, and the output was byte-for-byte unchanged (still SQL only). Verified programmatically too — the function that actually drives entity generate (resolve_targets / compute_supported_set in physical_space.py, see System 2 below) never reads this file at all. The edit was reverted (no trace left in the tree from this investigation).

System 2 (current, live): library/templates/ores.*.org physical-space nodes

  • One small .org file per facet/technical-space/archetype under projects/ores.codegen/library/templates/ (e.g. ores.cpp.domain.org, ores.cpp.repository.org, ores.cpp.generator.org, ores.cpp.service.org, ores.cpp.protocol.org, ores.cpp.qt.org, ores.sql.schema.org, …). Each carries its own #+model_types: keyword and a literate mustache source block, tangled to the sibling .mustache file actually consumed by the generator (compass build --direct tangle_codegen_templates re-tangles after editing the .org).
  • Parsed by load_graph() in projects/ores.codegen/src/codegen/physical_space.py (line ~75), which builds a Graph of facets/archetypes/model-type-admissibility from these files.
  • Driven by resolve_targets() in projects/ores.codegen/src/codegen/generate.py (line ~94), calling compute_supported_set() (physical_space.py line ~168) — intersecting "facets admissible for this model's type" with "facets requested via --address". This is what =compass codegen entity generate <name> [--address ...] actually calls.
  • Currently, ores.cpp.domain.org, ores.cpp.generator.org, and ores.cpp.repository.org all declare #+model_types: schema domain_entityno junction. Only ores.sql.schema.org (or wherever the SQL facet's node lives) includes junction, which is why entity generate on a junction produces SQL only.

The trap: junction C++ template mapping already exists, untested, in the dead system

projects/ores.codegen/src/codegen/core.py has get_cpp_junction_template_mappings() (line ~1094) mapping junction data onto the same entity-shaped C++ templates (domain class, JSON I/O, table I/O, generator, entity, mapper, repository — explicitly not service or protocol, per its own docstring: "Junction tables … do not generate service or protocol files (those are only for domain entities)"). There is also junction-specific data enrichment in core.py around line 2615-2660 (normalising junction.left=/=right=/ =columns, adding is_uuid=/=is_optional_uuid flags, etc. onto the junction dict — but not rewriting it into a domain_entity-shaped dict).

This code is reachable only through System 1 (the --profile path), which strongly suggests it was written and partially wired up, then abandoned mid-migration when the physical-space system (System 2) superseded profiles for the default path — without anyone porting the junction C++ support across. It has probably never been exercised end-to-end: nothing in the current default code path can reach it, and no generated junction C++ file exists anywhere in the tree to prove it once worked.

The templates it targets (cpp_domain_type_class.hpp.mustache etc.) read domain_entity.entity_singular, domain_entity.primary_key, and similar fields that assume a single primary key column — a junction has left=/=right FK columns instead, no independent primary key of its own conceptually (though the generated SQL PK is the composite of both FK columns plus valid_from). Whether the templates handle this correctly for junction data, or need real Mustache-level branching (domain_entity vs junction sections) or a normalisation shim that manufactures a synthetic composite-key "domain entity" view over a junction, is unknown and unverified — this is the actual work, not just flipping model_types.

Acceptance

  • facet_catalogue.org and core.py's _load_profiles_from_org() / --profile code path (System 1) are deleted outright — not deprecated, not left dormant. Anything that turns out to secretly depend on it breaks immediately and loudly (a missing function, a failed import, a CLI error) rather than silently no-op'ing the way the --profile-editing experiment above did. This is deliberate: the point is to surface every remaining dependency on the dead system by breaking it, not to guess at them in advance.
  • Every existing *_domain_entity.org=/=*_junction.org=/etc. model file and every already-generated =.hpp=/.cpp=/=.sql= file in the tree is left completely untouched by the deletion — this story removes dead tooling code, not generated artefacts or their source models. If deleting System 1 causes regeneration of any existing entity to produce a diff, that is a bug in this story's deletion, not an acceptable side effect.
  • compass codegen entity generate <any-existing-entity> (a sample of a few, across a few components) is re-run after the deletion and produces byte-identical output to before, proving System 1 truly wasn't load-bearing for anything currently generated.
  • ores.cpp.domain.org, ores.cpp.generator.org, and ores.cpp.repository.org (at minimum — service/protocol/qt deliberately excluded, matching get_cpp_junction_template_mappings's own documented scope decision, unless investigation says otherwise) gain junction in their #+model_types:, are re-tangled, and their templates are fixed/adapted as needed so junction generation actually produces compilable, correct C++.
  • Validated end-to-end against a real junction: projects/ores.marketdata/modeling/ores.marketdata.tenor_convention_resolution.org (id E1F5A9C3-6D2B-4E8A-B7F1-3C9D5A2E6B48) regenerates a domain struct + repository (at minimum read_all=/=read_latest-style queries; write support if the templates naturally produce it, though this junction's own data is Foundation-layer-managed and doesn't strictly need writes). Diff the generated repository/entity against the hand-authored reference files listed below to sanity-check field names, types, and query shape — then delete the hand-authored files and switch projects/ores.marketdata/core/src/messaging/registrar.cpp and the hand-authored service/protocol/handler (added in the Tenor management UI task) to use the generated repository instead.
  • No regression to any of the ~83 existing domain_entity models or ~8 existing junction models' generated output (see Acceptance bullet 3 above) — this is a strictly additive change to what junctions can generate, not a reshuffle of what entities already generate.

Tasks

Task State Start End Description
Delete legacy codegen profile system (facet_catalogue.org, core.py profile path) DONE 2026-07-15 2026-07-15 Delete facet_catalogue.org and core.py's _load_profiles_from_org()/–profile code path outright. Keep all existing org models and generated code untouched; regenerate a sample of existing entities to confirm no diff.
Add junction to model_types on ores.cpp.domain/generator/repository.org DONE 2026-07-15 2026-07-15 Add junction to #+model_types: on the three physical-space facet node docs, re-tangle, and attempt generation against tenor_convention_resolution to surface what breaks.
Fix domain/generator/repository Mustache templates for junction-shaped data DONE 2026-07-15 2026-07-15 Adapt the three facets' templates so junction generation (composite left/right key, no single primary_key) produces compilable, correct C++ – template branching or a data-normalisation shim in core.py.
Validate junction codegen against tenor_convention_resolution; retire hand-authored code DONE 2026-07-15 2026-07-15 Regenerate tenor_convention_resolution, diff against the hand-authored reference files, build, then delete the hand-authored entity/repository and repoint the hand-authored service/protocol/handler/registrar at the generated repository.
Sweep docs/skills/recipes for stale legacy-profile-system references DONE 2026-07-17 2026-07-18 Correct any remaining doc/knowledge/skill/recipe text describing the deleted facet_catalogue.org/–profile system, so documentation matches the physical-space system only.
B4: Collapse refdata and refdata-cpp into a single component DONE 2026-07-18 2026-07-18 Merge the refdata and refdata-cpp entries in the component catalogue into one component with a single discovery glob over ores.refdata.*.org. Remove the dead models_dir entries from both. Update all compass codegen entity commands that reference either component.
B6: Add compass codegen entity archetypes subcommand DONE 2026-07-19 2026-07-19 New 'archetypes' subcommand lists the full physical space catalogue in ores.ts.facet.archetype notation. With –entity shows the entity's supported set with per-archetype enabled/disabled status. Makes the address system self-documenting. Depends on B5.
B10: Retire facet_catalogue monolith + legacy literate facet/group docs DONE 2026-07-16 2026-07-16 Delete facet_catalogue.org and the 18 legacy literate facet/facet_group docs (cpp_qt.org, cpp_group.org, sql_schema.org, …), remove the now-dead load_profiles/resolve_profile_templates/validate_profile_for_model code, and repoint or remove the hundreds of inbound id: links (56 to facet_catalogue alone) across the org-roam graph to the new ores.* nodes.
B11: Migrate live codegen callers from –profile to –address DONE   2026-07-18 Switch the live caller surfaces (codegen/qt/sql recipes, skills, runbooks, shell scripts, compass help) from the deprecated –profile to –address; resolve composite-profile cases (all, all-cpp, non-temporal) that have no single-address equivalent; leave historical agile/plan/analysis docs untouched. Then optionally remove the –profile shim.
Migrate content: move _table.org sections into entity files and delete table files DONE 2026-07-19 2026-07-19 For each of the 9 dual-file entities, move the Validation function and Insert trigger sections from the _table.org file into the entity file; verify byte-identical generated SQL; delete the _table.org file. Moved in from the now-closed Decommission ores.codegen.table model type story — was originally out of scope here (Step 5 of the migration path), pulled in at sprint 22 close since it's real remaining work with no home elsewhere.

Decisions

  • Merged in "Resolve codegen model unification blockers" (2026-07-15): while starting task 1, discovered that story had two BACKLOG tasks (B10, B11) duplicating this story's tasks 1 and 5 almost exactly, plus six more BACKLOG blockers (B4, B6/archetypes subcommand, B7, B9, the _table.org content migration, table-model-type retirement) in the same "retire legacy codegen cruft" vein. Rather than track overlapping work in two stories, moved all 8 (via compass task move, UUIDs intact) into this story and closed the donor DONE. B10 was narrowed since task 1 already deleted its Python half (load_profiles=/=_load_profiles_from_org); B11 was left standing alongside task 5 with a delineation note, since it covers migrating working --profile callers to --address — a different, still-live shim (_PROFILE_TO_FACETS in generate.py) from the dead System 1 task 5 sweeps stale prose about.
  • Delete, don't deprecate. Explicit user instruction: rather than leaving System 1 in place as unused-but-harmless code, delete it outright so that anything still secretly depending on it breaks immediately and visibly, rather than continuing to silently work by accident (or silently not work, as the facet_catalogue.org edit-with-no-effect experiment demonstrated) until someone notices much later.
  • Keep every existing model and generated artefact untouched. This story is scoped to the codegen tooling, not to any entity's generated output. Explicit user instruction: "keep the legacy models and generated code" — only the dead resolution code path and the legacy catalogue doc are in scope for deletion.
  • Junction gets domain/generator/repository, not service/protocol/qt, as the initial scope — matching what get_cpp_junction_template_mappings's own docstring already documents as the intended design ("do not generate service or protocol files"). A junction's rows are meant to be read through a parent entity's service, not exposed as their own top-level NATS endpoint — the Tenor management UI task's hand-authored service is the concrete counter-example motivating this story, and if that pattern turns out to be needed generally (not just for this one screen), that is a separate, later decision about whether to widen junction codegen scope further, not something to fold into this story's initial pass.

Out of scope

  • Any change to what an existing domain_entity or already-supported junction (SQL-only) model generates today — covered by the regression-check acceptance bullet.
  • Widening junction generation to service/protocol/qt facets — out of scope unless the domain/generator/repository work reveals it's actually needed sooner than expected (see Decisions).
  • Fixing the IMM_ROLL tenor resolution algorithm or seeding credit/CDS resolution rows — unrelated, tracked separately in the capture Implement IMM_ROLL tenor resolution for credit/CDS conventions.
  • Finishing the Tenor management UI itself — tracked in Tenor management UI task, which this story unblocks (that task can proceed now with a hand-authored service/protocol/handler as a stopgap, per the user's explicit "continue your work manually" instruction, and switch to the generated repository once this story lands).

Result

Closed DONE: this story's own Acceptance is fully met by tasks 1-4. System 1 (facet_catalogue.org's dead resolution code) is deleted; junction C++ generation (domain/generator/repository) works and is validated end-to-end against a real junction (tenor_convention_resolution), which now runs on generated code with the hand-authored stopgap retired; no regression to any existing domain_entity=/=junction model's generated output (book verified zero-diff throughout, plus the full ores.refdata test suite passing). Task 4's manual Tenor management UI screen check was skipped per explicit user instruction (an unrelated build-target issue interrupted starting the client; the service/protocol contract is unchanged by this story's work, so the risk is low) – left for whoever next runs that screen to confirm.

Merged-in work retained here: task 5 (sweep docs/skills/recipes for stale legacy-profile-system references) and 4 of the 8 tasks merged in from "Resolve codegen model unification blockers" – B4, B6, B10, B11, and the _table.org content migration – all now DONE. None of these were required by this story's own Acceptance criteria; B10/B11 in particular were near-duplicates of tasks 1/5's scope.

Split into a follow-up story at health review 1 (2026-07-20, once DONE-heavy but not fully closed): B7 (split-model backward- compatibility, itself completed DONE and absorbing the duplicate table-model-type task as ABANDONED), B9 (mustache-to-address rename), and the remaining junction-codegen rollout task, moved via compass task move to Codegen legacy-profile retirement follow-ups so this story's own finish line didn't keep expanding.

Emacs 29.3 (Org mode 9.6.15)