Story: Codegen developer experience improvements

Table of Contents

This page documents a story in Product backlog — inbox, carried unfinished from Sprint 24 at close. It captures the goal, current status, acceptance criteria, and the tasks that compose it.

Carried from Sprint 22 as a not-yet-started BACKLOG story.

Goal

Fix three interrelated friction points in the codegen.sh regenerate workflow that were observed repeatedly during sprint 21 currency auxiliary work. All three make targeted codegen runs unnecessarily noisy, error-prone, and hard to scope to a logical entity group:

  1. Spurious ERRORs in --component mode. When an address is run against a component that contains entity types the address does not support (e.g. ores.cpp.qt against refdata-cpp, which contains table and junction models alongside domain_entity models), the generator emits one ERROR per incompatible entity and inflates the final error count. These are expected skips, not failures.
  2. No entity-level filter. --component refdata-cpp --address ores.cpp.qt generates Qt files for all 25+ entities in the component. There is no way to restrict generation to a named subset (e.g. rounding_type,monetary_nature, currency_market_tier). Every targeted run generates 20+ extra files for out-of-scope entities that the caller must manually discard.
  3. No logical entity group / scope concept. A task such as "sync Qt codegen for currency auxiliaries" involves a well-defined set of entities that share a domain boundary (the currency domain). There is currently no concept of an entity group or tag in the model that the codegen CLI could use to select --scope currency and regenerate only the relevant entities across any address without listing them by name. This also means it is impossible to run the codegen targeting "only currency entities" in a single command — you either regenerate the whole component (too wide) or enumerate entity names manually (fragile, must be updated when the domain grows).

Together these three gaps mean every focused codegen iteration in a task produces output the developer must manually inspect, filter, and clean up. The goal of this story is to eliminate that friction so that a targeted codegen run produces exactly what was requested — no more, no less — with a clean exit code.

Status

Field Value
State STARTED
Parent sprint Sprint 24
Now Not yet started.
Waiting on Nothing; no blockers.
Next Break into tasks; prioritise error downgrade first (cheapest).
Last touched 2026-06-25

Background

The three issues were surfaced during three separate codegen sessions in the commission_currency sprint:

  • task_sync_cpp_messaging_codegen_currency_auxiliaries — first observation of the component-wide over-generation and untracked-file cleanup burden.
  • task_sync_qt_codegen_currency_auxiliaries — second observation of over-generation (Qt profile); also first observation of the 13 spurious ERROR lines from qt profile against refdata-cpp (5 table + 8 junction models).
  • Both tasks — the impossibility of expressing "regenerate only currency entities" in a single command without listing all three entity names explicitly.

The --entity filter shortcoming was previously captured in codegen –component run generates all entities not just the target. The profile/type ERROR downgrade was captured in Downgrade profile/model-type incompatibility from error to skip in –component mode. This story groups both with the related entity-group feature into one deliverable.

Acceptance

  • Running codegen.sh regenerate --component refdata-cpp --address ores.cpp.qt produces zero ERROR or WARNING lines for models whose type (table, junction) is not supported by that address. Those models are silently skipped. Only genuine generation failures produce ERROR output.
  • A --entity (or --entities) flag is accepted by codegen.sh regenerate. Running --component refdata-cpp --address ores.cpp.qt --entity rounding_type,monetary_nature,currency_market_tier generates only files for those three entities and writes no other files to disk.
  • The ores.refdata model supports an entity group or tag annotation (e.g. :group: currency or a #+group: property in the entity org file). Running codegen.sh regenerate --component refdata-cpp --address ores.cpp.qt --group currency regenerates all and only the entities tagged currency, with no manual enumeration required. The currency group initially covers: currency, rounding_type, monetary_nature, currency_market_tier.
  • All existing codegen runs (--component, --address, without new flags) are unaffected and produce identical output to today.
  • codegen.sh --help and the profile documentation list the new flags and the group annotation syntax.
  • CI passes; no production source files are changed by this story.

Tasks

Task State Start End Description
Downgrade address/model-type incompatibility to skip in –component mode DONE 2026-07-26 2026-07-27 In generator.py, detect model-type/address incompatibility in –component mode and skip silently (DEBUG log) rather than emitting ERROR and incrementing the error counter. Keep ERROR for explicit single-entity invocations.
Add –entity filter flag to codegen.sh regenerate DONE 2026-07-27 2026-07-29 Implement an –entity (or –entities) flag that restricts a –component run to the named entities only. No other entity files are generated or written to disk.
Task: Add entity group / tag system to model and codegen CLI BACKLOG     Design and implement a group annotation (e.g. :group: in entity org files or a groups section in the component manifest). Expose –group flag on codegen.sh regenerate. Define the initial currency group in ores.refdata.
Split every codegen template org into commented literate blocks BLOCKED 2026-07-29   For every template org under ores.codegen/library/templates, split the single mustache tangle block into multiple same-target blocks interleaved with detailed human-level prose explaining what each block does and why; generated code stays comment-free.
Fix refdata entity model drift: update to latest format and regenerate DONE 2026-07-29 2026-07-30 Several refdata entities' checked-in generated code has drifted from what current codegen templates/models produce (discovered while verifying zero-diff for the literate-template-block-commentary task: country_repository.cpp, country_service.cpp, ClientCountryModel.cpp, CountryDetailDialog.cpp, CountryMdiWindow.cpp, country_protocol.hpp, country_handler.hpp all show diffs on regen even with unmodified templates). Update every refdata entity org model to the latest format/conventions, regenerate all of them, and commit the resulting drift so the tree is clean before further template work lands.
Bring counterparty's Qt logo/flag picker under codegen BACKLOG     counterparty's Qt detail dialog has a hand-added logo/flag picker (entityImageId(), initFlagButton, flagGroup box, flagEdited wiring, save-on-change) that isn't modeled/templated – discovered because regenerating counterparty via compass codegen entity generate silently deletes it. counterparty is excluded from the fix-refdata-model-drift task until this facet exists (mirrors the existing image_id/logo support other entities already have modeled).
Bring tenor_convention_resolution's messaging layer under codegen BACKLOG     tenor_convention_resolution (junction) has a hand-authored messaging layer (protocol/handler, read-only list) wired inline in registrar.cpp with the comment 'codegen doesn't generate a service/protocol/handler/sub-registrar for it'. Regenerating it via compass codegen entity generate produces a codegen-shaped handler with a different API (no .list(), different request-type naming) that breaks the hand-wired call site in registrar.cpp – a real compile error, not a diff nit. Excluded from the fix-refdata-model-drift task until junction messaging codegen (see the junction-messaging-codegen-missing-service capture) covers this shape.
Fix synthetic entity model drift: update to latest format and regenerate DONE 2026-07-30 2026-07-30 Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.synthetic domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix marketdata entity model drift: update to latest format and regenerate DONE 2026-07-30 2026-07-30 Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.marketdata domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix dq entity model drift: update to latest format and regenerate DONE 2026-07-30 2026-07-30 Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.dq domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix iam entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.iam domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix scheduler entity model drift: update to latest format and regenerate DONE 2026-07-31 2026-07-31 Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.scheduler domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix analytics entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.analytics domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix database entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.database domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix compute entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.compute domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix controller entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.controller domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix reporting entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.reporting domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix workflow entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.workflow domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix trading entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.trading domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.
Fix workspace entity model drift: update to latest format and regenerate BACKLOG     Following the same pattern as the refdata sweep (task fix-refdata-model-drift): regenerate every ores.workspace domain_entity and junction model against current codegen templates, updating any model found using an outdated format/convention before regenerating, and commit the resulting drift-free tree. compass build and rat must pass after.

Decisions

  • Wording updated at sprint 23 pull-in (2026-07-11). Originally written against --profile=/–component=; reworded to --address to match the CLI after Resolve codegen model unification blockers' B5 landed. If that story's B4 (collapse refdata=/=refdata-cpp) lands first, --component refdata-cpp collapses to --component refdata — update examples accordingly when picking up this story's first task.
  • Error vs. skip boundary. Incompatible-type encounters are an error only when the caller explicitly names an entity (--entity rounding_type --address ores.cpp.qt on a table model). In --component mode the codegen discovers entities automatically and must handle heterogeneous types gracefully.
  • Group granularity. The group concept should live in the entity model (org file property or a manifest section), not in the codegen CLI, so that it is co-located with the model and does not require CLI changes when the group membership changes.
  • –entity vs. –group. Both flags are needed. --entity is for ad-hoc one-off targeting without modifying the model. --group is for repeatable domain-scoped regeneration that reflects how the business thinks about entity ownership.
  • counterparty's logo/flag facet already existed; it just wasn't wired up. The "Bring counterparty's Qt logo/flag picker under codegen" backlog item assumed the facet needed building from scratch. It didn't – :flag_icon_column: already drives it (currency uses it); counterparty's model was simply missing the property. Adding it and regenerating restored the feature byte-for-byte. party had the identical gap, fixed the same way. That backlog item's premise is stale; worth re-triaging before picking it up.
  • scheduler's job_definition is hand-diverged across every layer, not just SQL. Unlike party (hand-maintained SQL only, other facets safe to regen), job_definition's domain column shapes (tenant_id=/=schedule_expression=/=is_active) don't match the hand-written struct's actual types (std::optional<boost::uuids::uuid>=/=cron_expression=/=bool), so even domain/repository-entity/mapper regen would emit wrong types. Its repository and messaging layers are also hand-maintained, with cross-component consumers (ores.reporting). No facet is currently safe to regenerate; the model file documents the exclusion in full rather than a full regen being attempted. Needs a proper resync (fixing column shapes, re-adding the hand-written operations to the generated surface) before any facet can be regenerated.

Out of scope

  • Changing which profiles support which model types (that is a profile design question, not a DX question).
  • Per-entity profile overrides (specifying a different profile for individual entities within a component run).
  • Automatic discovery of which entities in a component have drifted (that belongs to the refactor_codegen_cpp story).

Emacs 29.3 (Org mode 9.6.15)