Task: Fix marketdata entity model drift: update to latest format and regenerate

Table of Contents

This page documents a task in the Codegen developer experience improvements story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Mirror Fix refdata entity model drift for ores.marketdata: regenerate every ores.marketdata domain_entity and junction model (compass codegen entity list) against current codegen templates. Where a model uses an outdated format/convention rather than the templates having simply moved on, update the model to the current convention before regenerating. Commit the resulting clean tree.

Along the way, apply the same lessons learned fixing refdata's drift: check whether any entity's SQL/domain/messaging layer is explicitly hand-maintained (a doc comment flagging missing paste-block coverage) before doing a full regen – prefer --address-scoped regen for anything so flagged; grep synthetic generators for hardcoded/fabricated soft-FK values (coding scheme codes, lookup-table codes) that don't match what a freshly-recreated default database actually seeds.

2. Status

Field Value
State DONE
Parent story Codegen developer experience improvements
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-07-30

3. Acceptance

  • Every ores.marketdata domain_entity and junction model is regenerated via compass codegen entity generate and shows zero diff on a second regen (idempotent).
  • Any model found using an outdated format/convention is updated to the current one before regenerating.
  • Any entity whose SQL/domain/messaging layer is hand-maintained (flagged in its own model doc, mirroring party's) is excluded from a full regen and only its safe facets (e.g. Qt) are regenerated, same as party's treatment in the refdata task.
  • Synthetic generators for FK-referencing fields are checked against actual seed SQL (the default db recreate chain, not optional artefact/dataset scripts) and fixed if they fabricate values that don't exist there.
  • compass build and full rat pass after the regenerated files are committed.
  • No entity outside marketdata is touched.

4. Plan

(Implementation strategy. Written when work starts; key decisions are distilled into the parent story's * Decisions at close, but the plan itself stays — it is the historical record of what we did.)

5. Notes

6. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario, run through the QA Validation Runner panel) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

7. PRs

PR Title
#1764 [marketdata] Fix marketdata entity model drift

8. Review

# Comment summary File Decision Notes
1 market_series_generator.cpp regen reintroduced the uninitialized asset_class/series_subclass enum bug fixed in PR #1412 (hand-written fix never captured in the .org model) market_series_generator.cpp Fixed Added :name generator blocks cycling both enums deterministically, keyed off idx (65ee06b63).
2 market_series_handler.hpp's export_to_storage paste-block called log_handler_entry unqualified; broke when an unrelated handler_helpers-to-wire_codec migration on main moved it into ores::service::messaging market_series_handler.hpp Fixed Qualified the call in the .org paste-block, regenerated (65ee06b63).
3 Fix #1 only added a generator block, which feeds synthetic data but not the domain struct itself – cpp_domain_type_class.hpp.mustache only emits a member initializer when :default_value: is set, so the struct member was still genuinely uninitialized independent of the generator org_loader.py, market_series.org Fixed Added a validate_model check requiring :default_value:/:default: on every non-nullable is_enum column; added :default_value: fx/spot to asset_class/series_subclass (6c69432b2). Checked every is_enum column repo-wide – only these two entities use it, no other regressions.
4 Item 3's validator accepted :default: as satisfying the requirement (template only honors :default_value:, reopening the same bug for the next author); also didn't scan primary_key columns org_loader.py Fixed Dropped the :default: fallback, require :default_value: specifically; now also scans de["primary_key"]["columns"] (c6d24ccd9). Re-verified accept/reject repros.

9. Result

feed_binding regenerated clean (zero diff). The other 3 ores.marketdata entities (market_fixing, market_observation, market_series) had the same legacy top-level * Primary key / * Natural keys org-heading drift already fixed for ores.synthetic: none could be regenerated at all ("Missing primary key" validation error) until migrated to field-level :primary_key:=/:natural_key:= flags inside * Columns. No entity outside marketdata touched, no hand-maintained-SQL flags found on any of the 4 models.

Because these 3 had never successfully regenerated before, several downstream generated layers (messaging registrars, history-field-mappers) didn't exist on disk yet; committing the regen creates them for the first time. Also regenerated component_files.cmake for the 2 affected components.

compass build and full rat green.

Task-ID: 121F5727-E69C-4FA6-B8AD-40F286C22CD2

Emacs 29.3 (Org mode 9.6.15)