Task: Fix dq entity model drift: update to latest format and regenerate
Table of Contents
This page documents a task in the Codegen developer experience improvements story. It captures the goal, current status, acceptance, and any notes or results.
Goal
Mirror Fix refdata entity model drift for ores.dq: regenerate every
ores.dq domain_entity and junction model (compass codegen entity
list) against current codegen templates. Where a model uses an
outdated format/convention rather than the templates having simply
moved on, update the model to the current convention before
regenerating. Commit the resulting clean tree.
Along the way, apply the same lessons learned fixing refdata's
drift: check whether any entity's SQL/domain/messaging layer is
explicitly hand-maintained (a doc comment flagging missing
paste-block coverage) before doing a full regen – prefer
--address-scoped regen for anything so flagged; grep synthetic
generators for hardcoded/fabricated soft-FK values (coding scheme
codes, lookup-table codes) that don't match what a freshly-recreated
default database actually seeds.
Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Codegen developer experience improvements |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-07-30 |
Acceptance
- Every ores.dq
domain_entityandjunctionmodel is regenerated viacompass codegen entity generateand shows zero diff on a second regen (idempotent). - Any model found using an outdated format/convention is updated to the current one before regenerating.
- Any entity whose SQL/domain/messaging layer is hand-maintained
(flagged in its own model doc, mirroring
party's) is excluded from a full regen and only its safe facets (e.g. Qt) are regenerated, same asparty's treatment in the refdata task. - Synthetic generators for FK-referencing fields are checked against
actual seed SQL (the default
db recreatechain, not optional artefact/dataset scripts) and fixed if they fabricate values that don't exist there. compass buildand fullratpass after the regenerated files are committed.- No entity outside dq is touched.
Plan
(Implementation strategy. Written when work starts; key decisions
are distilled into the parent story's * Decisions at close, but the
plan itself stays — it is the historical record of what we did.)
Notes
Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario, run
through the QA Validation Runner panel) that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
PRs
| PR | Title |
|---|---|
| #1777 | [dq] Fix dq entity model drift |
Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | Junction generator template unconditionally declares counter/idx even when no generator_expr references idx (e.g. badge_mapping) | ores.cpp.generator.generator_impl.org, core.py | Fixed | Added a needs_counter flag for junctions mirroring domain_entity's, gated the declaration (a1c52579a). |
| 2 | badge_mapping's newly generated messaging/service stack is completely unwired – registrar.cpp still routes through the bespoke badge_handler | ores.dq.badge_mapping_junction.org, generated messaging/service files | Fixed | Disabled ores.cpp.nats-handler/nats-sub-registrar/protocol/service for badge_mapping via :ores.*.enabled: false; removed the dead generated files (a1c52579a). |
| 3 | Junction template drops boolean :default: initializers (dataset_bundle_member's optional field) | cpp_domain_type_class.hpp.mustache | Declined | Pre-existing gap, also present in refdata's calendar_date before this PR; every construction site already sets the field explicitly. Out of scope for this task. |
| 4 | NATS subject/response-field rename for dataset_bundle_members by-bundle lookup | dataset_bundle_member_protocol.hpp | Declined | Informational only; producer and consumer updated together in this PR, internally consistent. |
| 5 | No unit test coverage for the new codegen logic | core.py, generate.py | Fixed | Added projects/ores.codegen/tests/test_dq_drift_fixes.py covering composite-PK extra args and junction counter/idx gating (a1c52579a). |
Result
All ores.dq domain_entity and junction models regenerated clean.
Along the way, uncovered and fixed several codegen infrastructure
gaps that were classes of bugs rather than one-off drift:
- Junctions had no
<<paste:UUID>>mechanism at all (unlike domain_entity). Added it to the nats-handler and protocol junction templates and toload_org_junction_model=/=_substitute_paste_markers, and used it to restoredataset_bundle_member's hand-written plainlist()operation. Itsby_bundleoperation instead moved onto the existing generic:list_by:mechanism (already proven bycalendar_date) rather than a paste block, per "paste blocks only for what isn't already generic". - A junction or domain_entity with no
** Qtdrawer used to abort the entire generate the moment it reached the first Qt unit (junction), or silently emit broken Qt output – empty#include, empty class names (domain_entity, no guard at all) – starving every facet ordered after Qt (repository, service, SQL) of ever being generated. Both are now skipped cleanly under the default full generate; an explicit--address ores.cpp.qtstill fails loud. Removed the resulting untracked, never-successfully-generated Qt scaffolding forcatalog,subject_area,change_reason,change_reason_category,data_domain. - Composite text primary keys: the generated NATS handler's
history()=/=remove()only ever passed the first pk column to the service, silently dropping the rest of a compound key (caught viasubject_area'sname=+=domain_namekey). Addedhistory_request_extra_args=/=delete_request_extra_argsto carry the remaining columns. - Junction generators had no per-row uniqueness guarantee (unlike domain_entity's counter/idx mechanism) – batch-generating a junction with plain faker-word left/right codes could (and did) collide on the composite PK. Added the same counter/idx pattern to the junction generator template.
- Model-level fixes: stale pre-split
ores.dq/domain/.../ores.dq/messaging/...include paths incode_domain=/ =dataset_bundle=/=badge_severity's Qt drawers; an invalidIconenum value on two entities (Award,Grid– neither exists);change_reason.org's own auto-generated validation function collided by name with the pre-existing, widely-used sharedores_dq_validate_change_reason_fnhelper (removed the entity's redundant* Validation functionsection – nothing referenced it); thedataset_bundleseed upsert function'sON CONFLICTclause didn't match the entity's actual(tenant_id, code, name)composite natural key. code_domain's Qt regen was reverted: itsBadgeCachepassthrough (Badge Mappings tab) has no template support and isn't common to other entities, so it stays hand-maintained until a Qt paste-block seam exists – same treatment asparty's SQL in the refdata task.- Updated stale test expectations (generator
version=/ =change_reason_codedefaults) and hand-written call sites (registrar.cpp,PublishBundleWizard.cpp,DataLibrarianWindow.cpp,DatasetBundleDetailDialog.cpp) to match the regenerated shapes.
Verification: local build clean (linux-clang-debug-make); full
rat (build + ctest across every component) passed, after a
db recreate (the schema had drifted: the change_reason_fn name
collision and the dataset_bundle seed ON CONFLICT mismatch both
blocked schema setup).
Task-ID: B8FE7DA1-18D7-4F33-8402-21411A51A8E2