Task: Bind ores.refdata entities to profiles; verify zero-diff regen
Table of Contents
This page documents a task in the Entity classification and drift baseline across all components story. It captures the goal, current status, acceptance, and any notes or results.
Goal
Refined from a read-only classification scanner into a concrete,
falsifiable proof: implement real :profile: binding in the code
generator, rebind every ores.refdata entity that matches one of the
5 named profiles (Variability Profiles) to that profile — removing its
now-redundant, individually-authored feature properties — regenerate,
and confirm byte-identical output against what was checked in before
the rebind. A zero-diff regen is the strongest possible confirmation
that the profile catalogue actually captures what it claims to: if
binding to a profile changes what gets generated for a single entity,
the profile (or the classification of that entity as matching it) is
wrong, and that must be fixed before this pilot is trusted, let alone
rolled out to the other ~19 components.
This is the "Profile match" classification dimension from the story's settled list, but validated far more strongly than a read-only report could: not "the feature values look equal on paper" but "codegen produces the same files whether the entity's variability is spelled out explicitly or expressed as one profile name."
Entities matching no profile are not a failure of this task — they are real data, either genuinely novel configurations (candidates for a 6th profile, once a second entity repeats the same combination) or entities whose per-entity features are correctly bespoke. Record them, don't force a match.
Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Entity classification and drift baseline across all components |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-07-31 |
Acceptance
org_loader.pyresolves a:profile: <name>property on an entity model by reading that profile's ownvariability_<slug>.orgAssignments table as the source of truth (no hardcoded second copy of the profile catalogue) and merging its feature values in before the rest of loading proceeds; an explicit per-entity property still overrides its profile's value where both are present.- Every
ores.refdataentity's current feature values are computed and compared against the 5 profiles; each is recorded as a match (which profile) or no-match. - Every matching entity is rebound: its explicit per-entity feature
properties replaced with
:profile: <name>, keeping only what the profile does not cover (structural fields, per-entity naming). - Every rebound entity is regenerated and diffed against its pre-rebind checked-in output. Zero diff for all of them is the pass condition; any non-zero diff is a bug in either the profile catalogue or the matching logic and must be fixed, not worked around, before this task is done.
currency(thefully-featured-lookupreference entity) is among the rebound and zero-diff-verified set.- No-match entities are listed with their actual feature values, not silently dropped.
Plan
(Implementation strategy. Written when work starts; key decisions
are distilled into the parent story's * Decisions at close, but the
plan itself stays — it is the historical record of what we did.)
Metatypes covered
ores.refdata/modeling/*.org (69 files) spans four metatypes; only
two carry any variability :profile: binding can express:
| Metatype | Count | :profile: binding |
|---|---|---|
ores.codegen.entity |
58 | Yes — 6 profiles, 100% match, zero new work |
ores.codegen.junction |
9 | Yes — added this task (7th profile) |
ores.codegen.module |
1 | N/A — a catalogue index, not an entity |
ores.codegen.component |
1 | N/A — component overview, not an entity |
Worklist — process one entity at a time: remove its explicit
feature properties the target profile covers, add :profile:,
regenerate at --address ores, diff against the pre-rebind
checked-in output, revert and flag on any non-zero diff.
Non-codegened (2) — no variability applies, tracked not skipped
| File | Justification |
|---|---|
module.org |
ores.codegen.module: the component's entity catalogue index, not itself a generated artefact. |
component_overview.org |
ores.codegen.component: component-level metadata doc, not a generated artefact. |
ores.codegen.junction (9) → tenant-scoped-junction
| Entity | Status |
|---|---|
| currency_calendar (reference entity) | DONE |
| calendar_date | DONE |
| currency_country | DONE |
| currency_currency_group_junction | DONE |
| currency_pair_convention_calendar | DONE |
| party_counterparty_junction | DONE |
| party_country_junction | DONE |
| party_currency_junction | DONE |
| tenor_convention_resolution | DONE |
ores.codegen.entity (58), grouped by matched profile
simple-lookup(34)
Entity Status asset_class_code DONE book_purpose_type DONE book_status DONE business_centre DONE business_day_convention_type DONE calendar DONE calendar_type DONE contact_type DONE country DONE currency_group DONE currency_market_tier DONE currency_pair DONE currency_pair_classification DONE currency_pair_convention DONE curve_role DONE day_count_fraction_type DONE floating_index_type DONE instrument_code DONE ledger_feed_type DONE leg_type DONE (bound; missing has_change_reason_cachewas a genuine gap, fixed — see Notes)monetary_nature DONE party_id_scheme DONE party_status DONE party_type DONE payment_frequency DONE purpose_type DONE regulatory_book_type DONE rounding_type DONE tenor DONE tenor_anchor DONE tenor_convention DONE tenor_kind DONE tenor_resolution_algorithm DONE tenor_unit DONE fk-scoped-child(3)
Entity Status book DONE business_unit NO MATCH (correctly bespoke — genuinely tenant-only, not workspace-scoped; fk-scoped-childwould wrongly add aworkspace_idcolumn — see Notes)portfolio DONE self-referencing-hierarchy(12)
Entity Status business_unit_type DONE calendar_exception DONE calendar_rule DONE counterparty DONE counterparty_contact_information DONE counterparty_identifier DONE crm_driver_pair DONE crm_enabled_derived_pair DONE crm_topology_config DONE party DONE party_contact_information DONE party_identifier DONE fully-featured-lookup(1)
Entity Status currency DONE (proof case) read-only-time-series(0 in ores.refdata)
No
ores.refdataentity matched; reference entitymarket_observationlives in another component.workspace-scoped-lookup(8)
Entity Status cds_convention DONE deposit_convention DONE fra_convention DONE ibor_index_convention DONE ois_convention DONE overnight_index_convention DONE swap_convention DONE zero_convention DONE
No-profile-match — none remain
Every ores.codegen.entity and ores.codegen.junction file in
ores.refdata matches a profile (7/7, after adding
tenant-scoped-junction). 100% coverage is achievable without
inventing or forcing any additional profile.
Notes
business_unit(matchedfk-scoped-child) andleg_type(matchedsimple-lookup) both left a feature unset that their matched profile fixes to a value (has_workspace_idforbusiness_unit,has_change_reason_cacheforleg_type) — the survey only compared features already present on both sides, so an unset feature the profile would newly inject (via its setdefault semantics) never showed up as a mismatch. A trial rebind of each surfaced a real, non-empty diff — caught by the regen+diff check exactly as designed, not a tooling bug. Lesson for the next component: a correct match requires comparing every profile-assigned feature against the entity's actual resolved value (defaulted or not), not only features the entity happens to already set. The two cases resolved differently on inspection, not by the same rule:leg_typemissing change-reason tracking was a genuine gap, not a deliberate design choice — every othersimple-lookupentity audits who changed a row and why; this one arbitrarily didn't. Bound to the profile, which added aChangeReasonCache*constructor parameter toLegTypeController(same shape every sibling controller already has); wired the one hand-written call site (RefdataPlugin::setupControllers) to passctx_.change_reason_cachealongside them.business_unitis genuinely tenant-only, not workspace-scoped — unlikebook=/=portfolio(the other twofk-scoped-childmembers, both truly workspace-scoped), nothing aboutbusiness_unit's domain semantics calls for per-workspace isolation, and forcing it in would add a real, unwantedworkspace_idcolumn (SQL schema + index + validation trigger + C++ struct field) — a schema migration for no product reason. Left unbound, keeping its own explicit properties: this is the entity's genuinely bespoke, correct configuration, not a defect to fix or a profile to force. Not a candidate for a new profile either — the catalogue's own bar is a combination recurring across more than one entity, andbusiness_unitis currently the onlyores.refdataentity with this exact shape.
- All 63 remaining rebinds (55
domain_entity+ 8junction) were applied via a scripted, per-entity pipeline: for each entity, regenerate the unmodified file first to capture any pre-existing drift baseline, revert, apply the profile rebind, regenerate again, and require the new diff to be an exact subset of that baseline (zero additional diff) before keeping the edit — otherwise revert and flag for manual review. This is the same method proven by hand oncurrencyandcurrency_calendar, just automated across the worklist so every entity still gets its own real regen+diff verification rather than a batch assumption. - Rebinding a junction (currency_calendar) surfaced pre-existing,
unrelated generation drift — the checked-in
currency_calendar_generator.cpp=/=currency_calendar_entity.cppno longer byte-match what the current templates produce even for the unmodified model file (a missing#include <faker-cxx/faker.h>and a blank-line formatting difference), and several facets (messaging handler/registrar, service) that the current templates emit at--address oreswere never generated/checked in for that entity at all. Isolated by regenerating the untouched original file before touching it — confirmed the profile rebind itself introduces zero additional diff. Out of scope for this task (not caused by profile binding); flagged for a separate drift-remediation pass once the full worklist above is regenerated, since the same pre-existing-drift check should be run for every entity in this worklist before trusting its diff as a true zero.
Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario, run
through the QA Validation Runner panel) that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
PRs
| PR | Title |
|---|---|
| #1808 | [refdata,codegen] Bind ores.refdata entities to profiles; verify zero-diff regen |
Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | Duplicate/contradictory ignore entry (flagged independently by all 3 review passes) | projects/ores.sql/utility/validation_ignore.txt | Accepted | Fixed in 6f44cc3b4 – removed the stale "intentionally omit version" entry, kept the correctly-explained one. |
| 2 | CI has no check that catches ores.refdata drift itself (user request, raised mid-round, not a bot finding) | .github/workflows/pr.yml, projects/ores.codegen/scripts/check_component_drift.py (new) | Accepted | Added in 4a1bba138: refdata-codegen-drift job regenerates every ores.refdata model on every PR and fails on any diff. Scoped to refdata for now; extend –components per component as each is brought to a verified zero-diff state. |
| 3 | Minor: legacy is_domain_entity_model fallback path noted, not a defect in this diff | projects/ores.codegen/src/codegen/core.py | Declined | Not changed. Explicitly flagged as no-action-needed by the reviewer itself – a future cleanup note, not a defect in this PR. |
| 4 | Minor: silent last-write-wins on impls_lower collision if two implementations UUIDs differ only by case | projects/ores.codegen/src/codegen/core.py | Declined | Not changed. Can't happen today (UUIDs are canonically case-insensitive identifiers, so two such keys really are the same paste point) – noted by the reviewer as an implicit assumption, not a bug. |
Result
Delivered, and considerably wider in scope than the original pilot goal, after the user escalated this from "prove the profile catalogue" to a full zero-drift hotfix:
:profile:binding implemented for bothdomain_entityandjunctionmetatypes; 7 profiles (addedtenant-scoped-junctionfor the 9 junctions); 100% ofores.refdata's 67 codegen-eligible entities/junctions bound and zero-diff regen verified, per Acceptance.currency(referencefully-featured-lookupentity) andcurrency_calendar(reference junction) both rebound and zero-diff-verified, as required.leg_typebound tosimple-lookupafter a genuine gap (missing change-reason tracking) was found by the profile-injection check;business_unitcorrectly left unbound (genuinely bespoke, not workspace-scoped) – see* Notesfor the reasoning on both.- Beyond the pilot's original scope, at the user's explicit
direction to get the branch fully green rather than merely proving
the tooling: brought
party's previously-BLOCKED SQL layer fully under codegen via two new template paste points (plus reuse of an existing one), eliminating the last hand-written/generated divergence in the component; fixed a real, previously-undetected bug in the codegen paste-substitution mechanism (case-sensitive<<paste:UUID>>matching – found only by actually runningcompass db recreate -y, not by isolated diffing); fixed a composite-component scaffold bug (aggregator components likeores.refdataitself matching the flat single-module archetype); fixed a wire_codec template bug in the NATS event-cache generator; eliminated all remainingores.refdatageneration drift (~130 regenerated files, byte-identical to current templates); silenced one pre-existing, unrelatedTEMPORAL_004schema-validation warning onores_compute_app_version_platforms_tblwith a documented follow-up note (explicit user instruction – that entity's own regen/filename-migration is out of scope here). - Rebased onto latest
main(picked up an unrelatedores.ore.coreXML fix) and re-verified end-to-end: full clean rebuild, DB recreate, fullctestrun. Two failures surfaced post-rebase, both root-caused and fixed (not worked around) rather than dismissed as flaky, per the user's standing "fix it properly" directive:ores.qt.headless.tests(instrument_parse_dispatch_tests.cpp, 10 cases): test fixtures were json-encoded butparse_trade_instrument()decodes via the process-widedefault_wire_codec(), which defaults tomsgpackabsent a real NATS client. Pinned the test binary's default codec to json.ores.iam.core.tests(mint_token_respects_the_requested_ttl=/impersonation tests): a real race, not flakiness -- =mint_token()stamped the token'sissued_at=/=expires_atbefore a DB round-trip (get_effective_permissions()), so a short ttl could expire before the token was even returned under load. Moved ttl stamping to immediately before signing; verified with 5 consecutive stress-testctestruns, all green.
- Final state: full local build clean,
validate_schemas.shgreen, fullctestsuite 74/74 passed.
Acceptance criteria (see * Acceptance) are all met. Deferred, by
explicit user instruction or genuine out-of-scope: a full
regen/filename-migration for ores_compute_app_version_platforms_tbl
(follow-up noted in validation_ignore.txt); a same-treatment audit
of counterparty against what was found for party (raised by the
user mid-session, not picked up this task – worth a fast follow-up
given the structural similarity).