Task: Complete the market series asset class junction
Table of Contents
This page documents a task in the Complete the market series classification story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
Give ores_marketdata_market_series_asset_classes_tbl the two things
every other tenant-scoped table in the tree has: row level security and
a drop script. The junction must not be readable across tenants, and it
must be removable in the same order as its siblings.
Two defects are in scope, and they share a cause in the junction codegen path rather than in this one table.
- The table carries no row level security. Its sibling data tables in
ores.marketdataeach enable it inline, andrefdataapplies it to its own catalogues from a central policies file. The junction has neither, and a junction holds the same tenant-scoped data as the tables it joins. - The table has no drop script.
marketdata_drop.sqlwires every entity drop pair and omits the junction, whilerefdatahand-writes a drop file forcurrency_currency_groups. The two junctions are the same shape of object and get different treatment.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Complete the market series classification |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-09-20 |
3. Acceptance
- The junction template honours
:rls_tenant_isolation:the way the domain entity templates do, so a junction model opts in by flag rather than by hand-written SQL. - The market series asset class junction enables row level security with the tenant isolation policy its sibling tables carry, applied to the live database.
- Every junction in the tree that holds tenant-scoped data is brought in step, and the models that gain a policy are named in the result.
marketdata_drop.sqldrops the junction in reverse dependency order, and a drop file exists for it.- Both suites that touch the table are re-run green, and a query from another tenant returns no junction rows.
4. Plan
Two defects, one codegen cause. Fix the template, then the two consumers.
- Teach the junction create archetype the flag. Add the tenant-isolation
trailer to
ores.sql.schema.junction_create.org, gated on{{#junction.sql.rls_tenant_isolation}}, mirroring the block inores.sql.schema.domain_entity_create.org:alter table ... enable row level securityplus the permissive..._tbl_tenant_isolation_policy. Re-tangle the mustache withcompass build --direct tangle_codegen_templates; never edit the tangled file by hand. - Opt the one model in. The market series asset class junction gains
:rls_tenant_isolation: truein its* SQL / ** Flagsdrawer. The org loader lifts that drawer intojunction.sqlunchanged, so no Python change is needed. The iam account party junction already carries its policy from the centraliam_rls_policies_create.sql, so it does not opt in and no policy is duplicated. - The drop script. There is no junction drop archetype: the template
library holds
junction_createonly, and all 14 existing junction drop files are hand-written with noAUTO-GENERATEDbanner. So hand-writemarketdata_market_series_asset_class_drop.sqlin the shape the other junctions use, plus the policy drop the new trailer creates, and wire it intomarketdata_drop.sql. - Regenerate the marketdata component and confirm the only changed artefact is that junction's create file.
- Apply to the live database, verify a cross-tenant query returns no junction rows, then re-run the suites that touch the table and the schema validator.
5. Notes
Found during the review round of register-the-unmapped-series-types,
which added the table. That task fixed the same defect on the two
tables it owns, ores_ore_series_key_shapes_tbl and
ores_marketdata_series_classification_rules_tbl, by opting in with
:rls_tenant_isolation: true and regenerating.
The junction could not be fixed the same way, and that is why it is a
task rather than part of that review round.
sql_schema_junction_create.mustache and its drop twin never read the
flag, so no junction in the repository is tenant isolated whatever its
model says. Adding the flag would be a no-op, and the fix is a
template change that reaches every junction at once.
The absent drop file is the same shape of gap. marketdata_drop.sql
lists the five entity drop pairs and the two classification rules
drops; the junction appears in none of them. refdata treats its
junction as a first-class object with a hand-written drop file holding
drop rule, drop trigger, drop function and drop table, in that
order.
The table belongs to task 209's Unit 4, so this task inherits its consumers. Nothing in this task changes the classifier or the import path.
Reconnaissance on 2026-09-20 corrects two assumptions above. The note
calling sql_schema_junction_create.mustache "and its drop twin" is
wrong: no junction drop archetype exists at all, which is why every
junction drop script in the tree is hand-written. And the flag is read
nowhere in the junction path — the template has zero references to
rls_tenant_isolation, so the opt-in has to be taught rather than
merely exercised. The plan below reflects both.
6. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario, run
through the QA Validation Runner panel) that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
7. PRs
| PR | Title |
|---|---|
| #2112 | [codegen,sql] Honour tenant isolation in the junction template |
8. Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | The policy name is composed in the template with no identifier-length guard | junction_create.org | Deferred | The domain path derives rls_table_base in Python and truncates it so the policy name stays inside PostgreSQL's 63-byte identifier limit. The junction loader has no rls_table_base, so this template composes {{junction.name}}_tbl_tenant_isolation_policy in mustache instead. The longest junction today is currency_pair_convention_calendars at 62 bytes, one under the limit, and it does not opt in, so nothing can truncate today. A loader-side base is needed before a longer junction opts in. Captured to the inbox. |
| 2 | misspell-fixer . exits 1 |
three org records outside this change set |
Declined | All hits are the upstream identifier LOAD_PROFILE_1_EXPLICT, quoted verbatim in a sibling task's review table because the portfolio references that name and correcting it would make the record false. No file in this change set is among them, and two earlier review tables reached the same decision. |
| 3 | The drop file is named for the singular create basename, while refdata's junction drops use the plural | marketdata_market_series_asset_class_drop.sql | Accepted | The name pairs the drop with its create file's stem, which is what iam's junction drop already does. Refdata's junction drops use the plural, so the tree is not uniform either way and neither choice is wrong. |
| 4 | The junction template honours :rls_system_tenant_visible: although no junction sets it |
junction_create.org | Accepted | The :rls_*: features belong to the ores.sql.schema bundle, which junctions share through their own * SQL / ** Flags drawer, and the trailer is a verbatim copy of the domain one. Supporting the flag keeps the two templates from diverging on a metatype boundary; the marketdata feed bindings are the live consumer of that widening. |
| 5 | The drop chain is exercised only by hand, never by CI | marketdata_drop.sql | Declined | Nothing in the pipeline runs drop/drop.sql; the validator checks reachability instead. This change exercised the chain by running it inside a rolled-back transaction and asserting the tables were gone before the rollback. Making the drop path a gate is a wider change than this task. |
| 6 | The policy-name length guard is asymmetric with the domain entity path | junction_create.org | Deferred | The external review reached this independently and agreed it need not block: the longest junction today, currency_pair_convention_calendars, is 62 bytes against the 63-byte limit and does not opt in. It is filed in this same PR as doc/agile/product_backlog/inbox/junction-rls-policy-name-has-no-length-guard.org, so it is not lost. |
| 7 | :rls_party_isolation: is a silent no-op on a junction |
junction_create.org | Accepted | The junction template carries no party-isolation block, and validate_rls_isolation runs only on the domain entity path, so setting the flag on a junction does nothing instead of raising. Fixed by stating in the archetype that row level security here is opt-in and tenant-scoped only. The note sits outside the tangled block, so no generated SQL changes. |
9. Result
Done. Two defects, one codegen cause, and both consumers are fixed.
Template. ores.sql.schema.junction_create.org now emits the tenant
isolation trailer, gated on {{#junction.sql.rls_tenant_isolation}} and
mirroring ores.sql.schema.domain_entity_create.org: alter table ...
enable row level security plus the permissive
..._tbl_tenant_isolation_policy, with the system-tenant widening the
domain template also carries. The mustache was re-tangled with compass
build --direct tangle_codegen_templates; source and mustache each moved
21 lines and nothing else in the template library changed.
Model. ores.marketdata.market_series_asset_class_junction.org gains
:rls_tenant_isolation: true in its * SQL / ** Flags drawer. It is the
only junction that gains a policy. Just two junction models are
tenant-scoped at all: this one, and ores.iam.account_party_junction,
which already carries its policy in iam_rls_policies_create.sql. That
one does not opt in, so no policy is duplicated.
Drop. No junction drop archetype exists, so
marketdata_market_series_asset_class_drop.sql is hand-written in the
shape of its sibling marketdata_market_series_drop.sql: policy, rule,
trigger, function, table. It is named for the create file's basename, the
shape iam's junction drop already uses; refdata's junction drops use the
plural instead, so the tree is not uniform. It is wired into
marketdata_drop.sql ahead of market_series, which the junction
references, and the comment above that list now names it.
Evidence.
- Regenerating marketdata changed exactly one artefact: the junction's create file, plus the 14-line trailer. No other file in the component drifted.
check_component_drift.py --allreported no drift, with regenerated output matching the checked-in tree;check_model_drift.pyfound no model drift;check_populate_references.pyresolved every reference.- The schema validator reported 0 warnings. The two this table raised on
main,
DROP_001andRLS_001, are both gone. compass db recreate -y -kfinished with exit 0, so the whole create-then-populate path accepts the table. The FOMC populate still writes its two system-tenant junction rows under the newwith check.- On the live database
relrowsecurityis true andmarket_series_asset_classes_tbl_tenant_isolation_policyexists. Asores_swift_curie_marketdata_service, a session claiming the system tenant sees both rows; one claiming a foreign tenant sees none; one claiming no tenant sees none; and an update moving a row to another tenant fails withnew row violates row-level security policy. The blocked write left the table unchanged. - The drop chain was executed, not just read. Running
marketdata_drop.sqlinside a transaction dropped all seven marketdata tables, junction included, and the rollback restored them. ctest -R "marketdata|database|platform"passed 7 of 7, coveringores.database.testsand all four marketdata suites.- The full preset suite passed 71 of 72. The single failure,
ores.refdata.core.tests, was terminated mid-test with its XML truncated insidewrite_book_publishes_nats_changed_event, which is the eventing integration flake family sprint 25 already tracks rather than an assertion failure; re-run on its own it passed in 262s. The run took 3506s against 1634s for the previous full run, on a host whose load stayed above 10.
Scoping. The three other refdata junctions with a tenant_id column,
currency_currency_groups, party_counterparties, party_countries and
party_currencies, are stale relative to their models: their
#+profile: binding sits in front matter, which the junction loader
cannot read. They keep their central RLS policies either way, and the gap
stays filed at
doc/agile/product_backlog/inbox/refdata-junction-profile-binding-ignored.org.
Nothing in this task changes them.
Nothing is outstanding on this task. The branch has no CI run yet.