Task: Complete the market series asset class junction

Table of Contents

This page documents a task in the Complete the market series classification story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Give ores_marketdata_market_series_asset_classes_tbl the two things every other tenant-scoped table in the tree has: row level security and a drop script. The junction must not be readable across tenants, and it must be removable in the same order as its siblings.

Two defects are in scope, and they share a cause in the junction codegen path rather than in this one table.

  • The table carries no row level security. Its sibling data tables in ores.marketdata each enable it inline, and refdata applies it to its own catalogues from a central policies file. The junction has neither, and a junction holds the same tenant-scoped data as the tables it joins.
  • The table has no drop script. marketdata_drop.sql wires every entity drop pair and omits the junction, while refdata hand-writes a drop file for currency_currency_groups. The two junctions are the same shape of object and get different treatment.

2. Status

Field Value
State DONE
Parent story Complete the market series classification
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-09-20

3. Acceptance

  • The junction template honours :rls_tenant_isolation: the way the domain entity templates do, so a junction model opts in by flag rather than by hand-written SQL.
  • The market series asset class junction enables row level security with the tenant isolation policy its sibling tables carry, applied to the live database.
  • Every junction in the tree that holds tenant-scoped data is brought in step, and the models that gain a policy are named in the result.
  • marketdata_drop.sql drops the junction in reverse dependency order, and a drop file exists for it.
  • Both suites that touch the table are re-run green, and a query from another tenant returns no junction rows.

4. Plan

Two defects, one codegen cause. Fix the template, then the two consumers.

  1. Teach the junction create archetype the flag. Add the tenant-isolation trailer to ores.sql.schema.junction_create.org, gated on {{#junction.sql.rls_tenant_isolation}}, mirroring the block in ores.sql.schema.domain_entity_create.org: alter table ... enable row level security plus the permissive ..._tbl_tenant_isolation_policy. Re-tangle the mustache with compass build --direct tangle_codegen_templates; never edit the tangled file by hand.
  2. Opt the one model in. The market series asset class junction gains :rls_tenant_isolation: true in its * SQL / ** Flags drawer. The org loader lifts that drawer into junction.sql unchanged, so no Python change is needed. The iam account party junction already carries its policy from the central iam_rls_policies_create.sql, so it does not opt in and no policy is duplicated.
  3. The drop script. There is no junction drop archetype: the template library holds junction_create only, and all 14 existing junction drop files are hand-written with no AUTO-GENERATED banner. So hand-write marketdata_market_series_asset_class_drop.sql in the shape the other junctions use, plus the policy drop the new trailer creates, and wire it into marketdata_drop.sql.
  4. Regenerate the marketdata component and confirm the only changed artefact is that junction's create file.
  5. Apply to the live database, verify a cross-tenant query returns no junction rows, then re-run the suites that touch the table and the schema validator.

5. Notes

Found during the review round of register-the-unmapped-series-types, which added the table. That task fixed the same defect on the two tables it owns, ores_ore_series_key_shapes_tbl and ores_marketdata_series_classification_rules_tbl, by opting in with :rls_tenant_isolation: true and regenerating.

The junction could not be fixed the same way, and that is why it is a task rather than part of that review round. sql_schema_junction_create.mustache and its drop twin never read the flag, so no junction in the repository is tenant isolated whatever its model says. Adding the flag would be a no-op, and the fix is a template change that reaches every junction at once.

The absent drop file is the same shape of gap. marketdata_drop.sql lists the five entity drop pairs and the two classification rules drops; the junction appears in none of them. refdata treats its junction as a first-class object with a hand-written drop file holding drop rule, drop trigger, drop function and drop table, in that order.

The table belongs to task 209's Unit 4, so this task inherits its consumers. Nothing in this task changes the classifier or the import path.

Reconnaissance on 2026-09-20 corrects two assumptions above. The note calling sql_schema_junction_create.mustache "and its drop twin" is wrong: no junction drop archetype exists at all, which is why every junction drop script in the tree is hand-written. And the flag is read nowhere in the junction path — the template has zero references to rls_tenant_isolation, so the opt-in has to be taught rather than merely exercised. The plan below reflects both.

6. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario, run through the QA Validation Runner panel) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

7. PRs

PR Title
#2112 [codegen,sql] Honour tenant isolation in the junction template

8. Review

# Comment summary File Decision Notes
1 The policy name is composed in the template with no identifier-length guard junction_create.org Deferred The domain path derives rls_table_base in Python and truncates it so the policy name stays inside PostgreSQL's 63-byte identifier limit. The junction loader has no rls_table_base, so this template composes {{junction.name}}_tbl_tenant_isolation_policy in mustache instead. The longest junction today is currency_pair_convention_calendars at 62 bytes, one under the limit, and it does not opt in, so nothing can truncate today. A loader-side base is needed before a longer junction opts in. Captured to the inbox.
2 misspell-fixer . exits 1 three org records outside this change set Declined All hits are the upstream identifier LOAD_PROFILE_1_EXPLICT, quoted verbatim in a sibling task's review table because the portfolio references that name and correcting it would make the record false. No file in this change set is among them, and two earlier review tables reached the same decision.
3 The drop file is named for the singular create basename, while refdata's junction drops use the plural marketdata_market_series_asset_class_drop.sql Accepted The name pairs the drop with its create file's stem, which is what iam's junction drop already does. Refdata's junction drops use the plural, so the tree is not uniform either way and neither choice is wrong.
4 The junction template honours :rls_system_tenant_visible: although no junction sets it junction_create.org Accepted The :rls_*: features belong to the ores.sql.schema bundle, which junctions share through their own * SQL / ** Flags drawer, and the trailer is a verbatim copy of the domain one. Supporting the flag keeps the two templates from diverging on a metatype boundary; the marketdata feed bindings are the live consumer of that widening.
5 The drop chain is exercised only by hand, never by CI marketdata_drop.sql Declined Nothing in the pipeline runs drop/drop.sql; the validator checks reachability instead. This change exercised the chain by running it inside a rolled-back transaction and asserting the tables were gone before the rollback. Making the drop path a gate is a wider change than this task.
6 The policy-name length guard is asymmetric with the domain entity path junction_create.org Deferred The external review reached this independently and agreed it need not block: the longest junction today, currency_pair_convention_calendars, is 62 bytes against the 63-byte limit and does not opt in. It is filed in this same PR as doc/agile/product_backlog/inbox/junction-rls-policy-name-has-no-length-guard.org, so it is not lost.
7 :rls_party_isolation: is a silent no-op on a junction junction_create.org Accepted The junction template carries no party-isolation block, and validate_rls_isolation runs only on the domain entity path, so setting the flag on a junction does nothing instead of raising. Fixed by stating in the archetype that row level security here is opt-in and tenant-scoped only. The note sits outside the tangled block, so no generated SQL changes.

9. Result

Done. Two defects, one codegen cause, and both consumers are fixed.

Template. ores.sql.schema.junction_create.org now emits the tenant isolation trailer, gated on {{#junction.sql.rls_tenant_isolation}} and mirroring ores.sql.schema.domain_entity_create.org: alter table ... enable row level security plus the permissive ..._tbl_tenant_isolation_policy, with the system-tenant widening the domain template also carries. The mustache was re-tangled with compass build --direct tangle_codegen_templates; source and mustache each moved 21 lines and nothing else in the template library changed.

Model. ores.marketdata.market_series_asset_class_junction.org gains :rls_tenant_isolation: true in its * SQL / ** Flags drawer. It is the only junction that gains a policy. Just two junction models are tenant-scoped at all: this one, and ores.iam.account_party_junction, which already carries its policy in iam_rls_policies_create.sql. That one does not opt in, so no policy is duplicated.

Drop. No junction drop archetype exists, so marketdata_market_series_asset_class_drop.sql is hand-written in the shape of its sibling marketdata_market_series_drop.sql: policy, rule, trigger, function, table. It is named for the create file's basename, the shape iam's junction drop already uses; refdata's junction drops use the plural instead, so the tree is not uniform. It is wired into marketdata_drop.sql ahead of market_series, which the junction references, and the comment above that list now names it.

Evidence.

  • Regenerating marketdata changed exactly one artefact: the junction's create file, plus the 14-line trailer. No other file in the component drifted.
  • check_component_drift.py --all reported no drift, with regenerated output matching the checked-in tree; check_model_drift.py found no model drift; check_populate_references.py resolved every reference.
  • The schema validator reported 0 warnings. The two this table raised on main, DROP_001 and RLS_001, are both gone.
  • compass db recreate -y -k finished with exit 0, so the whole create-then-populate path accepts the table. The FOMC populate still writes its two system-tenant junction rows under the new with check.
  • On the live database relrowsecurity is true and market_series_asset_classes_tbl_tenant_isolation_policy exists. As ores_swift_curie_marketdata_service, a session claiming the system tenant sees both rows; one claiming a foreign tenant sees none; one claiming no tenant sees none; and an update moving a row to another tenant fails with new row violates row-level security policy. The blocked write left the table unchanged.
  • The drop chain was executed, not just read. Running marketdata_drop.sql inside a transaction dropped all seven marketdata tables, junction included, and the rollback restored them.
  • ctest -R "marketdata|database|platform" passed 7 of 7, covering ores.database.tests and all four marketdata suites.
  • The full preset suite passed 71 of 72. The single failure, ores.refdata.core.tests, was terminated mid-test with its XML truncated inside write_book_publishes_nats_changed_event, which is the eventing integration flake family sprint 25 already tracks rather than an assertion failure; re-run on its own it passed in 262s. The run took 3506s against 1634s for the previous full run, on a host whose load stayed above 10.

Scoping. The three other refdata junctions with a tenant_id column, currency_currency_groups, party_counterparties, party_countries and party_currencies, are stale relative to their models: their #+profile: binding sits in front matter, which the junction loader cannot read. They keep their central RLS policies either way, and the gap stays filed at doc/agile/product_backlog/inbox/refdata-junction-profile-binding-ignored.org. Nothing in this task changes them.

Nothing is outstanding on this task. The branch has no CI run yet.

Emacs 29.3 (Org mode 9.6.15)