Task: Turn the FX soft references into foreign keys

Table of Contents

This page documents a task in the Rework the FX instruments to the instrument table rule story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Give the FX family the keys its models promise. The seven tables declare no foreign key, and twenty columns are labelled as a soft reference to a named table, each deferring the real key to "PR 4". The identity-by-key check asks for a key, not a text code that only a non-empty check defends.

2. Status

Field Value
State DONE
Parent story Rework the FX instruments to the instrument table rule
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-18

3. Acceptance

  • Every one of the twenty soft references is either a real foreign key or carries a recorded reason that it stays soft.
  • A reference that becomes a key keeps its existing rows: the unit states how the existing codes map to keys, and no row is lost.
  • The closed vocabularies keep one home. A column that becomes a key to a type table must not also restate the closed set in its own check.
  • The trade anchor keeps its trigger-enforced key, or gains a declared one.

4. Plan

  1. List the twenty references with their target table and the code values in the corpus.
  2. Split them: the closed trading vocabularies become keys; a currency code becomes a key to ores_refdata_currencies_tbl; an FX index code has no target table yet and stays soft with a reason.
  3. Add the keys in the models, regenerate, and recreate the database.
  4. Import the corpus again and compare the stored rows with the document.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
Columns tightened into trigger-checked keys 14 =grep -oE "No active [a-z ]+ found with this code" projects/ores.sql/create/trading/trading_fx_*_instruments_create.sql \ grep -v "trade activity" \ wc -l=, in evidence/fx-keys.txt the PR head
Columns that stay soft with a recorded reason 21 the fourteen currency and seven trade-type references, in the models' prose the PR head
SQL files that gained a validation 7 git diff --stat origin/main -- 'projects/ores.sql/create/trading/*' the PR head
Trading core suite pass compass test run --preset linux-clang-release-ninja -- -R ores.trading.core.tests, summarised in evidence/trading-core-suite.txt the PR head
ORE core suite pass the same command with ores.ore.core.tests, summarised in evidence/ore-core-suite.txt the PR head
       

6. Notes

  • The evidence for this task is Investigation: FX family data model review, finding 5. Measured there: 0 declared foreign keys and 20 soft-reference claims across the seven models.
  • fx_index and underlying_code name an FX index that no ores.refdata entity holds. If that entity is not scaffolded in this unit, record the gap rather than inventing a key.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#3524 [agile] Close the soft references unit
#3523 [trading] Key the FX closed vocabularies to their lookup tables

9. Review

# Comment summary File Decision Notes
1 The measurement's command returns 17, because it also counts the pre-existing trade-activity message the task's Measurements Fixed The command filters the trade-activity line and returns 14, the number of tightened columns.
2 Four references were missed: settlement on the forward, vanilla and barrier options, and long_short on the Asian forward the models Fixed All four are trigger-checked keys now, with the lookup that holds their codes and a nil guard where the column is nullable.
3 The trade-type reason is false for four of the seven tables, whose check lists two to five codes the models' prose Fixed The reason now says the check lists exactly the codes the catalogue routes to the table.
4 The currency reason cites a validation the FX path never performs the models' prose Fixed The reason states the measured fact instead: a recreated database holds no EUR or USD in any tenant, and the FX rows state them.
5 Only three of the fourteen currency columns carried the reason the models' prose Fixed All fourteen carry it.
6 The barrier-type prose calls the set a closed ORE enumeration, but ORE states the leaf as free text the barrier and digital models Fixed The prose now says the lookup is the set this store accepts, and that every barrier value in the repository falls inside it.
7 The underlying-code gap the Notes require is not recorded the task's Result Fixed The Result names underlying_code and fx_index as ORE names no refdata entity holds.
8 The generated trigger comment still says "soft FK to …" for a column that is now keyed the generated SQL Rejected The comment is emitted by a shared template, so changing it rewrites every component's create SQL for a word. The model and the header state the key.
9 The ER diagram draws a nullable key as mandatory the trading diagram Rejected The cardinality comes from the diagram generator, which reads the key's presence, not the column's nullability. Changing it redraws every schema.

10. Result

Fourteen references became real keys and twenty-one stay soft with a recorded reason.

The fourteen are the family's closed vocabularies: long_short on the accumulator, the Asian forward, the digital option and the variance swap; option_type on the barrier, digital and vanilla options; barrier_type on the barrier and digital options; exercise_style on the vanilla option; moment_type on the variance swap; and settlement on the forward, vanilla and barrier options, whose lookup holds the two methods the documents state. Each now carries a foreign key block to its lookup table, so the generated insert trigger refuses a code the lookup does not hold. The nullable columns validate only when they state a value.

Twenty-one references stay soft, in two groups:

  • The currency columns, fourteen of them. A currency is ores.refdata's row, published from the data-quality pipeline per environment rather than by this component, and a trading database may hold only the codes its own imports loaded. A freshly recreated database holds no EUR and no USD in any tenant, while the FX rows state them, so a trigger check would refuse the documents the store is meant to accept. Each of the fourteen columns states that reason.
  • The seven trade-type codes. The column's check lists exactly the codes the trade-type catalogue routes to that table, and the catalogue is what decides the routing, so a foreign key would duplicate the router.

The other text columns name no table: underlying_code and fx_index name an ORE Underlying/Name that no ores.refdata entity holds, and description is free text.

11. Decisions

  • A key is the trigger check, not a SQL constraint. The tables are temporal, so a REFERENCES constraint would accept a closed version as current. The generated check reads the current row, which is what the rest of the family already does for its trade and activity keys.
  • A currency reference stays soft because its rows are not this component's. The unit's acceptance allows a recorded reason, and the alternative would be a foreign key that fails in every database whose currency set is loaded after the documents.
  • The trade-type code stays soft because the check is narrower. It lists the codes the routing sends to the table, where a key would accept any catalogue code. The trade envelope keys its own trade_type to the same lookup without a check, so the two tables state the same set differently; the discriminators unit decides which one wins.
  • The lookup's closed set is now stated twice. The SQL check and the lookup table both hold it. Removing the checks is the discriminators unit's decision, not this one's, so they stay.
  • A barrier type is not a closed ORE enumeration. ORE states the leaf as free text, so the lookup is the set this store accepts, not a schema set. Every barrier value in the repository falls inside it.

Emacs 29.3 (Org mode 9.6.15)