Story: Rework the rates instruments to the instrument table rule

Table of Contents

This page documents a story in Sprint 27, promoted from the product backlog's next bucket where it was scaffolded on 2026-10-02. It captures the goal, current status, acceptance criteria, and the tasks that compose it.

1. Goal

Rework swaps, FRAs, caps and floors, swaptions and the other rates trade types to the instrument table rule of the trade shape investigation (section 14), as the bonds were reworked, with the data-oriented review deciding the tables.

2. Status

Field Value
State DONE
Parent sprint Sprint 27
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-11

3. Acceptance

  • The family header's trade_type check lists exactly the family's codes, and each product fact table checks its own.
  • Every instrument row is keyed by its trade and pinned to the anchor's trade type and party by a composite foreign key.
  • The family's trade types route to their tables through the trade-type catalogue, not hand-written code.
  • The ORE round trip stays lossless across the family's corpus.
  • The blotter's common-date view is generated, mapping each family's dates to common names; this first family creates it and each later family adds to it.

4. Tasks

Task State Start End Description
Analyse the rates family and propose its relational data model DONE 2026-10-08 2026-10-08 Rates is the first family after the bond pilot for the relational target of the trade shape investigation, section 14. Survey every rates-related type and subtype in the tree (modeling orgs, SQL, domain C++, mappers, repositories, services, messaging, the ores.ore mapping), then propose the entity-relationship model based on the existing tables but in the clean data-oriented shape: a family header holding exactly its codes, product fact tables, shared children with role codes, and securities keyed by their reference datum. Deliverable is the analysis and ER proposal only; codegen of the types and SQL and the ore mapping update are follow-up tasks.
Repair the rates trade-type routing DONE 2026-10-08 2026-10-08 The generated routing, the ORE importer and the instrument tables disagree about three things. RiskParticipationAgreement routes to rpa_instruments while the importer writes credit_instruments, whose check refuses the code; RPA is classified swap in the trade-type seed while the domain note, the refdata instrument codes and the importer all call it credit; and three models generate trade type codes the catalogue and the seed do not hold (FRA, EuropeanSwaption, ZeroCouponInflationSwap). Repair all three so the routing is the single truth, and record the unproduced rpa_instruments table with its disposition.
Write the swap legs on import and record their direction DONE 2026-10-08 2026-10-08 The rates legs never reach the database. The ORE importer builds them and stamp_ids stamps them, and then the import drops them, so a stored swap exports with no legs; the table cannot even record a leg's direction, because legData/Payer is required by ORE and has no column. Add the payer column, read the direction in the mapper both ways, and write the legs the importer already builds.
Move the swap leg's notionals and rates into child rows DONE 2026-10-08 2026-10-08 A swap leg stores one notional, one fixed rate and one spread, so a leg whose notional amortises or whose rate steps is flattened at the boundary: map_leg reads first_notional, Rates.Rate.front() and Spreads.Spread.front() and drops the rest, and the amortisation list has nowhere to go at all. Move the leg's economics into keyed child rows, as the bond family's leg amounts are held, so the importing mapper can keep every step it reads. Scoped: the children also have to cross the wire, because the export rebuilds a leg from the payload and nothing else, so this unit carries the carrier and the hand-written codec with it.
Repair the swap leg children's insert trigger and isolation DONE 2026-10-08 2026-10-08 The swap leg's amount and rate tables bound the trading-instrument profile, whose insert trigger stamps party_id, but were flattened without that column, so every insert failed at run time. Both now declare the tenant feature their bond siblings declare, carry tenant isolation, and their drop files are wired.
Re-key the swap legs to the trade DONE 2026-10-09 2026-10-09 swap_legs kept a surrogate id that nothing supplied meaningfully: the trade and the leg ordinal already identify the row. It is keyed by (trade_id, leg_number) now, and the compound-key codegen defect that blocked the re-key is fixed.
Carry the cross-currency swap's own type DONE 2026-10-09 2026-10-09 The ORE importer wrote trade_type_code Swap for a cross-currency swap as well as for a plain swap, and the reverse always emitted SwapData, so the two products were one after a round trip. Both directions keep the type the document states.
Move the rates family onto its header DONE 2026-10-09 2026-10-09 Author ores.trading.rate_instruments: the family's coverage check, the shared dates and description, the trade-activity key, and the delete cascade that owns the family's rows. Convert the eight product tables from instrument entities into fact tables keyed by the trade, route the family's codes to the header through the catalogue, and wire the import and the export.
Move the rates family's shared children onto the bond shapes DONE 2026-10-09 2026-10-10 Move the cap/floor strikes and schedules and the swaption option block onto the shapes the bond pilot built, into the header's delete cascade, and move their registration to the root that owns them. Retype the option block's text dates and collections. (Staging unit 4; findings R5, R11.)
Pin the rates instruments to the anchor's key DONE   2026-10-10 Add the trade anchor's unique index and pin every rates instrument row to it with the composite foreign key on the party and the trade type. The one unit that changes the trade model. (Staging unit 5; finding R7.)
Close the rates round trip DONE   2026-10-10 Add the missing reverse mappers for InflationSwap, BalanceGuaranteedSwap and FlexiSwap, and decide the hand-written rates carrier's and codec's disposition: replace them with the generated per-entity protocols, or record why the carrier must stay hand-written. (Staging unit 6; findings R10, R13.)
Generate the blotter's common-date view DONE   2026-10-10 Finding R9: each product states its dates under its own name - start_date, maturity_date, expiry_date, end_date - so a deal grid cannot read one date column across the family. The view maps each product's dates to one set of names, and it is generated because the mapping is a property of the models. This first family creates the view; each later family adds its mapping. (Staging unit 7; finding R9.)
Prove the rates round trip over the corpus DONE 2026-10-10 2026-10-11 Run the ORE round trip over the family's corpus at the new shape, and add the repository, service and eventing coverage the family has none of. (Staging unit 8; findings R9, R14.)
Carry the FlexiSwap and BalanceGuaranteedSwap fields DONE 2026-10-10 2026-10-10 Add the fields the model lacks for these two products, so the export writes the elements the ORE schema requires.
Hold the FRA notional and strike as the document's digits DONE 2026-10-11 2026-10-11 The ORE binding declares the FRA notional and strike as bare floats, so the mapper stores a float artifact for a value the document wrote exactly. Keep the digits through import and export. (Follow-up of the rates corpus proof; same family as the leaf digits patch.)

5. Decisions

Distilled from design.org, which carries the argument and the evidence. The design stays in this folder as the record of why these decisions are what they are.

  • A money member of the ORE binding keeps the digits the document wrote. The FRA's strike and notional are a lexical leaf, patched in by the reapply script, and the mappers read and write the digits. Reason: a bare float holds only the nearest binary value, so 0.005 was stored and exported as 0.00499999989.
  • The round trip is proved by comparing stored rows over the corpus. The test imports each rates trade, exports it alone, imports it again and compares the rows the family stores, with a closed set of known differences that is empty. Reason: a document-level comparison hides a field the mapper drops, and the corpus is the only set of real products.
  • The family's eventing tests are generated and cover the service. Each model sets the eventing flag, and the fact tables set party_scoped_writes. Reason: the generated test writes, reads and updates a row through the service, so no hand-written repository case is needed.
  • A product's list fields are child rows and its schedule is a shared row. FlexiSwap and BalanceGuaranteedSwap carry their bounds, tranches and notionals as child tables keyed by the trade and an ordinal, and the tranche schedule is an instrument_schedule row under the owner role tranches. Reason: it is the shape callable_swap_call_date and the leg children already follow, and a second schedule table would repeat the shared one.
  • The anchor pin is a trigger check on the header alone. The trade anchor is versioned and has no unique key a database foreign key could name, so the header declares a Pinned keys row over (trade_id, party_id, trade_type_code). The fact and child tables hold neither column and join through the header (D3). The check runs at insert, so an amendment of the anchor does not re-check the header; the trade model's owner decides whether an amendment re-pins its children.
  • The family boundary is the asset class, not the catalogue's product_type. The rates family is the interest_rates codes: Swap, CrossCurrencySwap, FlexiSwap, ForwardRateAgreement, CapFloor, Swaption, BalanceGuaranteedSwap, CallableSwap and KnockOutSwap. RiskParticipationAgreement is a credit product and InflationSwap an inflation product; the refdata asset class catalogue and the two domain notes say so, and product_type is the misnamed column defect 10 records.
  • One family header, named rate_instruments. It carries the coverage check over the family's codes, the trade-activity key, the fields every product shares, and the * Delete cascade that owns the family's rows. It is a coverage and ownership device first: nothing owns the family today, so a delete leaves the legs and the children behind.
  • No security level. A rates trade has no issuer and no ISIN, so there is no master entity to normalise the economics onto.
  • In the target model the legs are re-keyed to the trade and given their schedules. (trade_id, leg_number) replaces the surrogate id as bond_legs is keyed, and amount and rate children carry the amortisation and step schedules the single columns cannot. payer has since joined the row; the re-key itself is deferred behind the compound-key template defect the capture below records.
  • netting_set_id leaves the instrument. trade_booking already holds it, as a uuid foreign key where the instrument's copy is bare text.
  • The ORE mapping repair is the family's largest unit, not a follow-up. The import never writes the legs it builds, overwrites a cross-currency swap's type to Swap, and leaves seven columns that exist permanently empty.
  • RPA is a credit product, and its catalogue row now says so. RiskParticipationAgreement is classified credit and routed to credit_instrument, which is the table the importer always wrote; the credit check gains the code and the trade-types seed carries the class. The old rpa_instruments table is unproduced and is recorded for retirement rather than removed here. Reason: the rpa.org domain note, the refdata instrument codes and the importer all called it credit already, and the routing was the only source that disagreed.
  • A leg records its direction, and the import writes the legs it builds. payer joins swap_leg and is mapped both ways, and the handler writes each leg after its instrument and fails the item when a leg fails. A stored swap is an instrument rather than a row.
  • The legs keep their surrogate key for now. Re-keying swap_legs to (trade_id, leg_number) fails to compile a generated test: the compound-key path inserts each extra key column's raw domain value where the generated service takes every key part as text. The fix is a template change that regenerates every component under test, so it belongs with the codegen work and the re-key is recorded as a capture. Reason: a family rework should not carry a tree-wide regeneration behind it.
  • A leg's notionals and rates are child rows, and they cross the wire. swap_leg_amount and swap_leg_rate hold the schedules; swap_leg no longer carries notional, fixed_rate or spread, because one column cannot hold a schedule. The mapper reads every entry of each ORE list instead of the first, and the carrier and its hand-written codec were extended in the same unit, because the export rebuilds a leg from the payload and nothing else: without that, removing the flat columns would have lost the leg's economics on the way out.
  • A common date is declared on the column, and the vocabulary is in the catalogue. The trade-type catalogue lists the four names the blotter reads. A column states which one it holds with :common_date: <name>, and the view for each routed header is generated from those declarations. A date no family owns, the trade date, names its entity and column in the catalogue instead. A later family adds its mapping on its models and edits no view.

6. Documents

Document What it is
design.org The design: the inventory, the findings and the proposed model. Transitional; expires when this story lands.
design_er.puml, design_er.png The proposed model's entity-relationship diagram.

7. Out of scope

  • The credit and inflation families' own reworks, which take rpa_instruments and inflation_swap_instruments once they leave.
  • The reference tables that turn the family's soft code references into real foreign keys.
  • The three unrouted codes' products, which need ORE importer support before a table can hold them.

8. Result

All fourteen tasks are merged or closed. The rates family sits on its header and fact tables, and the corpus round-trips with equal family rows.

Not closed. An earlier bookkeeping pass closed this story on its tasks' state alone, and that is how five of the design's eight staging units went unwritten: the tasks were done, so the story read as done. The five are tasks now, and this story closes when its acceptance does, not when its task list does.

The acceptance against the tree as unit 3 left it:

Acceptance State
The header's trade_type check lists exactly the family's codes met
Each product fact table checks its own not expressible: the catalogue states one routing per trade type, so the header carries the family's check and the facts carry none, as the bond pilot does
Every instrument row is keyed by its trade and pinned to the anchor's trade type and party by a composite foreign key met in a weaker form: the header is pinned by an insert-trigger check, because the anchor is versioned and no database key can name it; the fact tables carry neither column and join through the header
The family's trade types route through the catalogue, not hand-written code met
The ORE round trip stays lossless across the family's corpus met: all 47 rates documents (70 trades) of the corpus return the same family rows after an import, an export and a second import; Prove the rates round trip over the corpus holds the measurements
The blotter's common-date view is generated met: ores_trading_rate_instruments_common_dates_vw, generated from the :common_date: declarations

Emacs 29.3 (Org mode 9.6.15)