Task: Resolve a trade set's market data requirements

Table of Contents

This page documents a task in the Gather each trade set and its market data — the remaining work story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

gather_market_data exports every series the tenant holds, at whatever the database contains at the moment it runs. Nothing can say which market data a set of trades actually needs, so nothing can narrow the export, and nothing ties the data to the run's date.

Resolve it, for one trade set: from its trades to the market data requirements they imply, expressed as oresmd, and from those requirements to the series, observations and fixings that answer them. The as-of is the set's.

2. Status

Field Value
State BACKLOG
Parent story Gather each trade set and its market data — the remaining work
Now Not yet started.
Waiting on Partition a run's trade population into trade sets, which produces the batch keys this reads.
Next Begin implementation.
Last touched 2026-10-07

3. Acceptance

  • The resolution answers, for one trade set, which oresmd data a run requires.
  • A requirement that no series answers is reported as a gap, not skipped.
  • The resolution reads at the set's as-of date, so two runs that share a set's membership but not its date resolve against different rows.
  • Where the resolution lives is decided and recorded, with the reason.

4. Plan

The crux is where the requirement grammar lives. A trade names its instruments and those instruments name indices, currencies and tenors; the definition's curve configurations name the quotes a curve needs. Both halves already exist as rows. But an oresmd requirement is spelled by C++ codecs — oresmd_uri_codec and ore_key_codec — so "all in the database" means either restating that grammar in SQL or moving it into SQL and having C++ read it. That is the decision this task exists to take, and the two honest shapes are:

  • a SQL function that resolves a trade set to series and returns the rows, with the oresmd spelling left to the caller that already knows the grammar; or
  • the grammar expressed once in SQL and consumed from both sides.

A split — SQL for set to trades to instrument attributes, C++ for the requirement spelling — is the third shape and probably the first one built, because it needs no grammar to move and still removes the per-trade round trip. It should be recorded as a starting point rather than a resting place if the grammar then has two homes.

What the requirements are derived from is the other half. The candidates are the trades' instruments and the curve configurations the run binds; both are needed, because a portfolio's instruments need curves and the curves need quotes. The investigation should state which entity answers an oresmd requirement today, because if none does, that entity is part of this task.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
       

6. Notes

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
   

9. Review

Comment summary File Decision Notes
       

10. Result

Emacs 29.3 (Org mode 9.6.15)