Task: Export a trade set as an archive

Table of Contents

This page documents a task in the Gather each trade set and its market data — the remaining work story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

trading.v1.ops.export_trades_to_storage takes book ids and no date, pages trade_export_service::export_books, and stores one blob for all of them: the export_portfolio_response, holding the trades and their instrument batch. Carry the export's instruments as typed batches removed the opaque instrument_payload, so the batch is typed and the ORE mapper can be driven from it — which is how a run maps its portfolio today.

What the export cannot do is answer for one set. It names books and no as-of, so one object answers for every book in the report and for whatever the database holds now.

Export one trade set at a time, by its batch key, and source everything the portfolio needs — trade headers, instruments and envelopes — into a zip whose shape is declared. The as-of travels on the set, so the request does not restate it.

2. Status

Field Value
State BACKLOG
Parent story Gather each trade set and its market data — the remaining work
Now Not yet started.
Waiting on Partition a run's trade population into trade sets, which produces the batch keys this reads.
Next Begin implementation.
Last touched 2026-10-07

3. Acceptance

  • The request names a trade set's batch key, and the as-of comes from the set.
  • The archive holds everything the ORE portfolio mapper needs: each trade's anchor, its instrument typed and keyed by the trade id, and its envelope (counterparty, netting set, portfolios, additional fields).
  • The archive's shape is declared — the set of members, their names, their encodings, and a manifest to check them against — so a reader validates before it maps.
  • The archive is named by the batch key, so the package step can tell which sets have arrived and which are missing.

4. Plan

One archive per trade set, with members split by instrument type. The trade set is the unit the run fans out over and the unit the package counts, so the archive is per set. Inside it, a manifest plus one member per instrument type (and one for the headers and envelopes) is the shape that lets the mapper stream type by type, lets a new instrument family arrive without touching the others, and localises a mapping failure to a type rather than to a set.

The alternative — a separate archive per set per instrument type — multiplies the object count by the number of types and makes the fan-in set's size vary with each set's contents, which is the thing the waiting logic least wants. This is a decision to confirm when the shape is drawn: it is recorded here because it is the question, not because it is closed.

Where the as-of comes from. It is on the trade set, and every market and trade table is bitemporal, so reading it is a valid_at read rather than new machinery. Nothing in this path needs a date parameter.

What the encoding is. The members stay in an encoding this tree can read without a new parser; the point of the change is the shape and the completeness of the data, not the container's format. The manifest is what makes the shape declared rather than inferred.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
       

6. Notes

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
   

9. Review

Comment summary File Decision Notes
       

10. Result

Emacs 29.3 (Org mode 9.6.15)