Task: Partition a run's trade population into trade sets
Table of Contents
This page documents a task in the Gather each trade set and its market data — the remaining work story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
Nothing in ores.trading groups trades. The run resolves its scope once,
inside gather_trades, and treats it as one bag of books: the handler asks the
risk report configuration for its books and hands the whole list to one export.
A report covers hundreds to thousands of books, and each needs its own trades
archive, its own market data and its own portfolio XML.
Give ores.trading a batcher: a subsystem that partitions a report's trade
population into trade sets, driven by a strategy. The sets are what the
rest of the run reads.
2. Status
| Field | Value |
|---|---|
| State | BACKLOG |
| Parent story | Gather each trade set and its market data — the remaining work |
| Now | The partitioning landed: the strategy interface, batch by book, the bounded-set strategy, composition, the batch key and the partition invariants. The recording half is not built. |
| Waiting on | The trade group and its membership, Add the trade group and its membership, for the acceptance bullet that records the sets against the run. |
| Next | Wire the batcher to the run once the group exists, and load a strategy's parameters from tables rather than from the caller. |
| Last touched | 2026-10-07 |
3. Acceptance
- A batcher partitions a report's trade population into trade sets, and a strategy decides how it partitions.
- "Batch by book" is the first strategy, and it is one implementation behind an interface rather than a special case in the caller.
- A trade set carries a name and an as-of date. The name is the batch key,
a plain string of the form
REPORT_NAME:BOOK_ID:ASOFfor the book strategy, and nothing downstream parses it beyond using it as the artifact key. - The batcher records the sets it created against the run, so the run knows its fan-out width before any producer starts.
- A later strategy can partition differently — by grid behaviour, by size — without a producer or the package step changing.
4. Requirements the strategy machinery must carry
Recorded here while the batching subsystem's own documentation is pending (it arrives with Merry Newton's trade-set work). These are requirements, not a design; the design lands with the subsystem's docs.
- A strategy is associated with a report. Which batching a report uses is the report's to state, not a global setting.
- Strategies compose. "Simple report batching" is
batch by bookandat most 1000 trades— a composite over two strategies, not a third strategy that reimplements both. The simplest strategy is still the only leaf at the start, but the composition must already be expressible. - A strategy's parameters are configured in tables, so a parameter is data and changing it is not a deploy. What a strategy reads must therefore be declared the same way its behaviour is.
- A strategy may be marked default, for a report that states none.
- The leaf set starts at one.
batch by bookships first; the machinery around it ships with it rather than after it, because retrofitting composition onto a hard-coded first strategy is the thing this task exists to avoid.
5. Plan
The set is the unit, not the book. A book is one way to slice trades and it is the first way, not the shape of the system. Everything downstream addresses a trade set by its batch key, so a strategy that groups by desk, by counterparty or by whatever the grid is doing next needs no subscriber to change.
The dependency is real and it is not ours. Trade sets do not exist yet:
they are being built in ores.trading by Merry Newton. This task is blocked on
that landing, and the batcher is the second thing to be built rather than the
first. What the batcher needs from the set is small and worth stating so the
two efforts agree early: a name, an as-of, and the trades in it.
Why the strategy is an interface and not a switch. Grid behaviour is the stated reason for wanting more than one strategy — how work is best split depends on what the grid is doing — and that is a thing that changes while the system runs. A switch in the caller would put grid knowledge in the report path; a strategy boundary keeps it in trading where the population is known.
The batch key is opaque downstream. REPORT_NAME:BOOK_ID:ASOF is a
convention, not a grammar. Nothing downstream splits it: the key names an
artifact and the as-of travels on the set, so a future strategy may use any
string without a producer being able to tell.
6. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit |
|---|---|---|---|
7. Notes
Where the sets come from, and why that splits this task. The trade-set knowledge documents settled which sets need storage and which do not (Trade set membership).
A set whose context is a book is derived: the trades in a book are the trades whose booking names that book, so the set is a predicate over a key the trade already carries and needs no table. Batch by book is therefore pure partitioning over data that exists, and nothing about it waits on a new entity.
A set a run read is recorded, because a repeat has to reconstruct the input it read rather than today's. That is a trade group: a set with a foreign key to the run, and a membership table beside it. It is Add the trade group and its membership, in Trace a process run and the trade groups it creates, and that story is parked until a pilot process settles the run model.
So one acceptance bullet — that the batcher records the sets it created against the run — waits on that work, and the rest does not. The earlier blocker text said the sets did not exist to partition into; on the settled model they do, for the strategy this task ships first.
8. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
9. PRs
| PR | Title |
|---|---|
| 2875 | [trading] Partition a report's trade population into trade sets |
10. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
11. Result
Landed: the partitioning. ores.trading.core/service/trade_batcher.hpp holds
the vocabulary and the walk.
trade_populationis what a report asks to be partitioned: its name, the as-of, and every trade in scope with the book its current booking names.trade_setis what comes out: a name, an as-of and the trades. The name is the batch key and it is opaque downstream.trade_batching_strategyis the seam. A strategy splits the sets it is given and leaves the ones it cannot split unchanged, so composition is a sequence rather than a switch, and grid knowledge stays out of the report path.batch_by_bookis the first strategy. It is the derived set: it reads the book from the population's trade and stores nothing.at_most_tradesbounds a set's size. Withbatch_by_bookit makes the first real batching — "batch by book and at most a thousand trades" — two strategies composed rather than a third that reimplements both.composite_strategyapplies its members in order, andtrade_batcherruns one over the population.
The invariants are the batcher's, not each strategy's. A partition covers the population exactly once and gives no two sets the same batch key. A strategy that loses a trade, puts one in two sets, or names two sets alike would price a population nobody asked for, so the batcher refuses it rather than handing it on. The cases assert both refusals.
12. Acceptance, bullet by bullet.
- A batcher partitions a population and a strategy decides how – done.
- Batch by book is the first strategy, behind an interface – done.
- A set carries a name and an as-of, the key being
REPORT_NAME:BOOK_ID:ASOF– done. - The batcher records the sets against the run – not done. That is a trade group, which is Add the trade group and its membership.
- A later strategy partitions differently without a producer changing – the seam is in, and a case exercises a strategy that partitions on something other than the book.
What the design still owes. The requirements recorded here say a strategy's parameters are configured in tables, so that changing one is not a deploy, and that a strategy may be marked default for a report that states none. Neither is built: a strategy takes its parameters as a value, which is the shape a table can supply, but no table supplies them yet. That and the run wiring need the group, so they travel together.
Not verified. No build and no test run: the hourly validation owns them. The cases are written and the component file lists are regenerated, nothing more.