Task: Marshal the engine's output into result rows
Table of Contents
This page documents a task in the Prove the report grid run end to end — the remaining work story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
The node turns the engine's own output into one file of canonical rows before it uploads, so the service reads one shape and never the engine's files.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Prove the report grid run end to end — the remaining work |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-30 |
3. Acceptance
- The projection lives in
ores.ore.core/results, because the engine's headings are an ORE format. - It reads the engine's headings by name:
TradeId,NPV,NpvCurrency,NPV(Base),Notional,NotionalCurrencyandCounterParty. - It writes one row per measure:
npvandnpv_basewith their currency, andnotionalwith its own currency when the file states one. - It refuses a file with no
TradeIdand noNPVheading, a row with no trade id, a figure it cannot read and a currency-bearing measure with no currency. Any refusal uploads nothing. - The wrapper gzips the canonical rows and uploads them as the result's rows, because storage holds blobs compressed.
- The canonical header is
subject_type,subject_id,measure,currency,counterparty_id,figure,source.
4. Plan
The projection renames the engine's headings onto the canonical columns rather
than interpreting the file. The wrapper marshals npv.csv after the engine
finishes, writes results.csv.gz beside the log file and uploads it to the
result's rows location.
5. Measurements
| What | Value | Command | Commit |
|---|---|---|---|
| The canonical header the writer states | subject_type,subject_id,measure,currency,counterparty_id,figure,source |
grep -n "subject_type,subject_id" projects/ores.ore/core/src/results/canonical_rows.cpp |
55e53d5c56 |
| The engine file the wrapper projects | npv.csv, and no other |
grep -n "npv.csv" projects/ores.compute/wrapper/src/app/application.cpp |
55e53d5c56 |
6. Notes
- Only
npv.csvhas a projection. The other analytics write their own files (flows.csv,curves.csv,sensitivity.csv,scenario.csv,stresstest.csv,var.csv,pnl_explain.csvand others) and none is projected. Filed as Project each analytic's file into canonical rows. - A refusal is logged as a warning and the result carries no rows, rather than the run failing: the engine's output is still the run's output.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
| Run a report through the grid and read every result | PENDING | The scenario checks the rows a run of an NPV report loads. |
8. PRs
| PR | Title |
|---|---|
| #3916 | [reporting] Post-process a run's results into rows |
9. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
10. Result
The node writes the canonical rows from the engine's npv.csv before it uploads.
The projection is a rename of the engine's headings onto the canonical columns,
so a file whose headings are not the engine's is refused rather than guessed at,
and the refusal names the headings it saw. Three measures come from one file:
npv, npv_base and notional, each with the currency the engine stated beside
it, and the counterparty the file names travels on every row. The rows are
gzipped, because storage holds blobs compressed, and the service decompresses
them on read.
The projection reads only npv.csv. Every other analytic's file has no
projection yet, so a run of one of those analytics uploads no rows.