Task: Marshal the engine's output into result rows

Table of Contents

This page documents a task in the Prove the report grid run end to end — the remaining work story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

The node turns the engine's own output into one file of canonical rows before it uploads, so the service reads one shape and never the engine's files.

2. Status

Field Value
State DONE
Parent story Prove the report grid run end to end — the remaining work
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-30

3. Acceptance

  • The projection lives in ores.ore.core/results, because the engine's headings are an ORE format.
  • It reads the engine's headings by name: TradeId, NPV, NpvCurrency, NPV(Base), Notional, NotionalCurrency and CounterParty.
  • It writes one row per measure: npv and npv_base with their currency, and notional with its own currency when the file states one.
  • It refuses a file with no TradeId and no NPV heading, a row with no trade id, a figure it cannot read and a currency-bearing measure with no currency. Any refusal uploads nothing.
  • The wrapper gzips the canonical rows and uploads them as the result's rows, because storage holds blobs compressed.
  • The canonical header is subject_type,subject_id,measure,currency,counterparty_id,figure,source.

4. Plan

The projection renames the engine's headings onto the canonical columns rather than interpreting the file. The wrapper marshals npv.csv after the engine finishes, writes results.csv.gz beside the log file and uploads it to the result's rows location.

5. Measurements

What Value Command Commit
The canonical header the writer states subject_type,subject_id,measure,currency,counterparty_id,figure,source grep -n "subject_type,subject_id" projects/ores.ore/core/src/results/canonical_rows.cpp 55e53d5c56
The engine file the wrapper projects npv.csv, and no other grep -n "npv.csv" projects/ores.compute/wrapper/src/app/application.cpp 55e53d5c56

6. Notes

  • Only npv.csv has a projection. The other analytics write their own files (flows.csv, curves.csv, sensitivity.csv, scenario.csv, stresstest.csv, var.csv, pnl_explain.csv and others) and none is projected. Filed as Project each analytic's file into canonical rows.
  • A refusal is logged as a warning and the result carries no rows, rather than the run failing: the engine's output is still the run's output.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
Run a report through the grid and read every result PENDING The scenario checks the rows a run of an NPV report loads.

8. PRs

PR Title
#3916 [reporting] Post-process a run's results into rows

9. Review

Comment summary File Decision Notes
       

10. Result

The node writes the canonical rows from the engine's npv.csv before it uploads. The projection is a rename of the engine's headings onto the canonical columns, so a file whose headings are not the engine's is refused rather than guessed at, and the refusal names the headings it saw. Three measures come from one file: npv, npv_base and notional, each with the currency the engine stated beside it, and the counterparty the file names travels on every row. The rows are gzipped, because storage holds blobs compressed, and the service decompresses them on read.

The projection reads only npv.csv. Every other analytic's file has no projection yet, so a run of one of those analytics uploads no rows.

Emacs 29.3 (Org mode 9.6.15)