Task: Post-process a run's results into rows
Table of Contents
This page documents a task in the Prove the report grid run end to end — the remaining work story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
After the compute results are retrieved, the rows the node marshalled become the figures the run's screens read, and the run states what was written.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Prove the report grid run end to end — the remaining work |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-30 |
3. Acceptance
- The run's workflow carries a
post_process_resultsstep aftersubmit_compute, so the rows are loaded once the batch exists. - For each result of the batch the step downloads the canonical rows the node wrote, and reads them with one reader.
- A refusal names the file, the header or the row that caused it, and loads nothing: a partial load would read as a complete one.
- Every row of every result lands in one insert for the run.
- The step answers the number of rows it loaded and a message naming the run.
- The run's log rows are written once, before the rows are loaded, so a run whose rows are refused still states what the engine said.
- Nothing in the step parses the run's archive; the rows come from the canonical file.
4. Plan
The step asks the batch's workunits for their results, checks each result's
outcome, downloads each result's rows into a read_outcome, maps them onto
ores.reporting.api::domain::report_result_row with the instance's party,
definition and priced date, and inserts them once. The archive stays the run's
output and is not read here.
5. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit |
|---|---|---|---|
| Rows one run loaded | 24: 8 trades for each of npv, npv_base and notional |
compass sql -- -c "select measure, count(*) from ores_reporting_report_result_rows_tbl where report_instance_id = 'e31abdee-0dbe-404c-977d-80539b44fe17' group by measure"; the measurement is recorded on Run a report from its page and read its rows |
3ffd149319 |
6. Notes
- The description asks for "one reader per analytic". One reader serves every
analytic, because the canonical file states its measures as rows rather than as
columns; what is per analytic is the projection the node runs before upload.
Only
npv.csvhas one today, so one analytic's rows load and 34 of the 35 seeded analytic codes would load none. Filed as Project each analytic's file into canonical rows. - The step records the run's account through
record_run_logsbefore it loads the rows, so the ordering is a property of the step rather than of the archive.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
| Run a report through the grid and read every result | PENDING | The scenario checks the rows the step loaded against the run's figures. |
8. PRs
| PR | Title |
|---|---|
| #3916 | [reporting] Post-process a run's results into rows |
9. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
10. Result
The run's workflow loads its figures after the batch is submitted. The step asks each workunit for its results, refuses a result whose outcome is not success, and downloads the canonical rows the node wrote for each one. One reader parses the file; a header it does not know, a field it cannot read or a figure it cannot parse is refused with the reason, and a refusal loads nothing.
The rows are mapped onto the run's party, definition and priced date and inserted in one call, and the step answers the count it loaded. The run's log rows are written first, so a run whose rows are refused still carries the engine's account.
The service no longer parses the run's archive for errors: the account comes from the rows the node wrote, which is where the figures come from too.