Task: Implement report pre-processing

Table of Contents

This page documents a task in the Implement the full report pipeline story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Produce the ORE engine's input from a definition's scope: bulk scope reads on trading and market data, mapping to the engine's documents through ore's mappers in-process, streaming, and an archive addressed by a digest of its inputs so that concurrent runs over the same scope share one generation.

2. Status

Field Value
State BACKLOG
Parent story Implement the full report pipeline
Now Not yet started.
Waiting on Nothing.
Next Begin implementation.
Last touched 2026-09-27

3. Acceptance

  • Trading and market data each expose a scope-limited bulk read, and pre-processing reads nothing else.
  • The artifact is addressed by a digest of its inputs and a repeated request over the same inputs is a lookup.
  • Peak memory is bounded and does not grow with the size of the input.
  • The archive contains Input/ore.xml and the files it names, so the grid can run it.

4. Plan

(Implementation strategy. Written when work starts; key decisions are distilled into the parent story's * Decisions at close, but the plan itself stays — it is the historical record of what we did.)

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
       

6. Notes

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
   

9. Review

Comment summary File Decision Notes
       

10. Result

Emacs 29.3 (Org mode 9.6.15)