Task: Carry the bulk observation reads in the models

Table of Contents

This page documents a task in the Run a report end to end on a synthetic bond population story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

The component drift gate passes for marketdata on main, with the bulk observation reads and the export handler carried by their models.

2. Status

Field Value
State DONE
Parent story Run a report end to end on a synthetic bond population
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-23

3. Acceptance

  • check_component_drift.py –component marketdata passes.
  • The generated files equal what the models regenerate, and the bulk reads and the export handler keep their behaviour.

4. Plan

Commit b06ddccb2f edited three generated files by hand: the observation repository (header and source) and the series handler header. The drift gate regenerates them and removes the edits. So the code moves into the paste blocks of the two models, ores.marketdata.market_observation.org and ores.marketdata.market_series.org, and the files are regenerated. The code does not change.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
       

6. Notes

Found by another session, which ran the gate on main and saw it fail for marketdata. The cause was mine. I ran no drift gate on the pull request of that commit, because I chose to run gates sparingly, and I edited generated files instead of the model blocks. A model-carried read needs its section, its slot blocks and its includes in the model. read_latest_all and read_as_of_all are new sections. The manual point helper joins the block of read_as_of_records. The handler block and its chrono include are updated.

The regenerated source files equal what was committed. The header differs in order only: the generator orders the declarations by the order of the model sections.

The gate compares the tree with the committed state, so a task page or story row that is not yet committed reads as drift.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#3694 Carry the bulk observation reads and the export handler in their models

9. Review

# Comment summary File Decision Notes
1 The bulk reads hold the whole tenant in memory. market_observation.org No change needed Known trade-off, stated in the doc comment.
2 read_manual_points_all filters by date in C++. market_observation.org Skipped The manual set is small. The filter can move into the SQL if it grows.
3 The map key allocates a string for each row. market_observation.org Skipped A micro-optimisation with no measured cost.
4 Track the flaky catalogue tests in a separate issue. repository_market_data_catalogue_tests.cpp Open Two cases fail in some runs when a generated code repeats in a database that holds many rows. To be raised as its own task.
5 Run the drift gate earlier, as a CI step. build Open I skipped the local gate on the earlier pull request. The gate that runs before a pull request is raised covers this set of components, and I should have run it.

10. Result

The bulk reads and the export handler live in their models, and the generated files are what the models regenerate. The drift gate for marketdata passes.

Emacs 29.3 (Org mode 9.6.15)