Task: Collapse the remaining synthetic control-plane forks

Table of Contents

This page documents a task in the Implement synthetic data generation against the new market data model story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Collapse the spurious asset-class forks that remain in the synthetic generation control plane onto the one pipeline, so an asset class varies only in its datum shape, its configuration fields and its process maths. Every other concern — the tick clock, the vintage read, the registry instance, and the handler plumbing — exists once over registered kinds instead of once per asset class. A collapse deletes one of two copies; it does not add a branch that moves the fork.

2. Status

Field Value
State DONE
Parent story Implement synthetic data generation against the new market data model
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-07

3. Acceptance

  • The tick clock, the vintage read and the registry instance exist once, and both asset classes run through them.
  • No asset class's observable behaviour changes. Any difference that could not be removed without choosing a side is recorded under * Notes.
  • ores.synthetic.api.tests, ores.synthetic.core.tests, ores.synthetic.service.tests and ores.marketdata.service.tests pass.
  • check_component_drift.py --all and link_structure_notes.py --check pass.
  • Every spurious fork left uncollapsed is named with the reason it was not collapsed in this pass.

4. Plan

  1. Re-verify every ** ... — SPURIOUS entry of the fork inventory against the code at the base commit b72fc67e44, because the survey predates it. Result: of 13 spurious forks, two were already fixed and one was already gone; ten were still present.
  2. Collapse the smallest, most self-contained forks first, each as its own verifiable commit: the duplicated tick clock, then the triplicated vintage observation lookup, then the registry rebuilt at every call site. Each is a deletion of a second copy, not a new branch.
  3. Run the two suites, the component-drift gate and the link-structure gate after the collapses.
  4. The ten-minus-three remaining forks all turn on one missing seam: a per-kind registry that supplies, alongside the factory builder, the kind's config permission and a config_id -> feed_build_input resolver. Build that seam once and collapse feed_config_handler, the folder handler and the permission strings together; do not attempt them one at a time, because each would otherwise grow its own private registry. vintage_validity_handler, the simulate/preview envelope and the config plumbing fields hang off the same seam and follow it.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
Spurious forks in the inventory 13 count of ** ... — SPURIOUS entries in evidence/asset-class-fork-inventory.org 9c1c4af10e (survey)
Spurious forks already fixed or gone at base 3 grep for producer_subject(, binding_mode, ensure_feed_binding b72fc67e44
Spurious forks still present at base 10 as above, minus the three b72fc67e44
Forks collapsed in this pass 3 tick clock, vintage read, factory instance 20a45d6a50
Lines removed vs added by the collapses 120 insertions, 280 deletions git diff --stat b72fc67e44..20a45d6a50 20a45d6a50
ores.synthetic test suites 3 suites, 3 passed ctest --test-dir build/output/linux-clang-debug-make -R 'ores.synthetic' --output-on-failure 20a45d6a50
ores.marketdata.service test suites 1 suite, 1 passed ctest --test-dir build/output/linux-clang-debug-make -R 'ores.marketdata.service' --output-on-failure 20a45d6a50
New tick-clock test cases 3 ores.synthetic.api.tests, tick_clock_tests.cpp 20a45d6a50
Component drift none reported projects/ores.codegen/venv/bin/python projects/ores.codegen/scripts/check_component_drift.py --all 20a45d6a50
Link-structure notes 16 notes, all linked back python3 build/scripts/link_structure_notes.py --check 20a45d6a50

6. Notes

6.1. Behaviour deltas introduced by the collapses

  • The shared tick clock's catch-and-continue guard now covers the whole tick, including the process step. Both producers previously advanced the process outside the guard, so a process that threw would have terminated the service rather than skipping the tick. Neither process can throw on any exercised path, and no test covers it, but the change is a real one and is recorded rather than chosen silently.
  • Unifying the vintage read's lookup-failure text changed one string for IR only: "Failed to look up series '<uri>':" became "Failed to look up series for '<uri>':". FX and feed_controller already used the "for" form, so the helper matches the majority. No test asserts either string.
  • The "no vintage data" message stays caller-owned and is unchanged for all three callers: they name different anchors (FX's SPOT datum, IR's anchor pillar, an ORE key), and the helper takes that message as an argument.

6.2. Coverage limits

  • find_vintage_observation()'s paged scan needs a live marketdata service, so it has no unit test; it is covered by the build and by the handler suites, which exercise the fixed/disabled/error surface but not the paging loop. select_vintage_anchor_entry() remains unit-tested.
  • The tick clock has three new unit cases in tick_clock_tests.cpp; the producers' own start() loops are still only covered indirectly.

**Spurious forks confirmed present and not yet collapsed

  • feed_config_handler resolves the kind by probing the FX then the IR repository, in both start() and stop().
  • feed_config_handler::start_fx and start_ir each re-implement the auth, enabled-and-container gate, child loading and error reply.
  • folder_feed_control_handler runs one start loop and one stop loop per kind, and assembles by_kind=/=stopped_by_kind from two hardcoded entries.
  • The config-read permission pair is hardcoded at every verb.
  • vintage_validity_handler reads only FX rows and returns an FX-shaped response.
  • The simulate and preview sampling loop and protocol envelope are duplicated between simulate_handler and ir_curve_preview_handler.
  • The common config plumbing fields (source_name, ticks_per_hour, process_type, enabled, auto_start, price_source, vintage_source, vintage_date, folder_id and the audit block) are copied across the two config entities. Collapsing these changes the persisted schema, so it needs a model change and regeneration rather than a code edit.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#2766 [synthetic] Collapse three control-plane forks onto one pipeline

9. Review

Comment summary File Decision Notes
       

10. Result

First pass. Three of the thirteen spurious forks are collapsed; seven remain and moved to Give the synthetic control plane one per-kind registry, because they all need the same missing seam rather than four separate fixes.

Collapsed. The duplicated tick clock is now one tick_clock in the synthetic api: the period, the sliced stop-responsive sleep, the publish counter and the catch-and-continue guard exist once, and a producer supplies only what one tick does. The triplicated vintage lookup is now one paged (source, datum, date) read and one date_part. make_default_feed_factory is rebuilt per call site no longer: there is one default_feed_factory() shared by the four sites. Net 120 insertions against 280 deletions in code, so the collapse removes more than it adds.

Already done before this task started, and recorded so nobody reopens them: the IR producer subject honours binding_mode, IR auto-start forwards it, and the FX-only start(ore_key, ...) path is gone along with ensure_feed_binding.

Two behaviour deltas, reported rather than hidden. The shared guard now covers the whole tick including the process step, where both producers previously ran the process outside it, so a throwing process no longer terminates the service; no test or exercised path throws. And unifying the lookup-failure text changed IR only, from "Failed to look up series '<uri>'" to "…for '<uri>'", matching what FX and the feed controller already said.

Why the rest is one unit and not seven. Forks 4 to 8 and 11 all need a per-kind registry carrying, beside the factory builder, the kind's config permission and a config_id to feed_build_input resolver. Collapsed one at a time, each handler grows a private registry, which is the duplication this task exists to remove. The last fork, the duplicated config plumbing fields, changes the persisted schema and needs a model change plus a regeneration.

Measured at 20a45d6a50. ctest -R ores.synthetic passes 3 of 3; ctest -R ores.marketdata.service passes 1 of 1; the codegen drift check reports no drift; the structure-note link check passes. Not covered: find_vintage_observation's paging path has no unit test because it needs a live marketdata service.

Emacs 29.3 (Org mode 9.6.15)