Task: Vintage-aware seeding for ir_curve_feed

Table of Contents

This page documents a task in the IR Rates synthetic data: dataset seeding, index cleanup, dual-curve, quoting conventions story. It captures the goal, current status, acceptance, and any notes or results.

Goal

Give IR curve synthetic feeds the same "grounded in real data, easy to demo" quality FX spot feeds already have. Today an ir_curve_generation_config must be hand-entered with arbitrary kappa=/=theta=/=sigma=/=initial_rate values — nothing stops (and nothing guides away from) unrealistic numbers, since there's no reference. ir_curve_feed=/=make_ir_curve_feed has no vintage_source=/=vintage_date plumbing at all today — initial_rate always comes straight from the hand-entered config, unlike FX's feed_controller::start(), which resolves it from a real imported observation when a vintage is requested. This task adds the equivalent path for IR curves, consuming the dataset sourced by the sibling sourcing task.

Mirrors synthetic_fx_spot_configs_basic_populate.sql / synthetic_fx_spot_configs_realistic_populate.sql, which both seed initial_price from a real vintage (2016-02-05 Fed H.10), differing only in per-pair-calibrated vs exaggerated-uniform volatility.

Already delivered by prior tasks (not this one)

  • synthetic.ir_curve_configs.basic=/.realistic= DQ datasets, day- scaled default kappa=/=sigma — tick-batch-publishing task (615FD100).
  • The manual per-tenant start/stop/list control-plane (ir_curve_feed_config_handler, request-scoped context via JWT) and its Qt toolbar entry point on IrCurveGenerationConfigMdiWindow — delivered directly ahead of this task's own start, see commit 19c74d4d9. A dedicated "IR Simulator" window (mirroring the FX Market Simulator) remains future work, not required by this task's acceptance.

Split out of this task

Status

Field Value
State DONE
Parent story IR Rates synthetic data: dataset seeding, index cleanup, dual-curve, quoting conventions
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-07-18

Acceptance

  • ir_curve_feed=/=make_ir_curve_feed gains a vintage-aware seeding path (mirroring feed_controller::start()'s vintage_source=/ =vintage_date), so initial_rate comes from a real imported observation rather than an arbitrary config value.
  • Both the auto-start path and the on-demand ir_curve_feed_config_handler::start() path use this vintage resolution identically, so they can never drift (same pattern make_ir_curve_feed already uses to keep both paths in sync).
  • Missing vintage data is rejected with an actionable error, mirroring feed_controller's vintage_data_missing result — not a silent fallback to an arbitrary default.

Plan

(Implementation strategy. Written when work starts; key decisions are distilled into the parent story's * Decisions at close, but the plan itself stays — it is the historical record of what we did.)

Mirrors fx_spot_generation_config's price_source=/=vintage_source=/ =vintage_date discriminator (see feed_controller::start()'s vintage_data_available()) onto ir_curve_generation_config and curve_feed_controller. IR's initial_rate plays the role FX's gmm_initial_price plays.

1. Model the three new fields (codegen source of truth)

ir_curve_generation_config is codegen'd (SQL/domain/table/entity/ mapper files all carry "AUTO-GENERATED — DO NOT EDIT MANUALLY" banners) — the fields go into projects/ores.synthetic/modeling/ores.synthetic.ir_curve_generation_config.org first, mirroring ores.synthetic.fx_spot_generation_config.org's own price_source=/=vintage_source=/=vintage_date sections and Checks table entries, then regenerate (codegen-add-domain-type / code-run-build codegen step) rather than hand-editing generated output:

  • price_source (text, default "fixed", not "vintage" — IR curve configs today are all hand-entered kappa/theta/sigma/initial_rate, unlike FX which defaults new configs to vintage-seeded).
  • vintage_source (text, default empty).
  • vintage_date (text, default empty).
  • SQL check mirroring FX's: "price_source" in ('fixed', 'vintage') and the paired "fixed" and "initial_rate" > 0 and vintage cols empty / "vintage" and "initial_rate" = 0 and vintage cols non-empty constraint — swapping gmm_initial_price for initial_rate.
  • Regeneration touches (confirm the generated diff matches this list): ir_curve_generation_config.hpp (domain struct), _table.hpp/cpp, _json_io.hpp/cpp, _entity.hpp (core repository), _mapper.cpp, _generator.cpp, the SQL create script (synthetic_ir_curve_generation_configs_create.sql), and the Qt generated pieces (Detail fields=/=Columns tables already list auto_start from the sibling task — add the three new rows there too, same pattern: priceSourceEdit=/=vintageSourceEdit=/ =vintageDateEdit widgets, PriceSource=/=VintageSource=/ =VintageDate model columns).

2. curve_feed_controller: vintage resolution, mirroring feed_controller

Port feed_controller's vintage_data_available() (parse ORE key → market_data_client::find_series() → paged list_observations_page() scan matching source=/=point_id=/ =observation_datetime date part) onto curve configs. Differences from FX to account for:

  • Decision: resolve a single scalar, not a per-entry vector. initial_rate is a parameter of the short-rate process itself (kappa=/=theta=/=sigma=/=initial_rate, see process_factory::make_yield_curve_process()), not of each published tenor point — every entries_ tick is derived from that one process via price_ir_curve_entry() (ir_curve_feed.cpp:129), mirroring exactly how FX's single gmm_initial_price scalar seeds the process that all subsequent ticks derive from. So there is nothing to resolve per-entry.
  • Anchor point: after resolve(entries, refctx, ...) builds the ir_curve_resolved_entry list, pick the entry with curve_role = "DEPO"= and the smallest tenor (the point instrument closest to an overnight/short rate — the natural real-world analog to a short-rate model's initial_rate), and look up its vintage observation by (vintage_source, vintage_date, point_id) — same find_series + paged list_observations_page() scan as FX, keyed on that entry's own point_id instead of a hardcoded "SPOT". A template with no DEPO entry is an actionable vintage_data_missing error, not a fallback to a different role.
  • Add curve_feed_controller::start_result::vintage_data_missing (alongside the existing started=/=already_running=/ =qualifier_conflict), returned with an actionable error_detail when the vintage lookup fails — never a silent fallback to the config's hand-entered initial_rate.
  • add() (auto-start path) and start() (on-demand path) both run the same resolution before constructing the IYieldCurveProcess, so they can never drift — same invariant the task's Acceptance calls out, same shape make_ir_curve_feed already keeps in sync today.

3. make_ir_curve_feed=/=ir_curve_feed_config_handler wiring

  • make_ir_curve_feed gains the vintage resolution call (or takes a pre-resolved initial_rate from the caller, matching whichever split curve_feed_controller ends up owning per step 2) instead of reading cfg.initial_rate unconditionally as it does today (ir_curve_feed.cpp:187).
  • ir_curve_feed_config_handler::start() threads through error_detail=/=caller_bearer_token the same way market_feed_config_handler does today for FX, so a vintage_data_missing result reaches the Qt/shell caller as an actionable message rather than a generic failure.
  • vintage_validity_handler.hpp's Validate Vintage action already covers IR curve configs (delivered by the sibling sourcing task) — confirm it composes with this task's resolution path rather than duplicating it (ideally both call the same underlying resolver).

4. Verification

  • Unit tests around the new curve_feed_controller vintage resolution (missing vintage → vintage_data_missing with actionable detail; present vintage → correct seed value(s) reach the constructed process) — mirroring whatever coverage (or documented gap) exists for feed_controller's own vintage_data_available().
  • Manual QA scenario via the QA Validation Runner: start an auto_start=false IR curve config in vintage mode against the legacy/recent vintage datasets the sibling task seeded, confirm initial_rate is seeded from the real imported observation and a deliberately-wrong vintage_date produces the actionable rejection message.

Notes

Implementation status (code complete, manual QA scenario still pending):

  • price_source=/=vintage_source=/=vintage_date added to ir_curve_generation_config via the codegen model and regenerated (domain/table/entity/mapper/SQL/Qt Detail+Columns all updated). price_source defaults to "fixed" (non-breaking for existing rows), unlike FX's "vintage" default.
  • make_ir_curve_feed resolves initial_rate from a real market_observation when price_source is "vintage", anchored on the resolved Curve Template's shortest-tenor DEPOSIT entry (see select_vintage_anchor_entry(), unit-tested in isolation since the full resolution path needs a live market_data_client). Throws the new vintage_data_missing_error with an actionable detail on failure – mirrors feed_controller's vintage_data_missing result contract for FX, as an exception instead of a result enum since make_ir_curve_feed already throws for other construction failures.
  • Both call sites (auto-start in application.cpp, on-demand in ir_curve_feed_config_handler) go through this same function and now thread an authenticated nats_client (svc_nats=/=auth_nats) through registrar, so the on-demand path can delegate the lookup with the caller's own bearer token.
  • Full build green; ores.synthetic.service.tests extended with ir_curve_feed_vintage_tests.cpp (4 new cases covering anchor selection: shortest-tenor pick, non-DEPOSIT entries ignored, empty template, tie-breaking) – 21/21 cases passing.

Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario, run through the QA Validation Runner panel) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
Verify vintage-aware seeding for ir_curve_feed PASSED All 6 steps pass against a fresh Barclays provision

PRs

PR Title
#1705 [dq,sql,qt] Vintage-aware seeding for ir_curve_feed

Review

Comment summary File Decision Notes
       

Result

All acceptance criteria met and manually verified end-to-end against a freshly provisioned Barclays Plc tenant:

  • ir_curve_feed=/=make_ir_curve_feed resolves initial_rate from a real market_observation when price_source=vintage, anchored on the resolved Curve Template's shortest-tenor DEPOSIT entry; vintage_data_missing_error on a missing observation, mirroring feed_controller's contract for FX.
  • Both the auto-start and on-demand (ir_curve_feed_config_handler) paths share this resolution, so they can't drift.
  • IrCurveEditor gained the price_source=/=vintage_source=/ =vintage_date fields (mirroring FX), including a Browse-available- vintages picker.
  • ores.synthetic.service.tests extended with ir_curve_feed_vintage_tests.cpp (4 new cases); full suite green (74/74).

Two real bugs surfaced and fixed along the way, both required for the scenario to even reach a runnable state:

  • publication_service::list_bundle_publishable_datasets didn't resolve a bundle member's own dependencies, so a vintage-mode config's dependency on the real market data it resolves against was silently dropped from bundle publish.
  • refdata.calendars had no declared dependency on iso.countries in ores_dq_dataset_dependencies_tbl, so the dependency-graph resolution above could (and did) order calendars before the country rows (including the ZZ supranational sentinel) their own country_code column requires – breaking real GLEIF tenant provisioning, not just this story's synthetic bundles. Also fixed a duplicate account_contact_informations_tenant_isolation_policy RLS policy block in iam_rls_policies_create.sql blocking DB recreate entirely, unrelated pre-existing bug hit along the way.

Three follow-up gaps discovered during manual QA, filed as separate BACKLOG tasks in this story rather than fixed here (out of scope for this task's acceptance):

Emacs 29.3 (Org mode 9.6.15)