Task: Build ORE curve-config and market-data XML round-trip via oresmd

Table of Contents

This page documents a task in the Migrate to oresmd, delete market_series qualifier and ore_key story. It captures the goal, current status, acceptance, and any notes or results.

Goal

Give ORE's curveconfig.xml (IR yield curves) and market-data quote-string files a real, working import/export round-trip that is keyed entirely through oresmd – today only currency config, calendar adjustments, conventions, and portfolio/trades round-trip through ORE's XML at all; curve config and market-data quotes do not.

Status

Field Value
State ABANDONED
Parent story Migrate to oresmd, delete market_series qualifier and ore_key
Now Abandoned in favour of the proper follow-on story.
Waiting on Nothing.
Next Nothing – see * Result.
Last touched 2026-07-30

Acceptance

  • [ ] curveconfig.xml's YieldCurve segments round-trip: import produces oresmd-keyed domain objects (curve id -> ccy~/~tenor derivable via oresmd_projections::to_curve_key 's own Yield/CCY/CCYTENOR shape), export regenerates an equivalent YieldCurve segment from those same oresmd-keyed objects, covering every YieldCurveSegment subtype found across the full external/ore/examples corpus (Direct, Simple, AverageOIS, TenorBasis, CrossCurrency, FittedBond, DiscountRatio, and any others the survey below finds) – full depth on the one asset class (IR) oresmd fully represents today, not full breadth across every asset class the corpus also contains (that gap is tracked separately, see * Notes).
  • [ ] Market-data quote-string export: given market_series=/=market_observation rows, generate market*.txt lines whose keys are produced via oresmd_projections::to_quote_key=/=to_curve_key, the exact inverse of the existing import path (market_data_parser.cpp -> series_key_registry::decompose_key -> import_service.cpp).
  • [ ] Confirmed and documented: the existing import path (market_data_parser.cpp, series_key_registry.cpp, import_service.cpp) is untouched – it already round-trips externally-authored ORE key strings correctly (structural decomposition via a per-type registry, not oresmd parsing); this task adds the missing export direction and the curve-config (as opposed to quote-string) round-trip, it does not replace or reimplement the import side.
  • [ ] todaysmarket.xml curve-configuration-ID mapping: scoped down to a documented follow-on (see * Notes) unless Step 1 finds it trivial to include – see narrower-first-slice reasoning below.
  • [ ] Full compass build clean; ctest full suite green (barring the known, separately-owned ores.refdata.core.tests calendar-rule regression, if still unresolved at merge time).

Plan

(Implementation strategy. Written when work starts; key decisions are distilled into the parent story's * Decisions at close, but the plan itself stays — it is the historical record of what we did.)

Landscape survey (as found)

  • Raw XML plumbing already exists: ores.ore.core/domain/domain.hpp (XSD-codegen'd) already has curveconfiguration and todaysmarket raw structs with load_file=/=load_data=/=save_file=/=save_data, the same low-level pattern already used for conventions. No new XSD/XML-binding work is needed – only a mapping layer on top, exactly like conventions_mapper.hpp=/.cpp= does for importer::import_conventions.
  • series_key_registry is a separate, more sophisticated utility than oresmd_projections::split_market_series_key (added in the prior market_series task): it decomposes a key into {series_type, metric, qualifier, point_id} via a per-type registry that knows how many trailing segments are the point_id vs. the qualifier (e.g. DISCOUNT/RATE/EUR/2Y -> qualifier="EUR", point_id="2Y"), with a lossless-roundtrip fallback for unregistered types. oresmd_projections::split_market_series_key only ever splits into 3 parts (qualifier absorbs everything after metric) – deliberately simpler, for market_series's storage columns, not for reconstructing a point-level series. These are different tools for different directions and both stay: registry for real-ORE-file parsing (reverse direction, already correct, already tested), oresmd_projections for internal oresmd-URI generation (forward direction). Confirms the prior task's scoping decision was right.
  • No existing export path: import_service.cpp writes market_series=/=market_observation rows from parsed file content; there is no code anywhere that goes the other way (DB rows -> market*.txt lines). This is new work, not a migration.
  • No existing curve-config mapping layer: nothing maps curveconfiguration's raw YieldCurve XSD struct to/from any ORES domain type today – ir_curve_generation_config=/=ir_curve_feed (this story's second task) is the closest existing oresmd-keyed IR curve model to map onto/from.
  • oresmd_resolver solves a different problem: it resolves a possibly-partial market_data_requirement into a full market_data_identifier (e.g. for a UI's "what feed satisfies this requirement" question) – not directly reusable for curve-config import/export, though the resulting identifier's to_curve_key=/=to_quote_key projections are exactly what curve-config export needs to produce ORE-shaped key strings.

Narrower-first-slice reasoning

todaysmarket.xml maps configuration IDs (strings) to curve-configuration IDs referenced inside curveconfig.xml – it is largely a thin ID-to-ID lookup table once curve-config round-trip exists, and doesn't have its own oresmd-keying concerns the way curveconfig.xml's YieldCurve segments and market-data quote keys do. Landscape survey in Step 1 will confirm whether it is small enough to fold in, or genuinely deserves its own follow-on task; either way this task does not block on it.

Step 1 – Curve-config (curveconfig.xml) import/export

Add importer::import_curve_config=/=exporter::export_curve_config (mirroring import_conventions=/=export_conventions's existing shape) mapping curveconfiguration's YieldCurve segments to/from an oresmd-keyed domain representation. Survey every YieldCurve segment Type actually used across external/ore/examples' curveconfig.xml corpus first (Direct, Simple, AverageOIS, TenorBasis, CrossCurrency, FittedBond, DiscountRatio, …) and cover all of them, not just the discount/projection-via-direct-quotes shape ORE Studio's own synthetic generation produces – full round-trip fidelity against real-world curveconfig.xml files is the acceptance bar, not just against our own generated ones.

Step 2 – Market-data quote-string export

Add the missing export direction: given market_series=/=market_observation rows, produce market*.txt-shaped lines using oresmd_projections::to_quote_key=/=to_curve_key to regenerate the key string, the exact inverse of today's import path. Confirm round-trip fidelity against a sample file already used by the existing import tests (parse -> resolve to oresmd identifier -> project back to key -> byte-identical to the original key string).

Step 3 – todaysmarket.xml (scope TBD per Step 1 findings)

Either fold in as a thin ID-mapping addition, or split into its own follow-on task – decided once Step 1 clarifies how curveconfiguration IDs are actually referenced.

Step 4 – Verification

Full compass build + ctest; unit tests for the new mapping functions and the quote-string export round-trip, following this story's established "extract pure, testable functions" pattern.

Notes

While scoping "support every curve config in sample data" against the full external/ore/examples corpus, found oresmd's current design (../../../../../knowledge/domain/market_data_urn.org) only structurally represents ~7 of the 49 quote types the market data catalogue documents – most of the corpus's curve/vol types (inflation, commodity vol, correlation, equity vol, credit default curves, etc.) have no oresmd representation and no ORES domain model at all. That representation gap is real (these are gaps in what oresmd can express, not just what this task converts) but is design-doc-level work well beyond one XML-round-trip task, so it was split off into its own story: Extend oresmd to full ORE quote-type coverage, with one task per family. This task stays scoped to YieldCurve (full segment-subtype depth) plus market-data quote-string export, for the asset classes/types oresmd already represents.

Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario, run through the QA Validation Runner panel) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

PRs

PR Title
#1782 [agile] Abandon ORE curve-config XML round-trip task

Review

# Comment summary File Decision Notes
5136419848 Used markdown ~~strikethrough~~ instead of org-mode's +strikethrough+ story.org Accepted Fixed in 60ab72332

Result

Abandoned before implementation. Landscape survey (see * Plan=/=* Notes) found two compounding problems that make this task the wrong vehicle to finish in:

  • Scoping "support every curve config in sample data" against the real external/ore/examples corpus surfaced that oresmd's design represents only ~7 of the 49 ORE quote types the market data catalogue documents – a design-doc-level gap, split into its own story, Extend oresmd to full ORE quote-type coverage, with one task per family.
  • Even narrowed to YieldCurve alone, most segment subtypes (Simple-Deposit/FRA/Future/BMA, TenorBasis, CrossCurrency, ZeroSpread) reference quote types (MM/RATE, FRA/RATE, BASIS_SWAP/BASIS_SPREAD, CC_BASIS_SWAP/BASIS_SPREAD, ZERO/YIELD_SPREAD, …) that are exactly the gap the new story's IR-coverage task is meant to close – a genuinely oresmd-keyed curve-config round-trip is blocked on that task landing first, not something this task could finish independently.

With the sprint closing, the right move is to abandon this task rather than force a narrowed, partially-oresmd-keyed slice through, and pick the work back up properly via the new story once its IR-coverage task (and however many of the others are needed) has landed. No code was written; only the agile docs (this task, the sibling story split, the sprint/story tables) are part of this task's history. The feature/ore-curve-config-market-data-xml-roundtrip branch's doc commits land via a docs-only PR rather than being discarded, since they correctly capture the investigation and the story split that came out of it.

Emacs 29.3 (Org mode 9.6.15)