Task: Add oresmd support to ores.marketdata

Table of Contents

This page documents a task in the Migrate to oresmd, delete market_series qualifier and ore_key story. It captures the goal, current status, acceptance, and any notes or results.

Goal

Add oresmd support directly to oresmd's home component, ores.marketdatanot as a separate standalone library/component – so every downstream migration task on this story (ore_key, ir_curve_generation_config, market_series, and the ORE XML round-trip) has a single place to depend on:

  • market_data_identifier=/=market_data_requirement variants and the five per-asset-class concrete structs, per the design doc's data model, living in ores.marketdata.api/domain under the normal ores::marketdata namespace – no dedicated oresmd C++ namespace; oresmd is just the URI scheme string (a constexpr constant), not a code-organisation boundary.
  • parse_oresmd(uri) -> market_data_identifier, to_uri(market_data_identifier) -> oresmd_uri, and resolve(market_data_requirement, ...) -> market_data_identifier, dispatching via std::visit on the URI's asset_class – living in ores.marketdata.core alongside market_series_service and the other identifier-facing logic that component already owns.
  • The projection rules from the design doc (into ORE's index name, curve key, and quote key) as free functions callable from any other component that already depends on ores.marketdata (ores.synthetic, ores.qt.*, ores.reporting) without a new inter-component dependency edge.

No consumer migration happens in this task – it only adds the capability ores.marketdata exposes for the story's remaining tasks to consume.

Status

Field Value
State DONE
Parent story Migrate to oresmd, delete market_series qualifier and ore_key
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-07-28

Acceptance

  • market_data_identifier=/=market_data_requirement and the five per-asset-class structs are added to ores.marketdata.api, with no abstract base class/virtual dispatch (per the design doc's rfl-driven decision).
  • parse_oresmd, to_uri, and resolve are added to ores.marketdata.core, covering every asset class in the design doc's worked examples.
  • Projection rules into ORE's index name, curve key, and quote key are implemented and unit-tested against the design doc's worked examples (FX, IR, swaption, equity, credit, commodity).
  • No new component/library is created – the capability lives entirely inside ores.marketdata, reachable by any existing consumer of that component without a new dependency edge.
  • No consumer (ore_key, market_series, ir_curve_generation_config, ORE XML) is migrated as part of this task.

Plan

Added the data model to ores.marketdata.api/domain: oresmd_enums.hpp (instrument_type, curve_role, metric, index_family), five identifier structs plus their _requirement counterparts (market_data_identifier.hpp=/=market_data_requirement.hpp), and oresmd_uri.hpp (a constexpr oresmd_scheme string constant and a thin oresmd_uri wrapper type). Every struct has a defaulted operator== to support round-trip tests. Reused the existing domain::asset_class enum (fx=/=rates=/=credit=/=equity=/=commodity) rather than inventing a parallel one; the URI authority token ir maps onto asset_class::rates at the parser boundary, not as a separate enum value.

Added ores.marketdata.core/oresmd/ (oresmd_parser, oresmd_projections, oresmd_resolver, oresmd_exception), using boost::urls::url (already a vcpkg dependency; added the missing find_package(Boost ... url) + Boost::url link to this component's CMakeLists, following ores.http's existing pattern) and magic_enum for enum token parsing/serialisation. oresmd_parser::parse=/=to_uri dispatch on the URI's asset-class authority via a plain if-chain (parse) and std::visit (to_uri); oresmd_resolver::resolve dispatches via std::visit over the requirement variant, merging each field from the requirement first, the supplied defaults identifier second, throwing oresmd_exception naming any mandatory field left unset in both.

Per explicit feedback during implementation:

  • The library was rescoped from a proposed separate component into ores.marketdata itself before implementation started (see the task's own history) – no new library/CMake target was created.
  • Added explicit per-asset-class query-key validation (reject_if_present=/ =validate_fx=/=validate_ir=/etc.) so a field belonging to a different asset class (e.g. ~tenor=~ on an FX URI) throws =oresmd_exception rather than being silently ignored, and added a comprehensive test suite covering every worked example in the design doc plus invalid-input rejection cases for every asset class.

Two real bugs surfaced by writing that test suite before it passed (not found by manual inspection):

  • to_index_name=/=to_curve_key were initially gated on the wrong instrument_type value (curve instead of fixing), and separately on no type check at all in an intermediate fix – the design doc's own worked-examples table settles this: a single type=fixing URI produces both an index name and a curve key, while type=quote=/ =type=vol rows read "–" for both. Fixed to gate both projections on type == instrument_type::fixing, matching the table exactly.
  • oresmd_resolver's fallback logic treated a defaults identifier's default-constructed empty string (e.g. an unset ticker=/=ccy) as a present value, so a mandatory field left unset in both the requirement and defaults silently resolved to "" instead of throwing. Added a dedicated pick_mandatory_string helper that treats an empty string as absent on both sides, used for every asset class's genuinely-mandatory string fields (pair=/=ccy=/ =ticker=/=reference_entity=/=commodity_code); the type field keeps the original pick() helper since defaulting it to instrument_type::quote when unset is intentional, matching oresmd_parser's own default.

Verified locally: ores.marketdata.core.tests (77 test cases, 167 assertions, all passing, including the 8 that initially failed and exposed the two bugs above), ores.marketdata.api.tests, and ores.marketdata.client.tests all pass; ores.marketdata.service.exe builds clean. clang-format applied to all new files.

Notes

Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario, run through the QA Validation Runner panel) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

PRs

PR Title
#1721 [marketdata] Add oresmd parsing/projection/resolution support

Review

# Comment summary File Decision Notes
1 Unknown/misspelled query keys silently ignored, not rejected oresmd_parser.cpp Accepted query_params::from now throws on any key outside the seven known ones
2 FX pair length isn't validated at parse time oresmd_parser.cpp Accepted parse_fx now requires a 6-letter alphabetic entity
3 Duplicated to_upper=/=to_lower helpers between parser and projections oresmd_parser.cpp, oresmd_projections.cpp Accepted Extracted to shared oresmd/detail/oresmd_string_utils.hpp
4 index_name_ir permits a tenor-less term-index fixing without validation oresmd_parser.cpp Accepted parse_ir now rejects a term index (libor=/=euribor) type=fixing without a tenor; also extracted the duplicated is_overnight classification to oresmd/detail/oresmd_index_family_utils.hpp
5 Resolver docstring implies UB on asset-class mismatch, but behaviour is safe oresmd_resolver.hpp Accepted Reworded to describe the actual (safe, throwing) behaviour

Result

Delivered oresmd support inside ores.marketdata (no new component/library), meeting every Acceptance bullet:

  • market_data_identifier=/=market_data_requirement and the five per-asset-class structs (FX, IR, equity, credit, commodity) are in ores.marketdata.api/domain, tied together only via std::variant – no abstract base class or virtual dispatch anywhere.
  • parse_oresmd, to_uri, and resolve are in ores.marketdata.core, covering every asset class.
  • Projection rules into ORE's index name, curve key, and quote key are implemented (oresmd_projections) and unit-tested against every row of the design doc's worked-examples table, byte-for-byte.
  • No new component/library was created; the capability is reachable by any existing ores.marketdata consumer with no new dependency edge.
  • No consumer (ore_key, market_series, ir_curve_generation_config, ORE XML) was migrated – that is explicitly the remaining tasks' scope, per the story's Decisions.

Also delivered, beyond the original Acceptance bullets: explicit per-asset-class query-key rejection (invalid fields throw rather than being silently ignored) and a 77-test-case suite covering every worked example in the design doc plus invalid-input cases – both added in response to review feedback during implementation, surfacing and fixing two real bugs (wrong instrument_type gate on the index-name/ curve-key projections; empty-string-as-present bug in the resolver's fallback logic) that manual inspection alone had not caught.

Emacs 29.3 (Org mode 9.6.15)