Task: Declare the oresmd coordinates in the models
Table of Contents
This page documents a task in the Make the oresmd URI grammar explicit and typed story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
The oresmd grammar is declared in its models rather than in the generated
parser. Each of the twelve quote-type models states, for every query key, its
role (identity or coordinate), its value kind and its order; and it states, per
quote type, which coordinate keys that type carries. The point bag is removed.
No generated C++ changes in this unit. The point of the unit is that the next unit can generate the parser, the writer and the series identity from a declaration a reviewer can read, instead of from a hand-written visitor.
The findings and the target grammar are recorded in The oresmd URI grammar: silent defaults and the untyped coordinate.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Make the oresmd URI grammar explicit and typed |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-01 |
3. Acceptance
- Every dimension a
point=-bearing URI carries is a declared coordinate key. - Every query key of every quote type is either a declared coordinate or identity, and the generated series identity is the identity keys alone.
- The four volatility families with no quote type of their own gain one:
swaptionfor IR, and anoptionmember for FX, equity and commodity. - A codegen test walks every
point=-bearing URI in the model test tables and fails while any is not expressible as declared keys. It passes at the end of this unit and its check is the record's evidence. - The generated parser and writer accept and produce exactly what they do today, except that the quote-type enums gain the four new members. No URI that parses today stops parsing.
4. Plan
4.1. The model change
One new table per model, one new column per quote type, and one row added:
Each model gains a
* Coordinatestable:query_key cpp_type value_kind notes value_kindisstringorenum.- Each row of the
* Quote typestable gains acoordinatescolumn: the ordered comma list of the coordinate keys that quote type carries. The order is per quote type, because one key sits at different positions in different types (tenoris first forcds_indexand second forindex_cds_option). - The
pointfield, the identifier structs and the parser are untouched here. Removingpointis unit 3's work, so this diff reads as a specification rather than as a grammar. - A quote type whose last coordinate is either a number or a moneyness token
lists both
strikeanddelta. The parser tries the enum first, soATMis a delta and a decimal is a strike. - The four surfaces that had no quote type of their own gain one:
swaptionfor IR, and anoptionmember for FX and equity. Commodity already had one. Each gains a round-trip case, so the coverage record that requires both directions of every modelled type stays empty rather than gaining an exemption.
4.2. The coordinate inventory
Union per asset class, from the census in the investigation:
| Asset class | Coordinate keys, in order |
|---|---|
| fx | maturity; expiry, delta |
| ir | maturity; tenor; contract_code, tenor (mm_future); contract_month, contract_code, tenor (oi_future); expiry, strike/delta, smile |
| equity | maturity; expiry, delta, premium, call_put, strike |
| commodity | maturity; expiry, delta, premium, call_put, strike |
| credit | seniority, restructuring, tenor; expiry, strike |
| inflation | maturity; month; expiry, cap_floor, strike |
| correlation | expiry, delta |
| rating | from, to |
| shape_profile | date, second, period, dst |
| security, generic, power | none |
Per quote type:
| Quote type | Coordinates, in order |
|---|---|
| fx spot | — |
| fx fwd | maturity |
| fx option (new) | expiry, delta |
| ir ir_swap, mm, fra, imm_fra, bma_swap, basis_swap, cc_basis_swap, cc_fix_float_swap, zero | maturity |
| ir discount | tenor |
| ir mm_future | contract_code, tenor |
| ir oi_future | contract_month, contract_code, tenor |
| ir capfloor | expiry, strike |
| ir swaption (new), bond_option | expiry, smile, strike/delta |
| equity spot | — |
| equity dividend, fwd | maturity |
| equity option (new) | expiry, delta, premium, call_put, strike |
| commodity spot | — |
| commodity fwd | maturity |
| commodity option (new) | expiry, delta, premium, call_put, strike |
| credit cds, hazard_rate | seniority, restructuring, tenor |
| credit recovery_rate | seniority, restructuring |
| credit cds_index, index_cds_tranche | tenor, strike |
| credit index_cds_option | tenor, expiry, strike |
| inflation zc_swap, yy_swap | maturity |
| inflation seasonality | month |
| inflation zc_capfloor, yy_capfloor, cf_price | expiry, cap_floor, strike |
| correlation pairwise | expiry, delta |
| rating transition_probability | from, to |
| shape_profile shape_factor | date, second, period, dst |
4.3. The loader
load_org_oresmd_quote_type_model() reads the new table into
result["coordinate_keys"] (per asset class), splits each quote type's
coordinates cell into an ordered list, and refuses a key that is declared as a
coordinate in one table and as another type in the other, or that a quote type
lists and nothing declares. The templates that consume the keys are unit 3's
work.
4.4. The check
projects/ores.codegen/tests/test_oresmd_coordinates.py reads the same models
the generator reads and asks, of every declared test-case URI that still spells
a point: does it name a quote type, once the asset class's own default is
applied, and does that type declare the coordinates its key carries? It pins a
floor on the number of URIs it looked at, so a loader that returns nothing fails
there instead of passing over an empty set.
It does not map a token to the key it lands on, because the old point mixes
coordinate and identity dimensions and its arity varies with the key's shape, as
a swaption's smile form shows. What it does assert is the bound: the token count
never exceeds the keys the type's point_keys column declares, and every one of
those keys is a declared coordinate or a field. A key the point carries and the
model does not declare fails there. The generated parser of unit 3 is what
replaces the positional mapping.
5. Measurements
| What | Value | Command | Commit |
|---|---|---|---|
Market payloads under external/ore/examples |
96 | python predicate, is_market_payload in corpus_files.hpp |
80d03bab8f |
Fixing payloads under external/ore/examples |
66 | python predicate, is_fixing_payload |
80d03bab8f |
| Payloads carrying a lognormal and a normal vol at one date and coordinate | 22 | python scan of *market*.txt/csv |
80d03bab8f |
| Such groups across the corpus | 23,828 | same scan | 80d03bab8f |
| Whole-corpus import | Fails on its 32nd file, duplicate key | ores.marketdata.core.tests "every_line_of_the_whole_corpus_reaches_the_database" |
80d03bab8f |
6. Notes
7. Test Scenarios
| Scenario | State | Notes |
|---|---|---|
8. PRs
| PR | Title |
|---|---|
| #2464 | [marketdata] Declare the oresmd coordinate dimensions |
9. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
10. Result
The twelve quote-type models declare their coordinates, and the generator reads
them. Every point-bearing test URI resolves to a quote type that declares what
its key carries.
Delivered:
- A
* Coordinatestable in each of the twelve models, and acoordinatescolumn on every* Quote typesrow. load_org_oresmd_quote_type_model()reads both, and refuses a key declared as a coordinate in one table and another type in the other, or listed by a quote type and declared nowhere.test_oresmd_coordinates.py, the declaration check described above.- Three quote types:
ir.swaption,fx.optionandequity.option.commodity.optionalready existed. Each gained a round-trip case, soTEST_COVERAGE_EXEMPTstays empty. - A generator defect fixed at its source: the
* Quote typesheader ofcredit_quote_type.organdcommodity_quote_type.orgdeclared nonotescolumn while their rows carried one, so the notes text landed in thecommentslot and the generated enum documented the key shape where the meaning belonged. The header now declaresnotes, and the regenerated enums carry the real comments. - The one hand-written switch the new members reached,
default_metric(ir_quote_type), gained its case inores.cpp.oresmd.projections.org, from which the template is tangled.
Evidence:
| What | Value | Command | Commit |
|---|---|---|---|
| codegen suite | 1,049 passed | venv/bin/python -m pytest projects/ores.codegen/tests -q |
uncommitted at measurement |
| oresmd core tests | 1,940 assertions in 420 cases, all passed | ores.marketdata.core.tests |
uncommitted at measurement |
Not done here, and unit 3's work: the point field still exists, the parser
still reads the old spelling, and the series identity still clears the volatility
struct. The corpus import therefore still fails, and the investigation's
measurement of that failure stands.
11. Result
Delivered through PR #2464 and merged as the first half of the story.
What shipped: the twelve quote-type models declare their coordinate dimensions
and the keys the old point corresponds to; the identifier holds a typed member
per declared coordinate and a smile dimension of its own; the loader reads and
cross-checks both tables; three surfaces that had no quote type of their own
gained one; and test_oresmd_coordinates.py is the gate.
Verified before the merge: full tree build, full ctest 89 of 89, database recreated, 24 services started, codegen suite 1,049, oresmd core suite 1,940 assertions in 420 cases.
The merge was forced. GitHub was slow, the run was cancelled at the author's request, and the two checks still running (Codegen drift, Codegen sweep) were cancelled rather than waited for. The other twelve had passed. The local verification above covers what no check covers.