Task: Round trip the curve configuration

Curves are referenced by name from five of the analytics documents, so a curve that does not round trip breaks every document that names it.

Field	Value
State	DONE
Parent story	Model the ORE report configuration as entities
Now	Nothing.
Waiting on	Nothing.
Next	Nothing.
Last touched	2026-09-27

• Every curveconfig.xml in the corpus imports, exports and matches.
• A curve resolves its conventions, calendar, day counter and index by foreign key, not by string.
• The segment types are a seeded lookup, so a new segment type is a row and not a struct.

Measured on 2026-10-03 at fe244a3331 before any change:

• 89 corpus documents have the root CurveConfiguration, with 22 section kinds. An entry carries from 2 to 30 distinct child elements, by section. Yield curves are the largest: 1193 entries and 10 segment kinds in use.
• The binding already holds curveconfiguration with a loader and saver. No binding work is needed.
• The binding groups a yield curve's segments by kind, one vector per kind, so the order of segments across kinds is lost before a mapper runs. A mapper can keep only the order within a kind.
• The curve tables on main (curve_definition, curve_section, curve_segment, curve_quote) store each entry's common settings as text and the rest as a semicolon and pipe encoded extras column. Nothing reads, writes or seeds them. That shape fails this task's acceptance and the story's (no name and value pairs, references by foreign key).
• The day counters the corpus writes do not match the seeded day count codes: Actual/365 (Fixed), ACT, Actual/360, ActActISDA and 30E/360E have no row, 657 values in all. The schema's dayCounter enum lists every spelling ORE accepts, about seventy.
• Curve segments name 219 distinct convention ids. Conventions are one table per category, each keyed by its text id, with no shared identity table.

Shape:

• A curve entry is a header row, curve_definition (section, CurveId, description, position), which is the one identity other documents refer to, plus a detail row per section family with that family's settings as typed columns. Nested lists (segments, quotes, volatility configurations) are child rows with typed columns and a position within their kind.
• An ORE enumeration becomes a seeded lookup derived from the schema by a script, keyed by ORE's own spelling, so a column keeps the exact spelling and resolves by foreign key. A day counter spelling also names its canonical day count code.
• A convention reference is checked against every convention table by one validation function, because there is no shared convention identity. An index reference resolves against the floating index types.

Delivery, one PR per step, each with its own check:

1. Lookups. The curve sections, the segment kinds and types, and the ORE day counter spellings, seeded by a script that reads the schema. Check: the seed counts equal the schema's enumerations.
2. Yield curves. Replace the generic columns with the header and the yield curve detail, segments and quotes; the mapper for the YieldCurves section. Check: every YieldCurves section in the corpus round trips in memory and through the database.
3. The other curve sections (default, equity, inflation, commodity, securities, FX spots, intraday power). Same check per section.
4. The volatility and correlation sections and the report configuration. Check: every whole document in the corpus round trips.

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What	Value	Command	Commit
Corpus documents with root CurveConfiguration	89	Python ElementTree walk of external/ore/examples/**/*.xml counting roots	fe244a3331
Entry sections in the schema (excluding ReportConfiguration)	19	children of curveconfiguration in external/ore/xsd/curveconfig.xsd	fe244a3331
Yield curve segment types in the schema	21	Type enumerations and fixed values under segmentsType in curveconfig.xsd	fe244a3331
Day counter spellings in the schema	71	dayCounter enumeration in external/ore/xsd/ore_types.xsd	fe244a3331
Corpus day counter values with no day_count_fraction_type code	657 (Actual/365 (Fixed) 533, ACT 113, Actual/360 8, ActActISDA 2, 30E/360E 1)	corpus walk of DayCounter, YieldCurveDayCounter, DayCountConvention against the seeded codes	fe244a3331
Distinct convention ids named by curve segments	219	corpus walk of Conventions elements	fe244a3331

Step 2, decided while building it:

• A document is a curve_configuration row, as today's market is a todays_market_config row, so a tenant may hold several and a report reaches one through its reporting configuration.
• Sixteen of the nineteen sections appear empty somewhere in the corpus, so the sections a document writes are rows of curve_configuration_section, with or without entries.
• BootstrapConfig is shared by five section types, so it is one table keyed to the curve entry. A row exists exactly when the element does; fourteen corpus yield curves write it empty.
• Every corpus Report on a yield curve is present and empty (121), so yield_curve.has_report holds its presence.
• A segment stores only its type; the type fixes the segment element. The mapper keeps the same table as the seed, and the seed test ties the seed to the schema.
• ORE enumerations other than the day counter and the segment type – the ORE boolean, the interpolation method and variable, the extrapolation method – are held as the text the schema spells, and the binding refuses an unknown value on export. Only the two the acceptance names are lookups.
• A convention is checked by ores_refdata_validate_convention_id_fn, which looks in all twenty-four convention tables keyed by id. FX conventions are held per currency pair and their id is not stored, so only the id the conventions export writes, BASE-QUOTE-FX-CONVENTIONS, resolves. 600 or more corpus FX ids have another form; Keep each FX convention's ORE id fixes that.
• Four index names the corpus yield curves use were not seeded floating index types: EUR-EONIA, EUR-ESTER, GBP-LIBOR-3M and USD-LIBOR-1M. They are seeded now.

Step 3a, decided while building it:

• Step 3 is split by section family: 3a the equity curves, securities, FX spots and intraday power curves; 3b inflation; 3c default curves, whose nested configurations need child rows; 3d commodity.
• A calendar is an expression, not a name: TARGET,UK and JoinHolidays(TARGET, US settlement) are both one calendar. The schema lists 471 names and allows two open forms, any four-letter exchange code and any CUSTOM_ name. calendar_name holds the 471, and ores_refdata_validate_calendar_fn splits an expression and checks each part against them or against the two forms. The refdata calendar table's 61 codes miss most of the corpus (XNYS, USD, weekends only), so it is not the target.
• 101 corpus equity curves write an empty Calendar and three an empty Quotes; the empty calendar is stored as empty text and the quotes element's presence as equity_curve.has_quotes.
• intraday_power_curve.convention is nullable in the table because the generated event test cannot seed a convention; the mapper refuses to export an entry without one.

Step 3b, decided while building it:

• An inflation curve's settings are inflation_curve and its seasonality factors inflation_seasonality_factor; its quotes sit on the entry, as an equity curve's do. has_quotes and has_seasonality hold the presence of the two optional elements.
• The schema allows inflation Segments and no corpus document writes them. The mapper refuses an entry that does, rather than lose it; modelling them waits for a document that needs them.

Step 3c, decided while building it:

• A default curve writes its settings inline (144 corpus curves) or as a list of prioritised Configurations (189, nine with two), never both. One table, default_curve_configuration, holds both forms: an inline curve is one row with is_inline set. default_curve holds only the currency.
• A configuration's quotes and BootstrapConfig are rows of curve_quote and curve_bootstrap_config naming the configuration, through a new nullable default_curve_configuration_id on each.
• Pillars and States are comma lists in ORE's own text and are held as written. SourceCurves and SwitchDates appear in no corpus document and are refused, as inflation segments are.

Step 3d, decided while building it:

• A commodity curve's BasisConfiguration is one per curve, so its settings are basis_ columns on commodity_curve; its PriceSegments are rows of commodity_price_segment. Each appears in one corpus document.
• A quote's owner can now also be a price segment, and a parent that writes more than one quote list names the list: curve_quote.quote_list is BasisQuotes, OffPeakQuotes or PeakQuotes, and null for the main Quotes. That is the fourth way a quote attaches, and step 4 should decide whether a single owner reference reads better than four nullable columns.

Step 4a, decided while building it:

• Step 4 is split by surface shape: 4a the flat sections (FX volatilities, yield volatilities, base correlations, correlations), whose entries have no child lists; 4b CDS and inflation cap and floor volatilities; 4c swaption and cap and floor volatilities; 4d equity, commodity and bond future volatilities and the global ReportConfiguration.
• Ten entry types carry the same Report element, so it is one table, curve_report_configuration, keyed to the entry's curve_definition. A row exists exactly when the entry writes the element. No corpus entry writes its own Report; the database test adds one to an FX volatility.
• ParametricSmileConfiguration on an FX volatility, and RecoveryGrid, RecoveryProbabilities and QuoteTypes on a base correlation, appear in no corpus document and are refused.
• The FX smile interpolation is free text in the schema, not an enumeration, and is held as written.

Step 4b, decided while building it:

• CDS, equity and commodity volatilities all name one or more volatility configurations (Constant, Curve, StrikeSurface, ProxySurface and, for equity and commodity, more surface kinds), each with an optional priority. They share one table, curve_volatility_config, with a kind column and the columns each kind uses. Only StrikeSurface is modelled now, because it is the only kind a corpus CDS volatility writes; steps 4c and 4d add the other kinds with the sections that write them.
• A CDS volatility's Terms are rows of cds_volatility_term, with their position. PriceInfo, an empty Terms element and the unmodelled surface kinds are refused.
• An inflation cap and floor volatility is flat. Its Report and BootstrapConfig reuse curve_report_configuration and curve_bootstrap_config.
• The operator asked whether these tables hold curves or curve configurations. They hold configurations only; the naming question waits in Discuss the _config suffix for the curve configuration tables.

Step 4c, decided while building it:

• Swaption and cap and floor volatilities are flat. A ProxyConfig is one per entry, so its fields are proxy_ columns with has_proxy_config holding its presence. The corpus writes two swaption and four cap and floor proxies.
• FX, swaption and cap and floor volatilities write a ParametricSmileConfiguration on the entry. It is one row of curve_parametric_smile per entry, with its parameters as ordered rows of curve_parametric_smile_parameter. That lifts the step 4a refusal for FX. A smile inside a strike surface stays refused, because it would hang off the volatility configuration, not the entry, and no corpus document writes one.
• The cap and floor schema has both UseEffeciveVolatility, a misspelling kept for old documents, and UseEffectiveVolatility. Each has a column, so either round trips.

Step 4d, decided while building it:

• Step 4 gains a fifth part. 4d is the equity, commodity and bond future volatilities; 4e is the global ReportConfiguration, the whole-document corpus round trip, and closing this task.
• An entry writes each volatility configuration kind either directly or inside a VolatilityConfig element, which holds at most one of each kind. curve_volatility_config gains is_wrapped and the columns of the kinds the corpus writes: Constant, Curve, DeltaSurface and ProxySurface. MoneynessSurface, ApoFutureSurface, OneDimSolverConfig and a smile inside a surface appear in no corpus document and are refused.
• A Curve kind's quotes are rows of curve_quote on the entry, in the list Curve or VolatilityConfig/Curve, which reuses quote_list instead of adding a fifth owner column to curve_quote. The owner question noted in step 3d stays open.
• No corpus document writes BondFutureVolatilities. It is modelled from the schema, because it is flat with a strike surface and costs little, so a document that writes it is held rather than refused.
• Every section is now modelled, so the test of an unmodelled section is replaced by refusals of a solver configuration and an APO future surface.

Step 4e, decided while building it:

• The document's ReportConfiguration gives report settings per curve family. Each family element is one row of curve_global_report, keyed to the document, with has_report and the columns of curve_report_configuration. The yield curve family's report writes only PillarDates, which those columns already hold. A ReportConfiguration that writes no family is refused, because rows cannot record it.
• An entry's Report and a family's Report share one pair of mapper functions over the shared columns.
• The corpus walk now compares every whole document, with nothing projected away.

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario	State	Notes

PR	Title
#2566	[refdata] Map the curve report configuration and close the round trip
#2564	[refdata] Map the equity, commodity and bond future volatility sections
#2562	[refdata] Map the swaption and cap and floor volatility sections
#2560	[refdata] Map the CDS and inflation cap and floor volatility sections
#2558	[refdata] Map the flat volatility and correlation sections
#2557	[refdata] Map the commodity curve section
#2556	[refdata] Map the default curve section
#2555	[refdata] Map the inflation curve section
#2554	[refdata] Map the equity, security, FX spot and power curve sections
#2551	[refdata] Hold yield curves as typed rows and map them in both directions
#2536	[refdata] Seed the curve configuration vocabulary from the schemas

#	Comment summary	File	Decision	Notes
1	PR 2536: schema regexes depend on attribute order and fail without saying why	repository_curve_vocabulary_seed_tests.cpp	Accept	Each size check now states the pattern's assumption.
2	PR 2536: nothing checks that a new tenant receives the vocabularies	iam_tenant_provisioner_create.sql	No change	No test fixture counts provisioned lookups. Verified instead: a tenant provisioned in a rolled-back transaction received 71, 19 and 21 rows.
3	PR 2536: new permission rows not column-aligned	iam_permissions_populate.sql	Accept	Aligned.
4	PR 2536: note that day counters are not unique by meaning	ores.refdata.day_counter.org	No change	The model already states that ORE accepts several spellings of one day counter.

Every curveconfig.xml in the vendored corpus imports to typed rows and exports back to a document that matches the original, with nothing projected away: the corpus walk in xml_curve_configuration_mapper_roundtrip_tests.cpp compares whole documents and passed on every file in the 2026-10-03 run (ctest 90/90 on the step 4e branch).

Shipped in ten PRs, listed above, one per step:

• Step 1: the curve section, segment type and ORE day counter vocabularies, seeded by a script that reads the ORE schemas.
• Step 2: the document, section and curve header tables, and yield curves with their bootstrap settings, segments and quotes; conventions are checked against every convention table by ores_refdata_validate_convention_id_fn.
• Steps 3a to 3d: equity curves, securities, FX spots, intraday power curves, inflation curves and seasonality, default curves and their configurations, and commodity curves with price segments; calendars are checked by ores_refdata_validate_calendar_fn against the 471 names the schema lists.
• Steps 4a to 4e: every volatility and correlation section, the shared entry Report, the shared volatility configurations and parametric smile, and the document's ReportConfiguration.

Acceptance met: every corpus document round trips; a curve resolves its day counter, segment type and index by foreign key and its conventions and calendar through validation functions, so a name the database does not hold is refused; segment types are a seeded lookup.

Each step also round trips at least one corpus document through the database. Elements no corpus document writes (for example PriceInfo, MoneynessSurface, ApoFutureSurface, OneDimSolverConfig) are refused rather than dropped, and are listed in each step's notes above.

Left open, each with its own task: the owner columns of curve_quote (noted in step 3d), keeping each FX convention's ORE id, and the _config suffix question for these tables.