Analysis: where ores.marketdata stands

check_component_drift.py --component marketdata --dry-run exits 1:

Measure	Count
Files that would change	82
Files that would be created	14
Total	96

By area:

Area	Files	Nature of the difference
ores.sql/create/marketdata	10	table shapes and notify triggers
core/src/repository, core/include/.../repository	16	generated repository/mapper drift
api/include/.../domain	7	equality operator, doc comments
core/include/.../service, core/src/service	12	service drift
core/include/.../messaging, core/src/messaging	12	handler and registrar drift
api/include/.../messaging	6	the six generated protocol headers
service/src/messaging	6	event registrar drift
web wire-protocol (ts)	8	Get* to List* rename, domain twins
core/src/presentation	2	history field mappers
api/src/generators	2	test-data generators
core/tests	1	eventing integration test
ores.shell/marketdata	6	per-entity command units (new)
doc/recipes/shell	2	recipe documents (new)

check_model_drift.py --summary exits 1. See the model survey for the per-entity binding detail.

check_protocol_twin_coverage.py, check_handler_permissions.py and check_populate_references.py only judge components listed in COMPONENTS_UNDER_TEST, and marketdata is not listed. Their current green output says nothing about this component; item V05 is a claim about the state after it joins, and must be re-run then.

Gate	Result
regenerate_cmake_component_files.py --all --check	all four modules stale (G06), part of the tree-wide staleness
validate_docs.sh	passes for marketdata, but only because MISSING_PUML ores.marketdata.client is registered in docs_exceptions.txt

compass build cannot run in this environment: its systemd-run --user --scope wrapper fails with "Failed to connect to user scope bus". The same targets build through cmake --build --preset linux-clang-debug-make directly.

All four test targets build. With the test database up and .env sourced:

Suite	Result
ores.marketdata.api.tests	pass, 71 assertions in 14 test cases
ores.marketdata.client.tests	pass, 52 assertions in 30 test cases
ores.marketdata.service.tests	pass, 164 assertions in 24 test cases
ores.marketdata.core.tests	203 of 204 pass, 951 of 952 assertions

The single failure is series_classification_rule_eventing_integration_tests.cpp:184 – the known unwired event registrar (../../../../product_backlog/next/ores_marketdata_generates_register_series_classifi.org), confirmed here rather than taken on trust. Nothing else in the component is red.

So the component compiles and its suites are green apart from a defect that is already captured. That matters for scoping: this is not a rescue, it is a clean-up.

compass build itself cannot run in this environment (see above), and compass db recreate and the fleet are a prerequisite for V02 and V03 rather than a formality.

Metatype	Count	Files	Variability
ores.codegen.entity	6	feed_binding, market_fixing, market_observation, market_series, observation_lineage, series_classification_rule	yes – must bind
ores.codegen.junction	1	market_series_asset_class_junction	yes – must bind
ores.codegen.oresmd_quote_type	8	model.org (manifest) + the seven asset-class specs	no – carries no variability
ores.codegen.component	4	the four component_overview.org files	no

The oresmd_quote_type metatype is the one the story flagged: a metatype of its own, generating the identifier, requirement, parser, resolver and projection families rather than an entity stack. It classifies under B02 and is excluded from the profile binding, as the loader never reads :profile: for it.

Item	Result	Finding
M01	PASS	No legacy JSON, no lookup_entity, no retired table type; all 15 entity/junction models load through the current loader
M02	FAIL	Five of seven models carry no :profile:
M03	FAIL (minor)	The junction restates :has_tenant_id: true, which tenant-scoped-junction already fixes; check_model_drift.py does not detect a same-value restatement
M04	PASS	Every stated feature sits in the namespace the loader reads
M05	PASS	All seven describe one real table; none is joined or denormalised; all seven tables carry valid_from=/=valid_to
M07	PASS	Singular named, key column types real, tenant-scoped row-level security in the generated SQL; the read_only=/=client_read_only=/=party_id_is_target flags are genuinely not applicable
M08	PASS	The one custom type, std::chrono::year_month_day, is bound to <chrono>; SQL spellings are canonical; enabled is a boolean
M09	FAIL	feed_binding.org states ten legacy Presentation properties naming retired message types – get_feed_bindings_request and save_feed_binding_request – where the canonical verbs are get, get_many, list, list_by, put, put_many, delete, delete_many
M10	FAIL	No component-root overview and no group model, so no root CMakeLists.txt is generated

Bindings that should be declared and are not:

Model	Profile it should bind	Why
market_observation	readonly-timeseries	the profile's reference entity is market_observation
market_fixing	readonly-timeseries	same shape
market_series	uuid-surrogate-lookup	UUID primary key, tenant-scoped, no workspace and no parent
observation_lineage	uuid-surrogate-lookup	same
series_classification_rule	simple-lookup	already bound
market_series_asset_class	tenant-scoped-junction	already bound

feed_binding is a genuine no-match: it needs workspace=true, a UUID primary key and a standard presentation tier at once, and no profile combines those three without also requiring a parent or an explorer. This is the same gap already recorded for reporting.report_definition. It must be recorded as a no-match with its actual feature values and the reason, which is what M02 requires – not force-bound to the nearest profile.

The committed protocol headers still use the retired get_* CRUD naming where regeneration emits list_* and =put_*. That is drift, not a model defect: the models already declare the canonical verbs, which is why the drift check wants to rewrite the headers.

Two model families live under projects/ores.marketdata/modeling/:

• Seven ores.codegen.entity / junction models under modeling/ – feed_binding, market_fixing, market_observation, market_series, market_series_asset_class (junction), observation_lineage, series_classification_rule.
• Eight ores.codegen.oresmd_quote_type models under modeling/oresmd/ – one per asset class (ir, credit, equity, commodity, fx, inflation, correlation) plus model.org, the batch manifest. This is a metatype of its own: it carries no variability and produces no entity stack. It generates oresmd_enums, oresmd_identifiers, oresmd_requirements, the parser, the resolver, the projections and four test files.

The ORE series-type universe is 43 types, seeded in projects/ores.sql/populate/ore/ore_series_key_shapes_populate.sql. By type count oresmd projects 30 of the 43. The types from_ore_key() deliberately rejects are the option, capfloor, bond and fixing families: BOND, BOND_OPTION, FX_OPTION, CAPFLOOR, INDEX_CDS_OPTION, EQUITY_OPTION, COMMODITY_OPTION, ZC_INFLATIONCAPFLOOR, YY_INFLATIONCAPFLOOR, FIXING, plus SHAPE_PROFILE, RATING and GENERIC.

Type count flatters it. Measured against the whole committed corpus – 100 fixtures, 477,013 key lines, 107,948 distinct keys – oresmd represents 26.0% of the data: 124,194 lines project to a URI and 352,819 do not.

The rejects split in two, and the second half is the surprise.

Half one: 232,545 lines (48.8%) belong to the 13 of 43 registry types that have no oresmd coverage at all. EQUITY_OPTION 119,137, CAPFLOOR 78,270, ZC_INFLATIONCAPFLOOR 14,058, FX_OPTION 10,735, YY_INFLATIONCAPFLOOR 4,366, INDEX_CDS_OPTION 3,026, COMMODITY_OPTION 2,346, SHAPE_PROFILE 171, BOND 154, BOND_OPTION 154, RATING 128, and GENERIC and FIXING with no corpus lines. This is the gap the abandoned story already recorded: oresmd models seven asset classes and ORE carries more.

Half two: quote types oresmd does claim cannot hold the keys ORE writes. Ten of the 29 oresmd quote types have zero corpus coverage and six are partial:

ORE type	Lines projected	Why
BASIS_SWAP	0 / 5049	real shape is long_tenor/short_tenor/ccy/maturity, not ccy/index/tenor/point
CC_BASIS_SWAP	0 / 2720	four qualifier segments, not three
FRA	0 / 1678	index and tenor order differs from the model
ZERO	0 / 498	shape differs
MM	10 / 1436 (2.6%)	shape differs
CC_FIX_FLOAT_SWAP	0 / 106	four qualifier segments
IMM_FRA	0 / 108	shape differs
MM_FUTURE, OI_FUTURE	0 / 109	real shape is ccy/contract_month/exchange:code/tenor
CORRELATION	0 / 61	real shape is operand1/operand2/expiry/strike, six segments
CPR	0 / 1	real key is CPR/RATE/ISIN:...
IR_SWAP	84.0%	7-segment keys rejected
SWAPTION	37.7%	7/8-segment smile keys, 91,689 lines
DISCOUNT	80.0%	DISCOUNT/<CCY>/<CCY> rejected by an off-by-one
CDS	19.4%	shape differs
RECOVERY_RATE	54.6%	shape differs

The oresmd quote-type models were derived from ORE's documented shapes, not from the corpus. Where the two disagree, the corpus is the authority and the model is the defect.

The registry fallback is healthy. decompose followed by reconstruct_key is verbatim-lossless for 100% of the corpus, and neither the unknown-type nor the short-key fallback fires on a single line. So the shape table itself is sound; the losses are in the projection layer above it.

Four further findings from the same walk:

• A silent rewrite on the live import path. The IR 6-segment path discards the settlement segment and rebuilds it as the hardcoded "2D". The corpus carries 0D on 1,338 six-segment keys, and every one of them is imported as 2D; a further 2,221 seven-segment keys are rejected outright. This is not an export problem: the stored row already differs from the file.
• A derivation, not a guard, costs 9,075 lines. from_ir_discount treats the key's fourth segment as the currency with a tenor appended, so it strips the currency back off to recover a tenor. The segment is really the name curveconfig.xml gives the curve – the corpus carries USD3M beside USD-SOFR-3M, USD-FedFunds, USD-DUMMY and the bare currency – so a curve named after its own currency leaves nothing behind and the guard rejects the key. It is not the <=/-vs-<=< comparison the first pass called it: relaxing that guard would leave =id.tenor unset and the forward dereferences it. The other 36,948 keys project while carrying a tenor such as -SOFR-3M, and round-trip cleanly, which is why the round-trip measurement never showed this.
• A single stray comma aborts a whole file. The tokeniser needs two commas and a CSV line with exactly one loses the entire file – 19,170 lines across the large CSVs.
• Nine fixtures cannot be parsed at all: seven on date format (DD-MM-YYYY and DD.MM.YY, 23,497 lines) and two on that malformed CSV line. The two named fx_localvol/market.txt fixtures are among the seven.

FIXING deserves its own note: fixings are imported and stored as series series_type"FIXING", and =from_ore_key("FIXING/RATE/EUR-EONIA") returns nullopt. There is no type=fixing in either direction, and no type=curve either. The fixing key space cannot round-trip through oresmd at all today.

The measurements above are the baseline at origin/main 8a16c7a4f0 and stay as written. Six of the mis-modelled shapes are fixed:

Type	Lines	What was wrong	Fixed by
FRA	2,113	declared with an index; ORE writes ccy/start/length	no new field
IMM_FRA	160	same	no new field
MM	1,609	only the indexed form projected; ORE writes the index-less form for 94% of its lines	dispatch on segment count
BASIS_SWAP	5,994	declared ccy/index/tenor/point; ORE writes two tenors and no index	second_tenor
CC_BASIS_SWAP	3,824	declared one currency and one tenor; ORE writes two of each	second_ccy
CC_FIX_FLOAT_SWAP	106	same	second_ccy
MM_FUTURE	319	declared ccy/index/tenor/point; ORE writes ccy/contract-month/contract-code/tenor	contract_month, contract_code
OI_FUTURE	620	same	contract_month, contract_code
DISCOUNT	9,075	the curve segment is an opaque declared name, not ccy+tenor; the derivation rejected every currency-named curve	the curve kept verbatim, the trailing segment read as the tenor
CDS	1,717	ORE writes a seven-segment key carrying the credit derivatives' restructuring clause between the currency and the tenor; the model accepted only six	the clause as a third point part
RECOVERY_RATE	134	the same clause, on the keys that name one	same
IR_SWAP	2,323	two causes: the seven-segment form naming the index the swap is written against (ccy/index/settle/tenor/maturity) was rejected outright, and the index token ESTER has no family, because index_family spells it estr	the named-index form, and the producer's spelling carried in index_spelling
MM	42	the same ESTER spelling	same
SWAPTION	99,977	two optional qualifiers the model did not carry: the index the surface is quoted against, and the Smile convention marker whose shift takes the value slot, which together make the family six, seven or eight segments	the index, and delta_type for the marker
INDEX_CDS_OPTION	3,026	a volatility surface in an asset class that had no vol path at all: quote_key_credit refused type=vol outright, and there was no inverse	a quote-type row marked surface, a vol branch in the credit forward, and from_credit_index_option
COMMODITY_OPTION	2,346	the same gap in the commodity class, whose forward also refused type=vol	the same three changes, plus the equity option's three shapes
BASIS_SWAP	426	a seventh segment names the basis itself between the currency and the maturity (SOFR_FedFunds, LIBOR_PRIME); those tokens are not index families, so the model had nowhere to put them	the token carried in index_spelling, the field that already holds a producer's spelling
BOND	157	a whole asset class was missing: the key is TYPE/METRIC/SECURITY, with no currency, no tenor and no point	a new security asset class
RECOVERY_RATE	48	the same security-keyed shape, under a type that also has credit-keyed forms	the same class, one more quote type
SHAPE_PROFILE	171	a profile-keyed class was missing: the key is TYPE/METRIC/PROFILE/DATE/SECOND/PERIOD with an optional DST flag, and no class carried a time-of-day coordinate	a new shape_profile asset class
RATING	128	a provider-keyed class was missing: the key is TYPE/METRIC/PROVIDER/FROM/TO, with a provider-only form the corpus also writes	a new rating asset class

That is 13,806 corpus lines brought in by the first six, 939 by the two futures, 9,075 by the discount re-modelling, 1,851 by the credit restructuring clause, 2,365 by the swap's named-index form and the index spelling, 99,977 by the swaption's two qualifiers, 3,026 by the credit vol path, 2,346 by the commodity one, 426 by the basis swap's named form, 205 by the security class, 171 by the shape-profile one and 128 by the rating one. The shape record is down to CPR alone, and 99.99% of the corpus is named.

The corpus is named in full. 485,606 of 485,606 keys project to an oresmd identifier and every one of them reads back as itself, so there is no coverage gap left to close or to record.

PR 2 closed the last two by the records rather than by the shapes, and neither was a shape defect:

• CPR moved from commodity to the security class. It was the last entry in the shape-mismatch record and the only entry in the test-coverage exemption list, and both are now empty.
• F-TIIE joined index_family as an overnight benchmark, which meant widening both CHECK constraints on ores_synthetic_ir_curve_generation_configs_tbl and adding a one-shot migration. The 46 MXN keys it unblocked were the last ones.

Two further index tokens needed no family at all: a basis swap names the basis itself and a swap may name a curve marker rather than a benchmark. Both ride in index_spelling, the field that already carries a producer's spelling, and are emitted whole.

Neither was visible until a new spec file and a multi-member last spec existed. Both are fixed at source, and both are the kind that fail silently:

• Umbrella outputs were rendered once per spec file. The oresmd facet's manifest is meant to be the only generation entry point, and its loader reads every sibling spec itself. But discovery picks up any file whose frontmatter declares a codegen model type, and every spec declares the same type as the manifest, so each spec was generated separately into the same eight output files. Whichever path sorted last won, and model.org had been winning only because it sorts after every spec. security_quote_type.org sorts after it, so the generated headers collapsed to the new class alone. Discovery now treats a directory with a batch manifest as generated through the manifest alone.
• The outer last marker suppressed the inner one. A mustache section resolves a key it does not find in its own context from the enclosing one. The specs list was marked with last to keep the variant argument list from rendering a trailing comma, and the quote-type loop inside each spec also used {{^last}} for its own commas – so every member of the last spec lost its comma and the enum became one long line. It never showed because the last spec held a single member. The outer lists are now marked last_spec.

The first is the more dangerous: a new spec file could replace the whole generated set, and the failure surfaced only because the result would not compile. The second produced syntactically broken output for a spec that merely sorted last.

Every asset class that declares a volatility field now reaches a projection, so the gate's unwired list is empty – the first exception list this work has emptied rather than shortened.

Each of the remaining shapes needs the identifier grammar extended, and the oresmd parser is hand-written per asset class rather than table-driven – its generic branch handles the entity, a currency, the type, the quote type and a point, and nothing else – so a new field is a coordinated change across the model, the parser, the URI order and the projections.

MM_FUTURE and OI_FUTURE were recorded here as needing a registry correction first, because the shape table describes the two with different qualifier depths, so which segments identify the series looked unsettled. Reading the corpus settles it. Both are 6-segment and identical in shape – the corpus writes MM_FUTURE/PRICE/EUR/2024-04/XICE:FEI/3M and OI_FUTURE/PRICE/USD/2024-01/XCME:SRA/3M – for all 939 keys, with no 5-segment form anywhere in the corpus.

The contradiction was in the documents. The market data catalogue carried the abbreviated key (/USD/2025-09/3M, the contract code dropped), and the model had copied the registry's mislabelled CCY/INDEX/TENOR/POINT, which puts the contract month in a field called index and the contract code in a field called tenor. The catalogue examples are corrected, and the model now declares CCY/CONTRACT_MONTH/CONTRACT_CODE/TENOR. The contract_code is kept verbatim: ORE writes XICE:FEI and XCME:SRA, and the token names the contract rather than decomposing into fields we can check.

The URI layer is not only unwired; it has its own defects. model is an accepted query key on all seven asset classes and is discarded for six. parse_credit does not type-gate quote. The ir entity is never validated as a currency. And split_market_series_key keeps no point, contradicting the four-field key the registry defines – which is the function the ingest loop relies on.

Symbol	Production callers
oresmd_parser	none
oresmd_resolver	none
oresmd_projections::to_index_name	none
oresmd_projections::to_curve_key	none
oresmd_projections::to_quote_key	none
oresmd_projections::from_ore_key	none
oresmd_projections::split_market_series_key	feed_ingest_loop.cpp, ores.synthetic (fx_spot_feed, feed_controller)

Test-only, by the standard's B06 rule, is dead code. This is the one place where that rule should not be applied as a deletion: the layer is the decided foundation for the target state, so the finding is "wire it", recorded as an exception with the plan attached.

Sprint 25's adopt-oresmd-native-market-data-identity story (E62808C1-D692-4877-835D-3A512A9F4160) set the target state and was abandoned at sprint close on 2026-09-23:

Task	State
Analysis: identity inventory, consumers, fields-vs-URI	DONE
Inverse projection: ORE key to oresmd identifier	DONE
Identity core: market_series and feed_binding store URIs	ABANDONED
Observation identity: point_id carries the oresmd point	ABANDONED
Generation configs move to URI s	ABANDONED
The NATS wire carries URI s	ABANDONED
ores.shell talks oresmd natively	ABANDONED
Delete legacy ORE representations	ABANDONED
Design: the market data data model on oresmd	ABANDONED

The story's own * Result reads: "Abandoned at sprint 25 close, as part of the reset to one approach for sprint 26." The reset chose the component-clean-standard programme, one component at a time. The two are not in conflict – the clean standard is what makes the cutover safe – but the marketdata clean-up is the natural place to resume it.

• One oresmd_uri column per identity-bearing entity. Component fields are dropped. The hybrid is rejected: two writable copies of one identity drift and no constraint can detect it.
• market_observation.point_id stays but is repurposed to carry the serialised oresmd point; the "SPOT" sentinel dies and a scalar series gets a null point.
• Classification (asset_class, series_subclass, is_scalar) is derived, not identity.
• Validation moves from SQL check constraints into the oresmd layer.
• ORE text and CSV stay as boundary formats. The projection boundary is the ORE-format edge only. The NATS wire becomes oresmd-native, not a projection.
• Six sequenced steps, each keeping the system buildable: inverse projection (done), identity core, observation rows, generation configs, the wire, the shell, then deletion.

origin/feature/identity-core-series-and-bindings holds six commits implementing step 1 (77 files, +1433/-1299). Thirty-six of those files are ores.qt/ sources that no longer exist on main, so the branch cannot be merged as it stands. The other ~41 – the ores.marketdata, ores.sql and ores.synthetic halves, plus the from_index_name inverse projection – still apply and are worth reviving rather than rewriting.

ores.shell marketdata import reads the files client-side, sends their content over marketdata.v1.import, and import_service::import():

1. parse_market_data() / parse_fixings() (ores.ore.core/market/) with a series_key_registry read once per batch.
2. The FX key_swapped correction against ores.refdata's currency pairs, best-effort.
3. Find-or-create market_series by (series_type, metric, qualifier), classified by series_classifier into subclass and asset classes.
4. Bulk-write market_observation rows (value as text, point_id defaulted from the registry) and market_fixing rows.

serialize_market_data() / serialize_fixings() are the only code that writes ORE text. Both call sites are in ores.ore/core/tests/market_market_data_roundtrip_tests.cpp. There is no command, handler or service anywhere that turns stored rows back into an ORE file.

A round trip ORE file -> stored rows -> ORE file cannot be lossless today. In rough order of severity:

1. No export exists. 100% of stored rows. The serializer has no production caller, there is no rows -> market_datum mapper and no ORE-text protocol struct. Everything below is what a new exporter must also solve.
2. 74.0% of the corpus cannot become an oresmd URI at all, and the ingest path never calls the projection library. This is a coverage problem, not a storage one – see the two halves above.
3. A single-comma CSV line aborts its whole file. The tokeniser requires two commas, so one malformed line costs 19,170 lines across the large CSVs.
4. The verbatim key is not stored. market_datum keeps key verbatim; import_service decomposes it into series_type=/=metric=/=qualifier=/=point_id and discards the string. This is less dangerous than it looks, because decompose then reconstruct_key is verbatim-lossless for 100% of the corpus – but it is still a reconstruction where a copy would do, and it is the reason the point must be re-derived (item 6).
5. The point is defaulted on ingest with no inverse. A key with no point of its own is stored with default_point_for(series_type). An exporter that does not reproduce the registry's has_point_dimension rule emits FX/RATE/EUR/USD/SPOT for FX/RATE/EUR/USD. The corpus already carries 1,365 FX, 401 RECOVERY_RATE, 154 BOND, 105 EQUITY, 43 COMMODITY and 2 CPR lines in this shape.
6. A one-character projection bug rejects 9,075 lines. from_ir_discount refuses a curve_id equal to the currency that the forward projection can emit, because oresmd_projections.cpp:644 compares with "<"= where it means "<".
7. The IR par-rate settlement segment is destroyed. The 6-segment path rebuilds it as the hardcoded "2D", so the 1,338 corpus keys carrying 0D are imported as 2D. A further 2,221 seven-segment keys are rejected. This is the most serious of the fidelity findings because the stored row already differs from the file – it is not something an exporter can repair.
8. The reversal is a rewrite, not a round trip. The FX convention correction mutates the qualifier when the pair is reversed relative to refdata. The value is untouched, but the key is not.
9. No provenance at all. market_observation.source is the caller's --source string. There is no source file, line number, import id, ingest time or raw line, so no row can be traced to its origin.
10. source is not in the unique key. Re-importing under a new tag supersedes the old tag rather than adding a second observation.
11. Duplicates are resolved destructively at parse time. The last occurrence of a (date, key) pair wins and the earlier one is gone; duplicate_policy only chooses whether the loss is a warning or an error, and the strict mode is unreachable from the CLI.
12. Comments, blank lines, delimiter, spacing, date format and file order are dropped. The parser discards them by design and the tables have no column for them, so a byte-identical round trip is impossible without new columns. Both YYYYMMDD and YYYY-MM-DD parse; the market serializer writes YYYYMMDD and the fixings serializer writes YYYY-MM-DD.
13. Shape-table row descriptions disagree with the metrics oresmd emits. Latent today, because nothing validates a metric.

Items 1, 5, 6 and 7 decide whether "no data loss" is achievable at all. Item 7 is the only one where the damage is already done on the ingest path; 5 and 6 are properties of the projection layer, and 1 is the missing code.

The repository commits a real ORE corpus at external/ore/examples/. A fixture inventory of it counts 100 market-data fixtures (105 names matching market*/marketdata*, plus three dated MD_*.csv, less eight derived dumps and logs) and 70 fixings files. It is tracked in git, not downloaded at build time.

Ninety-one of the hundred import cleanly: 477,013 key lines, 107,948 distinct keys. Nine abort – seven on date format and two on one malformed CSV line. The narrower glob market*.txt alone is 64 files and 250,748 lines; the wider inventory is the one to quote.

One test walks it: classification_series_classifier_tests.cpp resolves external/ore/examples through ores::testing::project_root::resolve() and censuses the distinct series keys to check the classifier. Forty-eight of the 170 fixtures are reached by the 27 parser cases and 48 round-trip cases. Nothing tests a byte-identical market-data round trip – by design, the assertions are field-by-field – and no test links series_key_registry to the projections.

What no test does is walk the corpus against oresmd. Nothing asserts that a corpus key can be represented as a URI, which is how a 26.0% coverage figure and a silent 0D to 2D rewrite both survived a suite of 952 assertions.

Twelve protocol headers. Six are generated for the six entities – feed_binding, market_fixing, market_observation, market_series, observation_lineage, series_classification_rule – and six are hand-written for operations and custom payloads: crm_protocol, curve_republish_protocol, curve_snapshot_protocol, import_protocol, market_feed_config_protocol, market_series_export_protocol.

The generated six are committed in the retired shape – list=/=save=/=delete=/=history – while the models declare the canonical set: list, get, get_many, put, put_many, delete, delete_many, the *_versions. pair for the four entities with history, list_by_series_id for observations, and the three <entity>_events. subjects. This is the same 82-file drift, seen from the protocol side.

M06 fails: six hand-written headers carry wire types no model covers. P04 fails with it: there is no ores.codegen.operation model anywhere in the component, and eight operation families want one (crm, republish, compute, curve-snapshot, import, export-to-storage, feed-control, publish-from-dq). P01 is satisfied at the model level for all six entities, except that no model offers a point-in-time read.

W01 fails, and more broadly than the two handler families:

• The umbrella registrar.cpp constructs its messenger handlers inline and calls only register_market_observation_handlers; the other five generated per-entity registrars are bypassed.
• All four history-provider registrars are unwired.
• All six event-mapping registrars are unwired, and the regenerated event registrar has a different signature (register_entity_event_mapping plus publish_entity_event) from the hand-written mapping the service composes. The absence is what makes series_classification_rule_eventing_integration_tests the one failing core test.

No reason is recorded for any of these, which is what W01 requires. Consequences: observation_lineages.* and series_classification_rules.* answer nothing, and market_fixings.* and market_series.history are declared but unserved.

Found in PR 2, after the wiring above landed. series_classification_rule_eventing_integration_tests assembles its own DB-notify chain, and that chain breaks before the event bus:

• The committed notify trigger emits the retired payload – {entity, timestamp, entity_ids, tenant_id} (marketdata_series_classification_rules_notify_trigger_create.sql:49).
• The shared listener parses every payload as the canonical entity_event_notification and returns on a parse failure (postgres_event_source.cpp:38-52), so its legacy-mapping branch is unreachable for these channels.

Installing the canonical function the current template renders, with the test unmodified, makes it pass; restoring the legacy function makes it fail again on every run. The test's own wiring is therefore correct – channel, subject, subscribe-before-write and readiness – and the payload shape is the sole blocker. That excludes the reading V02 recorded, that the test was simply not exercising the service's mapping.

The blast radius is wider than the test. 31 generated notify triggers still emit entity_ids – marketdata 6, refdata 15, trading 8, dq 2 – and the listener now drops all of them. Marketdata's rule is the only one with test coverage, which is why it is the only visible failure. It also qualifies PR 1's wiring: the six generated event-mapping registrars that application.cpp now composes are the stale legacy ones, so the live service drops every marketdata entity event too. The wiring was necessary and is not yet sufficient.

The fix is the regeneration this plan already schedules for PR 3, and it is not a test fix. The template has emitted the canonical shape since d6701b43b6; the committed marketdata triggers predate it. A library-side legacy fallback in the listener would be smaller and is the wrong direction: it restores a path that was deliberately deleted, and leaves all 31 triggers stale.

check_handler_permissions.py prints a pass, but marketdata is not in its REGISTRY, so it checks nothing. Read by hand, W02 fails: 16 codes are checked by handlers, none is seeded exactly as checked, seven marketdata:: codes are seeded but checked by no handler, and SyntheticService and IamService hold grants that do not match what the handlers test. The four hand-written handlers check crm:read, import:write and curve_bootstrap:* codes that are not seeded at all.

• P02: six raw subject literals duplicate a constant or belong to a subject family with no model; marketdata.v1.market-data-observations.publish-from-dq is a raw literal in registrar.cpp and again in SQL. Two exceptions are legitimate (the tick family has no model; the feed-config subjects are another component's).
• P03: seven stale generated artefacts, including six *_changed_event.hpp headers and market_series_asset_class_protocol.ts.
• The junction materialises no protocol, handler, registrar or service at all – neither side of it sets :list_by:, which is the junction messaging gate's condition. That is also why its shell command includes a header nothing generates.
• market_data_architecture.org documents a subject set the code no longer matches.
• The NATS wire is ORE-shaped: tick subjects are marketdata.v1.tick.<tenant>.<workspace>.<party>.<lowercased ore key with '/' as '.'>, and feed_binding.ore_key is a raw ORE key string.
• feed_binding's market_feed_configs.* subjects live in the marketdata.v1 namespace but are served by ores.synthetic.
• Seven tables carry party_id with no restrictive party policy, silenced by RLS_002 entries in projects/ores.sql/utility/validation_ignore.txt lines 342-354.

285 .hpp=/.cpp= files. 188 carry the generated marker; 97 are hand-written, of which 20 are tests and 77 are not. The 77 non-test hand-written files divide into 24 that generation should supersede, 36 that are infrastructure and stay, and 17 that are dead.

All 188 marked files name a template that exists, and no hand-written file carries the marker or a near-miss spelling. The item passes as it stands. Worth recording because the suspicion going in was the opposite: the presentation history field mappers, the api *_table / *_table_io / *_json_io families, the eventing headers, the generators, and the entity handlers and registrars are all already generated.

The component's live client surface is exactly one class, market_data_client, consumed by ores.synthetic (fx_spot_feed.cpp, ir_curve_feed.cpp, application.cpp, feed_controller.hpp) and linked by ores.refdata.client. Everything else in client/ is self-satisfying:

Symbol	External consumers
crm_client	none
fx_spot_subscription	none (the one hit in feed_ingest_loop.hpp is a comment)
detail::ore_key_to_subject	none
crm_rate_display_service	none
crm_rate_formatter	none
crm_rate_table (convert_to_table, convert_to_matrix_table)	none
crm_rate_table_io	no includer at all

This is 14 files plus their tests. They were the presentation layer behind the retired Qt cross-rates matrix, and their own doc comments claim a consumer – "for any text-console consumer (ores.shell today)" – that does not exist. There is no client codegen facet anywhere in the tree, so a hand-written client layer is by design; a dead one is not.

market_fixings_repository and market_observations_repository (plural, hand-written) sit beside market_fixing_repository and market_observation_repository (singular, generated). They are not line-for-line duplicates: both halves share sql() and write() over the same entity and table, and they split the read path. The generated singular is production through the generated *_service and is untested; the hand-written plural carries read_as_of, read_as_of_buckets and the series/date reads, and is what import_service, curve_snapshot_handler, curve_republish_service and feed_ingest_loop actually call.

So H02's "two implementations of the same behaviour" applies to the shared half and the fix is the standard one: the generated singular survives, the extra queries move into the cpp_domain_type_repository template and are regenerated, then the plural is deleted and its four callers migrate.

• application_exception.hpp in service/app/ has zero consumers.
• marketdata_tick_subject_prefix and synthetic_sandbox_tick_subject_prefix in tick_subjects.hpp are unused constants.
• service/src/curve_republish_resolver.hpp appears in no component_files.cmake while two files include it directly (G06).

The census recommends deleting the oresmd layer, since by B06 a test-only consumer is dead code. That rule has an explicit escape – "Name the intended consumer on the task, or delete the symbol with its test" – and this is the case it exists for. The intended consumer is the import and export boundary, it is named in the decided target state, and it is what the owner asked for. Record the exception with that consumer named, and wire it in PR 4. The symbols do not go.

The client CRM stack has no such defence: nothing intends to consume it.

Scope. Every defect that must be fixed at its source before the component is regenerated. Nothing is regenerated here, so the tree is unchanged and the evidence is the codegen suite plus a diff that shows marketdata untouched.

• The model fixes (M02, M03, M09): bind the four unbound models, record feed_binding as a no-match with its actual feature values, drop the restated :has_tenant_id: from the junction, delete the ten legacy Presentation properties from feed_binding.org.
• The junction shell-command defect (G05). The junction model enables the ores.cpp.shell-command facet, but the junction messaging gate emits no protocol header, so the generated market_series_asset_class_commands.cpp includes ores.marketdata.api/messaging/market_series_asset_class_protocol.hpp – a file the render never produces. Either the junction gains a protocol or the shell facet is gated off for it. Regression test in the codegen suite.
• The event-registrar signature mismatch (W01/G05). The regenerated event registrar exposes register_entity_event_mapping plus publish_entity_event, while the service composes a hand-written mapping of a different shape. Fix at the template.

Why first. Regenerating before these are fixed will not compile, so a "just regenerate" PR cannot exist. Three hand-written files reference types the models remove – market_data_client.cpp on get_market_series_request, save_market_series_request and save_market_observation_request; the umbrella registrar.cpp on the same names; and ores.iam's provisioning handler – and the junction shell unit includes a header nothing generates.

Why separate. It is the only PR whose whole content is "the generator was wrong", it changes no generated output for marketdata, and it is what makes PR 3 reviewable as a mechanical regeneration rather than a mix of mechanical and semantic change.

Scope. Point the test suite at external/ore/examples/: a test that parses every fixture and asserts the resulting series and observation rows, plus a coverage test that fails when a key line the corpus carries cannot be represented. Then fix what it finds. The first two fixes are under twenty lines and recover 42,667 corpus lines:

• The CSV tokeniser needs two commas, so a line with exactly one aborts the whole file. Change the requirement to at least one comma (market_data_parser.cpp:106-118).
• time_utils::parse_date gains DD-MM-YYYY and DD.MM.YY branches (about fifteen lines), which unblocks seven fixtures.

The third is the live import defect: the IR 6-segment path silently rewrites 0D to 2D. PR 2 stops the silent rewrite by preserving the segment through the parse; the real per-currency lookup follows in PR 4.

Why now and why separate. It is the only way to exercise the data types the owner named, it needs no model or schema change, and it converts the 26.0% coverage figure into a gate. Without it there is no way to show a later cutover did not change what a file means.

Depends on. Nothing but PR 1 for a stable tree. It also produces the evidence PR 4 and PR 5 need.

Scope. Items G01 to G08, B03, B06, H02, H03, H05, W01, V05 to V08. The mechanical work the previous two PRs made safe:

• Regenerate all four modules: 82 changed files, 14 created, across SQL, C++, TypeScript, shell and docs.
• Migrate every consumer of the retired wire types in the same piece of work – market_data_client.cpp, the umbrella registrar.cpp, ores.iam's provisioning handler, and transitively ores.synthetic's five market_data_client call sites.
• Delete the dead code the census found: the client CRM stack (14 files plus tests), application_exception.hpp, the two unused tick subject constants, and the plural repositories once their read_as_of queries move into the cpp_domain_type_repository template.
• Give the component its root component_overview.org and group model so the root CMakeLists.txt is generated (M10); write the client's diagram and replace its placeholder prose; clear the MISSING_PUML ores.marketdata.client exception.
• Compose the unwired registrar families or record why not (W01). The event mappings are the live case: their absence is the one failing core test.
• Refresh the CMake source lists and land the per-entity shell units and recipes (S01, S02).

Why separate from the cutover. It is large but entirely mechanical, and it is the PR that makes check_component_drift.py --component marketdata pass. Keeping it apart from PR 5 means the 96-file diff is reviewed once, in isolation from a schema change.

Known prerequisite inside it. The recorded_at member is dropped by the current template for :no_audit_columns: entities (Bring the no-audit-columns entities over with the domain member gone), so market_fixing and market_observation and the two hand-written tests that populate it move together.

Three different things are usually run together when this question is asked, and after PR 3 they sit at very different stages. Stated separately, because a single percentage hides which of the three it measures.

Naming and representing every type: done. The gate reports 43 ORE series types, 41 represented in oresmd and 2 recorded with a reason (FIXING and GENERIC-MD), zero shape mismatches, 42 quote types over 38 distinct ORE types, and 6 of 6 volatility-declaring asset classes wired. The two allowlists in oresmd_ore_coverage_tests.cpp – types_oresmd_cannot_name() and types_with_round_trip_gaps() – are empty and asserted equal to empty. So no ORE type is unnamed, and no key projects without reading back.

The database is not yet on that path. import_service.cpp parses through ores::ore::market::parse_market_data in ores.ore.core. The import path names no oresmd symbol at all. The representation at 100% coverage is therefore not load-bearing at the write boundary, and it is reached by tests alone. This is B06's recorded exception, and wiring it is PR 4.

And there is no end-to-end evidence either way. service_import_service_tests.cpp, the only test that drives an import into the database, feeds it two hand-written lines. Nothing imports a real ORE sample file and asserts the rows. So "every sample file reads into the database correctly" is unmeasured rather than unfinished. That test is the first thing PR 4 builds.

Update: it is built, and the answer is yes. service_corpus_import_tests.cpp imports every market payload under external/ore/examples – 118 files – and reads the rows back, and all of them pass. It found three defects on the way that this analysis had recorded as fixed in PR 2, which is the argument for having built it before touching the import boundary:

• parse_date read two of the five date formats the corpus uses. DD-MM-YYYY (9,302 lines), DD.MM.YY (96) and DD.MM.YYYY (25) all threw unrecognised date format, so two 4,652-line files could not be imported at all. Ten characters is not enough to tell DD-MM-YYYY from YYYY-MM-DD; the separator positions decide.
• The tokeniser took its comma path only when a line held two commas or more, so a line like 2025-02-10,RECOVERY_RATE/RATE/BANK/SR/EUR 0.4 – one comma, a space before the value – fell through to the whitespace path and aborted its whole file.
• ORE's own examples repeat (date, key) pairs, and the import de-duplicates them by last-line-wins and reports each as a warning. The first draft of the test read that as data loss, which is a reminder that the expectation has to be the contract rather than the line count.

So the distance to "reads every sample file into the database" was three defects, not a programme of work. What remains for PR 4 is the boundary itself: oresmd is still not the thing at the write edge, so the rows are correct but not canonically named, and the export still has nowhere to stand.

The line counts, for the record and for the two bases they are quoted on: the corpus is 511,002 lines, 486,436 of them (95.2%) on modelled types. The remainder are lines whose first token is not a series type at all (Lufthansa, SP5, EURUSD, SwaptionVolatility), not market-data series. The 485,606-of-485,606 figure recorded earlier was of the keys the gate expected to name, which is a different denominator; both are true and neither is the answer to "how much of the corpus reaches the database", which is currently zero because nothing measures it.

Landed: the regenerate (97 files across four modules plus the web twin), the consumer migration, the dead-code deletion, the W01 wiring, M10 and H05. The gate is green – check_component_drift.py --component marketdata produces no diff – and so is every suite, on a database rebuilt from the tree.

Two items left the PR, and neither is a deferral of convenience. Both were sized first, and both share a shape the rest of PR 3 does not have: they change behaviour or grammar rather than bringing a tree up to its models.

• M06, the operation models. Six hand-written protocol headers carry 26 structs and 85 fields, and five of the six are mechanical to model. The sixth is not: market_feed_config_protocol.hpp returns std::map<std::string, feed_kind_counts> and std::map<std::string, int>, and org_loader._ts_type has no map branch, so a model stating them cannot even load – _reject_silent_ts_gap raises. The usual escape is closed: check_protocol_twin_coverage.py fails any model carrying :ores.ts.protocol.enabled: nil, globally, so the map branch has to exist. That is a generator feature with its own regression test, and the same six headers are consumed by five other components (ores.synthetic, ores.iam, ores.shell, ores.reporting, ores.ore), so it is a cross-component header swap. Modelling them is also what would generate the six missing TypeScript twins, which is why COMPONENTS_UNDER_TEST cannot take marketdata first.
• P01 and the plural repositories. PR 3 established that these are two different things, which the checklist had run together. The plural repositories hold read_as_of, read_as_of_buckets, a series-scoped read_latest and a series-scoped remove, and their consumers are three services and two test suites inside the component. Moving those methods into the generated singular twins needs no template change – the paste points already exist and the marketdata models already use them – but the series-scoped methods have no generated twin and their tests assert series-wide semantics, so it is a rewrite of test semantics rather than a rename. Separately, P01 proper is the canonical as_of field on the list request, driven by :has_as_of_lookup: true, which is a different read again and belongs with the as-of query patterns already in the backlog.

The order is unchanged: M06 before the registry entry, and both of these before the component is clean. What changed is that PR 3 no longer claims them, and the checklist says why on the row rather than in a footnote.

Scope. Make the oresmd layer load-bearing at the ORE-format edge, and close the fidelity gaps that stop an export being lossless. Re-read on main after PRs 1 to 3 landed: several items this section used to list are now closed, and they are recorded as closed below rather than repeated as work.

First, the measurement. No test in the tree imports a real ORE sample file and asserts the rows it produced. oresmd_ore_coverage_tests.cpp and classification_series_classifier_tests.cpp walk external/ore/examples and assert projection and classification in memory; service_import_service_tests.cpp, the only test that drives the import into the database, feeds it two hand-written lines. So "every sample file reads into the database correctly" is unmeasured rather than unfinished, and the first thing this PR builds is that test: import every file under external/ore/examples/ into a scratch tenant and assert the resulting series, observation and fixing rows. It is what turns the coverage figure into a statement about stored rows, and it will separate a real parsing gap from non-series noise in the 4.8% of corpus lines that are not on a modelled type.

Then, in order:

• The import boundary gains the inverse projection: parse_market_data() -> from_ore_key() -> market_data_identifier -> the series and observation rows it already writes. This is the consumer the B06 exception for the oresmd layer names, and the boundary is not half-wired: import_service.cpp names no oresmd symbol at all today.
• The point-elision rule in the export mapper: drop the point for series types that carry no point dimension. has_point_dimension no longer exists anywhere in the component, so this is a rule to write, not a predicate to call.
• The IR settlement segment: replace the std::string{"2D"} fallback in the ir_swap projection with the per-currency spot lag from refdata. The segment survives the parse now – PR 2 added settle to ir_market_data_identifier and stopped rewriting it – so what remains is the case where the input carried no settlement segment and the projection currently invents one. Re-checked and withdrawn; the source named is the wrong convention. See "The settlement segment is not a currency's spot days" below.
• The verbatim key, so the export is lossless for every case the reconstructed key cannot reproduce. A key text not null column on the observations and fixings tables, populated from market_datum::key in import_service.cpp. Absent today.
• A new export path. A rows -> market_datum / fixing mapper plus one serializer caller (a new core/src/service/ore_export_service.cpp), exposed as a shell verb and a subject, asserting a file round trip through the database.
• --duplicates-are-errors on the shell import verb, so the strict policy is reachable (marketdata_commands.cpp).
• The grammar holes, each re-checked against the current projections before it is claimed: model accepted on seven quote classes and discarded for six, parse_credit not type-gating quote, the ir entity never validated as a currency, and split_market_series_key dropping the point against the registry's four-field key.

What PR 4 delivered.

The boundary, the grammar and the export are in. Two items were re-checked and withdrawn, below. Two were built differently from the plan, and the reasons belong here rather than in a commit nobody re-reads.

• The boundary. import_service runs every parsed datum through from_ore_key() -> market_data_identifier -> to_quote_key() and builds the series and observation rows from the registry's decomposition of that canonical spelling. A key oresmd cannot name keeps the parser's own decomposition and is imported anyway. The FX reversed-pair correction moved onto the identifier with it. Proved on the corpus: all 118 market payload files still import and read back under [.corpus-full].
• The verbatim key, which the wiring made necessary: the import now deliberately rewrites a key it can name, so the series' columns hold the importer's text and an export reading them back cannot return the file's. Built as a nullable column on market_observation only. Nullable because the rows no file produced – feed ticks, curve bootstraps – have no producer key to preserve, and forcing every writer to invent one would give the column two meanings. Observations only because a fixing's index name is already stored verbatim as its series qualifier: there is no rewrite to undo, so a column there would carry nothing new.
• The export. ore_export_service reads the tenant's series, observations and fixings and hands them to ores.ore.core's serializer, so both ends agree on the format by construction rather than by two implementations kept in step. A key line is the producer's own text where the row has one, and otherwise rebuilt from the series with the point dropped for a type that has no point dimension – every point-free row stores one anyway, SPOT for an FX rate, and emitting it would turn FX/RATE/EUR/USD into FX/RATE/EUR/USD/SPOT, a key no producer writes.
• The subject and the verb. marketdata.v1.export and marketdata export [--market-data path] [--fixings path], mirroring the import pair so the two read as inverses. The handler asks for marketdata::observations:read rather than a code of its own, because the export only reads and W02 already records this component's permission seeding as failing – seeded codes no handler checks, and a handler checking a code that is not seeded. A seventh hand-written protocol header joins the six M06 tracks.

And the test that was missing. The first two export cases imported a real ORE file and asserted the exported keys were the file's own. Both passed with the point-elision rule disabled, because an imported row always carries a producer key and the rebuild path never ran – so the rule the export exists for was untested by the test written to test it. The third case writes observations directly, as a feed or a bootstrap does; disabling the rule makes it report FX/RATE/EUR/USD/SPOT against FX/RATE/EUR/USD. That is the lesson the corpus walk taught earlier in this task, in a new place: a test that exercises the path you had in mind is not the same as a test that exercises the rule.

What the re-checks found.

The grammar bullet asked for four holes to be re-checked against the current projections before any of them was claimed, and that was the right instruction: one of the four is not a defect, one of the four is a defect with no affected caller, and one of the seven items above rests on a convention that does not exist.

• model was accepted on every asset class and kept on none. It names a volatility surface's own subtype, so it means something only when the URI also asked for type=vol; a quote URI naming a model parsed successfully and dropped the value. The parser now emits one check per class – reject_model_unless_vol for the six that carry a surface, reject_if_present for the four that do not – because no class gives model a second meaning. Ten rejection cases, generated from the models.
• credit's quote key was the one class not gated on type, so type=curve&quote=cds parsed and the quote type was kept. Its model no longer sets quote_type_checked: false.
• An ir URI's entity segment is its currency and was never checked. It is now a currency for every IR type except the bond option, whose third key segment is the underlying bond curve's own name – EUR_GENERIC in the corpus, and the one IR identifier the round-trip walk produces that is not three letters. The claim as written needed that exemption to be true; it was not.
• split_market_series_key does fold a point-bearing key's trailing point into the qualifier, so DISCOUNT/RATE/EUR/CURVE/2Y yields the qualifier EUR/CURVE/2Y where the import stores EUR/CURVE and the point 2Y. But all four of its callers pass an FX key, and FX has no point dimension, so no stored series is looked up under a qualifier that never existed. The fix needs the shape table, and three of the four callers – two in ores.synthetic, one free function – have only a NATS client and no context to read it from. The docstring now states the precondition and points at series_key_registry::decompose() for the general case. Left as a recorded limitation rather than a signature change three callers cannot satisfy.

The bullet above proposed ores.refdata's per-currency spot lag as the source for the IR_SWAP settlement segment. It is the wrong convention, and the corpus says so twice over.

ores_refdata_currencies_tbl holds exactly two values: USD is 1 and every other currency is 2. ores_dq_currencies_artefact_tbl shows the same split, 1 currency at 1 and 12,510 at 2. That is FX spot settlement, which really is T+1 for USD.

The segment the corpus writes for an IR_SWAP key is not that. Across external/ore/examples the settle segment is 2D for 2,158 USD keys and 0D for 821, and 1D never appears for USD at all. It also appears as an explicit date – IR_SWAP/RATE/GBP/20220922/... – which no per-currency integer can produce. So the segment is an ORE key component that is sometimes a lag and sometimes a date, and the field refdata exposes is the FX spot convention.

Substituting it would also break the round trip the coverage gate measures. from_ir_swap records settle only when the segment is not 2D, so a USD 2D key carries no settle in the identifier and the forward's fallback is what reproduces it. A fallback of 1D for USD turns IR_SWAP/RATE/USD/2D/3M/PAR_RATE into IR_SWAP/RATE/USD/1D/3M/PAR_RATE, and oresmd_ore_coverage_tests.cpp asserts all 485,606 corpus keys project back to the spelling they arrived in. The item is withdrawn, not deferred: there is nothing here to wire.

Closed by PRs 2 and 3, and struck from this list.

• The modelled-but-unrepresentable quote types – BASIS_SWAP, CC_BASIS_SWAP, ZERO, FRA, IMM_FRA, MM, MM_FUTURE, OI_FUTURE, CORRELATION, CPR and the 7-segment IR and SWAPTION keys. All of them project and round-trip; types_oresmd_cannot_name() and types_with_round_trip_gaps() are both empty in oresmd_ore_coverage_tests.cpp and asserted to be.
• The one-character projection bug. The DISCOUNT projection emits exactly DISCOUNT/METRIC/CCY/CURVE/TENOR and its inverse requires exactly five parts, so the two agree; and the forward requires both curve_id and tenor, so the four-part form the inverse used to reject cannot be emitted. The bug is gone rather than merely unreachable from the corpus.
• CPR's shape mismatch, which was a recorded exemption. PR 2 moved CPR to the security class, where it asserts the ISIN-keyed shape it writes.

Why here. It delivers the owner's first two goals without the schema cutover, and it is the smallest change that makes the oresmd layer load-bearing. It also surfaces where the URI and the stored columns disagree, which is PR 5's input.

The honest limit, restated. The figure this section used to carry – "lossless only for the 26.0% of the corpus oresmd can name" – is spent: every ORE series type is now represented or recorded, and the corpus names 486,436 of its 511,002 lines. What limits a lossless export is no longer naming coverage but the two things above that are still unwired: the boundary itself, and the verbatim key. Until both land the export can be lossless in principle and still not lossless in fact, and until the end-to-end import test lands we cannot say which.

Note. split_market_series_key() currently serves as the identity split in the ingest loop. Under this PR it stays – the cutover that retires it is PR 5.

Scope. Revive feature/identity-core-series-and-bindings: the oresmd_uri column on market_series and feed_binding, the repos and registrars and ingest loops reading URIs, the point_id repurposing, then the deletion of series_type=/=metric=/=qualifier=/=ore_key and the import classification table. Hard cutover, no backfill, per the decided target state.

Why last of the substantive PRs. It is the only irreversible one – it drops columns – and everything before it makes it verifiable. It should not be attempted before the corpus exists and the projections are lossless.

Not startable as scoped, and the corpus says why. The cutover's premise is that every identity-bearing row can be given an oresmd_uri: the column is not null, the natural key moves to it, and the decided policy rejects a type oresmd cannot name. For a series that arrives in a market.txt that premise holds – from_ore_key() names all 485,606 corpus keys and the coverage gate asserts it. For fixings it does not, and the fixing half is the half the target state never measured.

A fixing's key space is the index name, not the TYPE/METRIC/QUALIFIER/POINT grammar, so it needs an inverse of its own. The abandoned branch wrote exactly one, from_index_name(), whose contract is CCY-FAMILY[-TENOR] against the index_family enum. Measured against the corpus with that contract:

files	distinct index names	nameable	not nameable
fixings*.txt	96	24	72
the same plus fixings*.csv	158	34	124

The 72 in the first row are not a residue. They are 48 POWER-ICE:PDQ-... contracts, 6 inflation indices (UKRPI, USCPI, EUHICP, EUHICPXT, FRHICP, ZACPI), 3 equity tickers (EQ-SP5, EQ-RIC:...), 2 FX fixings (FX-ECB-EUR-USD), one credit name (GENERIC-JuniorNote), and twelve interest-rate names whose family the enum does not declare – EUR-EONIA, the CIBOR=/=NIBOR=/=STIBOR=/=SIOR families, and USD-FedFunds, USD-CMS-5Y, USD-SIFMA.

The abandoned branch handled this by skipping: on a failed from_index_name it pushed ="Skipped fixing '...'" into warnings and wrote no row. On the corpus that drops three quarters of the distinct fixing series, and it drops them quietly – which is why it went unnoticed. The branch predates service_corpus_import_tests.cpp by five weeks, and the test this task built in PR 4's first half is what makes the loss visible, because it asserts the warning count is exactly the de-duplication count. The branch's own verification line, "ctest 69/71 passed", could not have caught it.

So PR 5 needs a predecessor the six-PR plan does not have: map the fixing key space to oresmd, asset class by asset class. That is a grammar question before it is a code question, and adding enum members does not answer it:

• an inflation fixing names an index the inflation asset class already models as index_code, so UKRPI is close to nameable already;
• an equity fixing (EQ-SP5) names a ticker and no currency, and the equity identifier requires one, so the name does not carry enough to build the URI;
• an FX fixing (FX-ECB-EUR-USD) carries its source as well as the pair, and the FX identifier has no field for a source;
• a power fixing (POWER-ICE:PDQ-2021-01-04-14400-18000) is an hourly contract rather than a point on a curve, and needs a shape the grammar lacks;
• GENERIC-JuniorNote is a curve name, not an index;
• and the twelve interest-rate names are producer spellings the index_family enum does not hold – the same spelling-versus-family gap the ESTER alias already papers over for one of them.

Until that lands the cutover cannot be attempted: adding oresmd_uri and dropping the triple would leave the fixing boundary unable to write the fixings ORE's own examples carry.

What landed from it meanwhile. from_index_name() and is_scalar() are in, generated from the projections templates, with corpus-driven cases that pin the round trip and the limit – EUR-EONIA, UKRPI and a power contract are asserted unnameable rather than left undocumented. is_scalar() is the primitive the point repurposing needs, and it is where the abandoned branch disagreed with its own plan: it kept the "SPOT" sentinel, while both the story and the deletion task say the sentinel dies and a scalar series carries a null point. That repurposing is still to do.

Update, 2026-09-28 (task AB2922AA, commit 495dc8d2e3). The interest-rate third of the gap is closed. The index_family enum gained the fifteen families the corpus uses and the two spelling aliases, and the tenor rule moved out of a hand-written header into the model: from_index_name() now names all 57 distinct interest-rate index names and projects every one back to the name it arrived as, from 34 named and 28 round-tripped. The table above stands as the measurement at the revision that made PR 5 unstartable; the total named is now 57 of 158.

A corpus-driven test now walks the fixing payloads class by class and fails when a class that named names stops naming them, so the remaining classes are measured rather than described: inflation 7 names, equity 8, FX 7, commodity 25, power 48, security 2, GENERIC 4. It also asks whether the reader reads each payload at all, which found CurveBuilding/Input/fixings_bondyieldshifted.csv: the fixingDataFile of an example ORE runs, semicolon-separated, and unreadable here. That is fixed too (commit 6fa10a87f4), so the census covers all 66 payloads rather than 65.

Update, 2026-09-28 (task AB2922AA, commit fb3c5a4f31). The FX class followed, and it needed a field rather than data. An FX fixing's identity now carries source (lower-cased) and source_spelling (the token as ORE wrote it), both meaningful only when type=fixing, and keeps the pair as it was published rather than a canonical order: the corpus writes FX-TR20H-EUR-GBP and FX-TR20H-GBP-EUR as one rate both ways on 122 dates, and folding them together would put two values under one date. All seven FX names name and round-trip. The corpus total is 64 of 158, and six classes remain.

Update, 2026-09-28 (task AB2922AA, commit 995efd81e2). The power and commodity halves are closed. ORE states the intraday power index grammar in its own parser, POWER-COMMNAME[-YYYY-MM-DD[-START-END[-DST]]], and its commodity parser reads a contract month the same way, so a fixing's delivery period is a delivery field of its own rather than a point: point is what a quote or a vol names, and a fixing whose coordinate is a field stays scalar, as the interest-rate tenor already does. Power is a class of its own because ORE accepts a bare POWER-ICE:PDQ beside a bare COMM-ICE:PDQ, and one class could not write both names back. Power is 48 of 48 and commodity 25 of 25, all round-tripped, so the corpus total is 137 of 158 and four classes remain: equity (8), inflation (7), security (2) and the four GENERIC tokens, of which two are market-data keys inside a fixings file rather than index names. The FIXING exemption in the oresmd coverage gate still stands: an index name is a key space of its own rather than a quote key, and the boundary that consumes it is the cutover this section describes.

Two of the four GENERIC tokens are market-data keys that sit inside a fixings file rather than index names, and the reader reads them as fixings because that is the file they are in. Whether it should refuse a key that is not an index name is part of what remains.

Update, 2026-09-28 (task AB2922AA, commit 7b3918b6fe). The generic class is closed, and the two market-data keys are settled. ORE resolves GENERIC- in a parser of its own, and the name is all the identity there is: the identifier carries it upper-cased and the case ORE wrote it in beside it in name_spelling, which is the FX source_spelling rule. Both names the corpus carries name and GENERIC-JuniorNote and GENERIC-PortfolioBasket#1, the second through the %23 its # becomes in a URI – so the corpus total is 139 of 158. The two GENERIC-MD/EQUITY_OPTION/PRICE/... rows are SPX call prices in a fixing column rather than index names; the index-name projection refuses any name that carries a slash, which keeps them out of the fixing space, so the target is 156 of 158 rather than 158. Three classes remain: inflation (7), equity (8) and security (2). The FIXING exemption in the oresmd coverage gate still stands, unchanged.

Update, 2026-09-28 (task AB2922AA, commit 48eb0e27aa). The fixing key space is closed. Inflation, equity and security are the last three classes. Inflation is the one class whose index name is a bare token, so its model carries the codes and a dash-less name resolves only when it is one of them; equity needs no new field, because a fixing carries no currency and the ticker is the tail ORE writes, identifier scheme included; security reuses delivery for a bond future's expiry month, which is the shape commodity already gives its contract month. Every class the corpus puts in the fixing space resolves, so 156 of 156 index names round-trip and the two GENERIC-MD rows stay refused. The corpus-driven test names the classes with an empty allowlist, and it pins those two keys in a bucket of their own, so the census is whole without an exception standing in for a class. What remains is the cutover itself: the import still writes the index name into the series qualifier.

Update, 2026-09-29 (PR 5's scoping round). The blocker this section records is cleared, and the claim above is now false: the fixing half names, so the cutover's premise – every identity-bearing row can be given an oresmd_uri – holds over the whole corpus. Measured on the merged trunk: from_ore_key() names 485,606 of 485,606 corpus instrument keys, which the ORESMD coverage gate asserts, and from_index_name() names 156 of 156 fixing index names, with the two GENERIC-MD/EQUITY_OPTION/... rows refused as market-data keys in a fixing column rather than as fixings. No identity-bearing row in the corpus has no way back.

What the import writes today. Two write paths, and neither stores an identity:

• Market data. import_service.cpp computes the identifier already –

canonical_key() calls from_ore_key(), projects it with to_quote_key(), and returns the canonical spelling plus the registry's decomposition – and then discards the identifier. The series is found or created on (series_type, metric, qualifier), and the observation keeps the file's own text in obs.key, because the rows above hold the canonical spelling rather than it.

• Fixings. The series is created as

find_or_create_series("FIXING", "RATE", f.qualifier), where the qualifier is the index name. from_index_name() has no production call site at all: 0 outside the projections and the tests. So a fixing series has no identifier to store and the index name is its whole identity.

Where the oresmd_uri column belongs, and what it costs. market_series and feed_binding are both identity-bearing: the series on its (series_type, metric, qualifier) triple, the binding on ore_key and source_name. The cutover adds the column to both, moves the natural key onto it, repurposes point_id, then deletes series_type, metric, qualifier, ore_key and the import classification table. The blast radius of deleting the triple is 63 files across ores.marketdata, ores.synthetic, ores.shell and ores.ore that name qualifier. The value the column needs is in hand on the market-data path already and one call away on the fixing path, so the work is the cutover and its consumers rather than a new projection.

Scope. Add marketdata to component_registry.py and run every gate: component drift, model drift, protocol twin coverage, handler permissions, populate references, recipe inventory, CMake source lists, plus compass build, the suites and compass db recreate. This is item V06 and the standard's own closing condition.

Why separate. Registration is a claim about the whole component. It should be its own commit so the claim can be reverted without reverting the work.