Task: Make the oresmd URI the series identity

Table of Contents

This page documents a task in the Clean ores.marketdata to the component clean standard story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Every identity-bearing market-data row carries the oresmd_uri that its key or its index name projects to, so the series identity is the URI rather than the registry's (series_type, metric, qualifier) triple, and the import creates a series for every row ORE's own examples carry – the fixing half included.

2. Status

Field Value
State DONE
Parent story Clean ores.marketdata to the component clean standard
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-09-30

3. Acceptance

  • market_series and feed_binding carry oresmd_uri, and it is the natural key for a series and for a feed binding.
  • import_service writes the URI for every market-data key and every fixing index name, and writes no row that has an identifier without one. from_index_name has production call sites at last.
  • The corpus import round-trips: every fixing row in external/ore/examples creates a series, with the skipped-name warning count at zero and the de-duplication count unchanged.
  • series_type, metric, qualifier and ore_key are deleted, and every consumer reads the URI. Delivered: the three left market_series, the DQ artefact and its publish, and the last four hand-written readers went with them. The series classification rule is not part of that deletion and the acceptance is corrected here: it classifies the tokens of a key – which the ORE shapes registry decomposes – rather than a series row, so the identity does not replace it. The ore_key column named here is the feed binding's, already become oresmd_uri.
  • The FOMC segment's instruments carry ORE identities: the raw grid is one series per pillar, named by the shape of the instrument the feed simulates – the spot-starting pillar spelled as the swap it is, so no pillar needs a new spelling – and the republish curve carries ORE's DISCOUNT key.

4. Scope, measured before the column

Read at 00c9cdd711 on the merged trunk. Nothing here is a claim about the abandoned branch; the two write paths are the code as it stands.

Neither import path stores an identity today.

  • Market data. import_service.cpp's canonical_key() already computes the identifier: it calls from_ore_key(), projects it back with to_quote_key(), and returns the canonical spelling and the registry's decomposition of it. The identifier itself is discarded. The series is found or created on (series_type, metric, qualifier), and the observation keeps the file's own text in obs.key, because the rows above hold the canonical spelling rather than it.
  • Fixings. The series is created as find_or_create_series("FIXING", "RATE", f.qualifier), where the qualifier is the index name. from_index_name() has no production call site at all: zero outside the projections and their tests. A fixing series therefore has no identifier to store, and the index name is the whole of its identity.

Both identity-bearing tables are known, and the blast radius is measured. market_series keys on the triple; feed_binding keys on ore_key and source_name. 63 files across ores.marketdata, ores.synthetic, ores.shell and ores.ore name qualifier, which is the set the deletion has to carry. The value the new column needs is in hand on the market-data path and one call away on the fixing path, so this task is the cutover and its consumers rather than a new projection.

5. What the deletion has to carry, counted

Counted at ce3c6f32bd on this branch, with member-access and column patterns rather than the bare word, because metric alone matches 481 files and most of them are somebody else's metric:

What Count Pattern
C++ files reaching a series' qualifier 21 \.qualifier\b
C++ files reaching a series' series_type or metric 32 =\.(series_type\ metric)\b=, in the three components that hold series
SQL files naming the "qualifier" column 2 the market series create and populate scripts
SQL files naming the "ore_key" column 2 the feed binding create and populate scripts
Generated TypeScript carrying the field 2 the market series domain and protocol twins
Org documents naming qualifier 80 models, recipes and manuals

The organs split by component: ores.marketdata 33, ores.synthetic 16, ores.ore 12, then two in the shell and one each in ores.trading and ores.testing. The deletion is therefore a cross-component change, which is why the plan kept it as its own acceptance rather than folding it into the column's.

feed_binding's column is deferred, and the reason is structural. The plan gives the binding an oresmd_uri beside its ore_key, and the column was added and regenerated: ores_marketdata_feed_bindings_tbl took it and the entity's layers followed. It was then taken back out, because nothing can write it today. The row is assembled by the generated service – feed_binding_service.cpp carries the AUTO-GENERATED banner – and the service template has no per-column derivation hook; the #+begin_src cpp :name generator blocks a model can declare feed the test-data generators under api/src/generators/, not a real write. Every other writer is a generated shell command. So the binding's identity arrives with the unit that replaces ore_key, where the same code that supplies the key today supplies the URI; until then the column would be null in every row, which is worse than absent. The natural key stays ore_key and source_name meanwhile.

The identity the writers derived named a datum, not a series. The column sits on the series row, and the value came from a key that names one datum: IR_SWAP/RATE/ USD/0D/1D/5Y is the 5Y point of the USD/0D/1D series. The import projected the file's whole key, so a series row's identity carried whichever point arrived first, and two databases built from the same files in a different order would disagree about the identity of one series. The ingest loop projected the triple alone, which names no class whose key is longer than the triple, so a swap, curve or future series got no identity at all from a live tick. oresmd_parser::to_series_uri() projects the identifier with its point, and a volatility surface with it, dropped; both writers call it, and the ingest loop builds the datum's full key – the triple plus the tick's point – before falling back to the triple. No row changes shape: the series and its points were already separate rows.

The deletion needs no backfill, and the fallback is what says so. The plan records a hard cutover with no backfill, which only works because this schema is applied by recreation rather than by migration: a database is built from the create scripts, so it holds no series written before the identity column. The triple fallback exists for a database that does have one, and it is the thing the deletion removes – at which point a series with no identity is unreachable, which is why the create scripts and the writers have to agree before the columns go.

The curve snapshot requests name the series by the triple, and the identity move was taken back. Making them carry the identity is two hand edits, because neither the protocol header nor the handler is generated, and the edit was made and then reverted before the merge: a request carrying only an identity cannot reach a row that has none, and this path must read the FOMC grid, which no projection can name. So the handler's two lookups are the production readers left on the triple alone; the import's and the ingest loop's calls ask by identity first and fall back to the triple. The move waits for the rows to have identities. The path is verified by the compiler and the suites and by nothing else: it has no sender anywhere in the tree and no case exercises it, which is worth knowing before the cutover leans on it.

Not every series row in the table can be given an identity, and the constraint says so. Read on the live database at ca23d34ef2: of 5,864 series rows, 4,431 carry a series type the ORE key grammar knows, and 1,433 across 28 types do not. Re-read at 3632865c32, after the generator had been run again: of 5,971 rows, 4,569 are nameable – by from_ore_key(), or by from_index_name() for the 188 FIXING rows, which the quote grammar has no type for – and 1,402 are not. All but two of the unnameable set is test-data generator output: CD-0 to CD-25, 1,400 rows, whose series_type, metric and qualifier are all the same string and whose subclass cycles through the list the model's generated generator draws from.

The remaining two rows are the FOMC segment, both seeded by marketdata_market_series_fomc_populate.sql, and both production: they carry the fixed uuids the FOMC bootstrap config references as its source_series_id and its output_series_id.

  • RATES / YIELD / 'USD/SOFR-FOMC', the raw grid "the identity the ingest loop resolves the ir_curve_feed's ticks by". RATES and YIELD are not ORE series types, so from_ore_key() cannot name it.
  • YieldCurve / DISCOUNT / 'USD/SOFR-FOMC', the curve the republish service writes into. YieldCurve names a curve in ORE's curve configuration rather than a market data type, so the grammar carries it in neither direction.

This task's first reading called the second row generator output and so counted one production row. It is the republish seed, and there are two.

The instrument set is ORE's, and the corpus spells it. The raw grid's ticks are simulated quotes for meeting-dated OIS, and its pillars pair FOMC meeting dates: SPOT to 1F, then 1F to 2F, through 8F to 1Y. ORE carries the same instrument class elsewhere in the corpus: external/ore/examples spells a meeting-dated OIS as IR_SWAP/RATE/GBP/20220922/1D/20221103 – a start date, the overnight index tenor, an end date – and the live table holds 58 series whose settle segment is 20220922. The curve row is nameable in the same vocabulary: DISCOUNT/RATE/CCY/CURVE/TENOR is how the corpus carries a discount curve's points (the USD-DUMMY curve), and DISCOUNT is already an oresmd class. So the invented RATES/YIELD vocabulary is not what the instrument class needs.

Decided: the feed publishes the grid as the instruments it simulates, and the bootstrap reads them per pillar. The synthetic feed resolves each entry's start and end tenor to dates already, because the schedule needs them, so the ORE key is a projection of what the feed holds. The grid stops being one series with a point per entry and becomes one series per instrument, which is how ORE models a curve's input; the bootstrap's own pillars carry the start tenor, the end tenor and the role, so its input is derived from the config rather than referenced by one source_series_id. The republish output keeps its one-series-per-curve shape and takes ORE's DISCOUNT key, with the pillar as the observation's point. The alternatives, recorded for the reader who asks why they were not taken:

  • oresmd grows a namespace for internally derived series. Rejected: it would make the identity space admit whatever the system invents, which is what the grammar exists to refuse, and the grid's instruments turn out to be ORE's after all.
  • the derived series keep the triple and the URI stays the key for observed rows only. Rejected: two keys in one table, which defeats the point of the cutover.

The shape decides the identity, and it closes the deposit pillar's open question. Every pillar's quote is an ORE swap key: IR_SWAP/RATE/CCY/<start>/1D/<end>, which the corpus spells for the same instrument class as IR_SWAP/RATE/GBP/20220922/1D/20221103 (external/ore/examples, 58 series in the live table whose settle segment is 20220922). The start slot holds 0D for the pillar that starts at spot and a meeting date for the rest, 1D is the overnight index tenor every dated OIS quote in the corpus carries, and the end date is last. One series per pillar – the key without its end date – and the end date is the observation's point. ORE's money-market family has no dated point, so the spot-starting pillar is spelled as the swap it is rather than as a deposit; the dated spelling the unit left open is therefore closed, and no pillar needs a new spelling.

The curve the republish service writes keeps the corpus's discount shape, DISCOUNT/RATE/CCY/CURVE/TENOR (DISCOUNT/RATE/USD/USD-DUMMY/1D in external/ore/examples). There the curve is the series and the TENOR is the coordinate, which is the slot the corpus's 1D fills: the series' key is DISCOUNT/RATE/CCY/CURVE and the maturity is the observation's point. That is also why the DISCOUNT family appears in the defect below – the projection has to drop a maturity the identifier holds in tenor – rather than because the shape is wrong.

Both shapes are pinned by a case – the spot-starting pillar, a dated pillar and the curve – so the grammar is proved before the feed and the seed are written against it. The case also pins the negative: two pillars are two series, and the curve's maturities are one.

The curve's half of the seed carries its identity now. The FOMC seed row the republish service writes into reads DISCOUNT / RATE / 'USD/USD-SOFR-FOMC' with oresmd://ir/usd?curve_id=USD-SOFR-FOMC&type=quote&metric=rate&quote=discount, so the triple and the identity agree and neither is an invented vocabulary. Nothing else had to move: the republish service and the bootstrap reach that row by the fixed uuid the config names, and the pillars stay the observations' points. The raw grid's half waits for the re-key, because its identity only exists once the feed publishes one series per pillar.

The bootstrap reads its tenor convention off the grid series, so the re-key has to give it somewhere else to look. curve_republish_service.cpp's tenor_convention_code_for() returns RATES_SPOT_FOMC for a source qualifier ending in -FOMC and RATES_SPOT_FORWARD otherwise, build_refdata_context() throws when the series is missing, and that convention decides which tenor resolutions the pillar dates come from. The grid's qualifier is USD/SOFR-FOMC today, which is what the rule matches; the per-pillar qualifiers the re-key writes (USD/20260128/1D) match nothing, so the convention would silently become RATES_SPOT_FORWARD and the FOMC pillars would resolve as ordinary spot-starting instruments. The unit is therefore two parts: the bootstrap config names its own tenor convention – a column seeded to RATES_SPOT_FOMC for the FOMC config and RATES_SPOT_FORWARD for the rest – and then the feed and the bootstrap move to the per-pillar identities. The first part stands alone and is where the unit starts, and it deletes tenor_convention_code_for() and its throw path rather than keeping them as a fallback: a fallback would leave the coupling between a series' spelling and the convention its pillars resolve under.

The two configs still disagree on the roles, and identity is no longer what exposes it. The synthetic config's FOMC entries are a DEPO, seven FRA=s and an =IRS; the bootstrap config's nine pillars are a DEPOSIT and eight =SWAP=s. Because the shape decides the key, a pillar's role no longer reaches its identity and the vocabularies do not need reconciling for one. The disagreement is a pricing question – whether the feed's FRA path and the bootstrap's single-period swap path agree on the same pillar – and the FOMC unit answers it on its own terms. The column stays nullable until the feed and the seed agree, and the identity is what a row gets if it has one: the constraint would not catch the generator output anyway, since random text satisfies a not null column. What the generator writes is now a generic index name – the one oresmd URI that names a series with a name and nothing else – rather than the default faker noun, so the rows it produces carry an identity the grammar reads.

Two IR families keep their coordinate where the projection did not look, and the rest do not. The decomposition splits a datum key into the series and the point, and to_series_uri() was written to drop the identifier's point and its surface. That is the whole of the coordinate for the swap, money-market, FRA, vol and commodity families – the MM series CHF/2D carries its maturities in point, the FRA series EUR/3M likewise – but not for two IR families:

  • a DISCOUNT curve's maturity is the identifier's tenor, because the series is the currency and the curve: DISCOUNT/RATE/EUR/EUR carries 1D through 35Y as points;
  • a future's coordinate is its contract_code and its tenor, because the series is the currency and the contract month: MM_FUTURE/PRICE/EUR/2024-04 carries XICE:FEI/3M as its point.

So those two families' series identities carried one observation's coordinate while the rest were right. The triple fallback keeps a row reachable and stops duplicates, so nothing broke; what was wrong was the value stored. At 94e2d8645d the live table held 36 DISCOUNT series and 70 in the two future families.

The fix is in the projection, and the survey is a test. Which fields hold the coordinate is a fact about each family, so to_series_uri() now clears the discount tenor and the future's contract code and tenor as well. A case walks one two-point pair per family – MM, FRA, swap, swaption, commodity forward, discount and future – and asserts that the two points project to one identity with no point in it, so a family that regresses fails the case rather than the cutover. The earlier reading of this defect named the no-index families too, which the case disproves: their coordinate is the point, and 232 of their series were never affected. The series-level inverse this record first asked for is not needed, because the identifier already holds every field the series needs.

6. Plan

The additive half is delivered (#2358), the identity is a series rather than a datum for every family (#2362, #2364), the FOMC shapes are pinned (#2365), the curve's seed row carries its identity (#2366), the generator writes one (#2368), the config names the tenor convention its pillars resolve under (#2370) and the currency they are quoted in (#2375), and the feed keys every pillar as the OIS quote it simulates (#2373) while the bootstrap reads those keys per pillar (#2384); the raw grid those ticks used to land in is gone from the seed and from the config that named it. What is left is the cutover the task takes its name from.

The re-key, in four steps. Each step ends green on its own.

  1. The feed keys each pillar. ir_curve_resolved_entry gains the entry's start_date and end_date strings, which resolve() already computes for the year fraction. A pillar is spot-starting when its start_tenor_code is SPOT – the config's own first entry – and meeting-dated otherwise, and that one rule lives in a single builder both sides call rather than being written twice. ir_curve_feed publishes one tick per entry with series_type"IR_SWAP", =metric"RATE", =qualifier <CCY>/<start> (the start slot holding 0D for the spot-starting pillar and the start date for the rest), 1D as the index tenor, and point_id the entry's end date. The ingest loop already derives the series identity from a tick's full key, so the series appear with their ORE identities and nothing else in the pipeline moves. ores.synthetic is where this step's test belongs: a tick per pillar carrying the shape above.
  2. The bootstrap reads its own pillars. curve_republish_service stops reading one source_series_id and reads, per pillar, the series the key above projects to: resolve the pillar's start and end tenor codes to dates through refctx, call the same builder, which takes the currency the config names (#2375). Then the reader finds the series by identity and takes its observation. The rate map is keyed by the pillar's end_tenor_code, which is what resolve_bootstrap_pillars() already matches on, so its contract does not change. The fallback and the lineage are in the note below, which also records what this step's proof is and what it is not.
  3. The seed stops seeding the grid, and the column goes with it. Delivered: marketdata_market_series_fomc_populate.sql seeds the curve row alone, and the bootstrap config's source_series_id left the model, so the domain, protocol, entity, mapper, history field mapper, generator, service, shell command, recipe, TypeScript twins and create script are regenerated and a migration drops the column with the check that kept it apart from output_series_id. The reader's grid fallback went with the column, because the config no longer names a grid to fall back to. The lineage's column of the same name stays.
  4. The constraints and the deletion. Started, and most of the reading half is done: oresmd_uri is not null and is the natural key with party_id, the triple no longer keys the row, the import and the ingest loop read by identity alone, the curve snapshot's requests carry the identity, feed_binding carries it rather than an ore_key, and the client's triple lookup has one caller left where it had three – the FX feed's vintage seeding and the feed controller's vintage check read by identity now (see the identity note below). That one caller is the IR curve's vintage seeding, and it cannot move the same way: it reads the DQ deposit grid, whose series the cutover names generically, so the synthetic config has to name the series it seeds from or the path retires. The generated read_latest_by_type has gone, because nothing called it, and the datum-key projection the export's rebuild was waiting on exists now: oresmd_parser::with_point() puts an observation's point back into the field its family keeps it in, so the republish service – the last writer that stored no key – projects one from the output series' identity and the pillar's point and keeps it on the row. The vintage decision is delivered too: the datasets name their rows' own ORE keys and the series those keys belong to, so the deposit grid is a real MM/RATE/USD/2D series and the DQ publish stores the observation's key, and the synthetic config names the series its vintage reads – which is what the client's last triple lookup was waiting for, so that lookup goes with its caller. That is delivered, and so is the deletion the paragraph below describes: the export reads a fixing's index name off the identity, series_type, metric and qualifier have gone from market_series, the DQ artefact and its publish, and step four is complete. The import classification table was never part of it: it classifies a key's own tokens rather than a series row, and the fourth acceptance is corrected to say so.

    Read off the tree, the hand-written half of that work is four places rather than twenty-one files, because most of the census's hits are the generated entity, mapper, history field mapper, protocol, generator, service and shell command, which a regeneration takes: import_service's find_or_create_series stops writing the three columns (the parameters that feed the classifier stay, since they name a key), the ingest loop stops writing them on the series it auto-creates, ore_export_service's rebuild goes and the datum is emitted under obs.key – which every writer supplies now, so an empty one is a writer that forgot and should say so – with a fixing's index name projected from the identity through to_index_name() instead of read off the qualifier column, and market_data_client::save_series stops copying the three and starts copying oresmd_uri, which it never did and which the write's own contract says clears the identity. The DQ artefact and its publish lose the three columns with the series row, and the FOMC seed's insert loses them too.

  5. The component joins the registry. Delivered. The seven hand-written protocol headers are one operation model, ores.marketdata.operations, so each interface's header and its TypeScript twin generate from one declaration and no header is left without a twin. The model holds the CRM rate reads, the curve bootstrap, republish and snapshot subjects, the whole-tenant ORE import and export, and the folder feed control: twenty-eight messages, every field keeping the type, the initialiser and the subject the hand-written header carried, so the wire does not move. The one shared codegen change it needed was a projection for std::map, which now renders a TypeScript Record. The seven headers are deleted and their sixteen includers read operations_protocol.hpp. With the twin gate green, marketdata joins COMPONENTS_UNDER_TEST and the standard's gates check it from now on.

The identity is the key, and what that costs. market_series.oresmd_uri is not null, non-empty, and unique with party_id; the registry's three columns no longer key the row. Two writers had to change with it, because both could previously store a row with no identity at all: the import drops a market-data key or an index name the grammar cannot name, and the ingest loop drops a tick whose key it cannot name, each reporting what it dropped. The corpus names every key it carries – 485606 of 485606 instrument keys and 156 of 156 index names – so no corpus row is dropped. What this loses is a key the registry accepts and the grammar does not, such as the short IR_SWAP/RATE/EUR/2D/1D the import used to store with no point: a row no reader could find once the identity keys the catalog is a row that should not be written, and the unit that pinned it now pins the drop. The generator's rows cannot collide on the identity, because the party is generated per row and the party is part of the key – which is the answer to the review question that asked whether its counter was enough.

The triple's readers went with it, and the fallback went too. Each of the two writing paths used to look a series up by identity and fall back to the triple, for a row written before the identity column existed; every row carries one now, so both fallbacks are gone and the identity is the only lookup. That also answers the identity review's finding about the synthetic names: a row an earlier database named oresmd://generic/migrated-<uuid> is not the series a key names, and reusing it would have kept that name for good, so the import no longer finds it and writes the real identity instead, leaving the synthetic row inert the way the grid's row is. The curve snapshot's two lookups were the last readers on the triple alone, kept there because a request carrying only an identity could not reach the FOMC grid no projection could name; the grid is gone, so both requests carry the identity and the handler reads by it. Nothing in production called the generated read_latest_by_type, which a later unit deleted outright rather than waiting for the columns it reads.

The rename crossed a component the pull-request checks cannot see. feed_binding.ore_key became oresmd_uri in the binding unit, and ores.iam's tenant provisioning handler still read and wrote the old field; ores.iam.core's registrar includes that header, so the tree stopped compiling and the break survived sixteen green PR checks. The check named tests runs pytest projects/ores.codegen/tests – the codegen suite – and every other job is a Python gate over generated output, so no pull request compiles C++ at all; the C++ build lives in the continuous and nightly workflows. Locally I built subsets of targets chosen by me, and the grep for the renamed field listed the iam header and I did not open it, reading the remaining hits as the synthetic config's own ore_key and the wire ticks. The whole tree builds again with the handler projecting the config's key to the identity, and the whole-tree build is now what a rename's evidence rests on.

The identity cannot name a datum alone, and the projection needs the grammar's own fact. The export rebuilds a datum's key from the series' registry columns today, and the replacement was attempted a second time: a helper that reads the series' identity and puts the observation's point into the identifier's coordinate field. It names an FX rate, whose series key is complete on its own, and it returns nothing for a discount curve – the family the republish service publishes – because a curve's maturity is the identifier's tenor while a money market's is its point, and one point string cannot know which. Filling whichever coordinate field the series left empty aborted the suite rather than working. Both attempts were reverted, because a key that comes out wrong is worse than one that does not come out at all: a reader would write it to a file. The fact that decides it – which identifier field holds which class's coordinate – belongs in the oresmd family tables, so the grammar exposes one projection from a series identifier and a point, and a hand-written helper cannot know it. Two things about that fact were checked and one of them surprises. The loader already hands every family row's columns to the templates verbatim, so nothing has to be parsed to reach them; and the row's point_semantics names the coordinate's kind rather than the field that holds it – tenor for both the discount curve and the money market, which store their maturity in different fields. What does know the field is the generator's own key emission: ores.cpp.oresmd.projections.org writes each family's key out of the identifier's members, so which member is the coordinate is written down there, per family, already. The next unit therefore adds a per-family arm beside that emission – a setter the writer calls to place a point before projecting – rather than reading point_semantics, inventing a rule, or extending a hand-written helper that cannot know it. That arm is delivered, one unit after this note was written: oresmd_parser::with_point() places the point in the field the family's own key emission reads, the family survey's pairs walk backwards through it, and the republish service – the writer that held no key – stores the one it projects from its output series and the pillar's point.

The export cannot rebuild a datum's key from the identity, and that was worth finding before the columns go. The export writes an observation under the key it was given when the row carries one, and rebuilds it from the series' registry columns when it does not. The rebuild was tried the other way – read the identity, project it back to a key – and the corpus case fails: the identity is the series' key, and a datum's key needs its point placed in a field that differs per family, because a future's coordinate is a contract code and a tenor rather than the URI's point. The experiment was reverted. What the deletion needs instead is for every writer to store the key with the datum, the way the import already does. The ingest loop does, and the republish service did with the projection above: it builds the key from the output series it already reads and the pillar's point, so the rebuild described here has nothing left to serve and goes with the columns.

That also closes the identity review's other recorded prerequisite. With the fallback gone there is no contest to settle: the import finds a series by the real identity when one holds it, and creates one when none does, so a synthetic row is never preferred and never collides – it simply stays, as inert as the grid's row, until an operator deletes it. The deletion has nothing to decide about which row wins.

The vintage is named by two rows that agree, and the decision was never between naming it and retiring it. The IR curve feed's vintage seeding was the last caller of the client's triple lookup, and the series it read was the DQ deposit grid, whose published identity was a generic name no projection rebuilds. The choice the record left open was whether the synthetic config names the series it seeds from or the dataset carries the ORE key its own source names; the answer is both, because each is true on its own. The dataset's rows come from ORE files – the deposit grid's row is MM/RATE/USD/2D/3M, as its own source_url already says – so the seed names that key and the identity its series projects to, the publish stores the key on the observation and creates the series under the identity, and the config names the same identity for its vintage. Retiring the path was never on the table: it is a working feature whose only defect was the vocabulary its series was filed under.

The DQ artefact is the one place the projection is stated rather than made, and that is the honest limit: a key becomes an identity through the C++ grammar, and SQL has no way back to it. The seed therefore carries the two strings its own source names, the way the import carries a file's key. The asset class a published series joins follows from the identity instead of from a type table – oresmd://fx/... is an FX series and oresmd://ir/... an interest-rates one – which is the one part of a URI every class shares, so a further class is one arm in that case.

The registry's three columns outlive their use in the artefact, and the reason is a constraint rather than an idea: market_series still refuses an empty series_type, so the publish keeps writing the decomposition beside the identity until the deletion takes both away in one step. One thing the deletion does not take is the series classification rule. It is keyed by a key's own tokens, which the ORE shapes registry decomposes, and it never reads a series row, so the identity does not replace it; the acceptance that named it is corrected above.

The binding names a series by identity now, and the constraint it inherited broke a writer. feed_binding.ore_key was the last column the task's goal names, and it held a key the ingest loop then decomposed; it holds the series' identity now, which the loop projects back to the key for the subject it republishes under and for the decomposition the observation is filed by. The synthetic service's feed controller projects the key its caller passed, so it supplies the identity from the same place it supplied the key.

The same constraint caught a writer that no test covers. The dq publish function inserts market_series rows for the datasets it publishes and wrote none of them an identity, so the not-null added by the identity unit refused every publish – calling it for the IR deposit dataset failed with null value in column "oresmd_uri" ... violates not-null constraint. It names the series generically now, because a DQ key is not an ORE key and the projection belongs to the grammar rather than to SQL, and the same call inserts its row. Two consequences follow, and neither is repaired here: a DQ-published series carries a generic identity rather than the ORE key its own source names (MM/RATE/USD/2D for the deposit grid), and because the feed no longer falls back to the triple, an instrument the dq path published and the feed ticks now has one series per writer rather than one between them. The unit that migrates the vintage lookup decides both.

Step two, delivered. The reader stands on four things, and all four exist now.

  • The key builder sits in ores.marketdata.core, as a hand-written source (oresmd/pillar_quote_key.cpp) beside the parser and the projections it goes through, registered with regenerate_cmake_component_files.py --component ores.marketdata.core. It returns the pillar's ORE key and, beside it, the series identity that key projects to, so the feed's writer and the bootstrap's reader cannot disagree about the spelling. The feed's own copy is gone: ir_curve_feed calls the shared builder. ores.synthetic.api links ores.marketdata.core.lib (projects/ores.synthetic/api/src/CMakeLists.txt), and the reachability gate is green on this tree with that link in it.
  • The currency exists now: the config names it (#2375), and the reader takes it from there rather than from a series' spelling.
  • The dates exist now: resolve_tenor_date() turns a pillar's start and end tenor codes into calendar dates, which is the path the discount curve's own points already take.
  • The fallback arrived with the reader and went with the column it read (step three): a pillar whose series is absent, or whose series carries no quote at the point the read derived, now gets no rate at all, and resolve_bootstrap_pillars() fails loudly on it. The plan said curve_republish_resolver_fomc_tests seeds the grid and would prove that path. It does not, and the correction stands: the case is a unit test of resolve_bootstrap_pillars() that hands the resolver an in-memory rate map, so it never reads a series at all. The reader's own path is proved by four database-backed cases instead – a pillar read from its own series, one with no series, one whose series has no quote at its point, and the lineage the mixture pins – added when the review asked for them.

The lineage follows the reader: it lists the series each pillar was taken from, so a curve's provenance names its quotes rather than the grid they used to land in. A PROJECTION config names its discount curve's own output series there too – the row the discount points are read from – where the earlier form named that curve's config id, which is not a series.

The one thing the plan does not do is keep two writers: the feed either publishes the grid's key or the pillars' keys, never both. The fallback above is a reader tolerance, so a database written before step one is still readable after step three, which is the same reason the identity lookups fall back to the triple.

7. Measurements

What Value Command Commit
Corpus instrument keys that name 485606 of 485606 ores.marketdata.core.tests "[oresmd]" 00c9cdd711
Fixing index names that name 156 of 156 ores.marketdata.core.tests "[fixing]" 00c9cdd711
from_index_name() production call sites 0 grep over projects/, excluding the projections and their tests 00c9cdd711
Files that name qualifier 63 grep over ores.marketdata, ores.synthetic, ores.shell, ores.ore 00c9cdd711
Production readers left on the triple alone 2: the curve snapshot handler's two lookups; the import's and the ingest loop's calls ask by identity first and fall back to the triple grep -rn "read_latest_by_type" projects/ --include=*.cpp --include=*.hpp, less the repository's own definition and the tests 664cae095d
ores.marketdata.core suite, with the identity reader 1782 assertions in 400 cases ores.marketdata.core.tests 47327e935b
Series rows that carry an identity 4569 of 5971; the 188 FIXING rows are named by from_index_name() count(*) filter (where series_type in (<the grammar's token table>) or series_type = 'FIXING') over ores_marketdata_market_series_tbl 3632865c32
Series rows that do not 1402: 1400 CD-0 to CD-25 generator rows plus the two FOMC rows the same query, negated 3632865c32
Production rows no projection can name 1: the FOMC raw grid RATES / YIELD / 'USD/SOFR-FOMC'; the curve row now reads DISCOUNT / RATE / 'USD/USD-SOFR-FOMC' with its ORE identity marketdata_market_series_fomc_populate.sql and a read of the live rows 48ab31abc8
Where the bootstrap's tenor convention comes from the source series' qualifier, through tenor_convention_code_for() grep -n "tenor_convention_code_for" projects/ores.marketdata/service/src/app/curve_republish_service.cpp 07d9052523
Pillars the feed keys nine per FOMC config, one series each: IR_SWAP/RATE/<CCY>/<start>/1D ores.synthetic.api.tests "a_pillar_key_names_the_instrument_the_feed_simulates" ff3b5a9e2c
Where the reader would get the pillar's currency nowhere: the config has no currency column and a pillar carries none grep -in currency projects/ores.refdata/modeling/ores.refdata.ir_curve_bootstrap_config.org 3767255a5b
Bootstrap configs that name their currency 1 of 1: USD, the only curve seeded select id, currency_code from ores_refdata_ir_curve_bootstrap_configs_tbl dbaa00c52b
ores.synthetic.api suite, with the pillar key 109 assertions in 30 cases ores.synthetic.api.tests ff3b5a9e2c
Identity the test-data generator writes oresmd://generic/<noun>-<idx>?type=fixing, where it wrote nothing grep -n oresmd_uri projects/ores.marketdata/api/src/generators/market_series_generator.cpp c9bbfb7923
ores.marketdata.core suite, with the identity written 1778 assertions in 399 cases ores.marketdata.core.tests ce3c6f32bd
Key spaces the import names one case each: FX/RATE/EUR/USD and UKRPI ores.marketdata.core.tests "import_gives_a_series_the_identity_its_key_projects_to" ce3c6f32bd
A key's points that project to one series identity oresmd://ir/usd?tenor=1d&settle=0D&type=quote&metric=rate&quote=ir_swap for both .../5Y and .../10Y ores.marketdata.core.tests "a_keys_points_project_to_one_series_identity" be9191357f
ores.marketdata.core suite, with the series projection 1798 assertions in 402 cases ores.marketdata.core.tests be9191357f
Families whose projection dropped the coordinate, and now do not 36 DISCOUNT series and 70 in MM_FUTURE=/=OI_FUTURE; the 232 no-index series were never affected count(*) filter (where series_type = 'DISCOUNT') and ... in ('MM_FUTURE','OI_FUTURE') over ores_marketdata_market_series_tbl 94e2d8645d, the merge of #2363
Families the case walks, two points each eight: MM, FRA, IR_SWAP, SWAPTION, COMMODITY_FWD, DISCOUNT, MM_FUTURE and OI_FUTURE, plus three pairs that must stay apart ores.marketdata.core.tests "every_family_that_carries_a_point_projects_one_series_identity" 0819a6d9b6
ores.marketdata.core suite, with the family survey 1839 assertions in 403 cases ores.marketdata.core.tests 0819a6d9b6
FOMC shapes the seed and the feed will write, pinned ahead of them three: the spot-starting pillar, a dated pillar, and the curve ores.marketdata.core.tests "the_fomc_segments_shapes_name_their_series" 960a6d81c0
ores.marketdata.core suite, with the FOMC shapes 1852 assertions in 404 cases ores.marketdata.core.tests 960a6d81c0
Where the pillar's key and its identity are built ores.marketdata.core, one hand-written source the feed and the bootstrap both call grep -rn "make_pillar_quote_key" projects/ --include=*.cpp --include=*.hpp e3e237ded1
Identity a pillar's key projects to oresmd://ir/usd?tenor=1d&settle=0D&type=quote&metric=rate&quote=ir_swap for the spot pillar and ...settle=20260128... for a dated one, with the end date leaving the identity ores.marketdata.core.tests "[pillar]" e3e237ded1
ores.marketdata.core suite, with the builder's cases 1865 assertions in 408 cases ores.marketdata.core.tests e3e237ded1
ores.synthetic.api suite, with the moved case gone 102 assertions in 29 cases ores.synthetic.api.tests e3e237ded1
ores.marketdata.service suite, with the resolver untouched 164 assertions in 24 cases ores.marketdata.service.tests e3e237ded1
ores.marketdata.service suite, with the reader's own cases 178 assertions in 27 cases, 14 of them the reader's three ores.marketdata.service.tests d8333211c8
Paths the reader's cases cover 3: the pillar's own series, the grid when no such series exists, and the mixture that also pins the lineage ores.marketdata.service.tests "[curve_pillar_reader]" d8333211c8
The missing-point fallback, shown red before it was believed with the check disabled the mixture case fails on the map size alone: 1 where 2 is required that case, with if (!found) replaced by if (!found && false) d8333211c8
Test-case reachability, with the reader's cases 7599 declared cases in 937 sources check_test_case_reachability.py d8333211c8
Grid rows the FOMC seed writes 0: the RATES/YIELD row and its asset-class junction row are gone, and the grid's fixed uuid is named nowhere in the script =grep -c "RATES\ e1c2d3f4" projects/ores.sql/populate/marketdata/marketdata_market_series_fomc_populate.sql= 9bfac75414
Columns on the config that name a series 1: output_series_id; source_series_id left the model grep -c "series_id" projects/ores.refdata/modeling/ores.refdata.ir_curve_bootstrap_config.org 9bfac75414
Grid readers left in the reader 0: the fallback went with the column it read grep -c "source_series_id" projects/ores.marketdata/service/src/curve_pillar_reader.cpp 9bfac75414
ores.marketdata.service suite, with the fallback gone 176 assertions in 28 cases ores.marketdata.service.tests 9bfac75414
ores.shell.refdata suite, with the regenerated command 4284 assertions in 1518 cases ores.shell.refdata.tests 9bfac75414
ores.refdata.api suite, with the regenerated protocol and domain 588 assertions in 81 cases ores.refdata.api.tests 9bfac75414
ores.refdata.core suite on this machine's database 61 failures, none of them naming this entity: the database predates columns and tables such as ore_currency_type and the *_conventions_tbl relations, which is the drift bearings already reports ores.refdata.core.tests 9bfac75414
reachability with the reader's four cases 7761 declared cases in 956 sources check_test_case_reachability.py 9bfac75414
Columns that key a series 2: party_id and oresmd_uri grep -c "natural_key" projects/ores.marketdata/modeling/ores.marketdata.market_series.org ef74df0ccf
Writers that could store a series with no identity, and now drop the row instead 2: the import, for a key or index name the grammar cannot name, and the ingest loop, for a tick grep -n "skipped: oresmd names no" projects/ores.marketdata/core/src/service/import_service.cpp and grep -n "Dropping tick" projects/ores.marketdata/service/src/app/feed_ingest_loop.cpp ef74df0ccf
Rows the import drops from the corpus 0: every instrument key (485606) and every index name (156) names ores.marketdata.core.tests "[fixing]" and the corpus coverage case ef74df0ccf
ores.marketdata.core suite, with the identity cases 1879 assertions in 412 cases ores.marketdata.core.tests ef74df0ccf
ores.marketdata.service suite, with the tick drop 176 assertions in 28 cases ores.marketdata.service.tests ef74df0ccf
ores.shell.marketdata suite, with the reordered command 144 assertions in 51 cases ores.shell.marketdata.tests ef74df0ccf
reachability, with the identity cases 7765 declared cases in 956 sources check_test_case_reachability.py ef74df0ccf
Production readers the repository-method grep found 0: the import's and the ingest loop's fallbacks are gone, and the curve snapshot's two lookups read by identity grep -rn "read_latest_by_type" projects/ --include=*.cpp --include=*.hpp less the repository and the tests ac4e03519b
Production readers the same unit missed 1: market_data_client::find_series() filters a whole series list by series_type, metric and qualifier in the client, and the synthetic service's vintage paths call it – a grep for the repository method cannot see it, and the claim above reads as zero readers only because of that grep -rn "find_series" projects/ --include=*.cpp --include=*.hpp 467c13f73a
Series the dq publish path could create under the identity's constraint 0 before the repair: the live publish function inserted a market_series with no identity, so #2387's not-null refused every dataset select * from ores_marketdata_publish_market_data_observations_from_dq_fn('31776ec1-d39f-474e-8423-17e47442d06e', ores_utility_system_tenant_id_fn(), 'upsert', jsonb_build_object('party_id', ...)), which failed with null value in column "oresmd_uri" ... violates not-null constraint 467c13f73a
Series the same call creates after the repair 1: inserted, with the generic identity the function now writes the same call 467c13f73a
Callers of the client's triple lookup 1: the IR curve's vintage seeding; the FX feed's vintage seeding and the feed controller's vintage check read by identity, where three called it grep -rn "find_series(" projects/ --include=*.cpp, less find_series_by_uri and the tests eba20b1d9d
The client's identity lookup 1 new method, find_series_by_uri(), matching the identity the list carries grep -n "find_series_by_uri" projects/ores.marketdata/client/ eba20b1d9d
ores.marketdata.core, client, synthetic.service and synthetic.api suites 1893 in 414, 6 in 6, 297 in 46 and 65 in 22 assertions and cases the four test binaries eba20b1d9d
Why synthetic.api counts fewer cases than the identity unit measured 7 cases gone with two test files: another environment's merge deleted the organisation generator and its tests (04f6092cfd), not this change git show --stat 04f6092cfd eba20b1d9d
reachability, with the one caller left 7739 declared cases in 952 sources check_test_case_reachability.py eba20b1d9d
Targets built before merging the binding rename the marketdata core, service, api and client suites and the synthetic api and service suites: no target that compiles ores.iam's registrar, and no whole-tree build cmake --build --preset linux-clang-debug-make --target <subset> 467c13f73a
The tree after the rename, built whole a hard error in tenant_provisioning_handler.hpp: no member named 'ore_key' in the binding and in its write, on the commit that merged as #2392 cmake --build --preset linux-clang-debug-make with no target 599a1a06db
Pull-request checks that compile C++ 0: tests runs the codegen suite, and the rest are Python gates; the C++ build is in the continuous and nightly workflows sed -n '/^ tests:/,/^ [a-z-]*:$/p' .github/workflows/codegen-drift.yml 599a1a06db
The tree after the fix builds whole, no error and no warning the same whole-tree command 599a1a06db
Families the identity-plus-point projection names FX, whose key is complete without a point; not the discount curve, whose maturity is the identifier's tenor, and filling whichever field the series left empty aborted the suite ores.marketdata.core.tests "[datum_key]" on the reverted branch e76dcf160b
Writers that store the producer's key with the datum 2 of 3: the import and the ingest loop; the republish service is the one left, and it publishes the family the projection cannot name grep -n "obs.key = " projects/ores.marketdata/core/src/service/import_service.cpp projects/ores.marketdata/service/src/app/feed_ingest_loop.cpp e76dcf160b
Datum keys the ingest loop stores 1 per tick now, where it stored none; the rule is the export's own rebuild rule grep -n "obs.key = series_key_registry_" projects/ores.marketdata/service/src/app/feed_ingest_loop.cpp b3f8c1194f
The export experiment's failure the corpus import-export case fails when the export rebuilds a key from the identity, because a datum's point sits in a per-family field ores.marketdata.core.tests "[ore_export]", on the reverted branch b3f8c1194f
ores.marketdata.service and core suites with the stored key 176 in 28 and 1893 in 414 assertions and cases the two binaries b3f8c1194f
References to read_latest_by_type left in the tree 0: gone from the model, the generated repository and the tests, where nothing in production called it grep -rn "read_latest_by_type" projects/ --include=*.cpp --include=*.hpp --include=*.org 9db9a2ac3d
ores.marketdata.core suite, with the dead reader gone 1893 assertions in 414 cases, the by-type repository case become a by-identity one ores.marketdata.core.tests 9db9a2ac3d
Fields a curve snapshot request names a series by 1: oresmd_uri, where it carried the registry's three grep -n "std::string" projects/ores.marketdata/api/include/ores.marketdata.api/messaging/curve_snapshot_protocol.hpp ac4e03519b
ores.marketdata.core, service and api suites with the retirement 1879 in 412, 176 in 28 and 71 in 14 assertions and cases the three test binaries ac4e03519b
reachability, with the retirement 7780 declared cases in 956 sources check_test_case_reachability.py ac4e03519b
Grid readers left in the reader 1, the fallback in read_pillar_rates; no case exercises it grep -n "read_grid_rates" projects/ores.marketdata/service/src/app/curve_republish_service.cpp e3e237ded1
Component file lists stale on the trunk this branch sits on 2, ores.iam/api/src and ores.workflow/api/src, neither of them this change's regenerate_cmake_component_files.py --all --check e3e237ded1
The projection from a series identity and a point one grammar function, oresmd_parser::with_point(), generated beside to_series_uri() grep -n "with_point" projects/ores.marketdata/core/include/ores.marketdata.core/oresmd/oresmd_parser.hpp a67cc26b8b
Families the round trip names seven: MM, FRA, IR_SWAP, COMMODITY_FWD, DISCOUNT, MM_FUTURE and OI_FUTURE, with a surface and a fixing pinned as refusals ores.marketdata.core.tests "a_series_identity_and_a_point_project_back_to_the_datum_key" a67cc26b8b
ores.marketdata.core suite, with the projection 1920 assertions in 415 cases ores.marketdata.core.tests a67cc26b8b
Writers that store the key with the datum 3 of 3: the import, the ingest loop and the republish service grep -n "obs.key = " projects/ores.marketdata/core/src/service/import_service.cpp projects/ores.marketdata/service/src/app/feed_ingest_loop.cpp projects/ores.marketdata/service/src/app/curve_republish_service.cpp 4fb044c271
Evidence for the republish service's key the grammar's own case and a whole build; the service has no harness, so the row it writes is not read back by any test grep -rln "curve_republish_service" projects/ --include=*tests*.cpp 4fb044c271
ores.marketdata.service suite, with the republish key 176 assertions in 28 cases ores.marketdata.service.tests 4fb044c271
codegen suite 1016 passed projects/ores.codegen/venv/bin/python -m pytest projects/ores.codegen/tests -q 4fb044c271
reachability, with the projection's case 7741 declared cases in 952 sources check_test_case_reachability.py 4fb044c271
Dataset rows that name their series and their key 339 rows across 14 series, in the deposit-grid and both FX driver seeds select count(*), count(distinct oresmd_uri) from ores_dq_market_data_observations_artefact_tbl where oresmd_uri <> '' e761a8ce03
Series the DQ publish creates for the deposit grid 1, under oresmd://ir/usd?tenor=2d&type=quote&metric=rate&quote=mm, where it wrote oresmd://generic/dq-... select s.oresmd_uri from ores_marketdata_market_observations_tbl o join ores_marketdata_market_series_tbl s on s.id = o.series_id where o.party_id = '68dd6600-80bf-410a-a0bf-f01f667f13a4' e761a8ce03
Key the same publish stores on the observation MM/RATE/USD/2D/3M, the key the dataset's own source names the same query e761a8ce03
The config's name for the series it seeds from vintage_series_uri = oresmd://ir/usd?tenor=2d&type=quote&metric=rate&quote=mm, the same string, on the USD vintage exemplar select vintage_series_uri from ores_synthetic_ir_curve_generation_configs_tbl where price_source = 'vintage' d44d64ba64
The vintage's anchor point, against the series it now reads 3M: the DEPOSIT entry's end_tenor_code, the published observation's point_id, and the source and date the config names select e.end_tenor_code ... where e.instrument_code = 'DEPO' and the observation query above d44d64ba64
Callers of the client's triple lookup 0: the last one was the IR vintage seeding, which asks by identity now grep -rn "find_series(" projects/ --include=*.cpp --include=*.hpp d44d64ba64
What the vintage path's proof does not cover the lookup and the observation it scans, verified against the database; the feed's own run of them is not, because it needs a started service and a published feed ores.marketdata.service.tests and the queries above d44d64ba64
ores.synthetic.api suite, with the config's new column 65 assertions in 22 cases ores.synthetic.api.tests d44d64ba64
ores.synthetic.service suite, with the feed's identity lookup 297 assertions in 46 cases ores.synthetic.service.tests d44d64ba64
ores.marketdata.client and core suites, with the lookup gone 6 in 6 and 1926 in 415 assertions and cases the two binaries d44d64ba64
ores.marketdata.service suite 176 assertions in 28 cases ores.marketdata.service.tests d44d64ba64
ores.shell.synthetic suite, with the regenerated command 432 assertions in 153 cases ores.shell.synthetic.tests d44d64ba64
reachability, with the added field 7741 declared cases in 952 sources check_test_case_reachability.py d44d64ba64
The tree, built whole no error and no warning, with the config's field and the feed's new lookup cmake --build --preset linux-clang-debug-make with no target d44d64ba64
The vintage config's migration, on a database that had a vintage row the row comes back as oresmd://ir/usd?tenor=2d&type=quote&metric=rate&quote=mm, from the artefact's own identity alter table ores_synthetic_ir_curve_generation_configs_tbl drop column vintage_series_uri, then the migration, then a read of the row 42ca72b950
The publish function's reinstall, with the one-lookup class table updated 1, the same series and key as before the change psql -f marketdata_publish_from_dq_create.sql then the publish call 42ca72b950
The build gate's heavy job, on the pull request that adds it COMPLETED/SUCCESS for build, test and start the services / linux-clang-debug-ninja: it recreates the database, configures, builds every target, runs ctest, and starts the services gh pr checks 2401, and the check-run API on a19b874c1d a19b874c1d
Pull-request checks that compile C++, before the gate 0: tests runs the codegen suite and every other job is a Python gate over generated output the check rollup on #2392's head 599a1a06db
The gate's triage, per class of change 14 cases: docs, web and check config start nothing heavy; C++, headers, the build description, models, codegen and SQL build and test; a service directory starts the services too; an unclassified path asks for everything projects/ores.codegen/tests/test_change_classification.py a19b874c1d
Columns a series row carries, after the deletion 15, and the triple is not among them; oresmd_uri is the only name select column_name from information_schema.columns where table_name'ores_marketdata_market_series_tbl'= on the recreated database dcbee2f195
Hand-written readers of the series triple 0: the import, the ingest loop, the export and the client's save all stopped reading or writing it =grep -rn "\.series_type\ \.qualifier\b" projects/ –include=*.cpp –include=*.hpp= less the key, classification and tick vocabularies 7430e08021
The DQ publish, with no triple columns anywhere the deposit dataset creates the series oresmd://ir/usd?tenor=2d&type=quote&metric=rate&quote=mm with MM/RATE/USD/2D/3M on the observation the publish call and the join query dcbee2f195
The database, recreated from the scripts with the deletion 39s, schema 0.0.25, the FOMC curve row and the DQ publish both writing identity-only series compass db recreate -y -k dcbee2f195
The tree, built whole, and the suites no error and no warning; 68 in 14 marketdata-api, 1923 in 415 marketdata-core, 176 in 28 marketdata-service, 6 in 6 client, 144 in 51 shell-marketdata, 65 in 22 synthetic-api, 297 in 46 synthetic-service, 432 in 153 shell-synthetic cmake --build --preset linux-clang-debug-make and the eight test binaries dcbee2f195
codegen suite and reachability, with the deletion 1030 passed; 7741 declared cases in 952 sources pytest projects/ores.codegen/tests -q and check_test_case_reachability.py dcbee2f195
The ER diagram, with the columns gone 355 tables, regenerated plantuml_er_generate.sh dcbee2f195

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
Messages the operation model declares, and the types they carry 28 messages, all 20 distinct field types project to TypeScript load_org_operation_model() over ores.marketdata.operations.org 53bb7a59fc
Protocol headers, over the listed component 199 in 13 committed components all have a twin, and no model opts out check_protocol_twin_coverage.py 25d58d7ca1
Components under test 15 listed, 5 accepted exceptions across 49 items check_registry_exceptions.py 25d58d7ca1
The component's regeneration, listed no drift at every address for marketdata, and --all --dry-run reports no drift check_component_drift.py --component marketdata and --all --dry-run 25d58d7ca1
The other codegen gates, over the component model drift, handler permissions (7 components, 290 codes), populate references, route auth declarations, shell menu collisions, the recipe inventory and validate_docs.sh (41 components) all exit 0 each gate by name 25d58d7ca1
CMake source lists marketdata is current; iam/api and workflow/api are stale on trunk and are not this branch's regenerate_cmake_component_files.py --all --check 25d58d7ca1
The codegen suite, with the map projection 1032 passed pytest projects/ores.codegen/tests -q 25d58d7ca1
The tree, built whole, with the model exit 0 cmake --build --preset linux-clang-debug-make -j 4 53bb7a59fc
The database, recreated from the scripts, with the model exit 0; the schema, every seed and the Acme publish ran (the pre-existing missing-image warnings aside) compass db recreate -y -k 53bb7a59fc
The fleet, after the recreate 24 running, 0 stopped, 0 failed, 0 missing; NATS running compass services start and compass services status 53bb7a59fc
The whole test suite, on the recreated database and the running fleet 89 of 89 targets pass ctest --preset linux-clang-debug-make --output-on-failure -j 4 6eceb87cc7
The console default's stale test ores.service.tests goes from 42 cases and 117 assertions with one failure to 43 cases and 123 assertions, all green the test binary 6eceb87cc7
The workflow component's CMake list adds workflow_retry_protocol.hpp, the header 45f36c0d66 left out; --all --check then exits 0 with no stale list regenerate_cmake_component_files.py --all --check b4bc41c59c
The pull-request check set, run locally on the rebased tree all 35 checks pass, and the TypeScript typecheck, format check and 51 tests pass beside them. The set covers header hygiene, the boost dependency check, populate references and its tests, component drift, the ER diagram, SQL hygiene, twin coverage (201 headers in 13 components), model drift, handler permissions, the shell menu check and its self-test, route auth, both recipe inventories, the registry, reachability, the codegen suite (1032), the sweep, paste blocks, validate_docs.sh (41 components), validate_schemas.sh (1802 files), compass lint, the org link check, the six skill checks, the shell tangle and its drift check each gate by name b4bc41c59c
The cutover's suites unchanged at 68 in 14 marketdata-api, 1923 in 415 marketdata-core, 176 in 28 marketdata-service, 144 in 51 shell-marketdata and 297 in 46 synthetic-service the five test binaries 53bb7a59fc

8. Notes

9. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

10. PRs

PR Title
#2461 [doc] Encode the local verification rule for pull requests
#2460 [ci] Remove the pull-request build gate
#2430 [agile] Close the series identity task
#2425 [marketdata,codegen] Join the component to the components under test
#2414 [marketdata,sql] Let the identity be the only name a series carries
#2410 [agile] Record the gate that compiles C++
#2409 [agile] Hand the deletion its four places
#2408 [marketdata,synthetic,sql] Let the config name the series its vintage reads
#2406 [marketdata,codegen] Name a datum from its series and its point
#2401 [ci] Build and test the tree when a change can break it
#2403 [agile] Record where the coordinate's field is modelled
#2402 [agile] Record what the datum-key projection cannot do
#2400 [iam] Let a written binding join the freshness list
#2399 [iam] Build the provisioning handler against the binding's identity
#2397 [marketdata] Keep the key a tick's datum is written under
#2395 [marketdata] Delete the reader nothing calls
#2394 [marketdata,synthetic] Read a series by identity
#2392 [marketdata,synthetic] Name the binding by identity, and repair the dq publish
#2391 [agile] Leave the handover naming only what is left
#2389 [marketdata] Retire the triple's last readers
#2388 [agile] Point the handover at the deletion
#2387 [marketdata] Make the series identity the key
#2385 [refdata,marketdata] Drop the grid the feed no longer writes
#2384 [marketdata,synthetic] Read the pillars by the identity the feed writes
#2378 [agile] The gate permits the builder's home
#2377 [agile] Answer where the shared key builder lives
#2376 [agile] Write the reader's step in detail
#2375 [refdata] Let the config name the currency its pillars are quoted in
#2374 [agile] Record that the reader has no currency to key with
#2373 [synthetic] Key every pillar as the OIS quote it simulates
#2372 [agile] Write the re-key's plan
#2370 [refdata,marketdata] Let the config name its tenor convention
#2369 [agile] Record where the bootstrap gets its tenor convention
#2368 [marketdata] Have the generator name the rows it writes
#2366 [sql,agile] Give the FOMC curve row its ORE identity
#2365 [marketdata,codegen] Pin the FOMC shapes before the feed writes them
#2364 [marketdata,codegen] Walk every family that carries a point
#2363 [agile] Pin the FOMC shapes and the series inverse
#2362 [marketdata,codegen] Name a series, not one of its points
#2361 [agile] Decide the FOMC segment's identity
#2359 [marketdata,agile] Settle the cutover's review record
#2358 [marketdata] Write the market series' oresmd identity

11. Review

# Comment summary File Decision Notes
1 A series with no identity is unreachable through the curve snapshot handler, which reads by identity alone curve_snapshot_protocol.hpp, curve_snapshot_handler.hpp taken back The requests and the handler carry the triple again: a request with only an identity has no fallback for a row that has none, and this path must read the FOMC grid. The move waits for the rows to have identities. The handler's two lookups are the production readers left on the triple alone.
2 Legacy rows are never backfilled, so the not-null unit needs a data step import_service.cpp, feed_ingest_loop.cpp, the migration recorded The not-null unit's prerequisites carry it, with a recreated database as the alternative: the schema is applied by recreation, so a rebuilt database holds no row written before the column, and the triple fallback exists for one that does.
3 A write that omits oresmd_uri clears an existing identity the market series model accepted, documented The model's description of the column now says that a write leaving it empty clears it, because the generated service copies the write's fields straight through. The structural fix is the not-null unit, where the column stops being optional.
4 The three shell commands gained a required positional argument the market series recipe, the shell library accepted, fixed The recipe and the three tangled library scripts carry the new argument (1cfba6cc49). The commands have no other caller: no CI workflow, script or recipe names them beyond the recipe and the shell's own tests.
5 Two different keys can resolve to one series import_service.cpp recorded Intended: the cache is keyed by the triple and the lookup is by the identity, so a key that projects to an identity another key already holds finds that series, and the reversed-pair warning still reports the difference. The unique index on the identity belongs to the cutover, and a case pinning the collision belongs with it.
6 The identity column has no index, while the triple is indexed marketdata_market_series_create.sql recorded The triple's index is the natural key's unique index. The column is not the natural key yet; the cutover moves the key onto it, and the same generated index follows.
7 Three small points: a line over the column limit, a doc-comment sentence that overstates, and mixed comment styles feed_ingest_loop.cpp, the market series model, curve_snapshot_protocol.hpp accepted, fixed in part The over-long line, and the same shape the change wrote in import_service.cpp and in its case, are reformatted by clang-format (d0d56cfc4e); the files carry older violations this change leaves alone. The model's sentence already qualifies itself in the same breath, and the regenerated header carries it. The comment styles stay as they are: the pair is cosmetic and the path is hand-written.
8 The census row cites 664cae095d while the review rows cite 1cfba6cc49 and d0d56cfc4e, and the reviewer could not resolve them the measurements and review tables answered All three resolve. 664cae095d is the tip of the branch that merged as 0989e6f243, and 1cfba6cc49 is on the trunk; d0d56cfc4e is the formatting commit on this branch, which the merge brings in.
9 #+pr: names 2359 while #+branch: still names the branch of #2358 this task answered The task spans two units and each has its own branch, so the field can only name one; it keeps the branch its first unit was built on, and the * PRs table carries both, newest first.
10 The row about the model's doc sentence cannot be checked from this diff the market series model answered The model is in the merged tree rather than in this diff. The sentence it names is the column's description, which qualifies "every row the corpus import writes" in the same breath.
11 The ingest loop's key building is untested, so the format it builds must match the grammar for every class with a point feed_ingest_loop.cpp recorded, no change The projection cases now carry both shapes the loop builds – a six-segment swap key and a six-segment vol key – and the corpus coverage test already names every per-class key the corpus carries. A helper would move four lines into one caller and would not exercise the loop, which has no harness; the loop's construction is recorded as a limit of the unit.
12 Dropping the point and the surface may merge series that should stay apart oresmd_parser.cpp answered, pinned The series row keeps the underlying and the point keeps the coordinates: the live table's CHF swaption series carries 10Y/10Y/ATM as one of its points, and Lufthansa/EUR carries 10Y/11.47. So dropping the surface merges nothing the registry keeps apart, and the case now pins that two swaption strikes project to one surface identity with no point in it.
13 Rows the import wrote earlier hold a datum identity, which the series lookup will not find import_service.cpp answered The import asks by identity and falls back to the triple, the same order the ingest loop uses, so an older row is still reachable; it keeps the identity it has. The schema is applied by recreation, so a rebuilt database holds no such row, and a backfill is a recorded prerequisite of the not-null unit rather than of this one.
14 The import case's qualifier assertion depends on the first point service_import_service_tests.cpp answered The qualifier is the registry's decomposition of the canonical key, and the unit changes only the identity the series stores. The assertion pins that both points land in one row under the series' own qualifier, which is what the case is for.
15 The key is concatenated on every tick feed_ingest_loop.cpp recorded, no change Negligible beside the database round trips the same function makes, as the reviewer notes.
16 The measurement cites 5e48f41fea without saying it is a merge the measurements table accepted Both new rows say it is the merge of #2362.
17 The DISCOUNT count is not covered by the row's command the measurements table accepted It has its own row and its own command.
18 The corpus claims carry no path the FOMC shapes accepted Both spellings name external/ore/examples, and the dated-OIS count is given.
19 The DISCOUNT shape could read as the tenor-in-identifier defect the FOMC shapes accepted A sentence says the curve is the series and the TENOR is the coordinate, which is why the family is in the defect list rather than the shape being wrong.
20 Which unit adds the shapes' tests is not stated Status accepted Next named the series-inverse unit at the time; the case landed with the projection fix instead, and the body records why the inverse is not needed.
21 OI_FUTURE is handled but untested oresmd_parser.cpp accepted, fixed The corpus's OI_FUTURE/PRICE/USD/2025-11/XCME:SRA/3M joins the survey, and it exposed a third split: an overnight-index future's series is the currency alone, so its contract month is part of the coordinate too and is now dropped.
22 The two resets were not separate, so a field could be cleared for a family that keeps it oresmd_parser.cpp accepted, fixed The resets are now one branch per family, each naming what its series holds.
23 Different contract codes can merge into one series oresmd_parser.cpp answered, pinned Intended: the registry gives a money-market future's series the currency and the contract month, so the code and the underlying tenor are the coordinate. A pair of futures differing only in month now pins that the series still separates.
24 No negative assertion, so over-clearing would pass the family case accepted, fixed Three pairs that must stay apart – two futures' months, two swaps' tenors, two curves' currencies – assert different identities.
25 One pair per family could hide a second shape the family case recorded The corpus coverage case and the per-class round trips remain the wider net; the survey is the identity rule's own guard.
26 std::is_same_v and std::decay_t need <type_traits> oresmd_parser.cpp answered The generated file compiles in CI and the nightly include check is the wider net; the template's own include list is where the header would go if it is ever named.
27 The curve's second maturity is compared but not round-tripped the FOMC shapes case accepted, fixed Both maturities now round-trip to their own keys, so the point is pinned as surviving the key as well as leaving the identity.
28 The case name says one series per pillar, but half of it is one series per curve the FOMC shapes case accepted, fixed Renamed the_fomc_segments_shapes_name_their_series.
29 The comment claims the republish service writes the curve the FOMC shapes case accepted, fixed The comment now says only what the case can show: the curve keeps ORE's discount shape.
30 A generated name is not percent-encoded, so a reserved character in a faker noun would not parse the model's generator recorded The limitation stands: the not-null unit is where a generated row's identity meets the parser, and a case there parses a few of them.
31 The URI is unique only if the generator's counter never repeats the model's generator recorded The unique index belongs to the cutover, and that unit has to say whether the counter is enough.
32 The model's paragraph was dense the model's generator accepted, fixed Reworded: the generator sets the identity explicitly, so the not-null is met by design rather than by a default value.
33 The migration backfills every config with the FOMC convention the migration accepted, fixed The default is RATES_SPOT_FORWARD for every null row, and only the seeded config's id takes RATES_SPOT_FOMC.
34 Historical rows would keep a null and fail the not-null the migration accepted, fixed The backfill drops the current-row predicate, so every row, history included, carries a convention before the constraint is set.
35 The convention code is free text the model recorded Validation belongs with the refdata convention lookups; the republish path already fails loudly on an unknown convention, and a key or a check is the refdata component's call.
36 No case covers a config that takes the forward convention the service tests recorded A second config or a fixture is the re-key unit's work; the record's limitation says the same.
37 A generated config's default convention must exist if a test republishes one the model's generator recorded No test republishes a generated config today, and the default matches the ordinary case.
38 The currency is derived from the qualifier on every tick ir_curve_feed.cpp recorded The substring runs once per tick beside a priced valuation and a publish; hoisting it is a one-line change the next time the feed is touched.
39 SPOT is a magic string in the builder ir_curve_template_resolver.cpp recorded The config's own spelling is SPOT and the resolver reads the same code; a shared constant belongs with the reader step that calls the builder from the other side.
40 std::format with a chrono calendar needs C++20 chrono formatting ir_curve_template_resolver.cpp answered The same call is already in ir_curve_feed.cpp for its schedule, so the toolchain's support is what the component already builds on.
41 The new case's name does not match the file's own style feed_factory_tests.cpp recorded The case reads as a sentence because it asserts a rule rather than a factory dispatch; renaming it is cosmetic and belongs with the next touch.
42 Nothing tests the whole feed path, so no tick is shown landing in a per-pillar series the feed recorded The reader step is where that path is exercised end to end, and the record says so in the plan.
43 currency_code is free text, so a typo fails only when the reader builds a key the model recorded The reader step owns the validation, as it does for the convention code; the model already says the field references currency.iso_code.
44 The migration's backfill is USD for every row the migration answered The migration's own comment says so, and the seeded config is the only row a database built here carries.
45 The generator's fixed USD gives no variety the model's generator recorded Generated configs are not data; a currency that varies would need the generator to read the currency table.
46 Other consumers of the config need no change curve_republish_service.cpp confirmed They read it rather than write it, and the reader that uses the column lands next.
47 The shell tests' arity change deserves a comment ir_curve_bootstrap_config_commands_tests.cpp recorded Cosmetic; the assertion's order is what the test file's own convention carries.
48 A pillar whose series exists but carries no quote at the point the read derived gets no rate and no fallback, so the bootstrap fails on a grid that could have answered curve_pillar_reader.cpp accepted, fixed A real gap: the read now treats that pillar as unresolved and takes it from the grid, and the new mixture case fails on the map size alone – 1 where 2 is required – when the check is disabled (d8333211c8).
49 read_latest_by_uri is called with no party_id, so two parties holding one URI make series.front() arbitrary curve_pillar_reader.cpp accepted, fixed The lookup is scoped to the config's own party, as the ingest loop scopes the series it writes, because the curve is bootstrapped for that party (d1708c0334).
50 The lineage now names the discount curve's output series where it named a config id, and no case covers it the lineage accepted, recorded Intended: what the list held before was a config id inside a list of series. The reader's cases pin the pillar ids; a PROJECTION config needs a second config and its own curve, so that case belongs with the next unit rather than this one.
51 The reader's path carries the most logic and has no test the reader accepted, fixed read_pillar_rates moved out of the service's anonymous namespace into its own header, and three database-backed cases cover it: the pillar's own series, the grid when it has none, and the mixture (d8333211c8).
52 compute() still logs while compute_curve does the work curve_republish_service.cpp confirmed The extraction gives both callers one read of the pillars and, now, one read of the config; the log line stays on compute(), where a caller sees it.
53 pillar_series_uri throws inside the read loop with no catch curve_pillar_reader.hpp accepted, documented The header now says that a tenor code which does not resolve and a key the grammar cannot name are configuration errors, and that they are deliberately left to propagate.
54 The header's "the series the first writes the second finds" reads awkwardly pillar_quote_key.hpp accepted, fixed Reworded to "the series one writes is the series the other looks for".
55 A database migrated before the feed has published per-pillar series will fail at bootstrap, where it used to fall back to the grid, and the PR should say operators run the migration and reseed the feed together the migration answered, recorded Intended and stated: the fallback was the tolerance for a half-published database, and the record says a pillar with no series gets no rate and the bootstrap fails loudly. The schema is applied by recreation, so an environment rebuilds from the create scripts and reseeds, which is the same order the plan gives the cutover.
56 An old database keeps the grid series and its observations, and the migration says nothing about them the migration accepted, fixed The migration now says the rows are inert, that nothing reads them, and why a temporal table's rows are not deleted by a column drop; a rebuilt database has none.
57 The two hand-maintained per-component diagrams still name source_series_id ores.refdata.api.puml, ores.refdata.core.puml recorded They are refreshed by a manual pass, and already lag the model by the currency_code and tenor_convention_code columns this task's earlier units added; the PR body discloses it and the next manual pass carries it.
58 ores.refdata.core was not clean on the author's machine, so CI is the authority for the create script the refdata cascade confirmed CI's tests job builds its database from the create scripts and passed; the local failures are the drift bearings already reported, and none names this entity.
59 The tick comment says a key it cannot name is dropped only when the tick has no series of its own, but the drop happens before any lookup feed_ingest_loop.cpp accepted, fixed The comment now says the drop precedes the lookup, and why a row written before the identities cannot rescue the tick: that row has no identity either.
60 An unnameable key warns on every tick, which a live feed can flood feed_ingest_loop.cpp accepted, fixed The drop is reported once per key, through a once-per-key set of the same shape the unbound-binding warning already uses, guarded by the loop's mutex.
61 A migrated row's synthetic identity survives a later import of the real key, because the triple fallback reuses the row and never writes the real identity onto it import_service.cpp recorded, deferred True, and the healing write needs a decision this unit cannot make: writing the real identity onto a row that already holds it collides on the identity's index when another row has it, so which row wins is the deletion's question. Recorded as its prerequisite.
62 The migration is not atomic and does not name the rows that would fail the identity's index the migration accepted, fixed The file is one transaction, so a failure leaves the triple's index and the nullability as they were, and it reports the shared identities before creating the index. Re-running it against this environment's database is a no-op that commits.
63 The backfill touches closed history rows, so a temporal or update trigger could move their validity the migration confirmed The table carries an insert trigger only, and the update ran against this environment's database over 6079 rows with no trigger error; versions of one id share the generated name, as the comment says.
64 find_or_create_series's cache is keyed by the triple, not the identity import_service.cpp accepted, fixed Keyed by the identity now, which is what the model keys the row by; an empty identity is no longer possible, so the key is never empty.
65 The review did not re-run the C++ suites the PR confirmed CI's tests job builds its database from the create scripts and passed, together with the other fifteen checks.
66 Dropping the fallback means an old row with an empty identity no longer matches, so the import creates a second series beside it the import and the ingest loop answered A row without an identity cannot exist: the column is not null with a non-empty check, the migration backfills what predated it, and a rebuilt database is created with both. The second series is the intended outcome rather than a risk – it carries the real identity, which the synthetic row does not.
67 The snapshot request's shape is a wire break, so an out-of-tree client must send the identity the snapshot protocol answered, recorded True, and the request has no sender in the tree, so the compiler and the protocol gates are what verify it. An external client is a real dependency this change cannot see, and the hand-back for PR 6/V06 carries it.
68 series_type, metric and qualifier may now be unused in the two writers the import and the ingest loop answered The suite builds with the project's warnings as errors, which failed this task's build earlier for one unused variable; both names are still read where the series is created.
69 The handler has no exercised path, so a URI lookup in it has no coverage curve_snapshot_handler.hpp recorded No harness reaches the handler: it takes a live NATS client and a database context, and nothing in the tree sends the request. It is this unit's recorded limitation and a V06 hand-back item.
70 The binding's identity is projected back to a key on every tick rather than once feed_ingest_loop.cpp accepted, fixed The projection is kept on the binding's stats entry when that entry is made, and the tick loop reads it; the PR text said once per binding and the code did not, which the fix makes true.
71 The key-to-identity-to-key round trip is untested, so a key that does not survive it would change the republished subject silently the binding's premise accepted, fixed Two cases: the FX keys a bound feed publishes survive the trip, and a family whose series key is not a quote key projects back to nothing. The second was not known before the case was written and is now pinned as the limit.
72 A non-projecting identity is handled two ways, one keeping a binding the status view can never show ticking feed_ingest_loop.cpp answered Deliberate: the stats map exists so a binding that never ticks is still visible, so refresh keeps it with an empty subject and reports it once, and the tick path skips it without repeating the report.
73 The migration's delete needs checking against the table's delete rules and its notify trigger the migration accepted, fixed Sharper than the finding. The delete is ruled to a validity close, so the rows stay and the not null applies to them, and the installed notify function reads the dropped column, so every write fails until the generated scripts are re-applied. The migration now backfills every row and names the two scripts that must follow it; both were proved by probe: a delete reports DELETE 0 and leaves one row with none current.
74 The dq publish's generic identity is unique only if no component holds its separator, and leaves URI-reserved characters unescaped marketdata_publish_from_dq_create.sql accepted, fixed The name is the key reduced to URI-safe characters with a hash of its own components appended: two keys that collapse to one name still differ in the hash, so a collision cannot fail the publish.
75 A dq publish test would be worth having sooner than V06, since the missing test is why the regression shipped green the dq publish recorded True, and it is the same missing harness as the handler's: the function needs a seeded dataset and the call shape the dq handler builds. Recorded as a V06 hand-back item; the repair itself is proved by calling the function against the database, before and after.
76 The FX vintage path's comment claims the projection is the one the ingest loop applies, but this file's own tick loop splits the key a different way fx_spot_feed.cpp accepted, fixed The comment now names the marketdata service's ingest loop as the reader it agrees with, rather than leaving "the ingest loop" to be read as this file's.
77 find_series_by_uri and find_series repeat the list, send and error handling market_data_client.cpp answered Both are the same ten lines around a different match predicate, and one of them goes with the triple when the IR caller does, so a helper would be an abstraction with one unit's lifetime. Recorded rather than taken, and the deletion revisits it.
78 find_series_by_uri could be tested with a mocked list response the client recorded The client suite covers the pure subject helpers only; a mocked response needs the NATS stub that market_data_client's send() calls through, which no case in the tree has. The same missing harness as the handler's and the dq publish's, recorded with them for V06.
79 The retained find_series's doc still says "natural key" where the identity is now the natural key market_data_client.hpp accepted, fixed It says "the same decomposition" now.
80 The test helper's comment says it goes with the columns; linking it to the task would stop it being orphaned service_import_service_tests.cpp accepted, fixed The comment names task 4050BF6C-DBC3-4E75-A204-A75CE4AFA323 now.
81 The revision could not be diffed against a merge base, so the build and tests were not re-run the PR answered CI's tests job builds its database from the create scripts and passed, together with the other fifteen checks.
82 A dated plan still mentions read_latest_by_type doc/plans/2026-03-31-ore-marketdata-domain-model.org recorded It is a historical plan document; the record's own measurement counts C++, headers and models, and the plan is left as the dated artefact it is.
83 The projection's helper could be exposed for a test, and a surface's coordinate is not a point curve_republish_service.cpp, oresmd_parser.cpp answered The helper is ten lines over the grammar's own projection, so exposing it would be an interface with one test as its caller; the service's missing harness is recorded with the others. The surface refusal is deliberate and pinned: its coordinate is the vol point, which is not one string.
84 The vintage config's migration could not land on a database that had a vintage row, because the check requires the column the migration had just left empty the migration accepted, fixed The migration names every vintage row before the check arrives, from the publishing dataset's own identity where it covers the row and from the currency's deposit-grid spelling otherwise, and it was proved by dropping the column on the live database and running it.
85 A force-push after a rebase can leave GitHub's mergeability stale, and a pull_request workflow then runs nothing the PR recorded mergeable: UNKNOWN and an empty check rollup after rebasing #2406 onto main; closing and reopening the pull request recomputed the merge state and dispatched all fifteen checks. Rebasing is still right, and the reopen is the step that follows it.
86 An entity's regeneration also moves the recipe its shell library is tangled from, and the tangled scripts are a separate gate doc/recipes/shell/ir_curve_generation_configs/ accepted, fixed drift failed on three ir_curve_generation_configs-*.ores scripts that still wrote the record without the new field; compass build --direct tangle_shell_scripts and a commit fixed it. A model field now costs a tangle beside the entity regeneration.
87 The map projection keys the Record by the C++ key type, and a numeric key would state Record<number, V> while the wire carries text org_loader.py accepted, fixed The Record is keyed by string always, because a JSON object's keys are strings whatever the C++ key type is. Only the value decides the projection, so a key type with no projection is no longer a gap; the test covers a non-string key and an unmapped value.
88 The ORE-side docs and backlog still name the deleted headers doc/plans/…, product_backlog answered The one current recipe that named a deleted header is updated to operations_protocol.hpp. The remaining hits are a dated plan, idea captures under product_backlog and sprint 23's task records, and one is a different component's ore_import_protocol.hpp; all are historical artefacts.
89 The claim that the wire does not move rests on the gates, and the reviewer did not diff each field against the deleted headers the seven headers, operations_protocol.hpp answered Checked by hand as well. A parser over the seven deleted headers and the generated one reports the same 28 structs with the same field names, types, initialisers and subjects; the only difference is std::chrono::year_month_day date{} rendering as date = {}, which value-initialises the same.
90 ores.service.tests fails under ctest standard_service_options_tests.cpp accepted, fixed The reviewer agreed it is unrelated to the model. 91cf5273ab on trunk made the console the always-on sink and file logging opt-in, updated the two logging suites that pinned the old defaults, and missed this third one. The test now asserts the new contract and keeps the file path covered with an explicit --log-filename; the two signatures that still promised a default now say the name is retained for the caller.
91 regenerate_cmake_component_files.py --all --check names iam/api and workflow/api component_files.cmake accepted, fixed The IAM list was fixed on trunk while this branch waited, so the rebase took it. The workflow list was not: 45f36c0d66 added workflow_retry_protocol.hpp and never regenerated the list. The regeneration adds it, and --all --check then exits 0.
92 The local check run was made before the branch was rebased, so it verified the branch rather than the merged tree the whole set accepted, fixed The branch was rebased onto trunk and the whole set re-run there, with the database recreated from the new scripts and the fleet started: the tree builds whole, ctest passes 89 of 89, all 35 local checks pass, and the TypeScript checks pass.

12. Result

The additive half of the identity cutover is delivered. market_series carries oresmd_uri, and both writers of series rows supply it: the import, from from_ore_key() for a market-data key and from from_index_name() for a fixing's index name – the latter's first production call site – and the ingest loop, which derives it from the ore_key it already builds. The repository reads by identity through read_latest_by_uri, declared in the model beside the triple reader, and the import's and the ingest loop's lookups ask by identity first and by the triple after it, for a row written before the column existed.

Nothing is deleted. The triple still keys the row, so a series with no identity is still reachable and the component cannot be stranded by this change. Measured on the rebuilt tree: 1782 assertions in 400 cases, including a case that imports one key from each space and reads the series back by its identity and by its triple, both agreeing. Component drift is clean for marketdata at all eight addresses, including the entity that gained the column.

A second unit, on the same column, made the identity name a series rather than a datum. The import projected the file's whole key and the ingest loop projected the triple alone, so a series' identity carried whichever of its points arrived first and a swap series got none from a live tick at all. to_series_uri() projects the identifier with its point dropped, both writers call it, and the ingest loop builds the datum's full key before falling back to the triple. Two IR families kept their coordinate where that projection did not look – a discount curve's maturity and a future's contract code and tenor – and it now drops those too, with a case walking one two-point pair per family and three pairs that must stay apart, which is what guards the drop against over-clearing. Measured on the rebuilt tree: 1839 assertions in 403 cases.

A third unit put the grid's pillars on their own identities. The builder that spells a pillar as the OIS quote the feed simulates moved into ores.marketdata.core, beside the parser its identity goes through, and the bootstrap reads each pillar's observation from the series that key names, scoped to the config's party. The lineage names the series each pillar was read from, in place of the grid they used to land in. Measured on the rebuilt tree: 1865 assertions in 408 cases, and 178 in 27 for the service, including the reader's database-backed cases. The plan's claim that the resolver's FOMC case seeds the grid was wrong and is corrected in place; the reader's path it was said to prove has cases of its own, added after the review.

A fourth unit dropped the grid and the column that named it. The FOMC seed writes the curve row alone, and source_series_id left the config's model, taking the reader's grid fallback with it: a pillar whose series is absent, or carries no quote at the point the read derived, now gets no rate and the bootstrap fails loudly on it. The removal regenerated the config's addresses – domain, protocol, entity, mapper, history field mapper, generator, service, shell command, recipe and the TypeScript twins – and a migration drops the column, with its check, from a database created before the change. Measured on the rebuilt tree: 176 assertions in 28 service cases, 4284 in 1518 shell-refdata cases, and 588 in 81 refdata-api cases, with the ER diagram regenerated.

A fifth unit made the identity the key. oresmd_uri is not null, non-empty and unique with party_id, and the registry's three columns no longer key the row; a migration backfills a row that has none from its own uuid and takes the triple's unique index with it. The import and the ingest loop each stop storing a row they have no identity for: the import drops a market-data key or an index name the grammar cannot name, and the loop drops a tick whose key it cannot name, both reporting what went. The corpus names every key it carries, so no corpus row is dropped, and the one case that pinned a short key's empty point now pins the drop instead. Measured on the rebuilt tree: 1879 assertions in 412 core cases, 176 in 28 service cases, 144 in 51 shell-marketdata cases and 71 in 14 marketdata-api cases.

A sixth unit retired every reader a repository-method grep could find. The import and the ingest loop dropped their triple fallbacks, because every row carries an identity now and a row an earlier database named synthetically is not the series a key names – which is also how the identity review's finding about those names resolves: the import writes the real identity instead of reusing the synthetic one. The curve snapshot's requests carry the identity where they carried the registry's decomposition, and the handler reads by it; the grid those requests were kept on has gone. The reading that came out of it at zero readers was too strong, and the seventh unit's measurements correct it: the marketdata client filters a whole series list by the triple in find_series(), which no grep for the repository's own method sees. Measured on the rebuilt tree: 1879 core assertions in 412 cases, 176 in 28 service cases and 71 in 14 marketdata-api cases, with the marketdata component drift clean.

A seventh unit named the binding by identity and repaired a writer the constraint broke. feed_binding.ore_key becomes the series' oresmd identity, which the ingest loop projects back to the key for the subject and for the decomposition; the synthetic service's feed controller projects the key its caller passed, so it supplies the identity from where it supplied the key. The identity's not-null then caught the dq publish function, which inserts market_series rows and named none of them an identity: calling it for the IR deposit dataset failed with null value in column "oresmd_uri"..., and after the repair the same call inserts its row under a generic name. Two consequences are recorded rather than repaired: a DQ-published series carries a generic identity rather than the ORE key its source names, and an instrument both the dq path and the feed write now has one series per writer. Measured on the rebuilt tree: 1879 core assertions in 412 cases, 176 in 28 service cases, 102 in 29 synthetic-api cases and 297 in 46 synthetic-service cases, with marketdata and synthetic component drift clean for the former and a pre-existing trunk drift in seven synthetic shell recipes for the latter.

An eighth unit left the client's triple lookup with one caller. Three called it; the FX feed's vintage seeding and the feed controller's vintage check hold keys the grammar names, so they project them and ask for the identity through a new find_series_by_uri(). The caller left is the IR curve's vintage seeding, and it is the one that cannot move the same way: the series it reads is the DQ deposit grid, whose key is RATES/YIELD rather than an ORE market-data key, so the cutover names it generically and this path cannot construct that name. Moving it means the synthetic config naming the series it seeds from, or the dataset carrying the ORE key its own source names; the call site says so and the handover carries it as the next decision. Measured on the rebuilt tree: 1893 core assertions in 414 cases, 6 client cases, and 297 in 46 with 65 in 22 for the synthetic suites – smaller than the seventh unit measured because another environment's merge deleted the organisation generator and its two test files (04f6092cfd), not because this change removed anything.

A ninth unit deleted the reader nothing called. read_latest_by_type was how a series was found before the identity keyed it, and every production caller had moved: the import, the ingest loop and the curve snapshot read by identity, the client lists and filters, and the IR vintage seeding goes through the client. Only its tests kept it alive, so it went from the model and the generated repository with them. The import tests keep every assertion – a helper filters the list for the key the import was given – and the repository's own by-type case became a by-identity one. Measured on the rebuilt tree: 1893 core assertions in 414 cases, and no reference to the method left.

A tenth unit stored the key a tick's datum is written under. The import keeps the file's own key on every observation; the ingest loop did not, which is why the export had to rebuild one for every feed row from the registry columns. It stores it now, by the export's own rule, so nothing the export writes changes and the row carries the key the deletion needs it to carry. Measured on the rebuilt tree: 176 service assertions in 28 cases and 1893 core assertions in 414.

An eleventh unit gave the grammar the projection the deletion was waiting on, and made the republish writer keep its key. The export rebuilt a datum's key from the series' registry columns because a datum's key needs its point, and which identifier field holds that point differs per family; two hand-written helpers guessed it and both were reverted. The fact now lives in the grammar, beside the to_series_uri() drop it inverts: oresmd_parser::with_point() places one observation's point in the field its family keeps it in – a discount curve's maturity in tenor, a future's coordinate split into its contract fields, every other indexed family in point – and refuses a surface, whose coordinate is the vol point rather than one string, and a fixing, whose key is an index name. The family survey's pairs walk backwards through it in a new case: each corpus key projects to its series, the point goes back in, and the key projects again. The republish service, the last writer that stored no key, projects its observations' keys from the output series' identity and the pillar's point and keeps them on the rows, so the export's rebuild has nothing left to serve. The service has no harness, so what is proved is the projection and the whole build, not a republished row read back; that is recorded with the other missing harnesses. Measured on the rebuilt tree: 1920 core assertions in 415 cases, 176 service assertions in 28, and the codegen suite at 1016 passed.

A twelfth unit settled the vintage and took the last triple lookup with it. The IR curve's vintage seeding read the DQ deposit grid through the registry's decomposition, and the grid's series was published under a generic identity no projection rebuilds – the one caller the client's triple lookup had left. Both halves of the open decision are true, so both are built: the datasets name each row's own ORE key and the series that key projects to, because the rows come from ORE files whose keys say so, and the synthetic config names the series its vintage reads, because the series is what the catalog is keyed by. The DQ publish creates the deposit grid's series under oresmd://ir/usd?tenor=2d&type=quote&metric=rate&quote=mm and stores MM/RATE/USD/2D/3M on the observation, and the config's vintage_series_uri is the same identity, so the feed asks for it and scans it at the DEPOSIT anchor's own point. The client's triple lookup goes with its caller, and the classification rule stays: it classifies a key's own tokens, never a series row, so the acceptance that named it is corrected here. The registry's three columns stay on the artefact and in the publish one step longer than their use, because market_series still refuses an empty series_type; the deletion takes both away at once. Measured on the rebuilt tree: the artefact names 339 rows across 14 series, the publish creates the series and the key above, and the suites hold at 65 synthetic-api assertions in 22 cases, 297 synthetic-service in 46, 6 client in 6 and 1926 core in 415.

A thirteenth unit closed the gap that let a rename break the tree. No pull-request check compiled C++: the job named tests runs the codegen suite, every other job is a Python gate over generated output, and codegen-drift.yml narrows itself to generated paths – so the binding rename that stopped ores.iam compiling passed sixteen green checks. pull-request-verify.yml now runs on every pull request, classifies the diff against fourteen tested cases, and calls cpp-verify.yml when a change can break a build; that workflow recreates the database, builds every target, runs ctest, and starts the services when the change can affect one. Its heavy job ran green on the pull request that adds it: build, test and start the services / linux-clang-debug-ninja completed successfully, which is the check this record had twice reported as unwritten or unproven. A documentation change now costs one light job, and a C++ change costs the build.

A fourteenth unit finished the cutover: the registry's triple is gone. The identity was the natural key, every writer supplied one and no reader asked by the triple, so series_type, metric and qualifier left market_series, the DQ artefact and the publish function. The census's 21 + 32 C++ files turned out to be one generated entity's layers – which a regeneration takes – plus four hand-written places, and those four carried a real defect: market_data_client::save_series never copied oresmd_uri, so every save through it cleared the name of the row it wrote. The export stopped rebuilding a key from columns that no longer exist: it emits each datum's own key, which every writer stores, and refuses a row a writer left without one; a fixing's index name is now a projection of the series' identity rather than a read of the qualifier column. The classification rule stayed, as the acceptance says, because it classifies a key's tokens. Measured on the recreated database: the series row carries no triple, the DQ publish creates the identity-only series and keeps the observation's key, the tree builds whole, and the suites hold at 68 in 14 marketdata-api cases, 1923 in 415 marketdata-core, 176 in 28 marketdata-service, 6 in 6 client, 144 in 51 shell-marketdata, 65 in 22 synthetic-api, 297 in 46 synthetic-service and 432 in 153 shell-synthetic, with the codegen suite at 1030 and reachability at 7741 declared cases. The fifteenth unit below delivers V06.

Three measurements taken here changed the plan rather than the code, and each is recorded with its number:

  • the feed_binding column cannot be written, because its only writer is a generated service with no derivation hook and a model's generator block feeds test data;
  • the column cannot be made not null yet, because the FOMC bootstrap seeds a series under the synthetic pipeline's own vocabulary, which no projection can name – the replacement is the instruments' own ORE keys, decided above;
  • marketdata is one modelling pass short of COMPONENTS_UNDER_TEST, because seven hand-written protocol headers carry no TypeScript twin and a twin is generated from an operation model. Listing the component was tried and reverted, and the modelling pattern to follow is read off ores.variability.operations.

What the deletion still owes is recorded where it belongs: the internal series' identity, a backfill or a recreated database, the unique index on the identity, and the behaviour that a write omitting the identity clears it until the column is not null.

A fifteenth unit joined the component to the registry. V06 was the whole of what stood between the pass and its close: seven hand-written protocol headers carried no TypeScript twin, and a twin generates from an operation model. They are one model now, ores.marketdata.operations – twenty-eight messages covering the CRM rate reads, the curve bootstrap, republish and snapshot subjects, the whole-tenant ORE import and export, and the folder feed control – and the C++ header and operations_protocol.ts both generate from it. Every field keeps the type, the initialiser and the subject the hand-written header carried, so no subject and no payload on the wire changed; the one shared codegen change was a projection for std::map, which now renders a TypeScript Record rather than leaving the model refused. The seven headers are deleted, and the sixteen files that included them read the one generated header. Measured on the branch: marketdata regenerates byte for byte at every address, the twin gate reports 199 headers in 13 committed components with no model opting out, the drift check reports no drift across the catalogue, the registry lists 15 components, and the codegen suite passes 1032. The whole pull-request check set was then run locally. The first run found two breaks, both pre-existing on trunk, and both were fixed rather than recorded:

  • ores.service.tests pinned a log filename that 91cf5273ab ("Make the console where logging goes, not a default filename") deliberately stopped defaulting. That commit updated the two logging suites and missed this third one, whose expectation was the old contract. The test now asserts the new one – --log-enabled alone gives the console and an empty filename – and keeps the file path covered through the composed options with an explicit --log-filename. Both signatures that still promised a default now say the name is retained for the caller and is not one.
  • regenerate_cmake_component_files.py --all --check named iam/api and workflow/api. The IAM list was already fixed on trunk while this branch waited, so the rebase took it; the workflow list was not, because 45f36c0d66 added workflow_retry_protocol.hpp and never regenerated the list. The regeneration adds the header.

On the rebased tree, with the database recreated and the 24-service fleet running, the tree builds whole, ctest passes 89 of 89 targets, all 35 local checks pass, and the TypeScript typecheck, format check and 51 tests pass beside them. V06 is met, and the component is under the gates from now on.

Emacs 29.3 (Org mode 9.6.15)