Task: Generate tables for every document family

Replace the generated domain with tables generated from ORE's twenty-eight schemas. This is the artefact a reviewer reads instead of 122,000 generated lines, so it has to be reproducible and it has to fail when it and the schema disagree.

Field	Value
State	DONE
Parent story	Decide the XML strategy for ORE documents
Now	Nothing.
Waiting on	Nothing.
Next	Nothing.
Last touched	2026-10-17

• Every complex type the corpus reaches has a table.
• Regeneration is one command and its output carries the generated marker.
• A drift check fails when a table and its schema disagree.
• The generator reads the schema with Python's xml.etree.ElementTree, a build-time tool that needs no project dependency.

(Implementation strategy. Written when work starts; key decisions are distilled into the parent story's * Decisions at close, but the plan itself stays — it is the historical record of what we did.)

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What	Value	Command	Commit
Families with a table	13 supported, 7 unused, over 20 root elements	python3 build/scripts/check_xml_family_coverage.py	7515c50292
Generator determinism	tables match the schemas, generator deterministic	python3 build/scripts/check_xml_bindings_drift.py	7515c50292

The earlier rows for corpus documents bound and suite size described a tree with 7 families. They are removed, not updated, because nothing re-measured them.

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario	State	Notes

PR	Title
#3574	[agile] Reconcile the open XML tasks with the tree
#3440	[ore] Generate one family's records and tables from its schema

Comment summary	File	Decision	Notes

The generator is projects/ores.ore/scripts/generate_xml_bindings.py, with regenerate_xml_bindings.sh as the lever that invokes it for each family. It reads the schema and emits both the C++ records and the codec tables, so the hand-written calendar table is gone and the family is entirely generated.

What it derives from the schema: which complexTypes exist, their order, which children are elements and which are attributes, cardinality, enumerations, and which simpleType restricts to which primitive. What it cannot derive, and takes from this project instead, is what a numeric element means: ORE types the same money concept as xs:float, xs:double and xs:string in three places, so the money list is an argument rather than a reading.

Verified: the binding cases pass against the generated records, the whole ore xml suite is 524 cases and 63,587 assertions, and re-running the generator produces byte-identical output, which the drift check proves by generating twice under different hash seeds and comparing the two runs.

A second family, currencyConfig, is generated as well. It is shaped differently on purpose: an xs:all rather than a sequence, an enumeration, and a repeated child two levels up. It binds the thirteen currency documents in the corpus, over a thousand currencies in all.

A third family, IborFallbackConfig, is generated for the shapes the first two do not have: an inline complexType at the root, and a money field the schema types as xs:float. That element, Spread, is passed to the generator as money, and the corpus document writes -0.00055099999999999995, which no double holds. It arrives as a decimal with every digit intact.

A fourth family, counterpartyInformation, is generated for the last shape the others do not have: xs:simpleContent with an extension, which means an element carrying text of its own beside its attributes. That is the shape the instrument and simulation schemas are built on, so it blocked all of them. The engine gained a text field per record type and the generator emits it.

A fifth family, todaysMarket, is generated, and it is the largest yet: a hundred records, one per curve and volatility section the document carries. It binds every todays-market document in the corpus, 99 of them.

Two defects came with it, both about names rather than shapes. ORE reuses element names across families, so two different types were both called Correlation and the header would not compile; the first keeps the plain name and the rest are qualified by the type that holds them. And two families each defined a correlation struct, so the records of one family now live in a namespace of their own and the two can be included together.

A sixth family, nettingSetDefinitions, is generated, and it needed the last structural feature: xs:element ref"nettingSetGroup"= points at an abstract element with two substitution members, so one position in the schema is two possible element names in a document. Each member becomes its own optional field, which is how the model states a variation without a polymorphic payload.

That family also found a real engine defect. ORE's currencyCode enumeration has more than a hundred spellings and the enumeration index was eight bits, so looking a spelling up walked off the end of the table and took the process with it. Sixteen bits is the floor.

The engine also refuses text that no field expects now, rather than dropping it, which is the same no-silent-loss rule applied to mixed content.

The drift check is build/scripts/check_xml_bindings_drift.py. It regenerates into a temporary root and compares, so a hand edit to a generated file and a schema change the tables did not follow fail the same way.

The drift check found a real defect in the generator the first time it ran against three families: emission order came from iterating a set of type names, and a set of strings iterates in an order that follows the interpreter's hash seed. Two runs with different seeds produced different files. Collection is now an ordered mapping, and the output is byte-identical across seeds.

The corpus found one real defect in the engine. ORE writes <Symbol></Symbol> for a required element with no value, and the presence bit was set only when text arrived, so an empty element read as missing. The bit now means "this element appeared", which is what it was always meant to mean.

Three defects the wiring found, both of the same kind as the earlier ones. The generated source was not in component_files.cmake, so it compiled nowhere. And the accessor needed the export macro, because the library is built with hidden visibility and a generated function has no more right to the symbol table than a hand-written one did.

conventions is generated now, the seventh family and the largest document model yet.

It needed a naming pass, which is what had blocked it. A generated struct name can collide with a name the engine already uses, and ORE has a type whose snake case is event_type, which is the scanner's own event enumeration. The generator now collects every struct name before emitting anything and qualifies any that collide with the engine's vocabulary. The table variables also carry a record suffix rather than type, so a struct called DatesType cannot collide with the table of a struct called Dates.

It also needed one modelling correction. conventions is an xs:choice at the top level, and choice members are alternatives, so none of them is required however the schema writes its minOccurs. Treating them as required refused sixty-seven documents.

Verified: 527 cases and 63,598 assertions.

The first version of this record claimed more than the code did. It said the generator reused the project's money vocabulary, when in fact money arrives as a hand-written --money argument with one entry in it. It said the drift check proved determinism, when the check ran the generator once. It said 106 todays-market documents when 99 carry that root. And it described the schema reader as pugixml when it is Python's ElementTree.

All four are corrected above. The money vocabulary is the one that still needs work: every family generated so far that carries money takes it from xs:decimal, which is the schema deciding the kind, and the investigation says the schema must not decide it.

Measured on main at 7515c50292. The census script reports 13 supported families and 7 unused, over 20 corpus root elements. The generated directory holds one table per supported family. The drift check passes.

The 7 unused families have no table by decision. The census checks that none gains one. The acceptance line "every complex type the corpus reaches has a table" holds for the families the product reads.

One point is still open. The exact-decimal list, the money vocabulary, is still an argument to the generator. The schema does not decide it. The review above named this, and it is a known limit, not a defect in the shipped tables.