Task: Model each convention family on its own

Table of Contents

This page documents a task in the Refactor conventions onto families and bundles story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Every convention family is declared once, in its own entity, and carries only its own fields. The list of families is generated from those entities, and so is the code that writes and reads them. No type holds every family, so no translation unit has to reflect them all.

2. Status

Field Value
State DONE
Parent story Refactor conventions onto families and bundles
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-19

3. Acceptance

  • Each convention family is its own entity with its own fields, and a family's field belongs to it alone.
  • One generated registry names every family, its scope, and its repository, and the hand-maintained list is gone.
  • The apply code that writes the families is generated from the same registry, so adding a family needs no hand edit.
  • No reflected type in refdata carries every convention family, and no translation unit instantiates one.
  • conventions_document, conventions_document_service and the whole-document protocol operations no longer exist.

4. Plan

The work ran in two halves, and the halves are why the task is still open.

The first half generated the family list and the per-family apply, and it is merged. Before the hand list was touched, the registry was compared with it. The two agreed on 26 families, on the same party and world scopes, and on every row the service wrote, which is what made the swap safe. The apply could not come from the registry alone, because a table of strings cannot call a typed repository, so the same generation emits a branch per family.

The second half is the deletion, and it forces its callers to move in one wave. conventions_document is the payload the ORE mapper builds and reverses, and the run store is the service's only production caller. Deleting it means moving the whole-document service, the six generated apply and read functions, the mapper pair, the legacy importer and exporter, the diff, the service import and the run store onto the bundle.

4.1. The shape the deletion builds towards

Chosen on 2026-10-18 from three designs scored against this task. The scored designs and the synthesis are in the-package-design.md.

A family is a registry key and one payload per row, where a payload is that row serialised as its own generated type. Nothing else identifies a family.

refdata::service::conventions_bundle holds rows of { family, sequence, payload }, and no member is a family's value type. It arrives hand-written in the first step, because the shape is still moving, and moves into generation when the per-family units land. conventions_bundle_service::apply(bundle) is the only writer and ::read(party) the only reader. Neither opens a unit of work, so the commit passes its transaction and the run store passes its own.

The two tenor basis swap obligations become the prologue and epilogue of apply, in the caller's transaction. The prologue resolves the identity after stamping, because the identity is the party, the kind and the name, and the mapper cannot state the party. The epilogue closes the omitted leg form after the writes. The writer decides the write order for the shared family: it writes the central row and then its two part families in code, so the foreign key that a part carries to its central row holds whatever order the bundle is in.

ores::ore::domain::conventions_package belongs to ores.ore. It holds one bundle and the FX projection, which is a projection of three entities, is never stored, and never crosses the boundary. split_conventions and join_conventions replace conventions_mapper::map and reverse.

Both directions are generated one family per translation unit, with a dispatch unit that includes only declarations, so no unit reflects two families. A convention_family_key enumeration is generated from the registry with a static_assert tying it to the table. A codegen gate holds a translation unit to one reflected family, so the aggregate cannot come back quietly.

The plan is five steps.

  1. Pin today's behaviour. Tests only, with byte equality on exported files and canonical row facts, so every later step is judged against them.
  2. The refdata write side and read side. The bundle, the per-family units, the obligations, and the commit on top of them.
  3. The ores.ore boundary. The package and the generated bindings, with both callers moved and a declared bridge through the transition.
  4. Delete the aggregate, the service, the six bulk functions and the model section, retiring the bridge.
  5. The translation-unit gate, and the measurements this task records.

Step 3 arrived in one change. The package, its two directions, the tests that compare it with the document and the move of both production callers all came together, so the new shape had callers from the first commit. The bridge is the mapper pair itself: map and reverse still turn a document into the rows the legacy file importer and exporter, the diff and the tests work in, and step 4 removes them with the type.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
Families in the generated registry 26 grep -c 'convention_write_mode::' projects/ores.refdata/api/include/ores.refdata.api/domain/convention_family_registry.hpp 8d7c38449c
Corpus files the package carries one row per element for 72 ctest -R ores.ore.core.tests --test-dir build/output/linux-clang-release-ninja ebcf187bca
Lines the aggregate deletion removed, net -1077 across 53 files git diff --shortstat origin/main...ebcf187bca ebcf187bca

6. Notes

  • Step 1 is merged: the export of all seventy-two corpus files is pinned by digest, and the rows an example import writes are pinned by reading the tables directly.
  • Step 2's write side is merged. conventions_bundle_service applies a bundle of family payloads in the caller's transaction, the two tenor basis swap obligations now live in conventions_identity and are called by both write paths. The writer gives the shared family its own order, and the commit sorts its staged rows into registry order so the rows and the reported outcomes do not depend on the order the stages landed in. The staged path gains the obligations it lacked, which is a behaviour change and the one this step declares.
  • The registry does not yet say which families are one bundle. The bundle service names the three tenor basis swap families by hand, and the commit sorts by the registry's alphabetical order. A registry field that states the dependency order would remove the hand list and make the ordering rule explicit.
  • The run store has no unit of work, so it applies a bundle without one and the import is still not atomic. The bundle service takes the caller's transaction, so the fix is the run store opening one, not a change in the service.
  • Step 2's read side is merged. The generated fragment now carries read_family_payloads, one branch per family, and conventions_bundle_service::read assembles every family the session sees into one bundle of payloads. The shared convention family is narrowed to the kind this bundle covers, because that one table carries every kind. A test proves the bundle read carries the same rows as the document read, which is the bridge the next step removes.
  • Step 3 merged as #3562. ores::ore::domain::conventions_package holds one bundle and the FX projection, and split_conventions and join_conventions are the two directions over it. The package is now what crosses the boundary: the run store and the run configuration operations both read into it and write from it, and neither of them assembles a document any more. The run configuration operations lost the whole-document reflection that turned a document into generic JSON and back; per-row rfl::json remains, because a payload is a row.
  • An import now applies one row at a time: apply_staged_family decodes one payload and writes it, and the row helpers read the family's table on every call. The document path read a family once and wrote its rows together, so a family of many rows costs a read per row where it used to cost one. That is the shape the staging path already had, and the fix is the per-family apply the next two steps write: one call per family carrying its rows.
  • The FX convention is the only part of the package that does not enter the bundle, and that is what the projection means. It comes back on the export only when the same package is joined, which is why the store never returns one: the three entities it is a projection of are the store, and the read does not assemble them.
  • Step 4 deleted the aggregate. conventions_document, conventions_document_service, the six bulk apply and read functions, conventions_mapper::map and reverse, and the model section no longer exist. The importer returns a package, the exporter writes one, and the diff compares two packages by their family rows. apply_staged_family lives in the bundle header, and conventions_save_result is gone because conventions_apply_result carried the only field callers read. The tests read a family through rows_of, which names the family and its type. The pinned export digests judge the package, so the test that compared the package with the aggregate was removed with the aggregate.
  • Step 5 split the refdata units. A new facet, ores.cpp.convention-family, renders one translation unit for each family entity that opts in, holding that family's apply and payload-read functions and reflecting its own type alone. The registry facet renders the declarations header and a dispatch that looks a family up in a table of function pointers, and the registry header gained a convention_family_key enumeration tied to the table. The dispatch asserts one row per registry family and the registry's order, so a family without a unit does not compile. The apply fragment, its includes fragment, the six groupings they read and the repositories list went with it.
  • The translation unit budget is a local check, translation-unit-budget. A file that reflects more than one convention family fails it. Three files are over budget and recorded with a ceiling that can only go down: the package's pack and join tables in conventions_mapper.cpp at 26, and the two units that handle the three tenor basis swap families together, which are one document element. The first is now closed by the task Split the ORE side into per-family units.
  • The pin holds. Comparing the service's hand-written calls with the generated registry gives 26 families against 26 entries, no disagreement about which families are party-scoped and which are world-scoped, and no registry row the service never writes. fx is skipped and unregistered, as decided. The registry is therefore equivalent to the hand list before the hand list is deleted.
  • The registry cannot replace the calls by itself. A table of strings cannot call replace_party_rows(ctx, zero_convention_repository(), ...), because each family needs its own repository type. The dispatch has to be generated. Codegen must emit one apply function per scope from the same registry: party families through replace_party_rows, world families through add_missing_world_rows, and the skipped families collected. The service then calls two generated functions instead of listing 26.
  • The work splits. First the generated apply functions, which delete the service's hand list with behaviour unchanged and the pin green. Then the aggregate itself, which is what forces the mapper, the run store and the BFF to move in one wave.
  • fx needs no entity and no registry entry. The document carries an fx_convention that is the pair, its currency_pair_convention, a spot lag and its advance calendars. All three of those are entities (currency_pair, currency_pair_convention, currency_pair_convention_calendar), so the ORE FX convention is a projection of existing rows, not a store of its own. Drop it from the document and derive it in the boundary mapper.
  • The tenor basis swap family registers as its central row plus its two parts, because that is what the document carried. Collapsing it to one family entry is a later decision, not part of this task.
  • The registry started as a python renderer and is now a codegen facet with three archetypes, so a template change replaces a script change. The emitted bytes did not change when the renderer was retired.
  • What the deletion still has to move, as the tree stands on 2026-10-18. The aggregate is declared in projects/ores.refdata/modeling/ores.refdata.configuration_document_messages.org and generated into ores.refdata.api/messaging/configuration_document_protocol.hpp. In refdata, conventions_document_service is the only service that builds it, and the run store is the service's only production caller. The service also drives the six generated bulk functions in convention_family_apply.inc, and those functions, not the per-family dispatch, are what reach the type. ores.ore holds the type besides, in the mapper pair, the legacy importer and exporter, the diff, and the service import. The tests that pin the current behaviour are conventions_document_service_tests and the conventions half of run_store_database_tests.
  • Two modelling comments still repeat claims this task's record no longer makes. bundle_import.hpp and ores.refdata.bundle_import.org describe a sweep that marks stale imports expired, and a failed commit that marks an import failed, and nothing performs either.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#3566 [refdata] Give each convention family a translation unit of its own
#3564 [ore] Delete the conventions aggregate and the whole-document service
#3562 [ore] Split the conventions document into a bundle of families
#3554 [refdata] Read a family's rows as the payloads the wire carries
#3542 [refdata] Apply a conventions bundle in the caller's transaction
#3459 [codegen] Generate the convention family registry
#3460 [agile] Record that the FX convention is a projection
#3462 [codegen] Generate the convention family apply fragment
#3469 [refdata] Give the generated conventions apply its own translation unit
#3474 [refdata] Compile the conventions apply in its own translation unit
#3481 [refdata] Apply the review of the conventions apply unit
#3500 [codegen] Render the conventions registry from templates

#3497 was closed rather than merged. It raised the same change as #3500 without the environment tag or the task trailers, so #3500 redid it.

9. Review

# Comment summary File Decision Notes
1 The python renderer dropped its 100 column wrap when the templates took over. convention_family_registry facet Accepted, no change The generated header's longest line is 80 columns, so nothing exceeds the limit today. The fix, if a name grows, is in the template.
2 The facet hardcodes the refdata output paths, so a second component opting in would overwrite them. convention-registry facets Accepted, recorded A second component needs the address parameterised before it enables the facet. The facet page states the limit.
3 #3562: join_conventions wrote a central convention row of any kind as a tenor basis swap, where reverse filters on the kind. The RPC read hands it rows of every kind, so an export could carry a leg-less element for a convention no document element names. conventions_mapper.cpp Fixed The join keeps only rows whose kind is the tenor basis swap, which is what the path it replaces did. A test writes a central row of another kind and asserts no element comes back.
4 #3562: the row-count test covered six of the twenty-six families. conventions_package_tests.cpp Fixed A test now walks the corpus, refuses a family the registry does not name, and requires the corpus to name every family the registry does.
5 #3562: an import applies one row at a time, so the row helpers read the family's table once per row where the document path read it once per family. conventions_row_helpers.hpp Accepted, recorded The staging path already had this shape. The fix is the per-family apply the next two steps write, and the note above says so.
6 #3562: the package header says one row per element, when a tenor basis swap element is a central row plus its part rows. conventions_package.hpp Fixed The header now says one row per stored row and names the element that is more than one.
7 #3562: conventions_document_service::save is production-dead, with only tests calling it. conventions_document_service.cpp Accepted, recorded Step 4 deletes it with the type it takes.
8 #3562: the benchmark rows and the generated shell script are unrelated to a conventions refactor. benchmarks, shell scripts Accepted, explained The gate asks for a benchmark row after a change to the XML code, and it fails on trunk without the shell script, so neither can wait for a separate branch.
9 #3562: the skipped FX convention name was spelled by hand in the run store and in the run configuration operations, where the mapper already owned the name. run_store.cpp, run_configuration_operations.cpp Fixed skipped_fx_conventions builds the list from a package, and both callers use it.
10 #3562: the row count test checked five of twenty-three families by name. conventions_package_tests.cpp Fixed The test now compares every row outside the three tenor basis swap families with the sum of all other elements in the document.
11 #3562: ores.ore.core links ores.refdata.core publicly because the package header exposes conventions_bundle. CMakeLists.txt Accepted, recorded The bundle is the boundary's own type. The link goes private again if the bundle moves to the api component.
12 #3564: the diff drops every row of the convention family, so a changed deposit, swap or FRA convention is no longer compared. conventions_diff.cpp Declined, proved In the package the convention family holds only tenor basis swap central rows. Every other kind packs into a family of its own, as it did in the aggregate. Two tests pin it: a document differs from itself by nothing, and a changed deposit id is reported.
13 #3564: the diff compares raw payload strings and the sequence, so a different row or field order gives a false difference. conventions_diff.cpp Declined Both sides come from the one split_conventions, whose order is the document order, and reflection writes fields in declaration order. The aggregate comparison was order sensitive in the same way.
14 #3564: apply now fills the caller-facing result directly, so nothing before the loop may write it. conventions_bundle_service.cpp Accepted, no change Nothing before the loop writes the result.
15 #3566: the dispatch names std::size_t and does not include <cstddef>. convention_family_dispatch.cpp Fixed The template and the generated unit include it.
16 #3566: the budget match is domain::<type> with no namespace, so another namespace's swap could collide. check_translation_unit_budget.py Declined The refdata services name the types unqualified inside their own namespace, so anchoring on refdata::domain:: would miss them. A collision gives a loud false positive, not a silent pass.
17 #3566: only .cpp and .inc files are measured, so a reflecting inline template in a header escapes. check_translation_unit_budget.py Fixed The scan covers headers and the two include trees. It finds no new file over budget.
18 #3566: a comment containing rfl:: makes a file count as reflecting. check_translation_unit_budget.py Declined It can only raise a count in a file that also names families, and the failure is loud.
19 #3566: the dispatch looks a family up by a linear scan of 26 rows. convention_family_dispatch.cpp Declined Nothing here is hot, and a map would add a type for no gain.
20 #3566: make sure the ceiling of 26 on the mapper is not forgotten. check_translation_unit_budget.py Accepted, recorded The task Split the ORE side into per-family units owns it, and the ceiling check fails if the file improves without the entry being lowered.
21 #3566: the 96 benchmark rows may be stray. benchmarks/xml/history.tsv Accepted, explained They are the one run the gate asked for after the codec move changed the XML code.

10. Result

Every convention family is now declared once, in its own entity, and carries only its own fields. The registry, the apply and the read are generated from those entities, and no type holds every family.

What shipped, by PR:

  • #3459, #3460, #3462, #3469, #3474, #3481 and #3500 generated the family registry and the per-family apply and moved them into their own translation unit.
  • #3542 and #3554 wrote and read a bundle of family payloads in the caller's transaction.
  • #3562 put the ORE boundary on a package of one bundle and the FX projection.
  • #3564 deleted conventions_document, its service, the six bulk functions, the mapper pair and the model section.
  • The step 5 PR split the refdata units, one translation unit for each family, with a dispatch that includes declarations only, and added the translation-unit-budget check.

Acceptance, point by point:

  • Each family is its own entity with its own fields. Met.
  • One generated registry names every family, its scope and its repository. Met.
  • The apply code is generated from the same registry. Met, and the read is too.
  • No reflected type in refdata carries every family, and no translation unit instantiates one. Met for the types, and for the refdata units. The ORE package's pack and join tables still name every family in one unit, so the budget check records conventions_mapper.cpp at a ceiling of 26. The task Split the ORE side into per-family units owns the rest.
  • conventions_document, its service and the whole-document protocol operations no longer exist. Met.

The decision to leave the ORE side for later is a dependency, not a gap. Another environment is moving the mapper onto the codec records, and splitting the same file at the same time would collide. The ceiling in the check keeps the debt visible and lets it only go down.

Emacs 29.3 (Org mode 9.6.15)