Investigation: Populating ACME with a trade population

The story starts with bonds, so the first path was to establish whether bond support is complete and whether it follows the data-oriented design. "What complete" means here is stated by doc/knowledge/architecture/data_oriented_design.org, which classifies ores.trading as Regime 2 (persisted domain data) plus Regime 3 (wire) at lines 14-16 and 87-99. Regime 1 does not apply. Completeness for a family is therefore three things together:

• the Regime 2 decision surface is fully modelled, so nothing lives only in the tree. Lines 212-227 name the artefacts a review reads.
• the entity's facts split by change reason, multiplicity or reader under the composite pattern, with the parent version bump. That is codegen_patterns.org lines 60-73.
• the wire is a flat projection of those rows, with no nested objects and no polymorphic payload fields. Lines 280-286.

A passing build is not the test, and neither is coverage alone. The criteria page states the distinction at lines 244-258. The decision surface can fail, and codegen does not rescue it.

The audit found the family broad and codegen-bound. ls projects/ores.trading/modeling/ores.trading.bond*.org returns 42 orgs. Nineteen are ores.codegen.entity and 23 are ores.codegen.field_group. The nineteen entities are bond_instrument, bond_issue, bond_issue_call_date, bond_issue_conversion_target, bond_issue_leg and its five children bond_issue_leg_amount, bond_issue_leg_rate, bond_issue_leg_amortization, bond_issue_leg_schedule and bond_issue_leg_schedule_date, bond_leg and its three children bond_leg_amount, bond_leg_rate and bond_leg_amortization, bond_forward, bond_future, bond_option, bond_repo and bond_trs. The separate ores.trading.ascot.org carries the eleventh code.

Coverage is declared once, in projects/ores.trading/modeling/ores.trading.trade_type_catalogue.org. Parsing its * Trade types table at lines 47-172 gives 121 trade types. Thirty have an empty instrument cell and are therefore routable nowhere. The bond family is nearly whole. Ten of the eleven bond-typed codes route to bond_instrument:

Bond ForwardBond BondFuture BondOption BondRepo
BondTRS BondPosition CallableBond ConvertibleBond Ascot

The eleventh, BondFutureOption, is unrouted. The catalogue states the consequence at lines 24-27. An empty instrument means nothing imports it, so nothing can export it. The routed column check in the generated schema agrees. grep '^ check ("trade_type_code" in' projects/ores.sql/create/trading/trading_bond_instruments_create.sql returns exactly the ten codes.

The export carries the whole family through one carrier. bond_instrument_data, modelled at ores.trading.bond_instrument_data_field_group.org, holds 28 members. They are the header, the issue, the issue's call dates and conversion targets, the product's fact row, the bond's legs, and the document residue the tables cannot hold. The catalogue's ** Instrument types table at lines 201-209 is the one row that names it, and the generator emits the container from the model rather than by hand.

The composite pattern is half-built, and that is the central finding of this path. The pattern requires a child foreign key declaring :bump_parent_version:, :list_by:, :list_by_uuid: and :list_by_as_of: (codegen_patterns.org lines 60-73). It states what the bump buys at lines 67-73. A write to any part bumps the parent's version in the same transaction, so the parent's windows partition time into spans in which the parts are stable, and a read assembles the entity from the timeline with no link table.

Measured on the bond orgs:

cd projects/ores.trading/modeling
grep -c '^:bump_parent_version:' ores.trading.bond_leg.org          -> 0
grep -c '^:bump_parent_version:' ores.trading.bond_instrument.org   -> 0
grep -c '^:list_by: *true'       ores.trading.bond_leg.org          -> 1
grep -rl '^:bump_parent_version:' *.org -> structure_member, trade_additional_field,
    trade_booking, trade_identifier, trade_party_role, trade_portfolio, trade_state
grep -rl '^:list_by_uuid:' *.org -> (none)
grep -rl '^:list_by_as_of:' *.org -> (none)

The bond children have the list-by-parent read and none of the bump. The two as-of flags exist nowhere in the component. The trade anchor's own parts do declare the bump, which shows the pattern is expressible and was applied to one family and not the other. The consequence is concrete. bond_leg, bond_leg_amount, bond_leg_rate and bond_leg_amortization can each be rewritten while bond_instrument.version stands still. An as-of read of the family therefore cannot say which legs were current when.

Three smaller gaps follow from the same audit.

• The declared foreign keys are not database foreign keys. Both bond_instrument.org at lines 145-210 and the generated create script describe the trade, activity and issue keys. ./compass.sh db sql -- -At -c "SELECT count(*) FROM pg_constraint WHERE contype'f';"= returns 25, and every one is a TimescaleDB catalogue constraint. The bond keys are trigger checks instead. That is correct for a temporal parent and wrong for the immutable lookups the same models describe as database foreign keys.
• bond_instrument.org line 36 records its own coverage limit. It says the real foreign key to ores_trading_trade_types_tbl is PR 4's defect
  1. It has not landed.
• The issue sits outside the economic digest. trade_type_catalogue.org ** Instrument components at lines 269-318 lists bond_instrument as its own only component, and bond_issue appears in no digest table. The issue row holds the bond's terms. An amendment to those terms therefore moves the trades' economics without moving their economic_digest or their external_version (trade.org lines 153-195). Whether that is intended is a decision rather than a defect, and nothing in the models states it.

The second path was to stage the population as a DQ artefact and publish it. It fails on three grounds, each of which survives a redesign.

doc/knowledge/architecture/dq_artefacts.org states the purpose in its first sentence at lines 14-16. An artefact is a staging table that holds an entity's rows by dataset until a publication writes them into the service that owns the entity. Line 210 makes the ownership explicit. It is the owning component's data in transit. Its guarantees are the publication contract at lines 99-130, which are idempotence, three accepted modes (upsert, insert_only, replace_all), a return of one row per action taken, clock_timestamp() for row closure, and refusal of any target but the system tenant for installation data.

A dataset is registered in ores_dq_datasets_tbl, whose columns are id, tenant_id, version, code, catalog_name, subject_area_name, domain_name, ..., as_of_date, ingestion_timestamp, ..., artefact_type, valid_from, valid_to. The model of time is stated at doc/knowledge/domain/data_quality.org lines 70-71, which names business time (as_of_date) and system time (ingestion_timestamp) and calls the pair bi-temporal reasoning.

A vintage is therefore the pair of business date and ingestion instant that a dataset row carries. What it is not is a date inside the staged rows. Nothing in the artefact pipeline rewrites a staged value relative to as_of_date. Publishing is a copy, optionally preceded by a close. A vintage says when the data was true of the world, not how to make it true of today.

Live confirmation. SELECT as_of_date, count(*) FROM ores_dq_datasets_tbl WHERE code LIKE 'acme%' GROUP BY 1 returns one row, 2026-10-15 00:00:00+01|72. All 72 ACME dataset rows carry the provisioning date and nothing else. Their codes are such as acme.acme_group.books and acme.acme_hk.portfolios, from SELECT d.code, d.as_of_date, d.version FROM ores_dq_datasets_tbl d. A re-provision at a later date either leaves the old vintage or mints a new one. It never moves a trade's dates.

Why the path fails for trades whose dates sit relative to today:

1. There is nothing to stage. No trade artefact table exists, and dq_artefacts.org lines 44-47 require the owning model to opt in. Building one means generating a staging table for every instrument family the population touches, plus a publish function per family against the contract at lines 99-130.
2. A vintage is the wrong axis. The requirement is that the population sits relative to today. A dataset vintage is a fixed business date. To make the dates relative you must either regenerate the staged rows at each provision, which is generation wearing DQ's clothes, or shift them after publication, which is mechanism B and carries its costs below.
3. The publish function cannot be the place the arithmetic lives. It is a data movement function with an idempotence contract. Putting date arithmetic in it makes idempotence depend on the arithmetic, which the contract does not contemplate.

Measured on the tenant, the DQ path also stages nothing relevant.

• DQ carries no trade artefact table. ./compass.sh db sql -- -At -c "SELECT table_name FROM information_schema.tables WHERE table_schema'public' AND table_name LIKE 'ores_dq%';"= lists 156 tables and not one of them stages a trade, a bond instrument or an instrument leg. The nearest thing is ./compass.sh db sql -- -At -c "SELECT DISTINCT code FROM ores_dq_datasets_tbl WHERE code ILIKE '%trade%' OR code ILIKE '%bond%' OR code ILIKE '%instrument%';", which returns one row, synthetic.feeds.tradeweb, a market data feed.
• No seed writes a trade. ls projects/ores.sql/populate/trading/ shows 23 files, all reference data. They are trading_trade_types_populate.sql, trading_activity_types_populate.sql, the type vocabularies and trading_structure_reference_data_populate.sql. find projects/ores.sql -path '*populate*' -iname '*trade*' finds only those lookups.
• The ACME seeder dataset stages no trades. projects/ores.seeder/datasets/acme_corporation/dataset_overview.org lines 33-60 describe the dataset's files, and ls projects/ores.seeder/datasets/acme_corporation/*.json returns 36 of them. They are companies, LEIs, business units, portfolios, books, accounts, netting agreements and sets, CSAs, report definitions, report configurations, risk report configs and today's market configuration. No trade file exists, and no artefact table stages one.
• No trade set exists. ./compass.sh db sql -- -At -c "SELECT table_name FROM information_schema.tables WHERE table_schema'public' AND table_name LIKE '%trade_set%';"= returns nothing, and the task that would create the partition, D054BDD3-351F-42D3-9638-7D3BAEEAD332 "Partition a run's trade population into trade sets", is in the backlog.

The third path, and the one recommended. The generator takes an as-of date as an input, and every date it emits is an offset from that anchor.

The machinery is present in outline. ores.utility/generation/generation_context.hpp constructs from a seed at line 43 or randomly at line 48, and generation_engine.hpp offers past_timepoint(int years_back = 3), which returns a point relative to std::chrono::system_clock::now() (generation_engine.cpp lines 76-80). What is missing is an anchor parameter rather than a hard read of the clock, and offset arithmetic per date field.

Date relativity is exactly what this buys, and it is what the tree already reaches for. It is also the only one of the two mechanisms that is coherent for a schedule stored as a rule.

The existing generator machinery is real and its output today is test fixtures. Every model declares generator_facet_name: generators, and ores.trading.api/src/generators/ holds a generate_synthetic_<entity> per entity. The only consumer of these in the tree is the parent_seed paste block, and grep -rln parent_seed_snippet projects/ores.codegen/library/templates/ returns exactly one template, cpp_nats_integration_test.cpp.mustache. So the machinery seeds an integration test's foreign key chain and nothing else. grep -rn "generate_synthetic_trade\b" projects/ --include=*.cpp finds no caller outside its own definition.

The fourth path. Take a static population from DQ or from the ORE sample corpus, insert it, then update the dates by a delta. This was tested on its merits rather than dismissed, and it loses.

The fifth path, and the one salvageable version of the DQ idea. Stage the parameters a generator reads, not the trades.

An artefact table holding the population shape (which books, which counterparties, which products, which tenors, which as-of) publishes through the ordinary pipeline and gives DQ's idempotence and vintage story for free. The generator then runs at population time and the dates are relative because the generator is the thing that emits them. The artefact is small, diffable and stable, and the dates are never staged.

Called that way, DQ is a good fit for the inputs and a poor fit for the trades.

Property	DQ, as it stands	DQ, parameters only	Mechanism A, relative at generation	Mechanism B, shift after insert
Stability across re-provisioning	Dataset vintage is stable; rows never move	Stable, because the parameters are	Exact for a fixed (seed, as-of); different as-of moves the population deliberately	Stable only if the shift is absolute and guarded
Date relativity	None. A vintage is a fixed date	Full, by the generator	Full	Full, but by rewriting stored dates
Referential integrity to parties and counterparties	Not addressed by the pipeline	Not addressed by the pipeline	Guaranteed if the generator picks from the tenant's real rows	Guaranteed, because the corpus already references real rows
Reviewability of a diff	Good. Generated SQL and JSON diff cleanly	Good. The parameter set is small	Good. Regenerate and diff	Poor. The diff is version 1 against version 2 of every shifted row
Business-day correctness	Not applicable	Not applicable	Guaranteed if the generator snaps to a calendar	Not guaranteed by anything in the schema
Cost to build	High. A staging table and a publish function per family	Low	Medium. Anchor plus offset arithmetic per field	Medium to high. A per-field date walk across the 50 date-bearing tables

The sixth path, explored for what it can be reused for. Two engine packages are vendored, at ls build/output/linux-clang-release-ninja/publish/vendor-packages/ and ls external/ore/packages/.

• ore-1.8.16.0-4-g3b62ba248-x64-linux.tar.gz, engine commit 3b62ba248e36ea92f408f0d863ede09639074836, dated 2026-06-11.
• ore-1.8.17.0-x64-linux.tar.gz, engine commit 3d75a69087911a7e2cf1e6882da7bc27135c0762, dated 2026-09-11.

external/ore/packages/manifest.json records both, including the note that the 1.8.17.0 tag was created on the fork because upstream had not tagged. Each package is a self-contained binary set, and its root manifest.json is minimal. external/ore/methodology.txt keeps the engine source tree outside the repo.

Which one runs. Both are registered in the live database, from SELECT * FROM ores_compute_app_versions_tbl, as two rows of app ORE with engine_version 1.8.16.0-4-g3b62ba248 and 1.8.17.0. Nothing marks either as default. The choice is made at submit time in projects/ores.compute/core/src/messaging/report_submit_handler.cpp lines 125-128, which takes std::ranges::max_element(candidates, {}, &domain::app_version::engine_version), and the comment above it says why, "the highest engine version stands in; a field on the configuration would make the choice explicit rather than implied". So the grid runs 1.8.17.0 today.

The seventh path, and the audit that supplies the generator's inputs. All figures are for tenant 47b4aff4-79e9-4ee9-936f-ee56db971bee (acme_corporation), current rows only, where current means valid_to = '9999-12-31 23:59:59+00'.

• Whether the Windows build is green. The C1202 defect recorded at doc/agile/product_backlog/inbox/msvc_c1202_bond_export_carrier.org named instrument_payload.cpp, which no longer exists, and grep -rn AddTagsToVariants projects/ now returns nothing, so the trigger appears retired by the typed-batch design. The MSVC job could not be run from this environment to confirm it.
• Whether the ACME report package was built and inspected. The story 441E3343-F754-49B1-9763-960BBC2D4532 and its build-the-packages task were STARTED, so the zero-trade premise was taken as given and only the trade count was confirmed, which is zero for every trading table at every tenant.
• The exact provenance of the 2454 counterparties. They are GLEIF names and ores_dq_lei_entities_artefact_tbl holds 38212 rows, but no publication row for a counterparty dataset came back from SELECT d.code, p.records_inserted ... FROM ores_dq_dataset_publications_tbl p JOIN ores_dq_datasets_tbl d ON d.id=p.dataset_id WHERE d.code LIKE '%counterpart%', so the load path could not be shown end to end.
• Whether faker-cxx exposes a seeding API in the version this build links. find_package(faker-cxx REQUIRED) at CMakeLists.txt line 449 locates it, and it is not vendored in the tree, so the only established fact is that no seed call exists, not what the available call is.
• The current CI state of the drift gate for ores.trading. The September review recorded the component as absent from the drift registry. That was not re-measured here.

Mechanism B was tested rather than dismissed, and it loses on three grounds, not one.

1. Its derived dates are already relative, so B does not add what A lacks. swap_instrument_mapper.cpp lines 89-113 show the header dates are copied from the schedule, and bond_schedule_rules_field_group.org lines 14-18 show the schedule is a rule. Shifting the anchor and keeping the tenor is exactly what A does, with a database write and a version row added.
2. Its constraints do not check what matters. The 162 check expressions in the trading schema constrain order and the spine. None can constrain a business day, because calendars are not in the schema. A shifted population is unverified where it most needs verifying.
3. Its history makes the population read as corrected. The trigger writes a second version, the history diff renders it, and the as-of read the trade set already carries answers for the pre-shift state. Separately, schedule dates are :economic: true, so a shift moves the economic_digest, and by the rule in trade_versioning.org lines 27-28 the external_version follows, which reports every trade as amended by agreement.

B's one advantage, keeping the ORE corpus recognisable, does not survive the date-list schedules or the calendar gap. A corpus that has been partially rewritten is neither generated nor verbatim.

Mechanism A keeps the property the requirement actually asks for. The population is a function of (seed, as-of), and moving the as-of moves it coherently.

The user's reasoning is right, and for a narrower reason than stated. Relative dates are not a synthetic-versus-DQ question. They are an anchor question, and the anchor belongs in the generator's signature.

The seed is necessary and it is not sufficient today. Three things must hold.

1. The seed must reach the engine. generation_context.hpp line 43 constructs from a uint64_t seed, so the parameter exists and the population must pass it explicitly rather than take the random constructor at line 48.
2. The identifier minting must stop reading the clock. generation_engine.cpp lines 42-53 build a UUID v7 whose first 48 bits are std::chrono::system_clock::now() in milliseconds. Two runs with the same seed produce different ids. Either the timestamp is derived from the seed and the anchor, or the population uses deterministic keys it computes itself.
3. The text generation must be seeded too. The generated entity generators call faker::lorem::sentence(), faker::number::integer() and similar, for example ores.trading.api/src/generators/activity_category_generator.cpp line 50 and bond_forward_generator.cpp line 50. grep -rn "faker::seed" projects/ returns nothing, so faker-cxx draws from its own unseeded engine. A seeded context alone does not pin a single description string.

With those three closed, "same seed and same anchor produce the same rows" is true and testable. Note what determinism does not mean. The same seed at a later anchor must produce a different population, moved forward coherently. Determinism is over the pair, not over the seed.

The as-of is an input to the generator, never a read of the clock inside a field generator.

• The context carries (seed, as_of).
• Every date field is emitted as as_of + offset(tenor), where the tenor comes from the product's own terms, so a swap's schedule rules keep their tenors and move their anchor.
• The generator emits schedule rules wherever the schema offers the choice, because a rule is relative and a date list is not, and check ("schedule_kind" in ('rules', 'dates')) lets it say so.
• Business-day snapping runs in the generator against the same calendars ORE will use, so the emitted dates are the dates the engine would compute. Nothing in the schema can check this, so the generator is the only place it can be guaranteed.
• A re-provision at a later as-of re-runs the generator. The population moves and nothing is shifted.

The database already enforces most of it, so the work is to feed it real keys.

• Party. Set app.current_party_id per write, because ores_trading_bond_instruments_insert_fn overwrites the column from session state. Generate one party's population in one pass, then the next.
• Counterparty. Pick from the links in ores_refdata_party_counterparties_tbl for the party being written, not from the whole counterparty table. Nothing checks this, so the generator must be the check, and the weighting page should carry the per-party list. The list is broadened by the task that follows this investigation, from 38 links over 23 counterparties to 916 links over 456, so the page should be built against the widened set rather than the narrow one.
• Book and netting set. Pick the book from the trade's party and the netting set from that party and the chosen counterparty, because the booking trigger refuses anything else.
• Staff. Set modified_by and performed_by to a real username, and pick it from the Trading-role holders in ores_iam_account_parties_tbl for the party being written. This is the one piece with no model behind it, so it is a decision to make and record rather than a lookup to perform.
• The prerequisite to close first. Give staff accounts a default_party_id, or declare the account_parties rows as the authoritative mapping. Today 2 of 61 current accounts have one, and the other 59 have no party of their own while their account_parties rows are many-to-many, so the question of which person entered a trade cannot be answered from the data.

Two layers, and the first is the one that matters for a seed.

• Regenerate and diff. Because the population is a function of (seed, as-of), a review is a rebuild at a stated pair and a diff of the emitted SQL or fixtures. This is the review the seeder already imagines for its datasets, and it is the one that catches an accidental change to the shape. It needs the emitted population to be committed or written where a diff can see it.
• Entity history. The generic history request, the per-entity handlers and ores.diff (doc/knowledge/architecture/history_diff_architecture.org lines 14-30) already render version-to-version changes. Under mechanism A the first version is the only version, and history stays quiet. That quiet is the signal.

Phase	Work	Verified by
0	Close the seed's three determinism gaps: anchor on generation_context, seed-derived ids, seeded faker.	A test that builds the same (seed, as-of) twice and compares byte-for-byte.
1	Close the prerequisite: decide and record how a person maps to a party, and populate default_party_id or declare account_parties authoritative.	A query showing every book's party has at least one Trading-role holder.
2	Bond population. A generator for bond_instrument, bond_issue and the leg children, driven by the 10 routed bond codes, the Liquidity Buffer Bonds books and the per-party counterparty list. Write through book_trade and the generated repositories.	SELECT count(*) FROM ores_trading_trades_tbl WHERE tenant_id…= returns the intended count, and the export of a book's trades reverse-maps without loss.
3	Prove it prices. Export the bonds as an ORE portfolio through export_portfolio and run one report.	The engine proceeds past Pricing: Build Portfolio.
4	Extend to the rates family, then FX, equity, credit, commodity and scripted, in the order the weights page already sets.	The population's product mix matches the weights table within tolerance.
5	Fill the coverage hole that blocks a family. 30 of 121 trade types have no instrument table, and each one is a family member a population cannot carry.	The catalogue's unrouted count falls.

Reused:

• the generators facet, generation_context, generation_engine and parent_seed blocks, as the skeleton;
• trading.v1.ops.book_trade and the generated repositories, as the write path;
• the trigger-enforced integrity for party, book, netting set and counterparty identifier, as the guarantee;
• the export walk, ores::ore::xml::exporter::export_portfolio, as the bridge to the engine;
• trade_population_weights.org, as the weighting data;
• ores.diff and the history renderer, as the review surface;
• the seeder's dataset convention, for the parameters if a staged form is wanted later.

Built:

• the as-of anchor and its offset arithmetic in ores.utility/generation;
• a seeded id scheme that does not read the clock;
• a faker seed call, or generated text that does not use faker;
• the bond population generator and its per-party counterparty and staff selection;
• a shell command or service operation that runs the generation for a party and a tenant;
• the person-to-party decision, which is a prerequisite and not a task.

Not built, and deliberately:

• no trade artefact table in DQ;
• no date-shift pass;
• no new populate script per table.