Task: Book a trade through the repositories

Table of Contents

This page documents a task in the Record trade activities and trade links story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Every application write to an entity table goes through its generated repository, booking included, and a check refuses code that writes an entity table any other way.

2. Status

Field Value
State DONE
Parent story Record trade activities and trade links
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-07

3. Acceptance

  • ores.database offers a unit of work: repository writes made through it share one connection and one transaction, and commit or roll back together.
  • book_trade writes the trade, the activity, the booking and the state through their repositories inside one unit of work; a failure in any write leaves none, and a second booking of the same trade is still reported as a conflict.
  • ores_trading_book_trade_fn is deleted; no SQL function or hand-written C++ inserts into, updates or deletes from an entity table outside the generated repositories and the sanctioned purge.
  • An architectural rule states the policy, and a check run locally and in CI fails on a direct entity-table write outside the repositories, with the purge and test fixtures as its stated exceptions.
  • The round trip stays at in-scope lost 0; build, ctest, pgTAP under RLS and the local check set pass.

4. Plan

4.1. The problem

Booking writes the trade, the activity, the booking and the state in one SQL function, ores_trading_book_trade_fn. The function bypasses the generated repositories. The task moves the four writes onto the repositories and keeps them atomic.

4.2. The seam

Every repository write calls execute_write_query, defined once in projects/ores.database/include/ores.database/repository/bitemporal_operations.hpp. The helper calls session(ctx.connection_pool()), begins a transaction, writes, and commits. The read helper opens its own session in the same way. Current-state entities are a second shape: 33 generated repositories inline the same session/begin/commit in the template at cpp_domain_type_repository.cpp.mustache, and current-state reads use the same read helper.

4.3. The chosen seam: a transaction bound to the context

The unit of work binds one transaction to the context that every repository already takes. This is the shape sqlc uses with WithTx. The service opens the boundary; the repositories join it.

  • ores::database::unit_of_work owns one pooled session and one sqlgen::Transaction. It yields a context bound to that transaction. commit() ends it; the destructor rolls back when the caller forgets.
  • context carries the transaction as an owning rfl::Ref, so a copy carries it too and no lifetime rule is needed.
  • execute_read_query, execute_write_query and the current-state template path use the active transaction when the context carries one. Without one they open a session as they do today.
  • Repository signatures and all 343 call sites stay.
  • Reads must join, because replace_claim reads before it writes.
  • A service sets the tenant and the party before it opens the unit of work. A held connection cannot change tenant inside a transaction.

4.4. Phases

  1. Unit of work in ores.database, as above.
  2. Booking onto the repositories. trade_operations_service::book_trade opens one unit of work and writes the trade with a must_not_exist claim, the activity, the booking and the state. A second booking still returns outcome::conflict.
  3. Delete ores_trading_book_trade_fn and update the drop file. Rewrite the two pgTAP files that call it.
  4. State the rule in projects/modeling/system_model.org. Add check_entity_table_writes.py with a ratchet baseline. Run it in codegen-drift.yml and locally.

4.5. Verification

Build, ctest, pgTAP under RLS, the round trip at in-scope lost 0, and the new check.

4.6. Measurement

Each number in the Measurements table states its command and its commit.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
ctest suites passing 93 of 94 ./compass.sh test run 8ea91a2421
ctest assertions passing 9370 of 9371 ./compass.sh test results 8ea91a2421
unit of work suite 39 cases, 138 assertions, 0 failures ./compass.sh test run (test-results-ores.database.tests.xml) 8ea91a2421
booking suite 127 cases, 886 assertions, 0 failures ./compass.sh test run (test-results-ores.trading.core.tests.xml) 8ea91a2421
previously crashing targets 18 of 18 built and passing ./compass.sh build then ./compass.sh test run 8ea91a2421
entity-table writers the ratchet knows 114 (110 SQL functions, 4 C++) python3 projects/ores.codegen/scripts/check_entity_table_writes.py 8ea91a2421
the one failing suite ores.scheduler.api.tests, cron 0 9 1 * 1 day-field semantics ./compass.sh test run 8ea91a2421
pgTAP trading suites 4 of 6 abort, 2 assertions fail; every failure is a NULL party or counterparty ./projects/ores.sql/test/run_tests.sh 8ea91a2421

6. Notes

6.1. Measurements taken at e7caee274b

  • 110 SQL functions under projects/ores.sql/create write an ores_*_tbl table directly. By component: refdata 39, dq 34, iam 19, mq 7, inbox 4, variability 2, and trading, synthetic, marketdata, assets and reporting one each.
  • ores_iam_purge_tenant_fn is the only purge-named writer, so it is the sanctioned purge exception.
  • projects/ores.sql/populate holds 2127 direct DML statements. These are seed data, not application code. The check excludes them.
  • ores_trading_book_trade_fn has one C++ caller (trade_operations_service.cpp), two pgTAP callers and the drop file.
  • 343 files call execute_write_query. 33 generated repositories instead inline the session/begin/commit in the repository template, because they are current-state entities.

6.2. Decision: a transaction bound to the context

Taken 2026-10-06 with the owner. The unit of work binds one transaction to the context the repositories already take. This is the sqlc WithTx shape. The alternatives were an ambient thread-local transaction (rejected: no DI framework, hidden control flow) and an explicit unit_of_work& parameter on every repository (rejected: it forces every single-statement write to open a transaction, and it changes 343 call sites).

The binding fact: rfl::Ref is an owning, copyable handle, so a context copy carries the transaction safely. No raw pointer and no lifetime rule are needed.

6.3. Implementation: the unit of work

  • ores.database/repository/unit_of_work.hpp owns one pooled session and one sqlgen::Transaction. It yields a context bound to that transaction, commits once, and rolls back on destruction.
  • context carries the transaction as an optional owning rfl::Ref. A copy carries it, and with_tenant and with_party preserve it.
  • execute_write_op is new. execute_read_query, execute_write_query and execute_delete_query join the context's transaction when it has one, and otherwise open their own session as before.
  • The repository template's current-state path now calls execute_write_op instead of opening its own session. 21 generated repositories in the drift-free components under test are regenerated.
  • ores.database.tests.unit_of_work_tests checks the binding and the rollback.

6.4. Known offenders for the architectural check

Three hand-written writes still open their own session and so do not join a unit of work:

  • ores.telemetry.database/repository/telemetry_repository.cpp
  • ores.reporting.core/repository/report_input_bundle_repository.cpp
  • ores.iam.core/repository/session_repository.cpp, its update_session_counters and end_session.

ores.synthetic and ores.scheduler carry current-state repositories whose generated output has pre-existing drift. This change leaves them alone. Regenerating them would mix their drift into the booking change.

6.5. The sqlgen read that commits the unit of work

Found 2026-10-06 while moving booking onto the repositories. sqlgen's postgres iterator wraps every read in its own BEGIN and END, and PostgreSQL reads END as COMMIT. A read inside a unit of work therefore commits it. The booking service's conflict read and every repository apply_claim read committed the transaction before the failing write, so the anchor and the activity stayed while the booking failed.

book_trade reads inside its unit of work again, and states a must_not_exist claim for each of the four writes.

The tracked overlay at build/cmake/overlays/common/sqlgen lists fix-iterator-transaction.patch. The patch asks PQtransactionStatus whether the iterator joined a transaction, and runs no BEGIN and no END when it did. sqlgen 0.8.0#2 carries it.

Verified 2026-10-06 with sqlgen 0.8.0#2: ores.database.tests passes, 138 assertions across 39 cases, including "a read inside a unit of work does not commit it"; ores.trading.core.tests passes, 886 assertions across 127 cases, with the NATS server running. The memory The memory "A read inside a unit of work commits it", retired with sqlgen, carried the rule.

Raised upstream as munich-data-engineering/sqlgen#140. The port's 0.8.0 is this project's own label for the pinned commit; upstream's tags stop at v0.6.0 and its CMakeLists.txt still declares 0.6.0. The portfile records that the commit is the thing to track. Raised as munich-data-engineering/sqlgen#141 and closed by the owner.

6.6. Next

Nothing. The task is merged. The open item is environmental: the pgTAP trading suites read a party and a counterparty from the system tenant, and this environment's system tenant carries one party and no counterparty, so order by id offset 1 limit 1 is NULL. That state predates this task; the suites passed before compass db recreate rebuilt the database, which reset the system tenant's seed data.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#2718 [database,trading] Book a trade through the repositories

9. Review

Comment summary File Decision Notes
       

10. Result

Merged on 2026-10-07 as #2718, on top of #2672 and #2680.

10.1. What landed

  • ores::database::unit_of_work binds one transaction to a context; the repository read, write and delete helpers join it when the context carries one and open their own session when it does not.
  • book_trade opens one unit of work and writes the trade, the activity, the booking and the state through their repositories, each with a must_not_exist claim.
  • ores_trading_book_trade_fn is deleted, and the two pgTAP files that called it are rewritten.
  • check_entity_table_writes.py states the rule with a ratchet baseline, runs on every pull request, and records the purge and the fixtures as its exceptions.
  • The sqlgen overlay carries fix-iterator-transaction.patch, which stops the postgres iterator from ending an enclosing transaction on a read. The port is 0.8.0#2.

10.2. What was verified

ctest is 93 of 94 suites and 9370 of 9371 assertions at 8ea91a2421. The unit of work suite passes 39 cases and 138 assertions; the booking suite passes 127 cases and 886 assertions. Each of the 18 test targets that crashed before this work now builds and passes.

10.3. What is open

ores.scheduler.api.tests fails one case. It asserts that 0 9 1 * 1 matches on either restricted day field, so the next match after Tuesday 2026-10-06 is Monday 2026-10-12; the engine returns Monday 2026-02-01, which is the and reading. No file in this task's diff touches ores.scheduler, and the test binary and the scheduler library were both rebuilt from this tree.

The pgTAP trading suites cannot run in this environment. Every failure is a NULL party or counterparty read from the system tenant, which carries one party and none of the counterparties the tests index with offset 1. The suites passed against the database that predates the compass db recreate this verification required.

Emacs 29.3 (Org mode 9.6.15)