Task: Bring ores.testing to the clean standard
Table of Contents
This page documents a task in the Clean ores.testing to the component clean standard story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
ores.testing passes the Component Clean Standard for a component of kind All.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Clean ores.testing to the component clean standard |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-09-26 |
3. Acceptance
- Every checklist item that applies to a component of kind All passes, with its evidence recorded on the task.
- Every item that does not apply is recorded as not applicable, with the reason.
- Every generator defect found is fixed at its source, with a regression test.
- ores.testing is listed in
COMPONENTS_UNDER_TEST, or the reason the entry cannot be added is recorded as an exception.
4. Plan
Work the Component Clean Standard in its section order and record each item's result in the table below. The standard holds 49 items.
- Baseline and inventory (B01 to B06). Record the baseline before any edit.
- Models (M01 to M10).
- Protocol (P01 to P06).
- Generation (G01 to G08).
- Wiring and data (W01 to W04).
- Shell (S01, S02).
- Structure and hygiene (H01 to H05).
- Verification (V01 to V08), ending with the registry entry.
| Item | Result | Evidence |
|---|---|---|
| B01 | pass | Baseline at `f7fc9396b7`, the branch point. `check_component_drift.py` cannot name the component: it has no `component_catalogue.org` row. `check_model_drift.py` reports one known exception, `ores.reporting.report_definition`, which predates this pass. `regenerate_cmake_component_files.py –component ores.testing –check` reports `src/component_files.cmake` stale; the repo-wide `–all –check` reports 113 stale files, all in other components. `validate_docs.sh` is green across all 40 components. The component has no test target. The branch was later rebased onto `7ce4d3ef09`; the figures in this table are re-measured at that head, while the baseline figures stay as the branch-point measurement. |
| B02 | pass | `modeling/` holds two org files. `component_overview.org` declares `#+type: ores.codegen.component`, the component model M10 reads. `database_integration.org` declares no codegen type, so the loader ignores it; it is the component's depth document and its claims match the tree. 0 entity, junction, operation, message, field-group and module models. |
| B03 | pass | 23 hand-written C++ files outside tests at the survey: 16 headers under `include/ores.testing/` and 7 sources under `src/`. None is generatable, because no model sits behind the component. Two are documentation or export only (`export.hpp`, `ores.testing.hpp`), four are header-only facilities (`nats_options_helper.hpp`, `sql_values_rows.hpp` and the two seed readers), and the rest are the listeners, helpers and managers. The pass leaves 22, which is 15 headers and 7 sources, and adds 8 files under `tests/`. |
| B04 | not applicable | The component serves and sends no NATS subject and has no protocol header. Its only NATS surface is `make_nats_options()`, which copies `ORES_NATS_URL`, `ORES_NATS_SUBJECT_PREFIX` and the `ORES_NATS_TLS_*` trio into an options struct. |
| B05 | pass | Five captures name the component. `sanitize_test_log_filename_from_test_name` records that the logging listener uses a raw Catch2 case name as a path component, with no cap and no sanitisation; it changes behaviour, so it stays its own task. `fast_minimal_test_tenant_provisioning` records the cost of provisioning the test tenant, which is a performance change and not a collision. `fix_component_doc_drift` predates this pass, and the validator it asks for is now green. `document_the_mechanisms_that_only_exist_in_the_code` and `geo_location_tests_fail_for_some_ip_addresses` only cite the component. No in-flight branch touches it: the fleet's `clean-*` branches belong to other components. |
| B06 | fixed | Consumer census by `grep -rl –include=*.cpp –include=*.hpp -e 'ores\.testing/<header>\.hpp' projects/` for each header, excluding the component's own directory. The fully-qualified pattern matters: a bare header name also matches another component's header of the same name, which is why this count is not the same as a repo-wide `git grep` for the file name. The count is therefore of C++ sources and headers that include this component's header, measured at this head: `make_generation_context.hpp` 169 files, `scoped_database_helper.hpp` 157, `nats_options_helper.hpp` 117, `logging_listener.hpp` 71, `database_helper.hpp` 58, `project_root.hpp` 53, `database_lifecycle_listener.hpp` 46, `test_timeout_listener.hpp` 18, `scoped_environment_override.hpp` 11, `series_key_shape_seed.hpp` 4, `series_classification_rule_seed.hpp` 1. 352 C++ files outside the component include at least one of its headers; a `git grep` without the include filter counts more, because the header path also appears in generated CMake lists. Four headers have no consumer outside the component: `sql_values_rows.hpp`, `test_database_manager.hpp`, `export.hpp`, and the namespace header `ores.testing.hpp`, which G08 exempts from a census. `run_coroutine_test.hpp` had no consumer anywhere, its own tests included, so it is deleted. `sql_values_rows` and `test_database_manager` have production consumers inside the component: the two seed readers call the scanner, and the helper, listener and manager translation units call the manager. A test is not a consumer, and neither is one. |
| M01 to M05, M07, M08 | not applicable | No org model and no entity. |
| M06 | not applicable | No entity, no junction, no operation and no hand-written wire type. |
| M09 | pass | `grep` over `modeling/` for Qt, Wt, presentation, retired message types and dead history properties returns only the PlantUML compile-command's `java.awt.headless`. |
| M10 | fixed | `component_overview.org` described a retired design: a database per test process, `TEST_ORES_DB_*` variables and table truncation. It is rewritten against the tree, with the entry points and dependencies corrected and `See also` filled. |
| P01 to P06 | not applicable | No entity and no protocol. |
| G01 | not applicable | An entity item. There is no model, and the component has no catalogue row. |
| G02 | fixed | No hand-written file is superseded by generation, because nothing generates for this component. The one dead file, `run_coroutine_test.hpp`, is deleted. |
| G03 | fixed | `src/component_files.cmake` and the new `tests/component_files.cmake` carry the generated marker. No hand-written file carries it. |
| G04 | not applicable | An entity item, and the component has no generated code. |
| G05 | pass | No generator defect was found in this component's output. The two template causes behind the stale source list, the missing marker and the retired Qt-era comment, were already fixed on main; this pass regenerates against them. |
| G06 | fixed | `regenerate_cmake_component_files.py –component ores.testing –check` is green for both lists. The repo-wide `–all –check` still reports 99 stale files that belong to other components, which is the pre-existing condition the standard records and each component's own pass fixes. |
| G07 | pass, tooling defect recorded | The generator's dry run was read line by line before the refresh. It adds `scoped_environment_override` and two members of `test_timeout_listener`, and changes nothing else. The manual pass then filled every box and drew the relationships, and a dry run afterwards reports no changes, so the sentinel section survives a refresh. The wrapped-base-clause defect the standard records does not reach this component: no class here has a wrapped base clause, and none holds a one-line `enum class`. |
| G08 | fixed | `ores.testing.hpp` keeps its outermost-namespace `@brief`, and a census does not apply to it. Its claims were stale and are corrected. The component has no facet headers, so the uneven facet coverage the item mentions is not this component's work. |
| W01 | not applicable | No entity, no registrar and no composition point. |
| W02 | not applicable | No handler and no permission. |
| W03 | not applicable | No populate script of its own. |
| W04 | not applicable | No SQL script of its own. |
| S01, S02 | not applicable | No entity and no shell unit. |
| H01 | pass, findings fixed | The audit ran in full, and this table is its record. Shape: a simple component. Folders: `CMakeLists.txt`, `include/`, `src/`, `tests/` and `modeling/`, all signed off; the pass adds `tests/`, which the catalogue already signs. Leftovers: none. Platform code: none, after H04. Names: every file and type snake_case and matching its type. Registries: the infrastructure layer page and the simple-component table are corrected below. Diagrams: both passes ran, and the rendered image was read. Fixed: no test suite, a stale component model, a stale registry row, an empty `See also`, an incomplete dependency list, a dead header and a narration-laden comment set. Recorded: `sql_values_rows` carries a cyclomatic complexity of 29, the component's only lizard warning. Complexity measure over `include/` and `src/`: 1035 NLOC, 61 functions, average NLOC 12.1, average cyclomatic complexity 2.3, average token count 88.6, one warning; top file by complexity `sql_values_rows.hpp`, average 29.0. No generated source is in these totals. |
| H02 | fixed | `run_coroutine_test.hpp` is deleted, because nothing consumed it. The bracket-stripping branch in `extract_suite_name` is deleted, because Catch2 stores a tag without its brackets and the branch could not run. The dangling doc comment for a `logger()` accessor that never existed is deleted. Two unused `boost/uuid/string_generator.hpp` includes are deleted. About twenty comments that restated the next line are gone. |
| H03 | fixed | Comment pass over the component's hand-written files. Narration removed, end-of-line comments removed, and two comments that stated the opposite of the code corrected. The comments that explain a why survive: the orphaned-row sweep, the best-effort cleanup, the `_Exit` watchdog, the configured database user and the null column keyword. |
| H04 | fixed | `test_database_manager.cpp` hand-rolled the `localtime_r` / `localtime_s` split behind an `#ifdef _WIN32`. `ores.platform::time::time_utils::localtime_safe` exists for exactly that, and `ores.platform` was already a private dependency, so the shim is gone. The audit's grep for platform macros over `include/` and `src/` is now empty. |
| H05 | pass | `component_overview.org` carries the six sections the Component Documentation Guide requires, each with real prose and no placeholder or empty bullet, and `See also` links the knowledge that gives the component its depth. Its `Diagram` section points at the committed image, and that image is a fresh render of the committed `.puml`: rendered again at this head and read, and hashed against a fresh render of the same source, which matched. The diagram was refreshed in this pass when the generator changed under the branch, which is the item's own rule. The component owns no schema, so the ER-diagram clause does not apply. Evidence: the overview, `validate_docs.sh`, and the rendered image read rather than merely regenerated. |
| V01 | pass | Whole-tree build 0 at 100 percent at the pre-rebase head. At the rebased head the component's target builds and its suite runs clean. The command is `ORES_USE_BUSCTL=0 ./compass.sh build test_ores.testing.tests`, and the variable is needed because trunk now wraps `cmake –build` in `systemd-run –user –scope`, which a busctl-only sandbox cannot reach. That defect is captured in the backlog with this workaround. |
| V02 | pass, with the sandbox caveat | `ctest -R ores.testing.tests` passes 1 of 1, and the binary reports 57 assertions in 26 test cases. The `ores.codegen` suite passes at 588, and the `ores.compass` suite at 251 passed and 1 skipped. The fleet is undeployed in this sandbox, where `compass services status` reports 24 units missing and 0 running, so the consumer suites that need it are outside this measurement; Reach systemd from a sandboxed compass session tracks that gap. The component's own suite needs neither the database nor the fleet. |
| V03, V04 | not applicable | No entity, no SQL and no generated shell command. |
| V05 | pass | Every gate green at the committed head: `check_component_drift.py –all –dry-run`, `check_model_drift.py`, `check_protocol_twin_coverage.py` (34 headers in 4 components), `check_handler_permissions.py` (166 codes in 3 components), `check_populate_references.py`, `regenerate_shell_recipe_inventory.py –check`, `regenerate_cmake_component_files.py –component ores.testing –check`, `validate_docs.sh`, `compass lint`, and `clang-format –dry-run –Werror` on every changed and added C++ file. |
| V06 | exception | ores.testing stays out of `COMPONENTS_UNDER_TEST`. It has no catalogue row and no codegen model, and the entry cannot be added safely. The dry run proves it. Add a `testing` row mapping to `projects/ores.testing/modeling` in `component_catalogue.org`, then run `check_component_drift.py –component testing –dry-run`. It reports 12 files. It would change 9 of the component's own committed files: `CMakeLists.txt`, `include/ores.testing/export.hpp`, `include/ores.testing/ores.testing.hpp`, `modeling/CMakeLists.txt`, `src/CMakeLists.txt`, `src/component_files.cmake`, `tests/CMakeLists.txt`, `tests/component_files.cmake` and `tests/main.cpp`. Three of those changes destroy the component: the regenerated `src/component_files.cmake` and `tests/component_files.cmake` lose every file entry, so the library and the suite would build nothing, and `tests/main.cpp` gains the `database_lifecycle_listener`, which makes a database mandatory for the component's own unit suite. It would create three files, `include/ores.testing/domain/stub.hpp`, `src/domain/stub.cpp` and `tests/stub_tests.cpp`, whose template text is "STUB code to be removed". The stub trio is dead code, which H02 forbids. The gate the entry feeds still passes for the current list: `check_component_drift.py –all –dry-run` is green. |
| V07 | pass | No vacuous test. Every case asserts a literal expected value, apart from the root-discovery case, which asserts the observable fact that the root it found holds `.git` and a `projects` directory. The standard's own proof holds, measured at this head: stub the row parser's `trim` to leave its input unchanged and `project_root::resolve` to return its argument, then rebuild, and 13 of the 26 cases fail on 15 assertions. The `project_root` case that fails is the one that resolves a named model, on its absolute-path and existence checks, and every parser case that reads a column with surrounding whitespace fails with it. The subjects were restored, and the suite is green at 57 assertions in 26 test cases. |
| V08 | pass | Per-file survey under `src/`, against the component's test run. Covered by the new suite: `logging_listener.cpp`, for the module name and the suite name, and `project_root.cpp`, for root discovery and resolve. Header-only facilities covered by the same suite: `nats_options_helper.hpp`, `scoped_environment_override.hpp`, and `sql_values_rows.hpp` with 16 cases. Covered through their consumers: `series_classification_rule_seed.hpp` and `series_key_shape_seed.hpp` by `ores.ore.core.tests` and `ores.marketdata.core.tests`. Untested and recorded: `database_helper.cpp`, `database_lifecycle_listener.cpp`, `make_generation_context.cpp` and `test_database_manager.cpp`, which need a live database and are exercised by every consumer suite that has one, and `test_timeout_listener.cpp`, whose only failure path calls `std::_Exit` and would kill the test binary. Those five files are the coverage work item this item names. |
5. Notes
Survey, 2026-09-26, at trunk `f7fc9396b7`, by the agent that wrote the component's clean-up.
- Scale. 23 hand-written C++ files in 16 headers and 7 sources. The component had no tests at all.
- Reach. 352 files outside the component include at least one of its headers. `make_generation_context.hpp` alone reaches 169 files, `scoped_database_helper.hpp` 157 and `nats_options_helper.hpp` 117. The consumers are the test binaries of the production components, which is what a test-support library exists for.
- Keep the component. It is the repo's only home for test-tenant isolation, the Catch2 listeners and the populate-script readers.
- The pass found one live defect and one dead API. The scanner trimmed a column before it stripped the row's opening parenthesis, so the first column could keep leading whitespace. `run_coroutine_test.hpp` had no consumer anywhere.
- Two more findings are recorded rather than fixed. The logging listener uses a raw Catch2 case name as a path component, which is `sanitize_test_log_filename_from_test_name` and changes behaviour. `sql_values_rows` carries a cyclomatic complexity of 29, which the complexity measure surfaces and a later pass refactors.
The component gained a test suite this pass. It covers the surface that needs neither a database nor a fleet: the SQL values-row scanner, `project_root`, `make_nats_options`, `scoped_environment_override`, and the listener's name helpers. The database-bound classes stay with the consumer suites that already exercise them, and V08 records them as the coverage work item.
One behaviour is now written down rather than discovered. `scoped_environment_override` swaps in a fake provider that answers every read, so a key the guard does not carry reads as unset even when the real environment holds it. Its doc comment says so, and a case pins it.
6. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
7. PRs
| PR | Title |
|---|---|
| #2171 | [testing] Clean ores.testing to the component clean standard |
8. Review
The local code review ran before the PR was raised, over `git diff origin/main…HEAD`. It found no Critical issue and raised eleven findings, five Important and six Minor, plus one scope note. All eleven findings are fixed and the scope note is declined, which is row 12; the table numbers twelve rows in all.
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | The custom test target's command omitted the environment wrapper, the Catch2 arguments and the XML reporter, so `run_all_tests` ran the suite without the `.env` environment and wrote no result file | projects/ores.testing/tests/CMakeLists.txt | fixed | The command now matches the generated template and the 68 components that follow it, so `make rat` and `compass test results` both work. |
| 2 | Two DONE pages still carried the unmet `COMPONENTS_UNDER_TEST` acceptance bullet, contradicting the task's own V06 evidence | doc/agile/versions/v0/sprint_26/clean-testing/task_clean_testing.org and story.org | fixed | Both bullets now read "or the reason the entry cannot be added is recorded as an exception", and the story Goal says the same. |
| 3 | The component model omitted `ores.utility` and `ores.logging`, so it disagreed with the infrastructure layer page | projects/ores.testing/modeling/component_overview.org | fixed | Both are direct dependencies, both are used by the component's headers, and both now appear. |
| 4 | The resolve-append case asserted one production run against another, so it pinned no literal | projects/ores.testing/tests/project_root_tests.cpp | fixed | First it asserted that the resolved path ends with the relative path it was given. The independent verification then showed that this too passed when resolve returned its argument unchanged, so the case was deleted in that round; row 17 records it. |
| 5 | No case followed the per-case logging convention, so a failing case left no diagnostic | the five suite files | fixed | Every case declares the suite constant, builds a logger and logs one line. |
| 6 | `trim` stripped four whitespace characters while the scanner skips six, so a vertical tab or a form feed survived into a column | projects/ores.testing/include/ores.testing/sql_values_rows.hpp | fixed | `trim` now strips all six characters `std::isspace` accepts, and a case covers the vertical tab and the form feed. |
| 7 | `catch_reporter_registrars.hpp` is unused in the listener translation unit | projects/ores.testing/src/logging_listener.cpp | fixed | Deleted. |
| 8 | The module-name case's closing set wrote the same value the case had just set, so it restored nothing | projects/ores.testing/tests/logging_listener_tests.cpp | fixed | The case sets a distinct value, asserts it, and then restores the module name. |
| 9 | The tags and case names did not follow the unit-test conventions | the five suite files | fixed | All five use the `[testing]` tag and snake_case case names. |
| 10 | The story's front matter said 2026-09-23 while the page said 2026-09-26 | doc/agile/versions/v0/sprint_26/clean-testing/story.org | fixed | Set to 2026-09-26. |
| 11 | The census figures were not reproducible as stated, and the Result counted seven fixed findings against nine rows | doc/agile/versions/v0/sprint_26/clean-testing/task_clean_testing.org | fixed | B06 states the command and the counting method, the union figure names C++ files, and the Result says nine. |
| 12 | The branch also carries two product-backlog files, which widen the PR | doc/agile/product_backlog/ | declined | The build failure they record blocks every sandboxed build, and the workaround is not discoverable from the failure. The capture stays. |
| 13 | The environment-override cases unset their probe by hand rather than through an RAII guard | projects/ores.testing/tests/scoped_environment_override_tests.cpp | declined | Raised by the on-demand review as non-blocking and explicitly not a leak: every case reaches its closing unset, because `CHECK` does not abort a case the way `REQUIRE` would. A guard type for a test-only variable buys nothing. |
| 14 | The record covered 48 standard items, but the standard had grown to 49: H05 was added under the branch | task and story pages | fixed | H05 has its own row now, and both pages say 49. The item passes on the overview's six sections, its committed image and a fresh render, and the ER clause does not apply to a component with no schema. |
| 15 | The record claimed the diagram's dry run reported no changes, but the generator changed under the branch and the dry run now adds a member | projects/ores.testing/modeling/ores.testing.puml | fixed | The automated pass ran again, the manual section was hashed before and after it and did not change, the dry run is now clean, and the image was re-rendered and read. |
| 16 | The consumer census figures were the branch-point counts while the record said they were re-measured at the rebased head | task page, B06 and Notes | fixed | Re-measured at the head: 169, 157, 117, 71, 58, 53, 46, 18, 11, 4 and 1, union 352. |
| 17 | One case asserted only that the resolved path ends with the path it was given, so it passed when resolve returned its argument unchanged | projects/ores.testing/tests/project_root_tests.cpp | fixed | The case is deleted: it asserted nothing the named-model case does not, and it was the third non-literal case behind a V07 claim of two. The suite is 57 assertions in 26 test cases. |
| 18 | V07 named a stub symbol that no longer exists, and row 6 described the trim as calling `std::isspace` | task page | fixed | V07 now names the no-op `trim` it was actually measured with, at 13 of 26 cases and 15 assertions, and row 6 describes the six-character set. |
| 19 | The H05 row carried four cells in a three-column table, because it kept the standard's own four-column shape | task page, H05 row | fixed | The item description and its evidence are one cell now, like every other row. |
| 20 | The B06 census command was under-specified: run literally it counts another component's header of the same name | task page, B06 | fixed | The row states the fully-qualified pattern it was measured with, and says why the bare name does not reproduce it. |
| 21 | The local review's size was stated two ways, eleven findings in the prose and twelve rows in the table | task page, Review intro and Verification | fixed | Both places now say eleven findings, five Important and six Minor, plus one scope note, twelve rows in all. |
| 22 | Row 4 still described the resolve-append case as fixed by asserting a path suffix, which the verification round then deleted | task page, row 4 | fixed | The row records both steps and points at row 17. |
| 23 | The standard's own text said it holds 47 items while it holds 49 | doc/knowledge/architecture/component_clean_standard.org | fixed | Corrected to 49. Out of this branch's component, and one word, but the count is the ground truth this task is measured against. |
9. Result
ores.testing meets the standard for the items that apply to a model-free test-support component. The applicable items pass, nine are fixed findings, V06 is the one recorded exception, and the rest are recorded as not applicable with their reason.
The component gained a Catch2 suite of 57 assertions in 26 test cases, its first. It lost a dead header and about twenty narration comments, and it gained one parser fix, one platform-facility fix, and a component model, a diagram and two registry rows that now describe the tree.
Two things stay as recorded. The scanner's complexity and the unsanitised log filename are findings with their own homes, not work this pass silently took on. And the registry gap stands: no catalogue row and no model, so enrolling the component would empty its source lists and add a stub trio, which V06 records with the dry run that proves it.
10. Verification
Three reviews ran outside the writing session, and none was written by the agent that wrote the branch.
The local review ran before the PR was raised and returned Ready with fixes: eleven findings, five Important and six Minor, plus the scope note that is row 12.
The on-demand review on the PR found no blocking issue. It traced the scanner fix by hand on a whitespace-bearing row, confirmed the `extract_suite_name` assumption about Catch2's tag representation against the case that pins it, re-grepped the deleted header and the two removed includes, and compared the `localtime_safe` replacement against the platform utility it now calls. Its one Minor finding is row 13.
An independent verifier then returned FAIL at head `815f37ec7a`, and it was right on every point. It reproduced the suite, the other two suites, the gate totals, the complexity figures and both proofs, and then found five defects the two reviews had missed. Four were record defects: the standard had grown to 49 items under the branch and H05 was missing; the diagram's dry run was no longer clean because the generator itself changed under the branch; the consumer census was still the branch-point count; and V07 named a stub symbol that no longer existed. One was a real test defect: a third non-literal case that passed when its subject returned its input unchanged.
All five are fixed and recorded as rows 14 to 18. The verifier's FAIL is the reason this pass has an H05 row and a diagram that survives its own dry run, and it is the second time this programme's record, not its patch, is what an independent check failed.
A second independent verifier then returned PASS with notes at head `ed153fb016`, which is the head under review. It reproduced the five fixes, the item coverage, the suite and the other two suites, the gate totals, the complexity figures, the diagram's dry run and image hash, both proofs and the V06 dry run, and it found no defect in the work. Its four notes were record defects the fixes themselves introduced: the H05 row kept the standard's four-column shape in a three-column table, the census command it stated did not reproduce its own figures, the local review's size was given two ways, and row 4 went stale when the verification round deleted the case it described. All four are fixed and recorded as rows 19 to 22, and the verifier's out-of-branch note about the standard's own item count is row 23.