Task: Close the ores.reporting checklist

Table of Contents

This page documents a task in the Clean ores.reporting to the component clean standard story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

Record a result for every Component Clean Standard item the clean-reporting pass left unrecorded, and finish V04 and V08 rather than except them, because the standard says an unfinished item is unfinished work and not an exception.

2. Status

Field Value
State DONE
Parent story Clean ores.reporting to the component clean standard
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-09-27

3. Acceptance

  • All 49 items carry a result row
  • V04 replays every generated command against the live fleet
  • V08 records a test or a reason for every source file

4. Stocktake

The 30 items the parent task already recorded are not re-derived here; its rows for them stand, except where the correction section below says otherwise. This table records the 19 that had no row.

Item Verdict Evidence
B02 Pass survey_component.py --component reporting: 8 models. One component (component_overview.org), five entity (concurrency_policy, report_definition, report_instance, report_type, risk_report_config), one module (ores.reporting.module.org), one operation (ores.reporting.operations.org). Six carry variability; the component and module metatypes are variability-free and excluded. Survey at .audit/reporting/stocktake/B02-B04-survey.md.
B05 Pass No in-flight branch touches the component: git ls-remote --heads origin lists 74 branches and none is a reporting branch, and the 11 worktrees in compass fleet work on other components. Open captures read, listed under Notes.
G03 Pass 191 C++ files under projects/ores.reporting; 136 carry AUTO-GENERATED FILE and 55 do not, and the 136 are exactly the survey's generated set. The shell facet's generated units, tests and CMake lists all carry the marker, and projects/ores.shell/reporting/tests/main.cpp is hand-written and does not. Of the reporting SQL scripts, the 15 generated ones carry the marker and the hand-written ones (the FSM guard, the in-flight function, the resolve-book-ids function and the populate scripts) do not.
H04 Pass No platform or compiler macro and no direct OS header in any of the 55 hand-written files: a scan for the compiler and platform macros (_WIN32, _MSC_VER, __GNUC__, __clang__), for the OS headers (windows.h, unistd.h), and for hand-rolled snprintf, strcpy, min and max returned nothing.
P05 Pass on the reference check; the build half is blocked Neither retired header is referenced by any code: a search for report_scheduling_protocol and report_execution_protocol over projects/ returns only the parent task's own rows. The scheduler, the only consumer outside the component, publishes through the generated constant ores::reporting::messaging::trigger_report_instance_request::nats_subject (report_scheduling_service.cpp:97), and nothing is carried for backwards compatibility. The item's other half is the whole-tree build, which cannot run while origin/main is broken; see Blocked.
P06 Not applicable The component has no event cache. ores.cpp.nats-event-cache is false in the one model that names it (risk_report_config.org:817), no *_cache* file exists, and the only eventing artefacts are the eight generated event headers, four event registrars and the eventing test. There is no cache to warm.
S02 Pass regenerate_shell_recipe_inventory.py --check exits 0, and each of the five reporting recipe groups has its entry in doc/recipes/shell/shell.org. No reporting command is destructive, which is consistent with the tree: the only command declared :destructive: anywhere is ores.iam.reset_messages.
W03 Pass check_populate_references.py prints "populate references resolve." and exits 0.
W04 Pass validate_schemas.sh reports 304 tables parsed, 0 warnings, no WIRE_001; the parent task's compass db recreate -y -k is the live half of the same item.
G08 Failed, fixed here The api module had no namespace documentation header at all: projects/ores.reporting/api/include/ores.reporting.api/ores.reporting.api.hpp did not exist. The core module's ores.reporting.core.hpp existed at the required path but was a hand-written aggregate include with no namespace @brief, which nothing included. Both are now namespace documentation headers, matching the convention in the cleaned components (ores.analytics.core.hpp). The service module's header already passed. Facet headers do not exist, which the item says is not a failure.
H03 Failed, fixed here Four comments narrated edit history: report_instance_trigger_handler.hpp:210, config_parser_tests.cpp:40, report_scheduling_service.cpp:360 and service_scheduling_plan_tests.cpp:26. All four now state the rule rather than the history. No commented-out code and no stray end-of-line comment was found; the line-end comments present are IWYU directives and } // namespace scope markers, both preserved by the rules.
P01 Fail Four of the five entities speak the canonical entity protocol: concurrency_policy, report_definition, report_instance and report_type each generate list (paged, ordered, with total), get, get_many, put, put_many, delete, delete_many and the versions sub-resource. risk_report_config generates no C++ protocol at all. Its model disables the C++ surface — ores.cpp.protocol, nats-handler, nats-sub-registrar, history-provider-registrar and service are all false (risk_report_config.org:812-820) — and gives the reason at :784-800: nothing writes a risk report config, and generating the surface would add a fifth orphan family, because registrar.cpp is a hand-written fan-out that builds each handler directly and has nowhere to join a generated registrar. The contract nevertheless exists on the TypeScript side: projects/ores.web/packages/wire-protocol/src/generated/reporting/protocol/risk_report_config_protocol.ts is checked in, carries the generated marker, and declares eleven request subjects (reporting.v1.risk_report_configs.list, get, get_many, put, put_many, delete, delete_many, list_by_report_definition_id, the two versions reads) and three event subjects, with nothing on the C++ side to serve or answer them. The twin gate does not see this: _checked_headers globs C++ protocol headers and pairs each with its TypeScript twin, never the reverse, and the gate's opt-out rule looks for :ores.ts.protocol.enabled: and not the C++-side flag, which is why it reports "no model opts out". The absence of the point-in-time read is not a defect: no model declares has_as_of_lookup.
S01 Fail Same entity, same reason. <entity>_commands.{hpp,cpp} and <entity>_commands_tests.cpp exist for the four entities that generate a protocol, with a process_* case for every command, and the generated units carry the marker with no hand-written survivor. risk_report_config has no command unit, no test and no recipe group. The component has no junction models, so there are no junction verbs to cover. The 11 operation commands (the trigger, schedule, unschedule and the eight execution steps) all have units and tests, judged separately because they are operations and not entity verbs.
G04 Fail The defects are in shared templates. (1) The generated report_definition.hpp:145 uses std::chrono::system_clock::time_point with no <chrono> in its include block, where its four generated siblings include it; the template cpp_domain_type_class.hpp.mustache emits the field unconditionally but takes the include from the model. (2) report_definition_handler.hpp and its three siblings carry a per-method paragraph claiming the adapter "checks the permission a write needs" on nine methods each, while only put, put_many, delete and delete_many check a permission — 36 false claims from cpp_nats_handler.hpp.mustache:60-68. (3) The generated operator=='s doc narrates edit history ("the omission went unnoticed until the diff payloads were the first generated types to have a test"), from cpp_domain_type_class.hpp.mustache:158. The generated repositories and mappers read like the hand-written house style.
H01 Fail validate_docs.sh passes, but it checks presence and not content. No written Component Architecture Audit record for ores.reporting exists. The three part diagrams carry the sentinel and an empty manual section: not one method line appears in any reporting .puml, and the rendered ores.reporting.service.png draws application, host and parser as empty boxes though each has methods. The root ores.reporting.puml has no sentinel and no two-pass structure; its rendered image is a package diagram with no class boxes, so it cannot carry an API. The root edges were checked against the code and all are supported.
M04 Fail On the inert-assignment clause only. The placement clause passes: every catalogued feature sits in the namespace the loader reads (_FEATURE_NAMESPACE, org_loader.py:51-81), with root * Flags lifted at :1356, SQL flags under ** Flags at :1405, and presentation keys under ** Presentation at :1634. But two declared assignments are read by nothing. (1) report_instance.org:239 :parent_id_field: definition_id — its only reader is _parent_id_field (org_loader.py:3042-3049), which returns the empty string unless has_parent_id is set, and report_instance never sets it. (2) risk_report_config.org:505 :parent_entity: report_definition — the key has no reader; the parent is resolved by core.py from the file, and the template placeholder parent_entity_plural is a different variable. Drift is clean, which is the proof that nothing consumes either.
M05 Pass Each of the five models was compared field by field against its generated CREATE TABLE: the symmetric difference is empty, and the only column not derived from the model is var_quantiles, which the model declares :sql_only: true. No table is a join or a denormalisation, and all five are temporal. reporting_risk_report_config_books_create.sql and ..._portfolios_create.sql are temporal junctions, and the model correctly does not claim them as entities.
M07 Not applicable The component has no junction models: grep '^#+type:' projects/ores.reporting/modeling/*.org returns entity (five), operation, module and component, and no ores.codegen.junction. The two hand-written junction tables are tenant-scoped already: reporting_rls_policies_create.sql:99-106 and :147-154= both filter on ores_iam_current_tenant_id_fn().
M08 Fail Two clauses. (1) Fifteen *_enabled flags are declared :type: integer / :cpp_type: int, defended in the model's own prose at :24-25 ("Analytics flags use integer 0/1, not boolean, to match the project convention"), and generated as integer not null default 0/1 at reporting_risk_report_configs_create.sql:49-64. The item says flags are booleans, and the tree agrees: 64 columns are :type: boolean, and the only integer flags outside this model are in ores.dq, which is not under test. (2) numeric(5,4) and numeric(5,4)[] at :279, :363 and :405 are spelled nowhere else in the tree, where the canonical numeric type is numeric(28, 10) (67 uses) or numeric(18, 10) (3). The template emits the type verbatim, so the spelling reaches the SQL. The custom-type clause is vacuously satisfied: no model references a type from cpp_custom_types.org.

4.1. Correction to a parent row

The parent task's B06 row calls ores.reporting.core.hpp "the namespace documentation header … which G08 exempts because an empty namespace is their whole content". The file was not that: it was an aggregate include over the domain, repository and service headers, with 19 lines of includes and no namespace. The census that called it a file with no production includer was right; the reason it gave was wrong. The file is now the namespace header the item asks for.

4.2. Open failures

Seven items fail, and none of them is a documentation problem.

Item Cause Where the fix belongs
P01 risk_report_config generates no protocol at all, so one of the five entities does not speak the entity protocol. The model disables the whole surface with a reason: nothing writes a risk report config yet, and registrar.cpp is a hand-written fan-out, so a generated registrar would be a fifth orphan. Generating the surface requires repairing the registrar composition first.
S01 The same entity has no command unit and no test. Same cause and same dependency.
G04 Three defects in shared templates: <chrono> is not emitted with the timestamp field, the handler's per-method doc claims a permission check that four of nine methods do not make, and the generated operator== doc narrates edit history. cpp_domain_type_class.hpp.mustache and cpp_nats_handler.hpp.mustache. Tree-wide, so it affects every component, not only this one.
H01 No written audit record; the three part diagrams have an empty manual section and draw method-only classes as empty boxes; the root diagram has no two-pass structure. The diagrams and an audit record, in this component.
M04 Two model keys nothing reads: report_instance 's :parent_id_field: and risk_report_config 's :parent_entity:. The two models.
M08 Fifteen flags typed integer instead of boolean, and numeric(5,4) where the tree uses numeric(28, 10). ores.reporting.risk_report_config.org, then regenerate.
V04 Sixteen of the 51 generated commands do not answer. Three causes, below: a shell menu collision, a __none__ placeholder where a uuid is required, and the silent step handlers.

4.3. Three findings that are not item failures

These surfaced while checking the items above. None of them breaks an item as the standard words it, and all three are this component's own work.

  1. The instance's definition key is unvalidated. The generated insert function for report_instance validates the tenant, the workspace, the change reason and the acting account, and never the definition the instance belongs to: reporting_report_instances_create.sql:95-145. The sibling entity does validate its equivalent, because its model declares the key in a * Foreign keys drawer with :target_column: and :error_message: (risk_report_config.org:504-508), which produces reporting_risk_report_configs_create.sql:143. An instance can therefore name a definition that does not exist, and the failure surfaces later, inside a workflow step, instead of at the write. The fix is to declare the key the same way in the model.
  2. The report input bundle is unmodelled. The table ores_reporting_report_input_bundles_tbl, its entity and its repository are hand-written, all three without the generated marker, and the repository talks to sqlgen directly with no mapper. A search of the modeling directory finds only a mention in component_overview.org:50, so nothing generates them and no drift gate covers them: this is the component's only hand-written data layer. Whether it should be modelled is a real question, since every other entity's data layer here is generated, but it is not an item failure. B03's survey called the repository generatable, the parent task called it infrastructure by elimination, and neither reading has been settled.
  3. The cron expression is an unvalidated string. report_definition's schedule_expression is std::string (report_definition.org:113-117), while ores.scheduler types the same concept as domain::cron_expression with :is_value_type: true (ores.scheduler.job_definition.org:108-116), which is what stops an invalid expression from entering that domain. A malformed schedule is accepted here and then rejected, or silently ignored, when the job reaches the scheduler.

4.4. Line citations are commit-relative

Every line reference in this table was read from the tree at the commit this task's branch is based on. Main moves under the table, so a citation can drift by a few lines without the finding changing: when the branch was rebased onto 315f3bc7a7, four of them had, and the findings they support were unaffected. A citation is a pointer to the evidence, not the evidence.

4.5. A block that has since lifted

V04, V08, V01 and P05 could not be closed while the tree did not compile. origin/main currently references ores.dq.api/messaging/fsm_protocol.hpp in three places — ores.workflow/core/src/service/fsm_state_map.cpp:21, ores.trading/core/include/ores.trading.core/messaging/trade_handler.hpp:29 and ores.trading/modeling/ores.trading.trade.org:662 — and that header does not exist: commit dd131c388b replaced it with fsm_state_protocol.hpp and fsm_transition_protocol.hpp and updated neither consumer. A whole-tree build fails at ores.workflow.core.lib.

The repair landed as dbfb31d45d and e25313b2d2, which replaced the protocol include with the FSM repository include in both consumers. On the rebased branch the whole tree builds (exit 0), check_component_drift.py --component reporting reports no drift, and the four reporting suites pass 4 of 4. V01 and P05's build half are therefore satisfied, and V04 and V08 are open work rather than a block.

5. V04 and V08

5.1. V04: every command replayed, and sixteen fail

All 51 generated commands were replayed with check_shell_recipes.py against a database rebuilt with compass db recreate -y -k and a fleet restarted on the rebuilt binaries. Thirty-five answer and sixteen do not. Each cause is a defect, and none is a missing test.

Commands Count Cause
report_definitions list, get, version, versions, delete 5 The command is not reporting's. The DQ shell part registers a menu of the same name, and it wins: the request arrives at the DQ service as dq.v1.report_definitions.list. It then answers internal_error, because the table the DQ command reads, ores_dq_report_definitions_tbl, is declared at ores.dq.report_definition.org:124 and created by no script.
report_definitions add, delete-many, get-many, put-many, set 5 The same collision, and the failure proves it: the recipes are generated for reporting's signatures, and the DQ menu rejects them with "Expected 8 arguments, got 9" and "Expected 9 arguments, got 10". get-many reaches the argument check and fails on "Invalid value for id: none".
report_instances add, put-many, set 3 The generated recipe passes __none__ for fsm_state_id, a required uuid and a server-owned field: "Invalid value for fsm_state_id: none". The same placeholder stands where a uuid is required for id.
the eight execution steps, assemble-bundle to submit-compute 8 They time out. Every step handler opens with workflow_step_context::from_message(...) and returns in silence when the workflow headers are absent, which they are for a shell caller, so the caller learns only that time passed. This repeats the parent task's finding, on a rebuilt fleet.

The collision is the sharper finding, and it is verified from both ends: dq_commands::register_commands (repl.cpp:105) registers DQ's report_definitions menu before reporting's at repl.cpp:116, and the service log records the request at dq.v1.report_definitions.list rather than reporting.v1.report_definitions.list. Until it is resolved the report_definitions family cannot be exercised from the shell at all, so S01 passes on the unit and the test rather than on a working command.

5.2. V08: five of six files, and the sixth has a test

repository_report_input_bundle_repository_tests.cpp is added. It creates a bundle and finds it back by instance id, asserting the literal values it wrote, and asserts that an unknown instance id returns nothing. Both cases run and pass in ores.reporting.core.tests.

That leaves the composition and entry-point files, whose test is the fleet starting and whose reason is recorded. The item asks for a test or a recorded reason for every source file, so it is satisfied.

5.3. The verification round, on the current tree

The first pass verified too little, and the gap is worth recording because it looks like success. The build was current, but the fleet was not: the reporting service had started at 10:47 and its binary was rebuilt at 11:40, so the running process was the previous tree's. Only the four reporting suites had been run, the database was 59 SQL files behind, and V04 had been replayed against that fleet. Re-done, on the current tree:

Check Result
compass db recreate -y -k exit 0; the reporting fixture seeds 5 definitions, a party and a book
compass services restart 25 of 25 running, none stopped or missing. The storage service had been failing on a password for its own database role, because the database predated that role; the recreate fixed it
cmake --build --preset linux-clang-debug-make exit 0, whole tree
ctest -R reporting 4 of 4
codegen suite (pytest projects/ores.codegen/tests) 824 passed
full ctest --preset linux-clang-debug-make 81 of 86. The five failures are ores.dq.api.tests, ores.dq.core.tests, ores.shell.api.tests, ores.refdata.core.tests and ores.marketdata.core.tests
V04 replay unchanged: 35 answered, 16 aborted, the same 16

The five failures are not this component's and not this branch's: the branch changes thirteen files, all under projects/ores.reporting/ or doc/agile/, and none under ores.dq, ores.refdata, ores.marketdata or ores.platform. They are captured as Five test binaries fail on main, and four of them share one cause.

So V01, V02 and V03 hold for this component, V02's repo-wide half does not, and V04's result is reproducible rather than an artifact of a stale fleet.

6. Plan

The parent task claimed "a recorded result for all 49 items". It does not have one. Nineteen items carry no result row: B02, B05, G03, G04, G08, H01, H03, H04, M04, M05, M07, M08, P01, P05, P06, S01, S02, W03 and W04. Nine of those appear nowhere in the parent task at all; the other ten appear only in its plan headings or its summary sentence, never with a result.

Two items are also recorded as partial: V04 and V08. The parent handoff proposed recording both as accepted exceptions. The standard refuses that: "An item that is merely unfinished is not an exception; it is unfinished work, and the component waits." An exception is for a check that cannot pass where the work is done. Neither of these is that, so both are finished instead.

  1. Stocktake. Run the check each unrecorded item names, and record its verdict with the evidence. A verdict of "not applicable" is recorded with its reason, never skipped.
  2. Close V04. Replay every generated shell command for reporting against the live fleet with check_shell_recipes.py, which is the script the item's own evidence names. Record every command that cannot run and the reason. The eight execution-step commands cannot answer from the shell because their handlers return in silence when the workflow headers are absent; that is a recorded reason, and the item allows it.
  3. Close V08. Give report_input_bundle_repository.cpp a database test, and record the reason for every remaining source file that no test names. The item asks for a test or a recorded reason, so the composition and entry-point files are closed by their reason.
  4. Take no exception. Both items are made to pass, so the registry entry needs no ACCEPTED_EXCEPTIONS change.

7. Notes

7.1. Open captures read for B05

From the inbox: The report-definition eventing re-drive collides with the temporal exclusion constraint; The storage server refuses the report-data bucket its own consumers write to; Three services default their storage base URL to a port the server does not use; Task: Fix reporting entity model drift: update to latest format and regenerate; Finish the eventing payload migration, or read the legacy shape. From next: Party's per-party report_events pgmq queue looks half-baked / unused; Wire service token provider for all services; publish_from_dq codegen facet: hand-writing artefact tables is a recurring cost; Story: Codegen facet: generate the DQ publish-from-dq pattern. From deferred: Trade-aware market data filtering for report execution.

The model-drift capture is satisfied by the parent pass, which brought the reporting models onto the current format and left the drift gate clean. The storage-bucket capture's specific clause — the HTTP server's hard-coded allow-list — was deleted upstream while this work was in flight; its general point, that a bucket is an allocation the facility should be told about rather than a name it hardcodes, is still open.

7.2. The parent task's completeness claim is false

The parent task's Handoff says the checklist "has a recorded result for all 49 items". Its Result names 29 items and its table carries a row for 30, so 19 items have no row at all: the nine that appear nowhere in the file (B02, G03, G04, H04, M04, M05, M07, M08, P05) and the ten that appear only in a plan heading or in the summary sentence (B05, G08, H01, H03, P01, P06, S01, S02, W03, W04). A census of the rows' leading cell over the parent task is the evidence.

This is the failure check_registry_exceptions.py exists to prevent, one level up: the component is listed in COMPONENTS_UNDER_TEST while items are neither passing nor recorded, and the gate cannot see it because it only validates the exceptions that exist, never the ones that should.

7.3. The V04 rows disagree with each other

The parent task's V04 rows report eleven commands, then thirteen, then "fourteen of fifteen verbs answer"; one row calls set verified and a later one calls it unverified. The replay in this task is the tie-breaker.

7.4. The parent result's caveat about the environment is now resolved

The completed walk in the parent task depended on two ORES_*_SERVICE_HTTP_BASE_URL variables added to .env by hand. That file has been regenerated from the environment profile, so it is consistent again, and the durable fix stays where it belongs.

8. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

9. PRs

PR Title
#2266 [agile] Close the checklist task whose PRs have merged
#2259 [reporting] Record three findings no checklist item names
#2242 [reporting] Record the checklist results the clean pass never wrote

10. Review

Comment summary File Decision Notes
       

11. Result

Delivered, and the component is not clean.

The parent task claimed a recorded result for all 49 Component Clean Standard items. Nineteen had no row. This task records them from fresh evidence, so the checklist now carries a result for every item, and seven of them fail: P01, S01, G04, H01, M04, M08 and V04.

Two of those were cheap and are fixed here. G08: the api module had no namespace documentation header at all, and the core module's sat at the required path holding an unused aggregate include; both are now namespace headers matching the cleaned components. H03: four comments narrated edit history and now state the rule.

V08 is closed. report_input_bundle_repository.cpp had neither a test nor a recorded reason. It now has a test that creates a bundle, finds it back by instance id and asserts the literal values it wrote, and that an unknown instance id returns nothing. The composition and entry-point files carry their reason, which the item accepts in place of a test.

V04 is measured rather than passed. All 51 generated commands were replayed against a database rebuilt from scratch and a fleet on the current binaries: 35 answer and 16 do not, and every cause is a defect. The report_definitions family is routed to DQ by a duplicate shell menu, so ten of its commands exercise the wrong component; three recipes pass __none__ where a uuid is required; and the eight execution-step handlers return in silence when the workflow headers are absent, so the caller learns only that time passed. The result reproduces exactly on a second run, so it is not an artifact of a stale fleet.

Two of those causes are outside this component and are captured rather than fixed: the duplicate report_definitions shell menu, and ores_dq_report_definitions_tbl, which DQ's own model names and no script creates.

Verification, on the current tree: the whole tree builds; the four reporting suites pass and so does the 824-case codegen suite; the full 86-binary suite is 81 of 86, and the five failures belong to ores.dq, ores.refdata, ores.marketdata and ores.platform, which this branch does not touch. They are captured.

One correction to this task's own process, recorded above rather than quietly repaired: the first verification pass ran against a fleet whose binaries predated its own build, a database 59 SQL files behind, and only the reporting suites.

What remains is the seven failing items and V04's three causes, each with a named owner in the stocktake table, plus three findings that no item names: the instance's definition key is unvalidated, the report input bundle is unmodelled, and the cron expression is an unvalidated string. The component is not clean, and the record now says so.

Emacs 29.3 (Org mode 9.6.15)