Task: Close the ores.reporting checklist
Table of Contents
This page documents a task in the Clean ores.reporting to the component clean standard story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
Record a result for every Component Clean Standard item the clean-reporting pass left unrecorded, and finish V04 and V08 rather than except them, because the standard says an unfinished item is unfinished work and not an exception.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Clean ores.reporting to the component clean standard |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-09-27 |
3. Acceptance
- All 49 items carry a result row
- V04 replays every generated command against the live fleet
- V08 records a test or a reason for every source file
4. Stocktake
The 30 items the parent task already recorded are not re-derived here; its rows for them stand, except where the correction section below says otherwise. This table records the 19 that had no row.
| Item | Verdict | Evidence |
|---|---|---|
| B02 | Pass | survey_component.py --component reporting: 8 models. One component (component_overview.org), five entity (concurrency_policy, report_definition, report_instance, report_type, risk_report_config), one module (ores.reporting.module.org), one operation (ores.reporting.operations.org). Six carry variability; the component and module metatypes are variability-free and excluded. Survey at .audit/reporting/stocktake/B02-B04-survey.md. |
| B05 | Pass | No in-flight branch touches the component: git ls-remote --heads origin lists 74 branches and none is a reporting branch, and the 11 worktrees in compass fleet work on other components. Open captures read, listed under Notes. |
| G03 | Pass | 191 C++ files under projects/ores.reporting; 136 carry AUTO-GENERATED FILE and 55 do not, and the 136 are exactly the survey's generated set. The shell facet's generated units, tests and CMake lists all carry the marker, and projects/ores.shell/reporting/tests/main.cpp is hand-written and does not. Of the reporting SQL scripts, the 15 generated ones carry the marker and the hand-written ones (the FSM guard, the in-flight function, the resolve-book-ids function and the populate scripts) do not. |
| H04 | Pass | No platform or compiler macro and no direct OS header in any of the 55 hand-written files: a scan for the compiler and platform macros (_WIN32, _MSC_VER, __GNUC__, __clang__), for the OS headers (windows.h, unistd.h), and for hand-rolled snprintf, strcpy, min and max returned nothing. |
| P05 | Pass on the reference check; the build half is blocked | Neither retired header is referenced by any code: a search for report_scheduling_protocol and report_execution_protocol over projects/ returns only the parent task's own rows. The scheduler, the only consumer outside the component, publishes through the generated constant ores::reporting::messaging::trigger_report_instance_request::nats_subject (report_scheduling_service.cpp:97), and nothing is carried for backwards compatibility. The item's other half is the whole-tree build, which cannot run while origin/main is broken; see Blocked. |
| P06 | Not applicable | The component has no event cache. ores.cpp.nats-event-cache is false in the one model that names it (risk_report_config.org:817), no *_cache* file exists, and the only eventing artefacts are the eight generated event headers, four event registrars and the eventing test. There is no cache to warm. |
| S02 | Pass | regenerate_shell_recipe_inventory.py --check exits 0, and each of the five reporting recipe groups has its entry in doc/recipes/shell/shell.org. No reporting command is destructive, which is consistent with the tree: the only command declared :destructive: anywhere is ores.iam.reset_messages. |
| W03 | Pass | check_populate_references.py prints "populate references resolve." and exits 0. |
| W04 | Pass | validate_schemas.sh reports 304 tables parsed, 0 warnings, no WIRE_001; the parent task's compass db recreate -y -k is the live half of the same item. |
| G08 | Failed, fixed here | The api module had no namespace documentation header at all: projects/ores.reporting/api/include/ores.reporting.api/ores.reporting.api.hpp did not exist. The core module's ores.reporting.core.hpp existed at the required path but was a hand-written aggregate include with no namespace @brief, which nothing included. Both are now namespace documentation headers, matching the convention in the cleaned components (ores.analytics.core.hpp). The service module's header already passed. Facet headers do not exist, which the item says is not a failure. |
| H03 | Failed, fixed here | Four comments narrated edit history: report_instance_trigger_handler.hpp:210, config_parser_tests.cpp:40, report_scheduling_service.cpp:360 and service_scheduling_plan_tests.cpp:26. All four now state the rule rather than the history. No commented-out code and no stray end-of-line comment was found; the line-end comments present are IWYU directives and } // namespace scope markers, both preserved by the rules. |
| P01 | Fail | Four of the five entities speak the canonical entity protocol: concurrency_policy, report_definition, report_instance and report_type each generate list (paged, ordered, with total), get, get_many, put, put_many, delete, delete_many and the versions sub-resource. risk_report_config generates no C++ protocol at all. Its model disables the C++ surface — ores.cpp.protocol, nats-handler, nats-sub-registrar, history-provider-registrar and service are all false (risk_report_config.org:812-820) — and gives the reason at :784-800: nothing writes a risk report config, and generating the surface would add a fifth orphan family, because registrar.cpp is a hand-written fan-out that builds each handler directly and has nowhere to join a generated registrar. The contract nevertheless exists on the TypeScript side: projects/ores.web/packages/wire-protocol/src/generated/reporting/protocol/risk_report_config_protocol.ts is checked in, carries the generated marker, and declares eleven request subjects (reporting.v1.risk_report_configs.list, get, get_many, put, put_many, delete, delete_many, list_by_report_definition_id, the two versions reads) and three event subjects, with nothing on the C++ side to serve or answer them. The twin gate does not see this: _checked_headers globs C++ protocol headers and pairs each with its TypeScript twin, never the reverse, and the gate's opt-out rule looks for :ores.ts.protocol.enabled: and not the C++-side flag, which is why it reports "no model opts out". The absence of the point-in-time read is not a defect: no model declares has_as_of_lookup. |
| S01 | Fail | Same entity, same reason. <entity>_commands.{hpp,cpp} and <entity>_commands_tests.cpp exist for the four entities that generate a protocol, with a process_* case for every command, and the generated units carry the marker with no hand-written survivor. risk_report_config has no command unit, no test and no recipe group. The component has no junction models, so there are no junction verbs to cover. The 11 operation commands (the trigger, schedule, unschedule and the eight execution steps) all have units and tests, judged separately because they are operations and not entity verbs. |
| G04 | Fail | The defects are in shared templates. (1) The generated report_definition.hpp:145 uses std::chrono::system_clock::time_point with no <chrono> in its include block, where its four generated siblings include it; the template cpp_domain_type_class.hpp.mustache emits the field unconditionally but takes the include from the model. (2) report_definition_handler.hpp and its three siblings carry a per-method paragraph claiming the adapter "checks the permission a write needs" on nine methods each, while only put, put_many, delete and delete_many check a permission — 36 false claims from cpp_nats_handler.hpp.mustache:60-68. (3) The generated operator=='s doc narrates edit history ("the omission went unnoticed until the diff payloads were the first generated types to have a test"), from cpp_domain_type_class.hpp.mustache:158. The generated repositories and mappers read like the hand-written house style. |
| H01 | Fail | validate_docs.sh passes, but it checks presence and not content. No written Component Architecture Audit record for ores.reporting exists. The three part diagrams carry the sentinel and an empty manual section: not one method line appears in any reporting .puml, and the rendered ores.reporting.service.png draws application, host and parser as empty boxes though each has methods. The root ores.reporting.puml has no sentinel and no two-pass structure; its rendered image is a package diagram with no class boxes, so it cannot carry an API. The root edges were checked against the code and all are supported. |
| M04 | Fail | On the inert-assignment clause only. The placement clause passes: every catalogued feature sits in the namespace the loader reads (_FEATURE_NAMESPACE, org_loader.py:51-81), with root * Flags lifted at :1356, SQL flags under ** Flags at :1405, and presentation keys under ** Presentation at :1634. But two declared assignments are read by nothing. (1) report_instance.org:239 :parent_id_field: definition_id — its only reader is _parent_id_field (org_loader.py:3042-3049), which returns the empty string unless has_parent_id is set, and report_instance never sets it. (2) risk_report_config.org:505 :parent_entity: report_definition — the key has no reader; the parent is resolved by core.py from the file, and the template placeholder parent_entity_plural is a different variable. Drift is clean, which is the proof that nothing consumes either. |
| M05 | Pass | Each of the five models was compared field by field against its generated CREATE TABLE: the symmetric difference is empty, and the only column not derived from the model is var_quantiles, which the model declares :sql_only: true. No table is a join or a denormalisation, and all five are temporal. reporting_risk_report_config_books_create.sql and ..._portfolios_create.sql are temporal junctions, and the model correctly does not claim them as entities. |
| M07 | Not applicable | The component has no junction models: grep '^#+type:' projects/ores.reporting/modeling/*.org returns entity (five), operation, module and component, and no ores.codegen.junction. The two hand-written junction tables are tenant-scoped already: reporting_rls_policies_create.sql:99-106 and :147-154= both filter on ores_iam_current_tenant_id_fn(). |
| M08 | Fail | Two clauses. (1) Fifteen *_enabled flags are declared :type: integer / :cpp_type: int, defended in the model's own prose at :24-25 ("Analytics flags use integer 0/1, not boolean, to match the project convention"), and generated as integer not null default 0/1 at reporting_risk_report_configs_create.sql:49-64. The item says flags are booleans, and the tree agrees: 64 columns are :type: boolean, and the only integer flags outside this model are in ores.dq, which is not under test. (2) numeric(5,4) and numeric(5,4)[] at :279, :363 and :405 are spelled nowhere else in the tree, where the canonical numeric type is numeric(28, 10) (67 uses) or numeric(18, 10) (3). The template emits the type verbatim, so the spelling reaches the SQL. The custom-type clause is vacuously satisfied: no model references a type from cpp_custom_types.org. |
4.1. Correction to a parent row
The parent task's B06 row calls ores.reporting.core.hpp "the namespace
documentation header … which G08 exempts because an empty namespace is their
whole content". The file was not that: it was an aggregate include over the
domain, repository and service headers, with 19 lines of includes and no
namespace. The census that called it a file with no production includer was
right; the reason it gave was wrong. The file is now the namespace header the
item asks for.
4.2. Open failures
Seven items fail, and none of them is a documentation problem.
| Item | Cause | Where the fix belongs |
|---|---|---|
| P01 | risk_report_config generates no protocol at all, so one of the five entities does not speak the entity protocol. |
The model disables the whole surface with a reason: nothing writes a risk report config yet, and registrar.cpp is a hand-written fan-out, so a generated registrar would be a fifth orphan. Generating the surface requires repairing the registrar composition first. |
| S01 | The same entity has no command unit and no test. | Same cause and same dependency. |
| G04 | Three defects in shared templates: <chrono> is not emitted with the timestamp field, the handler's per-method doc claims a permission check that four of nine methods do not make, and the generated operator== doc narrates edit history. |
cpp_domain_type_class.hpp.mustache and cpp_nats_handler.hpp.mustache. Tree-wide, so it affects every component, not only this one. |
| H01 | No written audit record; the three part diagrams have an empty manual section and draw method-only classes as empty boxes; the root diagram has no two-pass structure. | The diagrams and an audit record, in this component. |
| M04 | Two model keys nothing reads: report_instance 's :parent_id_field: and risk_report_config 's :parent_entity:. |
The two models. |
| M08 | Fifteen flags typed integer instead of boolean, and numeric(5,4) where the tree uses numeric(28, 10). |
ores.reporting.risk_report_config.org, then regenerate. |
| V04 | Sixteen of the 51 generated commands do not answer. | Three causes, below: a shell menu collision, a __none__ placeholder where a uuid is required, and the silent step handlers. |
4.3. Three findings that are not item failures
These surfaced while checking the items above. None of them breaks an item as the standard words it, and all three are this component's own work.
- The instance's definition key is unvalidated. The generated insert
function for
report_instancevalidates the tenant, the workspace, the change reason and the acting account, and never the definition the instance belongs to:reporting_report_instances_create.sql:95-145. The sibling entity does validate its equivalent, because its model declares the key in a* Foreign keysdrawer with:target_column:and:error_message:(risk_report_config.org:504-508), which producesreporting_risk_report_configs_create.sql:143. An instance can therefore name a definition that does not exist, and the failure surfaces later, inside a workflow step, instead of at the write. The fix is to declare the key the same way in the model. - The report input bundle is unmodelled. The table
ores_reporting_report_input_bundles_tbl, its entity and its repository are hand-written, all three without the generated marker, and the repository talks to sqlgen directly with no mapper. A search of the modeling directory finds only a mention incomponent_overview.org:50, so nothing generates them and no drift gate covers them: this is the component's only hand-written data layer. Whether it should be modelled is a real question, since every other entity's data layer here is generated, but it is not an item failure. B03's survey called the repository generatable, the parent task called it infrastructure by elimination, and neither reading has been settled. - The cron expression is an unvalidated string.
report_definition'sschedule_expressionisstd::string(report_definition.org:113-117), while ores.scheduler types the same concept asdomain::cron_expressionwith:is_value_type: true(ores.scheduler.job_definition.org:108-116), which is what stops an invalid expression from entering that domain. A malformed schedule is accepted here and then rejected, or silently ignored, when the job reaches the scheduler.
4.4. Line citations are commit-relative
Every line reference in this table was read from the tree at the commit this
task's branch is based on. Main moves under the table, so a citation can drift
by a few lines without the finding changing: when the branch was rebased onto
315f3bc7a7, four of them had, and the findings they support were unaffected.
A citation is a pointer to the evidence, not the evidence.
4.5. A block that has since lifted
V04, V08, V01 and P05 could not be closed while the tree did not compile.
origin/main currently references ores.dq.api/messaging/fsm_protocol.hpp in
three places — ores.workflow/core/src/service/fsm_state_map.cpp:21,
ores.trading/core/include/ores.trading.core/messaging/trade_handler.hpp:29
and ores.trading/modeling/ores.trading.trade.org:662 — and that header does
not exist: commit dd131c388b replaced it with fsm_state_protocol.hpp and
fsm_transition_protocol.hpp and updated neither consumer. A whole-tree build
fails at ores.workflow.core.lib.
The repair landed as dbfb31d45d and e25313b2d2, which replaced the protocol
include with the FSM repository include in both consumers. On the rebased branch
the whole tree builds (exit 0), check_component_drift.py --component reporting
reports no drift, and the four reporting suites pass 4 of 4. V01 and P05's build
half are therefore satisfied, and V04 and V08 are open work rather than a block.
5. V04 and V08
5.1. V04: every command replayed, and sixteen fail
All 51 generated commands were replayed with check_shell_recipes.py against a
database rebuilt with compass db recreate -y -k and a fleet restarted on the
rebuilt binaries. Thirty-five answer and sixteen do not. Each cause is a defect,
and none is a missing test.
| Commands | Count | Cause |
|---|---|---|
report_definitions list, get, version, versions, delete |
5 | The command is not reporting's. The DQ shell part registers a menu of the same name, and it wins: the request arrives at the DQ service as dq.v1.report_definitions.list. It then answers internal_error, because the table the DQ command reads, ores_dq_report_definitions_tbl, is declared at ores.dq.report_definition.org:124 and created by no script. |
report_definitions add, delete-many, get-many, put-many, set |
5 | The same collision, and the failure proves it: the recipes are generated for reporting's signatures, and the DQ menu rejects them with "Expected 8 arguments, got 9" and "Expected 9 arguments, got 10". get-many reaches the argument check and fails on "Invalid value for id: none". |
report_instances add, put-many, set |
3 | The generated recipe passes __none__ for fsm_state_id, a required uuid and a server-owned field: "Invalid value for fsm_state_id: none". The same placeholder stands where a uuid is required for id. |
the eight execution steps, assemble-bundle to submit-compute |
8 | They time out. Every step handler opens with workflow_step_context::from_message(...) and returns in silence when the workflow headers are absent, which they are for a shell caller, so the caller learns only that time passed. This repeats the parent task's finding, on a rebuilt fleet. |
The collision is the sharper finding, and it is verified from both ends:
dq_commands::register_commands (repl.cpp:105) registers DQ's
report_definitions menu before reporting's at repl.cpp:116, and the service
log records the request at dq.v1.report_definitions.list rather than
reporting.v1.report_definitions.list. Until it is resolved the
report_definitions family cannot be exercised from the shell at all, so S01
passes on the unit and the test rather than on a working command.
5.2. V08: five of six files, and the sixth has a test
repository_report_input_bundle_repository_tests.cpp is added. It creates a
bundle and finds it back by instance id, asserting the literal values it wrote,
and asserts that an unknown instance id returns nothing. Both cases run and pass
in ores.reporting.core.tests.
That leaves the composition and entry-point files, whose test is the fleet starting and whose reason is recorded. The item asks for a test or a recorded reason for every source file, so it is satisfied.
5.3. The verification round, on the current tree
The first pass verified too little, and the gap is worth recording because it looks like success. The build was current, but the fleet was not: the reporting service had started at 10:47 and its binary was rebuilt at 11:40, so the running process was the previous tree's. Only the four reporting suites had been run, the database was 59 SQL files behind, and V04 had been replayed against that fleet. Re-done, on the current tree:
| Check | Result |
|---|---|
compass db recreate -y -k |
exit 0; the reporting fixture seeds 5 definitions, a party and a book |
compass services restart |
25 of 25 running, none stopped or missing. The storage service had been failing on a password for its own database role, because the database predated that role; the recreate fixed it |
cmake --build --preset linux-clang-debug-make |
exit 0, whole tree |
ctest -R reporting |
4 of 4 |
codegen suite (pytest projects/ores.codegen/tests) |
824 passed |
full ctest --preset linux-clang-debug-make |
81 of 86. The five failures are ores.dq.api.tests, ores.dq.core.tests, ores.shell.api.tests, ores.refdata.core.tests and ores.marketdata.core.tests |
| V04 replay | unchanged: 35 answered, 16 aborted, the same 16 |
The five failures are not this component's and not this branch's: the branch
changes thirteen files, all under projects/ores.reporting/ or doc/agile/,
and none under ores.dq, ores.refdata, ores.marketdata or ores.platform. They
are captured as Five test binaries fail on main, and four of them share one cause.
So V01, V02 and V03 hold for this component, V02's repo-wide half does not, and V04's result is reproducible rather than an artifact of a stale fleet.
6. Plan
The parent task claimed "a recorded result for all 49 items". It does not have one. Nineteen items carry no result row: B02, B05, G03, G04, G08, H01, H03, H04, M04, M05, M07, M08, P01, P05, P06, S01, S02, W03 and W04. Nine of those appear nowhere in the parent task at all; the other ten appear only in its plan headings or its summary sentence, never with a result.
Two items are also recorded as partial: V04 and V08. The parent handoff proposed recording both as accepted exceptions. The standard refuses that: "An item that is merely unfinished is not an exception; it is unfinished work, and the component waits." An exception is for a check that cannot pass where the work is done. Neither of these is that, so both are finished instead.
- Stocktake. Run the check each unrecorded item names, and record its verdict with the evidence. A verdict of "not applicable" is recorded with its reason, never skipped.
- Close V04. Replay every generated shell command for reporting against the
live fleet with
check_shell_recipes.py, which is the script the item's own evidence names. Record every command that cannot run and the reason. The eight execution-step commands cannot answer from the shell because their handlers return in silence when the workflow headers are absent; that is a recorded reason, and the item allows it. - Close V08. Give
report_input_bundle_repository.cppa database test, and record the reason for every remaining source file that no test names. The item asks for a test or a recorded reason, so the composition and entry-point files are closed by their reason. - Take no exception. Both items are made to pass, so the registry entry
needs no
ACCEPTED_EXCEPTIONSchange.
7. Notes
7.1. Open captures read for B05
From the inbox: The report-definition eventing re-drive collides with the temporal exclusion constraint; The storage server refuses the report-data bucket its own consumers write to; Three services default their storage base URL to a port the server does not use; Task: Fix reporting entity model drift: update to latest format and regenerate; Finish the eventing payload migration, or read the legacy shape. From next: Party's per-party report_events pgmq queue looks half-baked / unused; Wire service token provider for all services; publish_from_dq codegen facet: hand-writing artefact tables is a recurring cost; Story: Codegen facet: generate the DQ publish-from-dq pattern. From deferred: Trade-aware market data filtering for report execution.
The model-drift capture is satisfied by the parent pass, which brought the reporting models onto the current format and left the drift gate clean. The storage-bucket capture's specific clause — the HTTP server's hard-coded allow-list — was deleted upstream while this work was in flight; its general point, that a bucket is an allocation the facility should be told about rather than a name it hardcodes, is still open.
7.2. The parent task's completeness claim is false
The parent task's Handoff says the checklist "has a recorded result for all 49 items". Its Result names 29 items and its table carries a row for 30, so 19 items have no row at all: the nine that appear nowhere in the file (B02, G03, G04, H04, M04, M05, M07, M08, P05) and the ten that appear only in a plan heading or in the summary sentence (B05, G08, H01, H03, P01, P06, S01, S02, W03, W04). A census of the rows' leading cell over the parent task is the evidence.
This is the failure check_registry_exceptions.py exists to prevent, one
level up: the component is listed in COMPONENTS_UNDER_TEST while items are
neither passing nor recorded, and the gate cannot see it because it only
validates the exceptions that exist, never the ones that should.
7.3. The V04 rows disagree with each other
The parent task's V04 rows report eleven commands, then thirteen, then
"fourteen of fifteen verbs answer"; one row calls set verified and a later
one calls it unverified. The replay in this task is the tie-breaker.
7.4. The parent result's caveat about the environment is now resolved
The completed walk in the parent task depended on two
ORES_*_SERVICE_HTTP_BASE_URL variables added to .env by hand. That file
has been regenerated from the environment profile, so it is consistent
again, and the durable fix stays where it belongs.
8. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
9. PRs
10. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
11. Result
Delivered, and the component is not clean.
The parent task claimed a recorded result for all 49 Component Clean Standard items. Nineteen had no row. This task records them from fresh evidence, so the checklist now carries a result for every item, and seven of them fail: P01, S01, G04, H01, M04, M08 and V04.
Two of those were cheap and are fixed here. G08: the api module had no namespace documentation header at all, and the core module's sat at the required path holding an unused aggregate include; both are now namespace headers matching the cleaned components. H03: four comments narrated edit history and now state the rule.
V08 is closed. report_input_bundle_repository.cpp had neither a test nor a
recorded reason. It now has a test that creates a bundle, finds it back by
instance id and asserts the literal values it wrote, and that an unknown
instance id returns nothing. The composition and entry-point files carry their
reason, which the item accepts in place of a test.
V04 is measured rather than passed. All 51 generated commands were replayed
against a database rebuilt from scratch and a fleet on the current binaries:
35 answer and 16 do not, and every cause is a defect. The
report_definitions family is routed to DQ by a duplicate shell menu, so ten
of its commands exercise the wrong component; three recipes pass __none__
where a uuid is required; and the eight execution-step handlers return in
silence when the workflow headers are absent, so the caller learns only that
time passed. The result reproduces exactly on a second run, so it is not an
artifact of a stale fleet.
Two of those causes are outside this component and are captured rather than
fixed: the duplicate report_definitions shell menu, and
ores_dq_report_definitions_tbl, which DQ's own model names and no script
creates.
Verification, on the current tree: the whole tree builds; the four reporting suites pass and so does the 824-case codegen suite; the full 86-binary suite is 81 of 86, and the five failures belong to ores.dq, ores.refdata, ores.marketdata and ores.platform, which this branch does not touch. They are captured.
One correction to this task's own process, recorded above rather than quietly repaired: the first verification pass ran against a fleet whose binaries predated its own build, a database 59 SQL files behind, and only the reporting suites.
What remains is the seven failing items and V04's three causes, each with a named owner in the stocktake table, plus three findings that no item names: the instance's definition key is unvalidated, the report input bundle is unmodelled, and the cron expression is an unvalidated string. The component is not clean, and the record now says so.