Task: Close the marketdata test coverage
Table of Contents
This page documents a task in the Rewrite the oresmd grammar as a hand-written codec checked against ORE story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
Every marketdata source file has a test that exercises it, or a recorded reason, and the review items no one checked are recorded.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Rewrite the oresmd grammar as a hand-written codec checked against ORE |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-02 |
3. Acceptance
- V08: a per-file survey of
src/against the test run is on the task. - Repository tests exist for
feed_binding,observation_lineageandseries_classification_rule; the generated handlers,publish_from_dq_handler,curve_republish_serviceandmarket_data_clientare tested or recorded. - Every entity has an eventing integration test, or a recorded reason.
- G04: the generated marketdata code has a review record.
- G07 and H04 are recorded.
4. Plan
Run the survey after the grammar cut-over, so the deleted files drop out of it.
Decided with the owner on 2026-10-05: the V08 survey is static. A script
follows each test's includes through marketdata's headers and sources, and
reports every src/ file with the tests that reach it. It measures reach, not
line coverage; the instrumented build that would measure lines does not exist.
- Turn on the generated eventing integration test for every entity that can take one.
- Write repository tests for
feed_binding,observation_lineage,series_classification_ruleand the junction's list by series. - Move the as-of row mapping into
as_of_rowsand test its refusals. - Move
on_tick's decision intoplan_tickand test it. - Test the service's config parser.
- Record the G04 review and the V08 survey, with a reason for every file no test reaches.
5. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit |
|---|---|---|---|
| Source files reached by a test, at the start | 43 of 114 by headers alone; 65 once sources are followed | evidence/marketdata_test_reach.py |
04640bfe8c |
| Source files reached by a test, at the end | 65 of 114 | evidence/marketdata_test_reach.py, output in evidence/marketdata_test_reach.txt |
04640bfe8c |
| Entities with an eventing integration test | 1 of 6 before, 4 of 6 after; the other 2 cannot compile it | ls projects/ores.marketdata/core/tests/*eventing* |
04640bfe8c |
6. Notes
Correct the marketdata clean-standard record recorded G07 and H04 as Pass at ec37b05f77. V08 and
G04 are still open and stay with this task.
A name match finds no test for 73 of 102 source files. See the 2026-10-02 review, section 3, "Test coverage".
The review of PR 2607 found no test for feed_ingest_loop::on_tick. Cover these
behaviours: a tick from an unbound source is dropped, a tick whose URI names no
datum is dropped, a tick whose value is not a number is dropped, and a tick from
a source with several bindings is stored and republished once for each binding.
The review of PR 2630 found no test for the as-of reads' refusals: a short row, a missing bucket ordinal, an ordinal that is not a number, and an ordinal outside the requested buckets. The row mapping sits inside the query call, so a test needs it split out first.
6.1. What the task added
- Generated eventing integration tests for
market_series,feed_bindingandobservation_lineage, joiningseries_classification_rule's. The template sets audit columns thatmarket_fixingandmarket_observationdo not have, so their test does not compile; it is the fifth finding in the capture below. Turning it on also needs the core tests to link refdata, for the entities that take their party from the session. repository_entity_repository_tests.cpp: read back by key and remove forfeed_binding(by id and by source),observation_lineageandseries_classification_rule; and the junction's paged list by series, with its total and its order.as_of_rows: the as-of row mapping, moved out of the model's two reads so a test reaches it. The move is not quite pure: the bucket ordinal must now be all digits, so1x,-1and = 1= are refused as not a number, wherestd::stoultook the leading digits, a sign or a space. A row that is both short and missing its ordinal now reports the missing ordinal first.repository_as_of_rows_tests.cppcovers a good row, a short row, a missing ordinal, an ordinal that is not a number (two,2x) and an ordinal outside the buckets.tick_plan:on_tick's decision, moved out of the loop.app_tick_plan_tests.cppcovers an unbound source, a URI that names no datum, a value that is not a number, and a source with two bindings, which gets two targets with their own republish subjects. The loop keeps the store, the stats, the CRM bridge and the republish, which need NATS and a database.config_parser_tests.cpp: the service's defaults, its--http-base-url, NATS and database options, and--help.- The
observation_lineagegenerator wrotesource_series_idsas[], which the table's check refuses, so no generated lineage row could be stored. The model's generator block now writes a one-element array, and the repository and eventing tests store generated rows.
6.2. V08: the files no test reaches
49 source files are not reached by any test. Each group has its reason:
| Files | Count | Reason |
|---|---|---|
api/src/domain/*_table.cpp, *_table_io.cpp |
14 generated | Shell and REPL display. No template generates a test for them; capture Defects in the generated C++ templates. |
core/src/messaging/*_registrar.cpp, *_history_provider_registrar.cpp |
11 generated | Subscription wiring that needs NATS and the running service. The live run of every marketdata shell command in Give every marketdata entity its shell commands reached each one. |
service/src/messaging/*_event_registrar.cpp |
6 generated | Service eventing wiring. The eventing integration tests assemble the same chain directly, not through these registrars. |
core/src/presentation/*_history_field_mapper.cpp |
6 generated | Version history display, reached by the versions and version commands in the live run; no template generates a test for them. |
core/src/service/market_series_asset_class_service.cpp |
1 generated | Reached by the live run. Its repository is tested. |
core/src/service/market_fixing_service.cpp, market_observation_service.cpp |
2 generated | Reached by the live run. Their repositories are tested; their eventing test does not compile, the fifth template finding. |
api/src/generators/market_series_asset_class_generator.cpp |
1 generated | Unused: it fills market_series_id with a random UUID, which fails the foreign key, so the junction test builds its rows by hand. Same capture. |
service/src/app/application.cpp, host.cpp, feed_ingest_loop.cpp |
3 hand-written | Service start-up and the subscription loop, reached by every live ACME run. The loop's decision is tested through tick_plan. |
core/src/messaging/registrar.cpp |
1 hand-written | Handler wiring, reached by the live run. |
core/src/messaging/publish_from_dq_handler.cpp |
1 hand-written | The workflow step that publishes marketdata datasets; it needs the workflow service. |
service/src/app/curve_republish_service.cpp |
1 hand-written | Needs a bootstrap config and a live curve feed. Its resolver and pillar reader are tested. |
client/src/market_data_client.cpp |
1 hand-written | A NATS client for other services; it needs a running marketdata service. |
api/src/domain/i_feed.cpp |
1 hand-written | The out-of-line destructor of an interface; there is no behaviour to test. |
6.3. G04: the generated code review
Read on 2026-10-05: the junction's generated service, handler, repository and protocol, which this story generated first, against the entities that had them already, and the eventing tests generated for the other entities. The code reads like the component's hand-written code. Six findings, all in shared templates:
- Namespace closers carry an end-of-line comment, which H03 forbids. Task Remove the namespace closer comments from the templates.
- A claim lost between the service's check and its write reports as
internal_error, notconflict: every handler catches any repository exception the same way. put_manyreads each written row back with its own query.- No template generates a test for the
table,table_ioorhistory_field_mapperoutput. - Generators fill foreign keys with random UUIDs, so their rows fail the foreign-key checks.
- The eventing integration test sets audit columns, so it does not compile for the time-series entities.
The last five are capture Defects in the generated C++ templates.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
8. PRs
| PR | Title |
|---|---|
| #2650 | [marketdata] Close the marketdata test coverage |
9. Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | PR 2650: the ordinal parse is stricter than before; say so. | as_of_rows.cpp |
Accepted | Noted under the task's notes. |
| 2 | PR 2650: a short row with no ordinal now reports the ordinal first. | as_of_rows.cpp |
Not changed | Both refusals still fire; no test depends on the order. Noted. |
| 3 | PR 2650: -1 wraps in std::stoul and reports as outside the buckets. |
as_of_rows.cpp |
Accepted | The ordinal must be all digits; -1 and = 1= are tested. |
| 4 | PR 2650: confirm bindings_by_source_ never holds an empty vector. |
feed_ingest_loop.cpp |
Not changed | refresh() creates an entry only by pushing an enabled binding. |
| 5 | PR 2650: would a foreign key refuse the generator's fixed source id? | observation_lineage model |
Not changed | source_series_ids is jsonb with one check, a non-empty array; no foreign key. |
10. Result
Every marketdata entity with audit columns has an eventing integration test,
and the two time-series entities are recorded with the template defect that
stops theirs. The three
repositories and the junction that had no repository test have one. The as-of
reads' refusals and on_tick's decision moved into small functions that unit
tests reach. The V08 survey is a script on the task: 65 of 114 source files are
reached by a test, and each of the other 49 is recorded with its reason. The G04
review found six template defects, one a task and five a capture.