Task: Read the engine log with one parser
Table of Contents
This page documents a task in the Make ores.ore the only owner of ORE formats story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
One parser in ores.ore reads the engine's log, and every consumer of the log reads it through that parser.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Make ores.ore the only owner of ORE formats |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-28 |
3. Acceptance
- ore_log_parser and its tests are gone.
- The compute node's log publisher reads engine_log_event, and the rows it publishes are unchanged.
- One suite covers both log.txt and log_structured.json.
4. Plan
engine_log already reads both logs, but it keeps less than its two clients read from the old parser, so it gains the rest of what the engine states on a line:
- the severity word as the engine wrote it, because the telemetry publisher forwards the engine's own level names and must keep sending the same ones;
- the location in parentheses, which the publisher sends as the entry's component;
- the time the engine stated, as a value, so no client parses the engine's timestamp again;
- whether the line stated the engine's fields at all. Both clients today read only lines the old parser recognised, and a line that is only text must not become a telemetry row or an error.
The verdict also carries a hand-written reader of the structured message: it
searches the line for name=/=value pairs to name the trade or the curve. The
event already states the subject and the fields, so the verdict reads those, and
the search goes.
Then the old parser, its header and its suite are deleted. Its cases are not dropped: the level mapping, the memory report, the microsecond scaling and the refusal of a line without the engine's shape move to the engine log's suite, which is the one suite that covers both logs.
5. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit | |
|---|---|---|---|---|
| Files that name the old parser | 0 | =grep -rln "ore_log_parser\ | parse_ore_log_line" projects/ –include=*.cpp –include=*.hpp= | (this branch) |
| Cases of the one suite | 27 (18 belonged to the deleted suite) | grep -c "^TEST_CASE" projects/ores.ore/core/tests/log_engine_log_tests.cpp |
(this branch) | |
ores.ore.core.tests |
passed, 23.94s | compass test run -- -R ores.ore.core.tests |
(this branch) | |
ores.compute.core.tests |
passed, 19.42s | compass test run -- -R ores.compute.core.tests |
(this branch) | |
ores.reporting.core.tests |
passed, 51.68s | compass test run -- -R ores.reporting.core.tests |
(this branch) |
6. Notes
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
8. PRs
9. Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | The reader stripped every space after the separator, so a message that starts with spaces came back changed | engine_log.cpp | Accepted | The line is now read once, in the engine's order, and one space after the separator is dropped, as the old parser did. A case states it. |
| 2 | A structured record with no category reached the store with an empty severity | engine_log.cpp | Accepted | It reads as notice, as before, and a case states it. |
| 3 | The contract said engine_format describes a line, but the structured log set it too |
engine_log.hpp | Accepted | Renamed to states_line_fields: true only for a line that states a level, a readable time, a location and the separator, in that order. A structured record states none. |
| 4 | The location and time were found anywhere in the line, so a parenthesis in the message became a location | engine_log.cpp | Accepted | The four fields are read in order, so a bracket or parenthesis after the separator cannot be mistaken for one. A case states it. |
| 5 | The level table was duplicated in the wrapper and untested | log_publisher.cpp | Accepted | store_level_of lives in ores.ore beside the severity vocabulary, and the table has cases for all five levels, a memory report and a line the engine did not write. |
| 6 | The verdict re-read tradeId=/=curveId from the raw fields |
engine_run_verdict.cpp | Accepted | It uses the subject and the cause the event already carries. A curve reason now names the curve a person reads rather than the engine's raw reference. |
| 7 | A structured failure with no message became the raw JSON in a user-facing sentence | engine_run_verdict.cpp | Accepted | The event states whether the engine stated the message, and a record that states none is not a reason. A case states it. |
| 8 | line_count walked the lines again, without the byte order mark and carriage return rules |
engine_log.cpp | Accepted | line_count moved beside the line rules and the verdict calls it. |
| 9 | Stale public-API comment, an unused flag test, a find where contains reads better, comments that restate names |
several | Accepted | All corrected. |
| 10 | The synthetic build fix is unrelated to this task's subject | trading_trade_population_writer.cpp | Accepted | It was needed to build; a parallel environment landed the same fix as c18fc190ad, so it is redundant history and cannot be removed from a merged commit. |
| — | The review bot did not run: three requests each failed within a second with an infrastructure error, at no cost, before reading anything | — | Not reviewed by the bot | The branch was reviewed by hand instead: the publisher keeps the old level table and the old skip set, the verdict's reasons keep their wording, and the comment on message_after_separator was moved back onto its function after the diff review found it orphaned. |
| — | The review bot did not run for #4026 either: the same infrastructure error, at no cost, before reading anything | — | Not reviewed by the bot | #4026 was reviewed by hand: the line parser was replayed against each fixture the old suite held, the level table against all five levels, and the whitespace and parenthesis cases each have a test. |
10. Result
The engine's log has one reader. ore_log_parser, its header and its 18-case
suite are deleted, and its two clients read engine_log:
- the compute node's log publisher, which still sends the same entries under the same telemetry level names: the engine's own level word is kept on the event for exactly that translation, and a line the engine did not write in its own shape is still not forwarded;
- the run verdict, which also loses its hand-written search of the structured
message for
tradeId=/=curveIdpairs. It asks the event for those fields now, so the JSON is read in one place.
engine_log gained what those clients read from the old parser and it did not
keep: the level word, the location that logged the line, the time as a value,
and whether the line stated the engine's fields at all. Two faults surfaced on
the way: a structured message with no category was downgraded to notice, and
the verdict's own fixture stated its sub-fields in a shape the engine never
writes. Both are fixed.
One break on main is fixed here because it stopped these suites from building:
the synthetic trade writer assigned a double into the decimal type that
carries a basket constituent's initial price. A parallel environment landed the
same fix as c18fc190ad, so this one is redundant history.
11. Follow-up
The review of this task found four defects in what it shipped and several clean-ups, all answered in one later PR: the reader stripped the spaces after the separator, a structured record without a category reached the store with an empty severity, the location could be read from a parenthesis inside the message, and the level table was duplicated in the wrapper with no test. The line is now read once, in the engine's order, and the level table lives with the severity vocabulary it translates.