How do I investigate a test failure?

A unit test is failing. How do I find the root cause from the results XML and the per-test logs?

1. Scope the investigation. Ask the user which suite(s) and whether they want one specific failure or all failures in the suite. Typical suites: ores.accounts.tests, ores.refdata.tests, ores.cli.tests, or the rat meta-target that runs everything.

2. Enable logging. Off by default for performance. Per How do I enable test logging?:

   cmake --preset linux-clang-debug-ninja -DORES_TEST_LOG_LEVEL=debug
   # or for more detail
   cmake --preset linux-clang-debug-ninja \
         -DORES_TEST_LOG_LEVEL=trace -DORES_TEST_LOG_CONSOLE=ON
   

3. Run the failing suite. Run only what you need to keep noise down:

   cmake --build --target test_ores.accounts.tests \
         --preset linux-clang-debug-ninja
   

   Replace ores.accounts with the target component. Use --target rat to run everything.

4. Parse the results. The script reads the test-result XML files alongside their log files and prints a per-suite summary with failure locations and the relevant log lines:

   ./scripts/parse_test_results.py \
       build/output/linux-clang-debug-ninja/publish/bin
   

   Output sections:

   • File summary — which XMLs were found and where logs live.
   • Per-suite results — per failed test: exception message, test line, duration, links to the per-test log, and any [ERROR] / [WARN] lines in the log.
   • Overall summary — total / pass / fail counts and rates.
5. Walk the investigation in this order:
   1. Read the Exception line — the assertion or thrown error.
   2. Open the test file at the reported file:line to see what the test expected.
   3. Read the per-test log (under build/output/<preset>/publish/log/<suite>/<category>/<test>.log) for [ERROR] / [WARN] entries.
   4. If the per-test log is not enough, read the suite-level log one folder up.
   5. Crank the log level to trace and re-run (step 2 + 3).
   6. As a last resort, add targeted BOOST_LOG_SEV(lg, debug) << lines into the code under test, rebuild, re-run.
6. Common surprises:
   • "No test logs found" → logging is disabled; go back to step 2.
   • XML parse error → the test binary crashed before writing a clean result XML; the script dumps the full suite log so you can find the crash site.

7. Disable logging when done, to keep the test suite quick:

   cmake --preset linux-clang-debug-ninja -DORES_TEST_LOG_LEVEL=OFF
   

scripts/parse_test_results.py does the XML + log parsing and report layout.

Manual. The script is exercised every time tests go red.

• How do I enable test logging? — the prerequisite knob.
• How do I run the tests? — the surrounding run loop.
• Unit test conventions — what to read in the test file once you find it.
• code-investigate-test-failure skill — the skill that drives this recipe.