How do I run the tests?

Table of Contents

For the test-logging knobs and output layout, see CMake setup.

Question

How do I run the ORE Studio test suite?

Answer

All tests

The rat target runs every test:

cmake --build --preset linux-clang-release-ninja --target rat

Specific component

For one component's tests, use the test_<component> target:

cmake --build --preset linux-clang-debug-ninja --target test_ores.accounts.tests

Replace ores.accounts.tests with the component you want.

Enabling test logging

Logging is OFF by default for performance. To enable it at configure time (preferred):

# Logs to files only
cmake --preset linux-clang-debug-ninja -DORES_TEST_LOG_LEVEL=debug

# Logs to files and console
cmake --preset linux-clang-debug-ninja -DORES_TEST_LOG_LEVEL=trace -DORES_TEST_LOG_CONSOLE=ON

Then run the tests as usual:

cmake --build --preset linux-clang-debug-ninja --target rat

To turn logging off again:

cmake --preset linux-clang-debug-ninja -DORES_TEST_LOG_LEVEL=OFF

The env-var equivalents (ORES_TEST_LOG_ENABLED, ORES_TEST_LOG_LEVEL, ORES_TEST_LOG_CONSOLE) work too if you want to override at run time without reconfiguring.

Analysing failures

After running tests with logging enabled:

./scripts/parse_test_results.py \
    build/output/linux-clang-debug-ninja/publish/bin

The script:

  • Parses all test-results*.xml files in the directory.
  • Reports total / passed / failed / duration.
  • For each failure: test name, location, exception info, and the matching ERROR / WARN log entries.

Script

Commands above. No wrapper.

Tested by

CI runs rat on every push. The parse_test_results.py script is itself exercised by use in CI failure analysis.

See also

Emacs 29.1 (Org mode 9.6.6)