How do I run the tests?

Question
Answer
Script
Tested by
See also

For the test-logging knobs and output layout, see CMake setup.

Question

How do I run the ORE Studio test suite?

Answer

Running tests builds the test binaries first, so use compass build (not raw cmake --build) — it takes the host-wide build lock described in How do I build the system?, keeping concurrent worktrees from corrupting each other's build outputs.

All tests

The rat target runs every test:

./compass.sh build rat --preset linux-clang-release-make

Specific component

For one component's tests, use the test_<component> target:

./compass.sh build test_ores.accounts.tests --preset linux-clang-debug-make

Replace ores.accounts.tests with the component you want. The target name is test_<component_full_name>.tests — e.g. for projects/ores.dq/api the target is test_ores.dq.api.tests, for projects/ores.iam/core it's test_ores.iam.core.tests. If unsure, list the available targets:

cmake --build build/output/linux-clang-debug-make --target help | grep test_

Enabling test logging

Logging is OFF by default for performance. To enable it at configure time (preferred):

# Logs to files only
cmake --preset linux-clang-debug-make -DORES_TEST_LOG_LEVEL=debug

# Logs to files and console
cmake --preset linux-clang-debug-make -DORES_TEST_LOG_LEVEL=trace -DORES_TEST_LOG_CONSOLE=ON

Then run the tests as usual:

./compass.sh build rat --preset linux-clang-debug-make

To turn logging off again:

cmake --preset linux-clang-debug-make -DORES_TEST_LOG_LEVEL=OFF

The env-var equivalents (ORES_TEST_LOG_ENABLED, ORES_TEST_LOG_LEVEL, ORES_TEST_LOG_CONSOLE) work too if you want to override at run time without reconfiguring.

Analysing failures

After running tests with logging enabled:

./scripts/parse_test_results.py \
    build/output/linux-clang-debug-make/publish/bin

The script:

Parses all test-results*.xml files in the directory.
Reports total / passed / failed / duration.
For each failure: test name, location, exception info, and the matching ERROR / WARN log entries.

Script

compass build (projects/ores.compass/src/compass.py) wraps cmake --build, adding the host-wide build lock. parse_test_results.py does the XML + log parsing.

Tested by

CI runs rat on every push. The parse_test_results.py script is itself exercised by use in CI failure analysis.