Surface step colour, prevent result clobbering, and add reload in QA Runner
Table of Contents
This page is a capture in the inbox bucket of the product backlog — a pre-sprint idea, not yet pulled into a sprint as a story.
What
Multi-client test_scenario docs already nest steps under a colour
sub-heading (** blue / ** red, per the
doc_test_scenario.org.mustache archetype), but that structure is only
visible in the raw org file — nothing in the QA Validation Runner
panel itself surfaces which running client instance a step belongs to
when a tester has several coloured windows open side by side. Add a
clear per-step colour indicator (badge, coloured left-border, grouped
sub-list — panel's call) so the tester never has to cross-reference
back to the .org source to know which window a step applies to.
Separately, when the same scenario doc is opened via --open-scenario
in more than one client instance (as happened testing
day_count_fraction_type's CRUD/eventing scenario across blue and red),
nothing stops both instances from independently saving * Results
back to the same file and clobbering each other's PASS/FAIL/notes.
Need a single-owner write mechanism: one instance (first to open, or a
designated primary) owns the * Results write-back; other instances
opening the same scenario should either open read-only or defer their
step outcomes to the owner instead of writing independently.
A third, related gap: there is no way to pick up edits to the
scenario file made outside the panel — e.g. a tester updating steps
from a different instance, or an LLM session editing the .org file
mid-run — without closing and reopening the panel (losing scroll
position and any in-progress tick state). Add a Reload button that
re-reads the scenario from disk in place. Pair it with a file-system
watch on the open scenario doc: when the on-disk file changes while
the panel has it open, the Reload button should turn yellow/gold —
reusing the existing color_constants::stale_indicator convention
already used for server-side change notifications on list windows
(EntityListMdiWindow::markAsStale(), see
ColorConstants.hpp) — so the tester has the same visual language for
"this view is stale, refresh it" whether the staleness comes from a
NATS event or an external file edit.
Why
Discovered while running the manual CRUD + eventing scenario for
day_count_fraction_type: the scenario needed two client instances
(blue driving CRUD, red observing eventing), and both had
--open-scenario pointed at the same doc so the tester could follow
along in either window. That is exactly the situation where a
colour-blind step list and a no-owner save both bite: it is easy to
tick a step in the wrong window, and if both windows later save
results, one tester's ticks silently overwrite the other's.
References
- scenario_verify_day_count_fraction_type_crud_and_eventing.org — the scenario that surfaced this.
See also
- Archetype: doc_test_scenario.org.mustache
- Devops Run Client —
--open-scenariomechanics. - UX Language — colour semantics; the stale-yellow reload convention the Reload button should reuse.