Task: Implement Hotfix: the workflow deadline pass reads an entry it has freed

Table of Contents

This page documents a task in the Hotfix: the workflow deadline pass reads an entry it has freed story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

expire_overdue_steps() erases a step's awaiting entry only after it has read and stopped the run the entry names.

2. Status

Field Value
State DONE
Parent story Hotfix: the workflow deadline pass reads an entry it has freed
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-02

3. Acceptance

  • No read through waiting happens after its entry is erased.
  • An exception after the step is failed erases the entry once, through the handler, not twice.
  • Valgrind reports no error on either deadline case, and the workflow suite passes.

4. Plan

Diagnosis: the loop binds const auto& waiting = it->second, fails the step, calls it = awaiting_.erase(it), and then reads waiting.instance_id to find the run. The node is freed by then. libstdc++ leaves the freed bytes in place, so Linux reads the right id by accident; on macOS the id reads back wrong, the run lookup returns nothing, the continue skips stop_on_failure, and the run keeps its state. That is every macOS symptom: the step is failed, the run is not, and a retry refuses a run that has not stopped.

A second fault sat beside it: if the run read or stop_on_failure threw, the handler erased it again, which by then pointed at the next entry or at end().

Fix: move the erase to the end of the expiry branch, after the run is read and stopped. The handler's erase is then the only one on that path.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
valgrind, before 1 error: invalid read of size 16 at workflow_engine.cpp:1040, freed at :1037 by awaiting_.erase valgrind ores.workflow.core.tests "workflow_engine stops a run whose step outlives its deadline" 1d1dde7ca6 plus the marketdata hotfix
valgrind, after 0 errors on both deadline cases; 12 and 15 assertions pass the same, for both cases cd34c7c753 plus this change
Workflow suite, after passed, 9.38 s ctest -R "^ores.workflow.core.tests$" cd34c7c753 plus this change
macOS CI, before both deadline cases fail on both jobs Continuous MacOS runs 36932809229, 36957593572, 36987873236 8f8789e0a3 to ee63ba0aa3

6. Notes

Found from the macOS logs while fixing the marketdata timeout, captured as The workflow step-deadline tests fail on macOS.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#2508 [workflow] Erase an overdue step's entry after its run is stopped

9. Review

Comment summary File Decision Notes
       

10. Result

The erase moved after the run is read and stopped. Valgrind went from one invalid read to none, and the workflow suite passes. macOS CI is the remaining check.

Emacs 29.3 (Org mode 9.6.15)