Task: Declare what a step consumes and produces

Table of Contents

This page documents a task in the Gather a run's inputs by declared dependency story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

A step carries an ordinal and nothing else: the engine dispatches step n+1 when step n succeeds, and no part of the system states what a step reads or what it leaves behind. A consumer therefore discovers a missing input when it runs and fails, which is why a run still gathering and a run that is broken look the same from outside.

Give a step a declared set of input artifacts and output artifacts, and make the engine consult that declaration before it dispatches.

2. Status

Field Value
State DONE
Parent story Gather a run's inputs by declared dependency
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-08

3. Acceptance

  • A step definition names the artifacts it consumes and the artifacts it produces.
  • The engine refuses to dispatch a step whose declared inputs do not all exist, and names the ones that are missing.
  • An artifact's state is readable per run: missing, in progress, or present.
  • A step that declares no inputs is dispatchable, so the declaration is not a tax on work that has none.

4. Plan

The declaration belongs beside the step definition the registry already holds, because that is where a step's identity and command subject live. The graph task persists it per run; this task is the vocabulary.

An artifact is named by a key the storage plan already builds — reporting/runs/{instance}/.... Reusing that namespace means the declaration and the object cannot drift, and no second registry of names appears.

An artifact's state is derived, never stored twice: present means the object exists and its producing step reached success, in progress means its producer is running, and missing means neither. A stored state column beside the objects would be a second truth that a crash could disagree with; the principle that makes the run recoverable is that the objects are the record.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
       

6. Notes

Landed so far. The vocabulary and the two refusals it buys.

A step definition now names the steps whose results it reads. Until it did, the only statement of that dependency lived inside the step's own command builder, which failed in its own words when a result was missing — so the engine could not tell a run that was waiting on work from a run that was waiting on nothing.

Two refusals follow from the declaration, and both are in:

  • The chain is incoherent. A definition that names a step no earlier step of the chain produces — a typo, or a step that comes after it — is a mistake in the definition, so on_start_workflow refuses it before an instance exists, beside the existing refusal for a step with no deadline.
  • A declared input has not answered. At dispatch the engine checks its own declaration against the steps that have completed, and fails the run naming the inputs it is waiting on, rather than letting the builder discover it.

The declaration travels with the run: materialised_step carries it into materialised_steps_json, for the same reason the deadline does — a definition edited after a run started must not change what that run was waiting for.

The reporting chain declares its own dependencies, and the whole chain was run live after the change: seven steps, completed.

What the task still owes, and how it was answered. The acceptance asks that an artifact's state be readable per run — missing, in progress, or present. The first draft answered it for a step result by reading the step's state, and left the artifacts a step writes beyond its result (produces, carried but unread) to the graph task's persistence. The graph task then settled it the other way: the artifact is the step's name, so there is no separate produces to carry and no second place for a key to live. The field was dropped rather than persisted.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
2781 Let a step declare the steps whose results it reads

9. Review

Comment summary File Decision Notes
       

10. Result

Landed in PR 2781. The declaration is consumes: a step definition names the steps whose results it reads, and the name is the artifact key.

  • A step names what it consumes. workflow_step_def and materialised_step carry consumes, and it travels with the run so a definition edited after a run started cannot change what that run was waiting for.
  • The engine refuses an incoherent chain, and names the inputs that have not answered. on_start_workflow refuses a chain naming a step no earlier step produces, before an instance exists. At dispatch the engine reads its own declaration against the steps that have completed and fails the run naming what it is waiting on, instead of leaving the step's command builder to discover it.
  • An artifact's state is readable per run. The final design made the step's name the artifact, so an artifact's state is its producing step's state and the engine already reads it. A separate produces field was drafted and dropped: two names for one artifact is the second truth this task set out to avoid, and the graph's doc states the rule that a fan-out keeps the names distinct by naming each producer for the batch key it produces.
  • A step that declares no inputs is dispatchable. An empty consumes dispatches immediately, so the declaration is no tax on work that has none.

The graph and dispatch cases cover this, including a fan-out and a fan-in, in ores.workflow.core.tests. The reporting chain was run live after the change: seven steps, completed.

Emacs 29.3 (Org mode 9.6.15)