The step protocol

The caller starts an instance by publishing start_workflow_message to workflow.v1.ops.start_workflow. The message is fire-and-forget, and the fields it carries are drawn below.

Source: workflow_start_message.puml. Regenerate with plantuml -tpng doc/knowledge/architecture/workflow_start_message.puml.

The caller-supplied identifier is what makes a retried start the same run. A non-empty identifier becomes the instance uuid, and a second start bearing an identifier that is already known is that same instance, which the engine notes and leaves unchanged; an empty identifier gets a fresh uuid.

A start the design cannot use is refused: an unknown workflow type, a tenant that is not a uuid, a step list the definition fails to build, or an empty step list. An accepted start keeps the request, the correlation identifier and the step count, so that every later builder call works from the same material.

Source: workflow_step_protocol.puml. Regenerate with plantuml -tpng doc/knowledge/architecture/workflow_step_protocol.puml.

The diagram places the process manager at the centre of the protocol. The caller starts a run with one message and is answered at once; the definition supplies the ordered step list; the process manager stamps each step command with the three identifiers the handler needs; the handler reports one outcome; and the reader follows the run from the state the platform persists.

A definition declares the workflow type its starts are matched against, a description, and the step list. Each declared step carries a name, the identity that addresses its result; a command subject; a compensation subject, absent when the step has no rollback; a builder for its command payload, which turns the request and the results so far into that payload; and a builder for its compensation payload, which turns the original command and the step's result into the rollback.

The engine, as the process manager, builds the step list at start and again on every advance, always from the instance's stored request, the tenant and the correlation identifier. The protocol admits nothing else, so no completed step's result reaches the builder. That is the guarantee the design demands: a definition cannot depend on anything the engine does not hold when it builds the list.

The list must be the same on every advance, because an advance dispatches the step at the next position out of the list just built. The engine guards that invariant by recording what the definition declared at start and comparing each rebuild against it, so a differing list is caught rather than obeyed.

The engine persists the step before it publishes, and sends the command to the subject the step declares, exactly as declared. The publish is fire-and-forget: it carries no reply inbox and no acknowledgement.

Source: workflow_step_command.puml. Regenerate with plantuml -tpng doc/knowledge/architecture/workflow_step_command.puml.

The command carries the three identifiers of the step, the instance and the tenant. It carries no caller token, so the handler cannot authenticate as the caller, and the actor stays on the step's record as its last modifier.

The handler publishes step_completed_event to workflow.v1.ops.step_completed, and the step identifier it carries echoes the one on the command. The three headers on the command tell it which run it serves and which step it answers for.

Source: workflow_step_completion.puml. Regenerate with plantuml -tpng doc/knowledge/architecture/workflow_step_completion.puml.

The completion reports one of three outcomes: success with a result, success with warnings, which adds per-item log entries, or failure, which carries an error and an empty result. The engine treats success and success with warnings as success and dispatches the next step, while failure begins compensation. It drops a completion it cannot place — one for a step it does not know, and one for a step that is no longer in progress — so a duplicate or a late completion is harmless. See idempotency in a workflow step.

A step's command builder receives the instance's request and the results so far. Each result carries the name of the step that produced it and a response, and a later step addresses an earlier result by that name, never by position; a step that returns nothing contributes nothing, and a failed step is never built past.

An advance takes the step after the instance's current position; once that position passes the last stored step, the engine marks the instance complete with the last step's result and publishes a status event. Completion is therefore positional: the run stops because it exhausted the steps its definition declared, and no further step is dispatched.

There is no completion hook. A definition that must act at the end declares that work as a step and places it last. The states are catalogued in the instance and step states.

A definition declares the policy its failed step follows. Under the compensate policy the engine moves the run to compensating, walks the completed forward steps in reverse, and publishes each one's compensation subject until the run is compensated, so the run undoes its completed work. Under the stop-and-retry policy the engine parks the run on the failed step, keeps the completed steps, and waits for a retry verb, so the work that succeeded is not thrown away.

Source: workflow_retry_verb.puml. Regenerate with plantuml -tpng doc/knowledge/architecture/workflow_retry_verb.puml.

A caller sends the retry verb to resume a stopped run. It names the run, and it may name the step to resume from. The run's state is the platform's record of what has already happened, so a retry resumes from that record rather than from the caller's memory of it. See failure, compensation and retry.