Workflows

Table of Contents

A workflow is how this platform does work that outlives a request: the caller is answered with an instance id, and a service drives an ordered list of steps, recording each one's outcome. The state lives with the platform and any client can read it, which is why a journey can be closed and reopened and why the browser and the shell can drive the same run. Read the cluster in the order below: how a definition declares its steps, the contract a step must honour, the states a run moves through, what happens when a step fails, how a step avoids doing its work twice, and how a step waits for work that takes minutes. The last note is the one most readers came for: how tenant provisioning is described as a plan of operations.

1. Reading order

1.1. The pages for individual workflows

Workflow pages — the inventory of the per-workflow pages, under doc/knowledge/workflows/. Each one is the durable contract for one named process: its purpose, participants, messages, steps, failure policy and progress read. Read a page there when the question is what does this workflow do; read the notes below when the question is how does the engine run any workflow at all.

1.2. The definition that declares the steps

The workflow definition — start here. What a definition declares, where it lives, how one is added or changed, and what protects a run that is already in progress: the step list is resolved once at start and persisted, so a deployment cannot reshape a run. Read it before adding a workflow or changing the steps of one.

1.3. The protocol a step honours

The step protocol — start here. What a caller sends, what a definition declares, how a step is dispatched, what a handler reports to complete it, and what a later step may read from the steps before it. Everything else in the cluster follows from this protocol, so the identifiers a step carries and the completion it reports are worth slowing down on.

The life of a step — the same protocol as a sequence diagram and a flowchart, from the caller's request to the finished run.

1.4. The states a run moves through

The workflow instance and step states — the machine a run and its steps must honour: the states, which are initial and which terminal, and what each transition means. Read it before changing either machine, because a state carries a promise to every reader of a run.

1.5. When a step fails

Failure, compensation and retry — the failure policy as the definition's own choice, with stop-and-retry as provisioning's: what a stopped run leaves behind, what a retry resumes from, and why the steps that completed are kept.

Idempotency in a workflow step — why a step can be re-published at all, and the two forms of safety a step may rely on: an identity that makes a repeat the same step, and a write that converges on repetition.

1.6. Waiting, deadlines and abandonment

Waiting, deadlines and cancellation — how a long-running operation waits by subscription rather than by polling, where its budget belongs, why expiry is a failure of the step, and what abandoning a run requires.

1.7. Provisioning as a plan

Provisioning operations and the plan — the starting point as registered data, the step kinds as code owned by the component that performs them, the dataset machinery that carries the bulk of the work, and what each operation is for. This is the note to consult before adding a step kind.

The tenant provisioning journey — the whole path as an interaction diagram: the starting point chosen, the generic provision request answered with a created tenant and a run to follow, the operations the run dispatches, and retry and discard as its terminal actions.

1.8. How a run starts at a stated time

Scheduler. Read it when the question is where a run comes from at a stated time. The structure note for the scheduler states what a cron expression defines, how the loop fires a job on it, and what a firing records.

Scheduling and workflows. Read it next. It defines the boundary between the two: the job starts the run and does not execute it, the run holds the work, and a step's expiry is a timer of its own rather than a schedule.

1.9. Where it is designed

The provisioning architecture — the investigation that decided this cluster's shape, and the record of the evidence behind it. It is a point-in-time document: the durable design lives in this cluster, and that page says how it was reached.

Emacs 29.3 (Org mode 9.6.15)