Report execution orchestration

The report execution workflow is neither a seeded row nor a model. It is a definition registered in C++ when the workflow service starts, declaring a type name, a function that returns the ordered phases, and for each phase a name, a command subject, a compensation subject and the payload to send.

The function that builds the phases receives the start message's payload, the tenant and a correlation identifier. It reads the run configuration out of the payload and returns the chain that configuration names, and it throws if the configuration names a chain reporting does not implement, so the run fails before a step is dispatched. The tenant and the correlation identifier go unused, because each step's command is built from the run's own request rather than from the envelope the engine was handed.

Because the function is re-run on every dispatch and on recovery, it must be a pure function of its inputs. A phase list that read anything else could be rebuilt differently after a restart, and the run would change shape midway.

A phase is invoked by publishing its command subject with the step identifier, the workflow instance identifier and the tenant in the message headers, and the phase's own payload in the body. A phase answers on the workflow's completion subject, naming the step and carrying its result.

That is the entire interface. A phase does not know its position in the chain, does not know what the next phase is, and does not call it. Adding a phase adds a participant in this contract; it does not add a new mechanism.

Source: report_execution_orchestration_loop.puml. Regenerate with plantuml -tpng doc/knowledge/architecture/report_execution_orchestration_loop.puml.

The engine's own statement of its rules is Workflows. The table below is only the part a report phase depends on.

The guarantee above that matters most is resumption: dispatch is at least once, with no acknowledgement and no timeout, so a phase can be invoked again after a crash or a restart. Every phase must therefore converge when repeated rather than accumulate.

The report phases do not meet this yet. No reporting handler checks whether the step it is answering has already run, so a re-dispatched phase does its work again: assemble_bundle writes a second bundle row, and gather_trades and prepare_ore_package re-export and re-upload what they exported before. The engine's own record keeps the double invocation from moving the run twice, which is why the defect is invisible in a green run. The fix is the idempotency check the engine offers a handler; until it is used, treat a recovered run as one that may have acted more than once. This gap is recorded on Run a full report: scheduling, reporting and compute.

A run has two records of its progress: the workflow's own instance and step records, and the report instance's state. They serve different readers and can disagree — a workflow that has failed while the instance still reads running is a reachable state in the current implementation.

The report instance is the authoritative record for a caller. The workflow's records are the executor's bookkeeping. The terminal transition is the join between them: the workflow's outcome is what moves the instance, through the state machine, and nothing else may write that state.

A phase fails by answering with a failure rather than by staying silent, and the engine then compensates: it walks the phases whose state is exactly completed, in reverse, and publishes each one's compensation subject.

For a report run, every phase's compensation is the same message, the one that marks the instance failed. Compensation here is therefore not an undo. It is a single terminal transition, and the first compensation to arrive wins, so the error a reader sees is the first phase's. What a completed phase produced, a stored object or a compute batch, stays where it is; nothing deletes it.

Note also that a phase which answered with warnings rather than plain completion is not compensated at all, because the engine walks only the steps whose state is exactly completed. No reporting phase answers with warnings today.

A phase that never answers is covered by the step's deadline: every step states how long it may run, the engine fails the one that outlives it and stops the run, and the failure names the subject and what the engine knows about the service that owns it. The two phases here that wait on the compute grid state a longer budget than the ones that prepare it, because a pricing run takes as long as it takes. Failure, compensation and retry states the rule.

One failure mode is not covered by this and must be designed for:

• A phase list the engine cannot satisfy. An unknown workflow type is logged and dropped while the trigger has already answered that the run started, so a mis-selected chain looks like a success. A run must fail when its chain cannot be built.