Report execution

Turns a report definition into a completed report instance. When it finishes, the instance has walked pending, running and completed, the engine's output archive is in storage, and the run's record says where it landed. The workflow is also what keeps reporting free of the grid's timing: submission is asynchronous, and the engine holds the run open until the compute batch closes.

The fields of each message are declared in projects/ores.reporting/api/include/ores.reporting.api/messaging/report_operations_protocol.hpp.

The chain is a function of the run configuration the definition carries, so there are two shapes. A run that executes pre-processing runs seven steps; a run that substitutes it runs four. Steps 1 to 4 below appear only in the executing shape, and step 6 appears in both shapes with two possible operations.

Each step's command is built from the run's request and from the earlier steps' results, which are addressed by the name of the step that produced them. A builder that cannot find or read what it consumes stops the run rather than handing the next phase an empty field.

Source: [[./report_execution_interaction.puml][report_execution_interaction.puml]]. Regenerate with plantuml -tpng doc/knowledge/workflows/report_execution_interaction.puml.

This workflow declares no failure policy of its own: the engine compensates the steps that completed, in reverse. Every step compensates through reporting.v1.ops.fail_report, so a compensation is not an undo. It marks the report instance failed with the failing step's message, and each further compensation leaves the first message in place. What a completed step produced, a stored object or a compute batch, stays where it is; nothing deletes it.

A retry resumes from the run's persisted record of completed steps: a restart re-dispatches every step left in progress, under its original step id. The handlers do not yet check whether they have already run, so a re-dispatched phase does work twice. Until they do, two facts follow. Treat a run that was recovered as having acted more than once, and prefer triggering a fresh run over retrying a failed one.

Two failure modes have no automatic recovery. A phase that never answers leaves the run in progress, because there is no acknowledgement and no deadline; the run then re-dispatches on every restart. And an unknown workflow type is logged and dropped while the trigger has already answered triggered, so a mistyped type looks like a successful trigger that produced nothing. Both are recorded on the story below.

The engine's own failure rules are on Workflows.

Two reads render progress, and they answer different questions.

• The run: workflow.v1.instances.list and workflow.v1.ops.get_workflow_steps, over the workflow instance and its steps. The workflow instance id is created by the trigger and carried in the start message, so a client can follow the engine's view before the first step answers.
• The report: reporting.v1.report_instances.list and reporting.v1.report_instances.get, over the report instance. One line shows the definition, the instance's state, when it started, when it completed, and the output message.

The report instance is the read a business user follows, and it is the one that carries the failure text. Its states are pending, queued, running, completed, failed, skipped and cancelled, defined on Report definition and schedule. The engine's instance states are a separate vocabulary, and the two are not the same table.

• Correlation Identifier — the start message carries the run's correlation id.
• Saga — every step compensates through reporting.v1.ops.fail_report, which marks the instance failed.
• Process Manager — the engine runs the report chain the run configuration selects.

• Workflow pages — the inventory this page is listed in.
• Report execution pipeline — the phases this workflow sequences.
• Report execution orchestration — why the engine executes a run.
• Report execution substitution — the two chain shapes and what each proves.
• Report processing interfaces — every message and storage key, with its publisher and consumer.
• Sample bundle smoke test — the report that exercises the substituted chain.
• Run a full report: scheduling, reporting and compute — the story that owns the workflow.
• Run an example bundle through the compute grid from a report definition — the story that owns the known failure modes above.

• Open Source Risk Engine — the engine the submitted package runs.