Report processing interfaces
Table of Contents
1. Summary
This page is the reference for what a report run sends and stores. It collects the messages of the pipeline, the messages that control it, and the objects a run reads and writes, so that an implementer has one list to work from rather than one per phase.
Subjects marked designed do not exist yet; an unmarked subject is implemented and its name is the one in the protocol header. A substitute's subject is dispatched instead of the phase it replaces, never alongside it.
2. Detail
2.1. The run's messages
| Subject | Publisher | Consumer | Payload |
|---|---|---|---|
reporting.v1.ops.trigger_report_instance |
the shell, or the scheduler on a firing | reporting | definition, tenant, and the firing's identity |
workflow.v1.ops.start_workflow |
reporting | the workflow engine | workflow type, the run's request, and its run configuration |
reporting.v1.ops.gather_trades |
the workflow engine | reporting | instance, definition, tenant |
reporting.v1.ops.gather_market_data |
the workflow engine | reporting | instance, definition, tenant |
reporting.v1.ops.assemble_bundle |
the workflow engine | reporting | the two storage keys and the counts they carry |
ore.v1.ops.prepare_ore_package |
the workflow engine | ores.ore | the two storage keys |
reporting.v1.ops.resolve_prepared_input (substitute) |
the workflow engine | reporting | instance, tenant, and the prepared input key |
compute.v1.ops.submit_compute |
the workflow engine | ores.compute | instance, tenant, correlation, and the tarball URIs |
compute.v1.ops.claim_work |
a worker node | ores.compute | the host and its platform; the reply holds the result, workunit, package and its digest, input, output and a storage token |
compute.v1.ops.submit_result |
the worker node | ores.compute | result, host, output location, outcome |
reporting.v1.ops.collect_compute_results |
the workflow engine | reporting | instance, tenant, batch (a stub: it ingests nothing yet) |
reporting.v1.ops.ignore_compute_results (substitute) |
the workflow engine | reporting | instance, tenant, batch |
reporting.v1.ops.finalise_report |
the workflow engine | reporting | instance, tenant |
reporting.v1.ops.fail_report |
the engine's compensation | reporting | instance, tenant, and the reason |
workflow.v1.ops.step_completed |
any phase | the workflow engine | step and its result |
2.2. The messages that control the run rather than perform it
| Subject | Publisher | Consumer | Purpose |
|---|---|---|---|
scheduler.v1.job_definitions.put |
reporting, when reconciling | the scheduler | create or update a definition's schedule |
scheduler.v1.job_definitions.delete |
reporting, when unscheduling | the scheduler | remove a definition's schedule |
workflow.v1.ops.get_step_result |
the workflow engine, and any reader | the workflow engine | read a completed step's result |
reporting.v1.ops.schedule_report_definitions |
the shell, or an operator | reporting | schedule on demand |
reporting.v1.ops.unschedule_report_definitions |
the shell, or an operator | reporting | unschedule on demand |
2.3. The bulk reads pre-processing depends on
These are not steps of the run, but the first phase cannot work without them. Both exist, as operation subjects on the component that owns the data.
| Subject | Publisher | Consumer | Payload |
|---|---|---|---|
trading.v1.ops.export_trades_to_storage |
pre-processing | ores.trading | the book scope and the target key; returns the trade count |
marketdata.v1.ops.export_market_data_to_storage |
pre-processing | ores.marketdata | the target key; returns the series count |
Both write into the platform bucket under the key the caller supplies, which is
reporting/runs/{instance}/trades.msgpack and
reporting/runs/{instance}/market_data.msgpack
(projects/ores.reporting/core/src/service/execution_storage_plan.cpp). Neither
filters by the run's scope beyond the trades call's book list, so a report whose
scope is narrower than its tenant still exports the tenant's data. That is what
the designed, scope-limited reads below would close; they do not exist.
| Subject | Publisher | Consumer | Payload |
|---|---|---|---|
trading.v1.ops.scope-read (designed) |
pre-processing | ores.trading | scope and as-of timestamp; returns the trades |
marketdata.v1.ops.resolve (designed) |
pre-processing | ores.marketdata | the selected trades; returns the items they require |
2.4. Storage
| Object | Key | Written by | Read by |
|---|---|---|---|
| the engine package | compute/packages/{app}-{version}/{app}-{version}-{platform}.tar.gz |
seeded, once per engine build | the worker node |
| the gathered trades | reporting/runs/{instance}/trades.msgpack |
ores.trading, asked by pre-processing | the packaging step |
| the gathered market data | reporting/runs/{instance}/market_data.msgpack |
ores.marketdata, asked by pre-processing | the packaging step |
| the package | ore/packages/{instance}/ore_package.tar.gz |
the packaging step | the worker node |
| a prepared input | whatever the definition's prepared_input_key names |
seeding, or an earlier run | the run, on the substitute path |
| the engine output | compute/output/{result_id}.tar.gz |
the worker node | retrieval, post-processing |
All keys are under the platform bucket ores, so the object path is
/api/v1/storage/ores/{key}. The engine package and the output path are built
by ores.compute.api/net/compute_storage.hpp, and the three reporting keys by
projects/ores.reporting/core/src/service/execution_storage_plan.cpp and
projects/ores.ore/service/src/messaging/report_package_handler.cpp.
The packaging step downloads the gathered blobs and lays the run's own data at
the paths the run document names. The market data and the fixings go there as
they are; the trades are MsgPack rows the engine cannot read, so they are
rebuilt into the ORE portfolio the engine prices
(ores.ore.core/xml/exporter.hpp). The result is the Input/ directory shape
the engine expects, which is why the prepared substitute for a test run is a
bundle a person could hand the engine.
Both object access paths are as specified in
Object Storage: the command subjects
storage.v1.objects.put/get/delete/list for control, and
PUT/GET /api/v1/storage/{bucket}/{key} for bytes.
2.5. Naming
- Entity verbs name a resource and a verb, as the entity protocol specifies.
- Not every subject here is an operation. The phase subjects are addressed by
the service that owns the phase (
reporting.v1.report.*,ore.v1.report.*,compute.v1.report.*) rather than under a sharedopssegment, because the phase a service implements is part of that service's own surface. - The subject of a substitute names what it does —
resolve-prepared-input,ignore-compute-results— rather than naming the phase it replaces plus a flag.
3. Patterns used
- Claim Check — the trades and market data exports write to storage and answer with the key.
4. See also
- Report Processing — the structure note that orders this cluster, and where to read this page in it.