Task: Light the intraday reports on the LIVE cut
Table of Contents
This page documents a task in the Rework the market data feeds into a market data service with adapters — the remaining work story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
The intraday reports select the LIVE cut, their configs say live market data, and a run resolves and records the cut.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Rework the market data feeds into a market data service with adapters — the remaining work |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-11-03 |
3. Acceptance
- The Intraday Risk Monitor and Headline Position select LIVE and their configs say live.
- The missing intraday definitions are created with their configs and analytics, dual-cycle ones split.
- A run of the headline report resolves LIVE, mints the cut and records it; the docs and seeds agree.
- The close twins bind run settings whose market data type is eod, so a close report values the portfolio on the weekday before the run date and a live report values it on the day it runs.
- Every configuration that switches the sensitivity analytic on names the two files that analytic reads, and no configuration asks the engine for sensitivities with no factors to shift.
- Every intraday report has a page, the vega and greek ladder family has a page each under the names the product owner settled, and the index states both counts.
- The tests and gates are green.
4. Plan
The work has three parts, and the first is the one the other two depend on.
The cut selection has to reach the definition. A catalogue states a report's
cycle, and the cycle is the kind of cut the report prices from. The DQ artefact
table had no column for it, so the chain gained one: the artefact model gains
cut_kind_code, the acme and northwind catalogues state it per report, the SQL
generator writes it into the artefact populates, and the publish copies it onto
the definition. The publish also gained an upsert mode. Until now a publish that
found any definition for the party returned "skipped", so a moved catalogue
could not reach a database that was already provisioned; the upsert writes the
moved statement as a new version of the definition the party holds, and leaves
the identity, the party and the pipeline phases alone.
A report on both cycles is two definitions. One definition states one
selection, so the reports the catalogue marks intraday · eod are seeded twice:
the plain name selects LIVE and runs during the day, and the same name with
"(EOD)" selects EOD and runs at the close. The existing Headline Position
definition became the LIVE one, because that is what its ten-minute schedule
already said, and the close definition is new beside it.
A run must be able to mint its body. The trigger resolved its definition's selection but passed no body, so a LIVE selection could never mint: the resolution refused for the market data snapshot and the deal population that no store materialises. Two changes settle that. The trigger now assembles the body from the store, through the same reader the hand take uses, reading the trade store, the published sets and the compute versions as the person who asked for the run; and the mint requires the parts the store answers (the reference data version and the engine version) rather than the two references no component keeps, which the cut records as its note instead. The design's own note on a body already said the population and the snapshot are references the store does not materialise, and the hand take writes them empty too.
A development environment is not rebuilt. The seeds changed and the database stayed the same one: the apply script adds the artefact column, stages the regenerated populates and publishes them in upsert mode, per party. It is idempotent and re-runnable, and it is what an environment that is not recreated needs.
The .1 family is out of scope, as instructed: five reports whose measures need an analytic type the platform does not bind.
5. The two bodies of work
5.1. The documentation
The folder holds one page per report, and every intraday report now has one. Twenty-four pages are new and three are extended. Sixteen describe a report the seeds hold. Thirteen describe the vega and greek ladder family, under the names the product owner settled, with the B&R suffix spelled out.
A page carries what a reader needs to run the report and to read it: the purpose and the decision it feeds, the audience, the cycle and the schedule with the reason for it, the inputs, the columns of every file the bound analytics write, the ORE fragment that configures them, the parameters a definition states, a worked read of the archive, and what is not built. The columns come from ORE output formats and the measures from Risk reporting, so a page states what those pages state rather than a second opinion.
The decision the work settled is that a report on two cycles is two pages and not one. A definition states one cut selection, so the close twin is a definition of its own with its own run settings, and a reader chooses between two reports. The frequency follows the report type: the monitors run through the trading day, the close twins run at the close, the maps and Trades Since Last In are on demand, and the ladder family is end of day, with an intraday variant worth naming for the ATM and surface vega ladders.
5.2. The setup
The reports were seeded but their run settings were not the ones the reports need. Every risk report configuration stated market data type live, so the close twins valued the portfolio at the instant the run started. The type now follows the cut the definitions that share the row select, and the close twins bind a row of their own. Four rows are added per tenant: UK gains two, US and HK one each.
The sensitivity analytic named no file, so a run asked the engine for
sensitivities with no factors to shift. Every configuration that switches the
analytic on now names sensitivityConfigFile sensitivity.xml and
marketConfigFile simulation.xml, the names the file types table gives those
parameters and the names the shipped example ore_sensi.xml states.
The intraday definitions also take the engine set their cycle is worth. A trading
desk report reads the live cut and binds Intraday Indicative; the oversight,
valuation and close reports keep EOD Official.
Northwind is generated from the same ACME rows, so the tenant it provisions carries all of it.
6. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit |
|---|---|---|---|
| Catalogue entries | 42 (was 31) | json count of report_catalogue.json |
deb15641d0 |
| Definition rows the four ACME parties hold | 101, unchanged | select count(*) from ores_reporting_report_definitions_tbl |
99d234e95c |
| Definitions that select LIVE | 10 in the catalogue; 22 published (UK 10, US 6, HK 6) | select p.short_code, count(*) ... group by 1 |
99d234e95c |
| Definitions that select EOD | 3 in the catalogue; 7 published | as above | 99d234e95c |
| Run settings rows of the ACME tenant | 50: 46 live, 4 eod | select market_data_type, count(*) ... group by 1 |
99d234e95c |
| Close twins that bind an eod run settings row | 7 of 7: UK 3, US 2, HK 2, onto 4 eod rows | select p.short_code, count(*) filter (where cfg.market_data_type'eod') …= |
99d234e95c |
| Sensitivity analytics that name both files | 13 of 13, across 5 run settings rows | select cfg.name ... max(case when ap.name = 'sensitivityConfigFile' ...) |
99d234e95c |
| Intraday definitions that bind Intraday Indicative | 22 of 29: UK 10, US 6, HK 6. The other 7 are the close twins on EOD Official | select p.short_code, pe.name, count(*) ... group by 1,2 |
99d234e95c |
| Report pages in the folder | 27, of which 24 are new and 1 structure note is new | ls doc/knowledge/reports/report_*.org |
8b528a1a31 |
| The cut a run minted | LIVE.ACME-LONDON.2026-11-03T15:39:32Z, and the run then failed in the package step |
the run's transcript | uncommitted |
| Publish suites (pgTAP) | 184 tests across 19 files, all passing | pg_prove as the test DDL role over projects/ores.sql/test/reporting_*.sql |
3ce7901845 |
| Bindings suite cases | 11, of which 4 are the rebinding cases added here | pg_prove -d $DB projects/ores.sql/test/reporting_publish_report_bindings_test.sql |
3ce7901845 |
| Reporting suites | core 1473 assertions in 260 cases; service 26 in 10; shell reporting 2113 in 762 | ores.reporting.core.tests, ores.reporting.service.tests, ores.shell.reporting.tests |
8b528a1a31 |
| Run document generator | 60 assertions in 27 cases, the eod and live valuation dates among them | ores.reporting.core.tests "[run_document_generator]" |
8b528a1a31 |
| SQL hygiene | 1921 files, 0 problems | python3 projects/ores.sql/utility/check_sql_hygiene.py projects/ores.sql/create projects/ores.sql/populate projects/ores.sql/test |
99d234e95c |
| Codegen documentation | 39 components pass | ./projects/ores.codegen/validate_docs.sh |
8b528a1a31 |
| Codegen drift | none: regenerating writes nothing | codegen.sh regenerate --component reporting --address ores.sql.schema |
99d234e95c |
| Seed drift | none: the two generators and generate_sql.py write what the tree holds |
generate_report_data.py, generate_sql.py in both datasets |
99d234e95c |
| Org lint | filetags and generator markers valid; no dangling id links | ./compass.sh lint |
8b528a1a31 |
| Ore core suite | does not run: the binary predates the branch and fails to load ores::ore::results::index_payload |
ores.ore.core.tests |
8b528a1a31 |
7. Notes
What the run proves, and what it does not. The transcript in
the evidence file shows the
whole resolution: the trigger reads the definition's selection, assembles the
body, mints a cut of kind LIVE at the instant the run started, labels it
LIVE.ACME-LONDON.2026-11-03T13:19:47Z, writes it, records the cut on the run
instance, and dispatches the workflow. Reading the cut back through the service
shows its badge: in_use, because the kind needs no authorisation and the body
is fixed; the earlier cuts of the same scope read superseded, which is the
supersession rule working on real rows.
The run then stops before the engine, on the environment's schema rather than on
the report: the database this environment runs was built from an older checkout
than this branch carries, and ores_reporting_file_types_tbl has no
ore_file_code column here. The reporting service selects that column, so the
run fails while resolving the ORE files its configuration binds. No figure is
produced, and the instance records the failure in its output message.
What the second pass proves, and what stops it. The transcript in
the new evidence file shows the same
chain with the run settings this task adds. A live definition mints its cut,
labels it LIVE.ACME-LONDON.2026-11-03T15:39:32Z, records it on the instance and
dispatches the workflow. The workflow gathers the trades, gathers the market
data and assembles the bundle, and the ORE package step then asks the reference
data service for the conventions families the run document needs. It stops on
refdata.v1.commodity_future_conventions.list, which no running service answers,
and the run records the timeout on itself.
The checkout names no such family. Its convention family registry names
commodity_future_averaging_terms, commodity_future_contract_terms,
commodity_future_expiry_anchor, commodity_future_off_peak_index and
commodity_future_option_expiry, each with a subject stem of its own, so the
exporter that runs and the registry in the tree were built from different
revisions. The services in this environment predate the branch, which is the
same class of fact the earlier pass recorded about the schema. The engine is
therefore not reached here, and the run records the step that stopped it.
What the close twin does instead. A definition that selects the EOD cut does not mint its cut; it names one, and the environment holds none for the day. The transcript shows the refusal, which names the scope, the rates centre and the date. That refusal is the change working: before this task the twin bound run settings whose market data type was live, so it priced from the instant it started and reported live figures under the close's name. A close report that cannot find its close now says so.
A publish that could not rebind. Writing the close twin's run settings found a second gap. The publish inserted a binding only where the party held none, so a catalogue that moved a definition onto another configuration document changed nothing in a database that was already provisioned. The upsert mode gained the missing path, and the suite gained four cases for it.
The environment's administrator could not act in a party. The first
triggers failed at the run grant, because the account's session party was a
System Party row that the parties dataset superseded the same morning. The
membership is a row of ores_iam_account_parties_tbl and removing the stale one
is what let the account offer the real parties at login. The evidence records the
command.
What the cut does not hold. The market data snapshot and the deal population are empty on the minted cut, because no component materialises either: market data answers which published sets were in use, and the trade store answers how many deals it held, and this environment holds no published set at all. The cut's commentary states both, so the record says what the run priced from rather than looking like a smaller body. Making those two references real is the trade cut and the market data cut work; it is not this task's.
The .1 family is not built. Five reports need a 0.1-unit bump series, and each needs an analytic type the platform does not bind, so a definition would state a run that cannot execute. They are named on the reports index under what is not built, with the matrix and remaining ladder reports.
The generic seed is behind the ACME one.
projects/ores.sql/populate/reporting/reporting_report_definitions_populate.sql
still seeds 31 definitions for the generic provisioning path with no cut
selection. It is a second statement of the same catalogue and was left alone, so
a tenant provisioned through it holds no intraday report; the ACME datasets carry
them.
8. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
9. PRs
| PR | Title |
|---|---|
10. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
11. Result
Every intraday report has a page, its cycle has its own run settings, and its sensitivity runs name the files they read.
The pages are twenty-seven, one per report the folder documents: sixteen for the reports the seeds hold, thirteen for the vega and greek ladder family, and the smoke test. The index states both counts and the structure note states which family a reader wants and in what order.
The setup is two changes to the seeds and one to the publish. The risk report
configuration's market data type follows the cut the definitions that share the
row select, so the seven close twins across the three entities bind a row of kind
eod and value the portfolio on the weekday before the run date. The sensitivity
analytic names sensitivity.xml and simulation.xml in thirteen of thirteen
configurations that switch it on. The publish can rebind a definition that a
catalogue moved onto another document.
A run reaches the ORE package step with the new settings, minting its cut and
recording it, and stops there on a conventions family that this environment's
older service binaries disagree about. The run records the step that stopped it,
and the engine is not reached here. Every gate that can run passes: 184 pgTAP
tests across the reporting suites, 1473 assertions in the core reporting suites,
26 in the reporting service, 2113 in the shell reporting suites, the 39 codegen
documentation checks, 1921 SQL files with no problem, and the org lint clean. The
ore core suite does not run at all, because its binary predates the branch and
fails to load ores::ore::results::index_payload.
What remains not built is the ladder family: no definition states one, no mapper
reads or writes a SensitivityAnalysis document, the store states no shift size,
no volatility market data is seeded, and no stored shape holds a bucketed ladder.
Each is named on the page that needs it and in the index.