Task: Set the intraday frequencies a desk expects
Table of Contents
This page documents a task in the Rework the market data feeds into a market data service with adapters — the remaining work story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
The intraday reports run by default at the agreed cadences; only the investigative maps stay on demand; the docs and the seeds agree and the live jobs fire at the new cadence.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Rework the market data feeds into a market data service with adapters — the remaining work |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-11-04 |
3. Acceptance
- The triple and the surface ladder run hourly; the spot ladder every thirty minutes; trades since last in every fifteen; the maps stay on demand; the rest keep their cadence.
- The seeds and the pages state the same expressions, and the index table agrees.
- The live environment runs the new cadence (definitions and jobs), and the reports tick.
4. Plan
The change has three layers, and each one is checked before the next is built on it. The cadence is one fact stated in three places, and a reader must be able to get from any of them to the same expression.
- The catalogue and its seeds. The four cadences move in
report_catalogue.json, which is the one statement the party sets are derived from, and the two descriptions that claimed a report ran on demand lose that claim. The generated files follow:report_definitions.jsonbygenerate_report_data.py, and the five populate scripts by the twogenerate_sql.pygenerators. Northwind reads the ACME catalogue, so it gains the same statements without a second edit. - The pages. Each page's Frequency section states the expression its seed states, in words, and the spot ladder and trades since last in lose the on-demand claim in the places it was made rather than in one place only. The index gains a Frequency column, because the index is what a reader opens first, and the families note reads the triple as hourly.
- The live environment. A development environment stays up while the seeds move, so the change is applied in the order the layers stand in: stage and publish the catalogue, then move the schedules. A job carries its own copy of the expression, so the publish does not move a schedule: the definitions are unscheduled and scheduled again as the tenant's administrator for each party, which is the operation that writes the job and asks IAM for the run grant. Then the scheduler is watched, and the fires are the evidence.
5. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit |
|---|---|---|---|
| Catalogue rows whose cadence moved | 4 of 60: the triple, the surface ladder, the spot ladder and trades since last in | json by name over report_catalogue.json |
c692970339 |
| Seeded definition rows the change moved | 15 across the five datasets: group 1, UK 4, US 3, HK 3, Northwind 4 | python3 generate_report_data.py; python3 generate_sql.py in both dataset directories |
c692970339 |
| Seed generators idempotent | a second run of all three writes nothing: the md5 of every changed file is unchanged | as above, then md5sum -c |
c692970339 |
| Definition rows the live publish moved | 12: group 1, UK 4, US 4, HK 3 | the apply script | ecb1a8db83 |
| Definitions scheduled by the operation | 11: 1 group, 4 UK, 3 US, 3 HK; none refused, no job row written by hand | the scheduling script | ecb1a8db83 |
| Definitions whose job states the definition's expression | 11 of 11, all active, all nats_publish |
the live transcript | ecb1a8db83 |
| Fires at the hour | 11 definitions, 15:00:00 to 15:00:04, no person acting | as above | ecb1a8db83 |
| Fires in the half hour after | trades 3 parties at 15:00, 15:15 and 15:30; spot ladder 3 parties at 15:00 and 15:30 and not at 15:15; neither triple after 15:00 | as above | ecb1a8db83 |
| The legacy triple's job | 1 ended, which stops the fifteen-minute firing of the same report | as above | ecb1a8db83 |
| pgTAP reporting suites | 19 files, 188 tests, all passing | ./projects/ores.sql/test/run_tests.sh projects/ores.sql/test/reporting_*.sql |
ecb1a8db83 |
| The pgTAP publish suites alone | 5 files, 54 tests, all passing | ./projects/ores.sql/test/run_tests.sh over the five reporting_publish_* files |
ecb1a8db83 |
| Local checks over the change | 9 of 17 pass as run; the other 8 fail only because this worktree holds no codegen venv, and all 8 pass when run with the main worktree's venv | ./compass.sh check --exclude-heavy |
ecb1a8db83 |
| SQL hygiene | 3342 files, 0 problems | python3 projects/ores.sql/utility/check_sql_hygiene.py projects/ores.sql |
ecb1a8db83 |
| Org lint, links and the site page | all pass, in the check above | ./compass.sh lint, org-links, site-page |
ecb1a8db83 |
6. Notes
The environment holds two naming families. The catalogue renamed three reports after the 2026-11-03 publish, so the live database holds both the old name and the new one for the triple, the spot greeks and the spot ladder. The triple's older row held the fifteen-minute job, and that job is the one this task ended. The spot greeks and the close triple keep jobs at the cadences the agreed table states, so they do not contradict it, but the report runs twice for each firing until the work that retires the older catalogue lands. That duplication was there before this task and is not created by it.
The job is not a view over the definition. A definition states when a report runs and the scheduler holds the expression the job was made with. A publish that moves a definition therefore leaves a schedule where it was, and the two operations in the shell are the only path that moves both. The scheduling operation also asks IAM for the run grant the runs act under, so writing job rows directly would leave runs that cannot act in their party; no job row was written by hand here.
The environment's own binaries are used from this worktree. This worktree
holds no compiled binaries and no vcpkg submodule, so it cannot build a shell.
compass shell resolves the binary under this worktree's build/output/<preset>/
publish, and that directory is a symlink to the publish directory the running
services use. Nothing in the main worktree was modified; the symlink is inside
build/, which the tree does not track.
What the runs came to. The spot ladder and trades since last in completed every phase. The triple vol spot ladder met the engine and the engine refused the book it was handed, with 235 errors, which is the failure the ladder family's pages state: the dataset holds no volatility market data. A cadence is not a figure.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
8. PRs
| PR | Title |
|---|---|
| #4456 | [reporting] Set the intraday cadences a desk expects |
9. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
10. Result
The product owner's table is stated in the catalogue, in the seeds, on the pages and in the live scheduler, and the scheduler fires the reports at it.
- The catalogue and the seeds. Four rows moved and every other row stands: the triple and the surface ladder run hourly at every hour, the spot ladder every half hour through the London day and trades since last in every fifteen minutes, and the spike, strike and option maps keep the never-firing placeholder. The two descriptions that claimed a report runs on demand lose that claim, because the seed now schedules it. The fifteen seeded rows across the five datasets state the same thing for every party, and a second run of the generators writes nothing.
- The pages. The four affected pages state the expression their seed states, with the reason for the hourly cadence where the product owner set it; the spot ladder loses the paragraph that argued a definition must not fire by itself and trades since last in loses the on-demand claim in four places. The index gains a Frequency column for all fourteen reports, and the families note reads the triple as hourly and the desk reports as scheduled or on demand rather than all on demand. The business catalogue's Spot Ladder and Trades Since Last In rows read "on-demand · scheduled".
- The live environment. The catalogue was staged and published, which moved twelve rows; eleven definitions were then unscheduled and scheduled again as the tenant's administrator for each party, which wrote eleven jobs and eleven run grants with nothing written by hand. Every job states its definition's expression, and all eleven fired within four seconds of the hour with no person acting. Trades since last in fired again at 15:15 and 15:30, the spot ladders at 15:30, and neither triple, which is the table working.
- The legacy triple. The environment still carried the older name of the same report with a job at the old fifteen-minute expression, and that job fired the report at 15:15. The job was ended, and no triple fired at 15:30. The definition itself stays, and the notes state the seven other jobs the older publish left on three other names, at cadences the table does not contradict.
- The gates. The reporting pgTAP suites pass, 188 tests over 19 files, and the five publish suites pass alone; SQL hygiene is clean over 3342 files; the org lint, the link check and the site page pass. The eight checks that failed did so because this worktree has no codegen venv, and each passes when run with the main worktree's venv, which the evidence names.
- What is not verified. No figure is produced: the triple still meets the engine on inputs the environment does not hold, and that is the accepted failure. The duplicated spot greeks and close triple runs are the older catalogue's and are left for the work that retires it. Nothing was pushed, merged or built in this worktree.
The transcript, the two apply scripts and the scheduling script are beside this page in the live evidence.