Market data ticks because ownership, demand and a smoke check agree
Table of Contents
1. Summary
On 2026-10-23 a fresh build, a new database and a provisioned Acme tenant produced no tick, although the database held the routes and the subscriptions. The marketdata service reads as the system tenant, and the policies of the two tables showed it only the system tenant's rows. The service stayed up and logged "accepted 0 of 0 ids", so nothing looked wrong. This page lists the seven shortcomings that the failure showed and the order of the work that removes them. The first fix is a select-only policy for the service role. The decision about where ticks are stored is the one open question.
2. Detail
2.1. What happened
- The supervisor computes the ids that each adapter must publish. It reads every party's routes and subscriptions.
- The ingest loop reads the same routes, to know which parties store a tick.
- Both read through
read_latest_all_tenants. The policies ofroutesandsubscriptionsadmitted the own tenant only. The service session is the system tenant, so both reads returned no row. - An end to end test passed before the merge, because its seed wrote the routes under the system tenant.
- A second defect hid behind the first. A restart with stored ticks made 6 of 26 generators fail, because the vintage read took the first series of an identity and the tick series of another provider shared that identity. Pull request 3690 fixed it.
2.2. The shortcomings
| # | Shortcoming | Evidence | Fix |
|---|---|---|---|
| 1 | A platform process needs the rows of every tenant, and the policy gave it a wider reach than it needs | Pull request 3693 added an "or the session is the system tenant" clause, for all commands, to two tables. The widening guard lists both as temporary. | Read routes through a role-scoped policy |
| 2 | The failure was silent | The service was up. The log said "accepted 0 of 0 ids". No view showed routes seen against ids demanded. | Make the demand self-correcting and visible |
| 3 | The demand depends on change events | A refresh runs on a route event or a subscription event only. A missed event leaves the snapshot old until a restart. During one run no event reached the service, and the cause is not known. | Make the demand self-correcting and visible |
| 4 | No test runs the real path | Tests put their rows under the system tenant. No check provisions a tenant and then waits for ticks. | Add a provisioning to ticks smoke check and Test the market data service in a non-system tenant |
| 5 | The simulator builds its adapters once | A source added after the start needs a restart. | Refresh the simulator adapters when the catalogue changes |
| 6 | One copy of each tick for each party | The ingest stores a tick for every party that routes to the source: 26 pairs for 4 parties gave 104 series. The copy ties the ingest to the routes of every tenant. | Decide whether ticks are stored once for each source |
| 7 | An item has no single series | A vintage and a tick stream of one item are two series, one for each provider, and a read by series must guess. | The owner is the public sources work, not a task of this plan: the reader of the fixings reads by item. See Public sources publish daily fixings through an adapter. |
2.3. The target
The page Tenant ownership and the information flow names three owners: a tenant, the system tenant and the installation. The target puts each part of the market data flow with one owner.
- The catalogue (providers, sources, vendor instruments, mappings) belongs to the system tenant and every tenant reads it. This part is right now.
- Routes and subscriptions belong to a party in a tenant. The one platform reader is the marketdata service. It reads them through a select-only policy for its own database role. The CRM configs already use this shape, in Data read across tenants. No other system tenant session gains a read or a write of another tenant's routes.
- Demand is a pure function of the catalogue, the routes and the subscriptions. The supervisor computes it on a timer and on events. A missed event delays demand by one interval. It never loses demand.
- Ticks are stored once for each source stream, owned by the installation with no tenant, and a party's entitlement is checked when it reads. See A source tick is stored once and entitled when it is read.
- Every claim of health is observable. One status shows, for each adapter, the routes seen, the ids demanded and accepted, the ticks of the last minute and the age of the last tick.
- One check proves the path. It provisions a non-system tenant on a fresh database and fails unless ticks arrive.
2.4. The order of the work
- First, because it unblocks the work that comes after it, Read routes through a role-scoped policy. It replaces the widening of pull request 3693 with a narrower policy and removes two exceptions from the guard.
- Second, Add a provisioning to ticks smoke check and Test the market data service in a non-system tenant. They are the safety net for every later step. The smoke check runs by hand and in the local gate when the marketdata, synthetic or provisioning code changes, because it needs a database and every service. The tenant test is an ordinary suite that runs in every ctest run. Neither replaces the other.
- Third, Make the demand self-correcting and visible.
- In parallel, Refresh the simulator adapters when the catalogue changes.
- Then Decide whether ticks are stored once for each source. It needs measured numbers from a provisioned tenant, and it can change the later tasks.
- The tasks Load published quotes from the public sources and Calibrate the synthetic ticks from free public sources follow, and they remove the vintage read of the generators.
3. See also
- The market data path: a public fixing, a curve, a report — the structure note that orders this cluster, and where to read this page in it.
- Market Data Architecture — the structure note that orders this cluster.
- Tenant ownership and the information flow — the three owners and the four mechanisms.
- Data read across tenants — the building blocks of a read across tenants.
- Plan the market data tenancy and the ticking guarantee — the task that wrote this page.