Task: Replace the asset-specific ticks with one market tick
Table of Contents
This page documents a task in the Rewrite the oresmd grammar as a hand-written codec checked against ORE story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
Every live market data feed, whatever its asset class, publishes and ingests one tick type keyed by the oresmd quote URI, and every feed takes its owner from one source.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Rewrite the oresmd grammar as a hand-written codec checked against ORE |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-04 |
3. Acceptance
- fx_spot_tick and ir_curve_tick are deleted; one modelled market_tick carries the oresmd quote URI, the value, the observation time and the source.
- Every feed takes tenant, workspace and party from its feed binding; no tick carries an owner.
- The ingest loop reads the URI with oresmd_uri_codec and classifies the datum with classification_key_of; it no longer rebuilds an ORE key from parts or reads a classification off the wire.
- P02: marketdata.v1.tick is declared once, by the market_tick model, and every use of it, the republish subject builder included, references the generated constant.
- The republished subject is
marketdata.v1.tick.<tenant>.<workspace>.<party>.<key>, where<key>is the canonical ORE key of the tick's datum, lower-cased with its slashes turned to dots, written by one builder that both the ingest loop and the client use. - A tick finds its consumers by
sourcethrough the enabled feed bindings of that source, for every feed; a tick from a source with no enabled binding is dropped with one warning per source. market_tick.valueis decimal text, as stored observations are.- Tenant provisioning creates bindings for the IR curve generation configs as it does for the FX spot ones.
- The synthetic FX and IR curve feeds publish market_tick, and a live feed run stores observations for both.
4. Plan
The wire changes producers and consumers together, so this is one pull request of commits that each build:
- The tick. Model
market_tickinores.marketdata.operations, with:subject: marketdata.v1.tick, and write the one republished-subject builder in the api against its generated constant. - The binding. A binding maps a source to the tenant, workspace and party
that consume it. Its
oresmd_uriandasset_classgo: the tick names its own datum, and a curve source publishes many datums across several series. Its natural key becomes the party and the source. - The ingest loop. One path for every tick: find the source's enabled
bindings, read the URI with
oresmd_uri_codec, classify withclassification_key_of, store the observation, republish under the builder's subject. - The producers. The synthetic FX and IR curve feeds publish
market_tick. Tenant provisioning creates a binding for every feed, FX and IR alike. - The deletions.
fx_spot_tick,ir_curve_tick, their JSON I/O, the two copies of the subject function, and the FX-only client subscription, which becomes a subscription tomarket_tick. - Proof. The fleet runs both synthetic feeds, and observations from both arrive under their datum URIs.
5. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit |
|---|---|---|---|
| Change size | 55 files, 549 insertions, 1,495 deletions | git show --stat |
c87a9789b9 |
| Test suites, recreated database | 90 of 90 passed | compass test run |
c87a9789b9 |
| Bindings after ACME provisioning | 34 sources (13 FX, 21 IR curve), 5 parties each | compass sql, current feed bindings by source |
c87a9789b9 |
| Sources ticking within five minutes of the feed start | 13 FX spot, 20 IR curve | compass sql, observations by instrument and source |
c87a9789b9 |
| New series, classified from the datum | 65 FX spot (fx, spot), 300 IR swap (interest_rates, yield) |
compass sql, series by instrument, subclass and asset class |
c87a9789b9 |
6. Notes
6.1. The live run
The fleet ran on a recreated database with the ACME demonstration profile,
whose last step starts the market feeds. Every FX source and 20 of the 21 curve
sources ticked, and both kinds were stored through the one path. The remaining
curve, usdsofrfomc, published nothing: its first pillar ended on the spot
date, and make_pillar_quote_key refused it, failing the whole batch. The
cause is older than this change: tenant provisioning drops the FOMC schedule,
so every meeting tenor resolves to spot. Provision the FOMC meeting schedule into every tenant
fixes it.
6.2. Deleted rather than ported
fx_spot_subscription and the client's subject_helpers.hpp had no production
consumer, so they went instead of being moved to market_tick.
The model and its reasons are in the market tick investigation. One point it leaves to this task:
- The feed binding stores an
asset_classthat itsoresmd_urialready implies. Derive it from the URI or drop it.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
8. PRs
9. Review
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | PR 2602: the republish line references look stale. | investigation | Accepted | They were correct at 2a000020ea (the two js_publish calls); the document now names the calls and functions instead of lines. |
| 2 | PR 2602: the tick design has no rule for the republished subject's suffix. | investigation, task | Accepted | The suffix is the datum's canonical ORE key, lower-cased with dots, written by one builder; an acceptance line says so. |
| 3 | PR 2602: how a binding is found, and what happens with none. | investigation, task | Accepted | By source, as FX does today, for every feed; no enabled binding drops the tick with one warning per source. |
| 4 | PR 2602: settle the value type before the model is written. | investigation, task | Accepted | Decimal text, as stored observations are. |
| 5 | PR 2602: the task's Status still shows template text, and #+pr names the investigation PR. |
task | Not changed | The task is in BACKLOG until its work starts; compass pr create adds the implementation PR as another row. |
| 6 | PR 2602: no step creates bindings for the IR feeds; is source unique? |
investigation, task | Accepted | Provisioning creates them as it does for FX; a source names a producer uniquely within a tenant. |
| 7 | PR 2607: classifier_for returns a reference into a map that refresh() clears on another thread. |
feed_ingest_loop |
Accepted | The map holds shared classifiers, and the caller keeps its own reference while it classifies. |
| 8 | PR 2607: the set of unnameable URIs is never cleared. | feed_ingest_loop |
Accepted | refresh() clears it, as it clears the unbound sources. |
| 9 | PR 2607: one series read per tick and binding. | feed_ingest_loop |
Not changed | The old paths read the series the same way, so it is not a regression. A series cache is a measured change for later. |
| 10 | PR 2607: the stats count a tick before it is stored. | feed_ingest_loop |
Accepted | A tick counts after its observation is stored. |
| 11 | PR 2607: feed_bindings changes with no migration. |
SQL | Not changed | No deployed database exists; the database is recreated, and the design decision allows no backwards compatibility. |
| 12 | PR 2607: the value is stored without a check. | feed_ingest_loop |
Accepted | A tick whose value is not a number is dropped, with one warning per source. |
| 13 | PR 2607: no tests for the new on_tick behaviour. |
tests | Not changed | on_tick needs NATS and a database; the test task, Close the marketdata test coverage, owns it and its notes say so. |
10. Result
A live tick is one type for every feed and asset class: market_tick, the
oresmd quote URI, the value as decimal text, the observation time and the
source. Every feed takes its owners from its feed binding, which now maps a
source to a party and nothing else, and the ingest loop has one path that
classifies a new series from its datum, as the file import does. P02 holds for
marketdata: marketdata.v1.tick is declared once, by the model, and the
republished subject and both stream wildcards are built from it. On a recreated
database, 90 of 90 suites passed, and a live ACME run stored FX and IR curve
ticks from 33 of 34 sources; the one that published nothing fails on a
pre-existing pillar defect, recorded as a capture.