A quote series is keyed by item, provider and party
Table of Contents
- 1. Summary
- 2. Detail
- 2.1. What each one holds
- 2.2. A tick is not an observation
- 2.3. The tick store is shared
- 2.4. Why the provider is in the key
- 2.5. Why the source is not in the key
- 2.6. The soft update rule
- 2.7. Why observations carry no audit columns
- 2.8. Why valid_from is projected onto the wire
- 2.9. The derivation marker on the series
- 3. See also
1. Summary
Two tables hold the stored result of the pipeline. A market_series catalogues
a quote series: one internal item from one provider for one party. A
market_observation records one value of that series that a person or a process
states. The series key is (item, provider, party). It holds no source, so a party
can change its source and keep one series. A live tick is not an observation. It lives
in the shared tick table, on a stream that every party of the provider reads, and its
own read returns it after checking the party's subscription.
2. Detail
2.1. What each one holds
market_series holds oresmd_uri, provider, party_id, series_subclass,
and the asset classes in market_series_asset_classes. The key is unique. The
table keeps no decomposed copy of the key, so a reader that wants a part of it
reads it through the codec.
market_observation holds series_id, observation_datetime, oresmd_uri,
value and source_id. The observation_datetime is the vendor time. The
source_id records the source that produced the value and is information only.
The URI is the row's only identity. The ORE key is not stored, because
ore_key_codec writes it from the datum.
2.2. A tick is not an observation
The ingest writes each tick once, as a point of its source's stream, and writes no observation row. A party's series points at the stream. See A source tick is stored once and entitled when it is read for the store and The tick ingest pipeline: freshness first for the lanes that write it. The observation table holds the points that people and processes state: an ORE import, a hand-keyed point and a derived point. The list of observations returns observations and never a tick. A client that wants ticks calls the history read. See The history call returns every tick of a range in one reply, or refuses the range.
2.3. The tick store is shared
A tick is an observation about a source, not a row of a party. The store holds one row for each tick of a source, on a stream named by an id derived from the provider code and the series URI, and a party's series names the same stream. The row carries the stream, the source, the time, the coordinate and the value, and no tenant or party. So the store does not grow with the number of parties, and two parties of one provider read the same row.
The history read is therefore the only door to the ticks. It resolves the party's series, checks that the party holds a subscription to the provider with display or non-display, and refuses a party with none or with the derived right alone. See One shared tick or one copy for each party. The coordinate dictionary is shared for the same reason: every party of a stream must agree on which number means which datum.
2.4. Why the provider is in the key
Two providers quote the same item with different values. A party that subscribes to both holds two series, one for each, and a composite can then blend them. See A composite attributes its use to the providers of its inputs.
2.5. Why the source is not in the key
The source is a route, and routes change. A key that held it would split one price history in two on each switch. See A source switch changes one route and nothing else.
2.6. The soft update rule
market_observations_tbl is a TimescaleDB hypertable partitioned by
observation_datetime with 30 day chunks. A correction replaces the previous
value through the soft update trigger. GIST exclusion and DELETE rules do not
work on hypertables, so a partial unique index and the soft update and soft
delete trigger pair enforce uniqueness.
2.7. Why observations carry no audit columns
The observation type has no version, modified_by, performed_by,
change_reason_code or change_commentary. Tick-level volume makes them
impractical. valid_from and valid_to carry the transaction time.
2.8. Why valid_from is projected onto the wire
Codegen drops the audit tail for a :no_audit_columns: true entity. The
database column exists and every as-of read selects it. read_as_of_records
returns each point with the valid_from of its row. The curve_snapshot.get
reply carries the record times as a parallel array, so a consumer can report how
stale a term structure is. Two ages are readable. The read's instant minus a
point's observation_datetime is the age of the market state. Now minus
valid_from is how long ago the value arrived.
2.9. The derivation marker on the series
market_series.derivation_kind defaults to OBSERVED. A derived series carries a
named mechanism such as IR_CURVE_BOOTSTRAP, with derivation_config_id and
derivation_config_version. This is a catalogue marker, and it differs from the
per-observation lineage row in Lineage and provenance.
3. See also
- One identity from fixing to report: provenance and publication of market data — the structure note that orders this cluster, and where to read this page in it.
- Market Data Architecture — the structure note that orders this cluster.
- Lineage and provenance — completes: the extra row of a derived observation.
- The oresmd datum identity — completes: where the stored URIs come from.
- The ingest loop and the one pipeline — completes: the writer of the tick table.
- One shared tick or one copy for each party — answers: why the tick store holds no tenant and no party, and why the read checks the subscription.
- The history call returns every tick of a range in one reply, or refuses the range — completes: the one read of ticks.
- The mapping belongs to the provider and not to the source — completes: the step that gives the item.
- A source switch changes one route and nothing else — answers: why a switch does not split a series.