Market Data Requirements and Resolution

Summary
The pipeline
Documents in this cluster
- The core pipeline
- Refinements on the resolved set
See also

Summary

Between a trader wanting to know the value of a book of trades and a pricing engine actually computing that value lies a pipeline this cluster exists to describe precisely. A trade population generates logical requirements for market data — symbolic descriptions of what is needed, independent of any physical source. Those requirements are resolved, under a named configuration, into physical identifiers — fully specified references that a database query or messaging system can act on directly. Each identifier's own further dependencies are expanded via dependency resolution; the resulting set is intersected against the market data universe — everything that actually exists and is reachable — to produce the concrete, valuation-scoped set a computation genuinely needs. That set can be filtered, in three genuinely distinct senses, and its resolved values snapshotted for reproducibility. Each of the seven concepts has its own document; this hub supplies the connective tissue and the reading order.

The pipeline

Requirements and identifiers are two views of the same entity: a requirement names what is wanted in logical vocabulary any pricing system would recognise; an identifier is that same description with its physical fields — source, calendar, day-count conventions, ticker-construction hints — filled in, so that a one-directional vendor projection can render it into Bloomberg's, Reuters', or ORE's own notation. DNS readers will recognise the split immediately: a requirement is a domain name, an identifier is the IP address it resolves to, and a configuration is the resolver whose rules determine which address is currently authoritative. The analogy is imperfect at one point — DNS has no counterpart to re-expressing an IP address in a second registrar's own notation, the way our identifier must be expressible as a Bloomberg ticker or an ORE CurveSpec — but it is otherwise close enough to serve as a running orientation throughout the cluster.

Requirements are not all generated at the same time or with the same information. A trade's type determines the structural shape of what it needs before any specific trade is booked. A specific trade instance narrows that shape by binding currencies, indices, and credit names to its own attributes. A specific valuation date narrows it further by localising every curve-level need to the exact tenor points the trade's cashflows actually touch on that date. These three levels — static/instance/dynamic — are covered in full in Market Data Requirement.

The central relationship the whole cluster builds toward is:

\[\mathrm{MDS}(T) \;=\; \mathrm{closure}\bigl(\rho\bigl(\mathrm{static}(\tau(T)) \cup \mathrm{dynamic}(T,d)\bigr)\bigr) \cap U\]

Reading left to right:

\(\mathrm{static}(\tau(T)) \cup \mathrm{dynamic}(T, d)\) — the union of every trade type's static requirement set with every individual trade's dynamic refinement, at valuation date \(d\). This is a set of logical requirements, not yet a set of identifiers.
\(\rho(\cdot)\) — resolution, mapping each requirement to exactly one physical identifier. The result is a set of candidate identifiers — not yet confirmed members of \(U\), and not yet complete: each identifier's own dependencies have not been expanded.
\(\mathrm{closure}(\cdot)\) — dependency closure, expanding the set from step 2 to include everything it transitively depends on.
\(\cap\, U\) — intersection with the market data universe, turning "everything the trades claim to need, resolved and expanded" into "everything they need that actually exists." Anything produced by steps 1–3 that is not a member of \(U\) is excluded — which is exactly why this must not be a silent step: a production system needs to surface the difference \(\mathrm{closure}(\rho(\ldots)) \setminus U\) (everything requested but missing) as an explicit error, rather than letting the intersection quietly discard it and proceed to value the trade against an incomplete data set.

Worked example: a EUR/USD FX Forward

Suppose \(T\) is a single EUR/USD FX Forward. Its static requirement set (fixed by the FX Forward product type) is "the EUR discount curve" and "the USD discount curve." Its dynamic refinement, given this trade's actual 87-day maturity and today's date, identifies the two pillar points that bracket that date on each curve — the 2M and 3M tenor points — since 87 days falls between those pillars and interpolation between them is needed to obtain the discount factor at the exact maturity date. Both pillar points are therefore required for each currency: four point-level requirements in total. Resolution (\(\rho\)) maps each to one physical identifier in oresmd notation:

oresmd://ir/eur?index=estr&tenor=1d&role=discount&type=quote&metric=discount_factor&point=2m
oresmd://ir/eur?index=estr&tenor=1d&role=discount&type=quote&metric=discount_factor&point=3m
oresmd://ir/usd?index=sofr&tenor=1d&role=discount&type=quote&metric=discount_factor&point=2m
oresmd://ir/usd?index=sofr&tenor=1d&role=discount&type=quote&metric=discount_factor&point=3m

Dependency closure then expands each of the four identifiers to include whatever instrument quotes were used to bootstrap those pillar points in the first place. Finally, intersecting with \(U\) confirms that every expanded identifier genuinely has data behind it; if any pillar point depends on an instrument quote that was never loaded into \(U\), that gap surfaces here as a member of \(\mathrm{closure}(\rho(\ldots))\) that intersection with \(U\) excludes.

In the DNS analogy: a browser loading one page resolves the page's domain name to an IP address, then discovers the page has <img> tags that name further domain names — the analogue of dependency closure. Each of those is resolved in turn, and the browser only successfully renders each image if that address is actually reachable — the network-level analogue of intersecting with \(U\) and finding the result present.

Documents in this cluster

Read in pipeline order for the first pass; jump to any document for a specific concept.

The core pipeline

Market Data Requirement — start here. Defines what a requirement is, what makes two requirements the same, and the static/instance/dynamic three-level structure by which requirement sets are generated. Leads with a purchase-specification (requisition) analogy as the primary layperson model; DNS and CIDR appear only as named aside boxes.
Market Data Identifier — read second. The same entity as a requirement, fully specified: logical fields plus physical fields. Covers the two-field-group design, the projection \(\pi\) from identifiers to requirements (many-to-one, no decision), and the one-directional vendor projection \(T_{\text{vendor}}\) that renders an identifier into Bloomberg's, Reuters', or ORE's own notation. Includes a state-of-the-art survey drawing on External market data identifiers for the convergence evidence that motivates the logical field structure.
Market Data Configuration — \(\rho\), the resolution function. Explains why the same requirement can resolve to different identifiers under different configurations, and why "configuration" — not "profile" or "policy" — is the term two independent reference systems (ORE, OpenGamma Strata) converge on.
Market Data Universe — \(U\). The global set every identifier is drawn from. Carries the argument for why set notation — rather than "environment" or similar terms — is the right vocabulary for this concept.
Market Data Dependency Resolution — \(\mathrm{closure}\). The transitive expansion of a resolved identifier set to include everything those identifiers depend on, grounded in ORE's DependencyGraph and OpenGamma Strata's MarketDataNode.

Refinements on the resolved set

Market Data Filtering — three genuinely distinct senses of "filtering" sharing one word: narrowing \(\mathrm{MDS}(T)\) by configured pricing engines, aggregating risk output by dimension, and ranking competing sources for a single identifier during IPV curation. Each operates on a different kind of object; they are kept deliberately separate rather than unified under one mechanism.
Market Data Snapshot — materialising \(\mathrm{MDS}(T)\)'s actual values at a point in time, fixed or floating, for reproducibility. The one concept in this cluster with no confirmed prior art in either ORE or OpenGamma Strata, documented candidly as this repository's own.

A note on research method

Every claim of prior art in this cluster — every citation of an ORE class or an OpenGamma Strata interface — comes from direct inspection of those two systems' actual source code, not from memory or inference. Terminology decisions recorded here (replacing "environment" with "market data universe," "profile" with "configuration") were made only once independently confirmed against production open-source systems. Where no confirmation could be found — most notably for snapshotting, and for two of the three senses of filtering — each affected document says so plainly rather than presenting an educated guess as settled prior art.