Story: Support volatility surfaces

The system holds, builds and serves volatility surfaces as market data, and pricing reads them. The first step states the architecture in the knowledge base, so every later task builds on one agreed design.

The story closes on the tasks it finished:

• Add the daily GJR-GARCH variance to the synthetic FX spot.
• Add the calibrated FX volatility surface series to oresmd.
• Apply an approved variance calibration to a feed.
• Approve a fit within the class policy's rules.
• Build the derived source of FX volatility quotes.
• Capture the inputs of a variance calibration.
• Design the calibration of running feeds.
• Design the synthetic calibration workflow.
• Document the variance calibration workflow.
• Fit a variance calibration from its inputs.
• Document the volatility surface architecture.
• Keep the derived FX volatility quotes for backtesting.
• Make provisioning explain itself, and fix the trade generation stall.
• Model the volatility surface blueprint.
• Model the variance calibration records.
• Read whole numbers as doubles on the wire.
• Review a variance calibration.
• Run the derived source in the system tenant.
• Run the FX spot feeds on GJR-GARCH.
• Scaffold story: Support volatility surfaces.
• Show a feed's run and its variance on one tab.
• Show the calibration chain of a feed.
• Show the variance calibration's history and its review state.

• The surface waits on published IR curves (2026-11-04). Context: a surface's strike labels resolve to strikes through the forward to each expiry, read from the published curves of the pair's two currencies, and no currency has a published curve in the system tenant; only USD has a bootstrap config. Decision: the curves are their own story, Build the IR curves of every supported currency from public data, and the task that checked IR calibration moved there. Consequences: calibrating a surface from its quotes starts after that story publishes the curves of AUD, CAD, CHF, EUR, GBP, JPY, NZD and USD.
• The system tenant holds its own copy of the instrument conventions (2026-11-04). Context: the derived source of FX volatility quotes runs in the system tenant, where the feeds run, and reads the FX option convention of each pair it quotes. The conventions were published only into each party of a tenant, so the system tenant held none, and a party-scoped publish into the system tenant was skipped, since it holds no party but its system party. Decision: system_core publishes ore.conventions, and a publish into the system tenant that names no party writes for its system party. Consequences: the system tenant carries one more copy of the conventions, as it already does of the ORE parameters; the capture One source for reference data that the system and each tenant read covers both copies.
• A bundle publishes in its member order where no dependency decides (2026-11-04). Context: the bundle publish sorted its datasets with a plain topological sort, so two datasets with no edge between them ran in an arbitrary order. ore.conventions ran before iso.currencies in system_core and its currency check failed. Declaring the edge was rejected because dependencies resolve transitively: every party_essentials publish would republish the currencies and write a version of each for every party. Decision: a sort that keeps the given order, so a dataset publishes after its dependencies and otherwise in the bundle's member order, and a dependency takes the place of the first dataset that needs it. Consequences: a bundle's member order is now its publish order, and an ordering that only worked by chance shows up. The first was the Bloomberg feed quoting CNH before any currency existed, so system_core now publishes fpml.non_iso_currency after iso.currencies.
• GJR-GARCH sets the daily variance of FX spot, and not the surface (2026-11-03). Context: the approach document used GARCH only to feed the MDN's surface, the volatility channel listed GARCH as not adopted, and FX spot draws i.i.d. GMM shocks with no memory from day to day. Decision: a GJR-GARCH variance scales the standardised GMM shock, is calibrated from five years of fixings by variance targeting and quasi maximum likelihood, and multiplies with the intraday volatility channel. Consequences: the feed gains volatility clustering across days and a per-pair leverage effect; the surface reads no GARCH state. See "Daily GJR-GARCH variance of synthetic FX spot" in Volatility clustering and GARCH models and "The daily variance" in Market volatility channel.
• The GJR-GARCH variance is a stored, reviewed calibration, not a fit at feed start (2026-11-03). Context: a feed fitted alpha, gamma and beta from its own window of fixings each time it started, kept the result only in that run's start record, and used it with no review, so a user could not see where the values came from, compare them across runs, or refuse a bad fit such as a negative gamma. Decision: the calibration has three stored and versioned stages. A capture stores the inputs, read under the class policy; a fit reads one stored input and stores a proposed calibration; a person approves or rejects it with a reason. A feed names the approved calibration it runs, and its start record names the calibration used. Consequences: the window and the provider order belong to the class policy, not to the feed; the feed's own GARCH columns and the fit at start are removed; an authored shape is a calibration of method authored and is reviewed like any other; history shows every stage from the fixings to the run.
• The variance capture and fit run inside the class calibration (2026-11-03). Context: the asset class already calibrates through one run, calibrate_feeds, which resolves each feed's item under the class policy and either previews or applies what it finds, from the calibration screens. Decision: the same run captures and fits the variance of each GJR-GARCH feed it resolves from a fixing, rather than two operations of their own; a preview reports the fit and stores nothing, and an applied run stores the input, its points and a proposed or rejected calibration, each once under an id named by what identifies it. Consequences: one action calibrates a class's level, spread and variance from the same item under the same policy; a repeat run converges on the records it wrote; the review is the one separate step.
• A feed runs one approved calibration of its pair, and approving moves the pair's feeds (2026-11-03). Context: a feed must run a reviewed shape from its first start, and the approval index admits one current approval per pair. Decision: the publish of the system feeds installs the artefact's authored shape as the pair's approved, authored calibration when the pair has none; approving a fit supersedes the pair's approval and names the new calibration on every GJR-GARCH feed of the pair, each a new version of its config. A feed whose calibration is not approved refuses to start and says why. Consequences: a fresh installation runs GJR-GARCH on reviewed shapes; the feed's own garch_* columns and the fit at start are gone; the review is one operation of the system tenant, under a permission of its own.
• A surface is SSVI and moves through its parameters; the MDN is abandoned (2026-11-03). Context: the MDN is not available, does not cover IR, and makes the surface a function of spot variance only. Decision: the surface is SSVI with \(\gamma = 1/2\); its forward variances follow CIR processes, its skew a tanh of an OU process and its curvature a bounded logistic of an OU process, so every emitted surface is free of static arbitrage; the shocks are correlated with spot. Consequences: the generator prices quotes from the state in the pair's own convention, a tenant's blueprint fits the smile model it chooses and checks it, and a calibrated surface still needs an identity of its own. See the option vol surface section of Synthetic market data generation: approach and SVI Volatility Parameterisation.
• A derived source supplies the starting surface (2026-11-03). Context: no free public source of FX implied volatilities was found. Decision: a publisher of our own backs ATM, RR and BF out of the spot fixings through the Gram-Charlier smile. Consequences: the values are plausible, not real; a real vendor is another provider of the same items and replaces the derived source by a change of route. See Derivation of FX volatility quotes from historical fixings and "How a volatility surface starts" in Synthetic prices are calibrated from free public sources.
• Derived values carry their own identity (2026-11-03). Context: the derived source and the cross rates compute from fixings whose terms, the ECB's among them, allow reuse of unmodified statistics with a credit. Decision: a public publisher's data may feed a computation, and the computation's output is never labelled, credited or described as that publisher's data; it names the derived publisher alone, and the inputs are stored as published. Consequences: the rule sits with the terms of each source in the calibration sources note.
• A tenant receives contributor quotes, not the surface (2026-11-03). Context: a clean generated surface tests none of the marking a desk does. Decision: synthetic contributors inside the simulator adapter publish noisy observations of the latent surface; the clean contributor is the truth. Every quote is a mid; two-way prices are a later extension of the contributor. See A synthetic contributor publishes noisy observations of a latent surface.
• Surface construction follows curve construction (2026-11-03). Context: the curve already goes from quotes through a blueprint, a candidate with lineage and a four-eyes publish to a report. Decision: a surface takes the same path; its method is a calibration where a curve's is a bootstrap, and its output is a calibrated surface series with an identity of its own. See Constructing a volatility surface: from quotes to a published set.
• Five years of public fixings ship as a static DQ snapshot (2026-11-03). Context: every environment backfilled its window from the publishers' API. Decision: DQ holds a five-year snapshot refreshed by hand, and the adapter fills each item's gap from its latest stored fixing to today. Consequences: the load on Frankfurter drops to one small request per item per new environment, and every provider in the snapshot must allow redistribution. The work is Seed the public fixings history from a DQ snapshot.

• Calibration runs as a workflow, after the public rates are loaded (2026-11-03). Context: on a fresh installation the seeded system routes start every synthetic feed the moment its adapter is accepted, at its authored level, while the official adapters backfill the public rates on their own acceptance a few seconds later; nothing orders the two, and calibrating, reviewing and restarting were three manual acts with no owner. In the provisioning run of 2026-11-03 the 26 Bloomberg fixing feeds started from 16:09:57 and the IMF fixings were stored at 16:10:03, so each run kept its authored level and the recorded spots jumped at the first calibrated restart. Decision: synthetic_calibration_workflow runs on the engine in the system tenant, in the order load_public_rates, calibrate_feeds, approve_within_policy, tell_reviewer. Loading the public rates becomes a step of the official service rather than a side effect of acceptance. Provisioning's system run starts the workflow after system_provision, an operator starts it from the Calibration screen, and the scheduler starts it each business day. The workflow's policy is stop: a step that fails or times out ends the run there, so calibrate_feeds never runs on public rates that were not loaded, and a release a profile orders runs only after the workflow has completed. A stopped run leaves its class unreleased until the engine's retry completes it. Consequences: every calibration is a recorded run with steps a person can follow and retry; the tenant run does not wait for it.

  

  Figure 1: How a fresh installation reaches calibrated, running feeds.

• A fit within the class policy's rules is approved by the policy (2026-11-03). Context: a person approving each pair's fit would mean dozens of clicks before a demonstration tenant is usable, and more each day. Measured on the data of 2026-11-03, 13 of the 24 proposed fits pass the rules below, and 11 fail on disagreeing starts, a persistence out of range, or no reaction to moves. Decision: each asset class's calibration policy holds an approval mode, manual or auto, and its own rules. The FX policy is seeded in auto mode with these values: at least 250 returns, starts that agree, persistence from 0.80 to 0.995, and an average reaction, alpha plus half of gamma, of at least 0.01. The reaction is a threshold and not merely above zero, because a fit can reach alpha and gamma of about 1e-11, a variance that does not react to moves (USD/CHF on 2026-11-03). In auto mode a passing fit is approved through the review verb with the policy, its id and its version as the decider; a failing fit stays proposed with the failed rule as its reason, and its pair keeps the shape it runs. Consequences: a fresh installation needs no click to run fitted shapes where the data supports them; a person sees only the exceptions; the history names who decided each approval, a person or a policy version; a change of a rule is a new policy version.
• One review verb serves a person and the policy, and there is no apply workflow (2026-11-03). Context: the project's proposal pattern starts an apply workflow from a person's approval, because the work after the decision spans services. Decision: approving a variance calibration stays one verb of the synthetic service, which supersedes the pair's earlier approval and older proposals, approves, and re-points the pair's feeds; a repeat of the verb completes an approval that stopped part way. A person and the approve_within_policy step call the same verb. Consequences: one path and one history for every approval; an approval restarts no feed, which takes its new calibration at its next market day roll; a workflow is added only if an approval comes to need another service.
• The synthetic feeds start only when their class is released (2026-11-03). Context: a feed that starts before calibration records spots at its authored level, and the first calibrated restart puts a jump in the series that every later pricing reads. Decision: each asset class carries a release state; the synthetic adapters of a class that is not released do not announce, so no demand reaches them. Releasing is an explicit, recorded act: the release step a demonstration profile such as Acme orders, or an operator's Start feeds. Consequences: every fixing-sourced feed starts once, from its calibrated level; on a fresh production installation every synthetic feed is silent until an operator presses Start feeds after calibration, and the Calibration screen shows each class's release state, so the silence is explained.
• A running feed is continuous, and takes a new calibration at its market day roll (2026-11-03). Context: a feed has no resume, so any restart, deploy or crash runs the start resolution again and begins at the newest fixing: the AUD/USD restarts of 2026-11-03 at 15:22, 15:45 and 16:04 each started at 0.702297 from IMF, and none read the price its run had reached. At a market speed of 1440 a market day passes each wall minute, so the path can move far between restarts, and each restart can put a jump in the recorded spots. A daily recalibration that needed a restart would add one more. Decision: a feed of a released class that has a stored spot resumes from it. The feed config's level is the process's stored initial state, and a start always reads it: calibration writes it from the fixing, and a restart first writes it from the feed's last stored mid, read from the series the feed publishes into, as a new version under system.restart whose commentary names the spot's price and time, so the history shows every restart and the price it began at. A restart uses a spot only if it was stored after the config's current version, so a re-anchor written while the service was down still wins; a spot that cannot be read stops the start rather than falling back to the level. A feed with no stored spot keeps the level its calibration wrote. A running feed takes a newly approved calibration, and a recalibrated spread, at its next market day roll, where the GJR-GARCH process already updates its variance and its state is only the variance, the day's open price and the tick count. Consequences: the stored series has no jump from a restart or a recalibration, and each restart is an entry in the feed's history naming the price it began at, with no record type of its own; the GARCH variance may restart at the first-day ratio, a step in volatility and not in price; the feed records which calibration each market day ran under; the daily workflow restarts no feed.
• A running feed's level follows its own path, and only an operator re-anchors it (2026-11-03). Context: with continuity, a feed's level runs freely from its release and drifts from the real rates; the daily calibration still writes a configured level, which a running feed no longer reads. Three options were weighed: a free-running level re-anchored by a person; a slow mean reversion toward the latest fixing; and a restart from the fixings each day. The synthetic market runs on its own clock, a day per wall minute, so it cannot follow a rate that moves once a real day without either a daily jump or a pull no real market has. Decision: the level is free-running. Re-anchoring a class to the public rates is an explicit operator act on the Calibration screen: it restarts the class's feeds from their calibrated levels and records who, when, why and each level before and after. The level of a released class has three writers: a restart, from the last stored spot; a re-anchor, from the fixing; and nothing else, so the daily calibration of a released class writes its spread and variance and leaves its level alone. Consequences: the only jumps in a stored series are a release and a re-anchor, each chosen and recorded; no restart, recalibration or approval moves a running level.
• Closed on 2026-11-08 at sprint 29 close with its done tasks. The open tasks and the original acceptance moved to Support volatility surfaces — the remaining work.

• The fixings history. Years of daily fixings in every environment are Seed the public fixings history from a DQ snapshot, a story of its own: this story works from the adapter's shorter window without it, with the priors of the derived quotes dominating and weak butterflies.
• Two-way pricing. Every quote, synthetic or real, is a mid, and valuation runs on mids. Bid and ask, for every input and not only vol surfaces, are a later level of work, taken up once mid valuation is reliable for every product.
• The rest of the scope is set by the architecture task.