Story: Add new stochastic processes

Table of Contents

This page documents a story in Sprint 25. It captures the goal, current status, acceptance criteria, and the tasks that compose it.

1. Goal

The system exposes a wider catalogue of stochastic process engines, wired end-to-end (factory, seed data, config storage, Qt UI): the six unwired IR engines become selectable, further IR models (two-factor Hull-White, shifted log-normal, normal/Bachelier) are added, and an FX jump/Poisson process engine joins the synthetic FX spot price-process family. Every engine is validated against QuantLib reference tests.

2. Status

Field Value
State ABANDONED
Parent sprint Sprint 25
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-09-23

3. Acceptance

  • All moved engine tasks reach DONE with their processes selectable in the system, not just implemented in the quant layer.
  • Every new process is validated against QuantLib reference vectors from /home/marco/Development/ORE/Engine/QuantLib.
  • Where QuantLib 1.42.1 carries no class for a model (BDT, HJM, QG), the task names the historical QuantLib reference and relies on the engine's own cross-validation (lattice vs Monte Carlo; affine/QG recursion identity at gamma == 0).
  • SABR and vol-surface models are out of scope.

4. Tasks

Task State Start End Effort Description
Scaffold story: Add new stochastic processes DONE 2026-08-11 2026-08-11 S Story scaffolding rides this task: documents, sprint wiring, and the scaffold PR. Close it before merging that PR.
Analysis: survey the stochastic process landscape and break the story into tasks DONE 2026-08-11 2026-08-12 M Survey the stochastic process landscape (six unwired IR engines, G2 wiring precedent, new IR models, FX jump/Poisson capture, QuantLib reference) and produce the story's task breakdown.
Implement Black-Karasinski log-normal short-rate process DONE 2026-08-12 2026-08-12 S Engine merged via PR #1932; wiring landed (mapping dispatch, seed rows, kEngines/tooltip). Four scalar parameters fit the EAV machinery as-is. Reference: QuantLib BlackKarasinski (onefactormodels/blackkarasinski.hpp; no dedicated test in this version's suite).
Extend process parameter storage to vector-valued parameters ABANDONED   2026-09-23 M Extend the parameter definition/value model to vector-valued parameters (serialized storage, parsing, validation), unblocking BDT, HJM, and LMM. No QuantLib reference: shared storage infrastructure.
Implement Black-Derman-Toy binomial tree short-rate process DONE   2026-09-23 M Engine merged via PR #1932; remaining: system wiring following the G2 precedent. Needs the vector-parameter groundwork: sigma_path is time-varying and the discount curve is a model input derived from the config's curve. Reference: no QuantLib class in this checkout (BlackDermanToy removed from QuantLib); historical reference only.
Implement multi-factor Affine Term Structure process framework DONE   2026-09-23 M Engine merged via PR #1932; remaining: system wiring following the G2 precedent. Eigen matrix/vector parameters via named-scalar expansion (fixed two-factor UI scope), rebuilt in the mapping layer. Reference: QuantLib G2 (twofactormodels/g2.hpp) and bermudanswaption.cpp testCachedG2Values as the two-factor special case.
Implement Quadratic Gaussian term-structure process DONE   2026-09-23 M Engine merged via PR #1932; remaining: system wiring following the G2 precedent. Affine's surface plus the gamma matrix; the most original engine. Reference: no QuantLib class in this checkout (QuadraticGaussianModel removed); Ahn-Dittmar-Gallant papers.
Implement Heath-Jarrow-Morton forward-curve process DONE   2026-09-23 M Engine merged via PR #1932; remaining: system wiring following the G2 precedent. Needs the vector-parameter groundwork; the forward grid derives from the config's curve. Reference: no QuantLib class in this checkout (HjmModel removed); historical ql/termstructures/forwardratefamily reference.
Implement LIBOR Market Model forward-rate process DONE   2026-09-23 L Engine merged via PR #1932; remaining: system wiring following the G2 precedent. Largest parameter surface (rates, volatilities, correlation matrix, displacements, spacings); needs the vector-parameter groundwork. Reference: QuantLib liborforwardmodel.hpp/lfmprocess.hpp and libormarketmodel.cpp (testCapletPricing, testSwaptionPricing, testCalibration).
Implement jump-diffusion (Merton) FX spot process engine ABANDONED   2026-09-23 S Merton 1976 jump-diffusion engine in ores.analytics.quant: GBM plus compound Poisson jumps with log-normal sizes. Reference: QuantLib Merton76Process and test-suite/jumpdiffusion.cpp (testMerton76, testGreeks).
Wire jump-diffusion into the system: factory, config storage, Qt UI ABANDONED   2026-09-23 M Factory branch, config storage, FxSpotRateEditor surface (replaces the Component Tools placeholder), shared validation. Reference: via the engine task.

5. Decisions

  • The six unwired IR engine tasks moved here from the IR curve follow-ups story on 2026-08-11. Their engines are merged (PR #1932, 2026-08-10); the system wiring is the remaining scope, following the G2 wiring precedent (Wire G2 task, DONE).
  • Scope is FX and IR only. SABR and any vol-surface model are out of scope.
  • Candidate additional IR models (from tmp/tmp.org): two-factor Hull-White (reuses the affine/G2 engine), shifted log-normal / displaced diffusion, normal (Bachelier). The FX jump/Poisson process comes from capture 84635A47 (deferred backlog).
  • QuantLib reference for validation: /home/marco/Development/ORE/Engine/QuantLib — its test-suite provides reference vectors for most target models; BDT, HJM, and QG have no class in this checkout (see the inventory finding below).
  • Analysis findings (2026-08-12): the new IR models resolve to existing engines — two-factor Hull-White is exactly G2++ (already wired as two_factor_gaussian_process; QuantLib anchor G2 + bermudanswaption.cpp testCachedG2Values); normal/Bachelier is the arithmetic engine (FX ABM) and Vasicek (normal short rate), both wired; shifted log-normal is already parameterised inside LMM (displacements). No new-engine tasks for the IR models.
  • QuantLib 1.42.1 inventory: Black-Karasinski and LMM classes and tests present; G2 present (affine anchor); Merton76Process + jumpdiffusion.cpp present (FX jump anchor). BDT, HJM, and QG classes are absent from this checkout (removed from QuantLib): their tasks name historical references and rely on the engines' own cross-validation (rate_tree lattice vs Monte Carlo; affine/QG recursion identity at gamma == 0).
  • EAV representation: the scalar value table covers Black-Karasinski as-is, and affine/QG via named-scalar expansion (fixed two-factor UI scope, matrices rebuilt in the mapping layer). Curve-conformant vectors (BDT sigma path, HJM/LMM per-tenor grids) need the vector-parameter extension — the groundwork task.
  • FX jump splits into an engine task (quant layer, Merton 1976 port) and a wiring task (factory, config, editor). The capture 84635A47 file paths are stale: IStochasticProcess and process_factory now live in ores.analytics.quant, and the FX flow runs FxSpotRateEditor → fx_spot_generation_config → make_fx_spot_feed → process_factory::make_process().
  • Effort scale in the Tasks table: S ≤ 1 dev-day, M 2–3 dev-days, L 4–5 dev-days. (No prior convention; the scale is recorded here.)
  • The parameter form generalises beyond the G2 one-off: IrCurveEditor's table is rebuilt from the definitions table on engine change, so new engines need no dialog work — only a kEngines row and tooltip entry in each editor's hardcoded engine table, plus seed rows on the IR side.
  • Black-Karasinski wiring (2026-08-12) confirmed the generalisation end-to-end: three commits (mapping dispatch, seed rows, kEngines/tooltip), no dialog work, no schema change, no migration, no flat-factory change — make_yield_curve_process stays flat-arg for its own tests only, and the "factory constructs a working engine" acceptance is satisfied by mapping-layer direct construction (the G2 precedent). Seed parameterization: theta is the long-run level of ln r, so its default is log(0.04) = -3.218876 (honest log-level default, description explains the correspondence); kappa is unbounded below (kappa <= 0 is a valid, degenerate driftless branch, unlike Vasicek's min 0); sigma min 0; initial_rate min 1e-6 — the catalogue expresses the engine's strict initial_rate > 0 bound as a small positive epsilon (review round 1), so the mapping layer rejects 0 with its consistent error instead of the engine constructor.

6. Out of scope

  • SABR and any volatility-surface model: FX and IR only.

7. Result

Abandoned at sprint 25 close. The IR work needs a refactor at the implementation level before more features land, and sprint 26 does not reach it. The capture Refactor the IR rates implementation before more IR features land holds the remaining work.

Emacs 29.3 (Org mode 9.6.15)