Task: Fix day-scaled kappa/sigma calibration across short-rate processes
Table of Contents
This page documents a task in the IR Rates synthetic data: dataset seeding, index cleanup, dual-curve, quoting conventions story. It captures the goal, current status, acceptance, and any notes or results.
Goal
Make the "day-scaled default kappa/sigma" claim from the
tick-batch-publishing task actually true — for every short-rate
process, not just the one that first surfaced the bug. Manual
verification of the published synthetic.ir_curve_configs.basic
dataset produced unrealistic (~150-250) published rates despite
day-scaled kappa=0.5/365.
Confirmed root cause on reading the code (not just the ad hoc
verification): every process's one-tick transition treats kappa as a
discrete per-tick AR(1) decay (exp(-kappa)), not as an
annualised kappa*dt. Day-scaling only divided kappa by 365 and left
sigma untouched. A correct discretisation of a continuous-time SDE
under dt = 1/365 needs both kappa_dt = kappa*dt and
sigma_dt = sigma*sqrt(dt) — omitting the sigma scaling means
per-tick variance stays roughly annual-sized regardless of how small
kappa gets, which is why every process's simulated rates move around
far more than a single day's move should. This is shared, not
Vasicek-specific:
vasicek_processcomposeshull_white_processdirectly (it is Hull-White with constanttheta, see its own docstring) — one fix covers both.cir_process::next_stochastic()'s per-tick noncentral-chi-squared scalec = sigma^2*(1-e^{-kappa})/(4*kappa)has the identical shape: askappa -> 0,c -> sigma^2/4, i.e. it does not shrink with a smaller day-scaledkappaeither.
Split out of Seed realistic IR curve sample data because this is a quant/numerics bug in the existing process recursions, independent of sourcing new vintage data or wiring it into the feed.
Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | IR Rates synthetic data: dataset seeding, index cleanup, dual-curve, quoting conventions |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-07-22 |
Acceptance
hull_white_process=/=vasicek_process=/=cir_processgain an explicitdt(year-fraction per tick) parameter, defaulting to1.0(full backward compatibility);kappa=/=theta=/=sigma=/ =initial_ratestay in their natural annualised units, never pre-scaled by callers.discount_factor()'s per-tick bond-time accumulation isdt-aware for both Hull-White's recursion and CIR's closed form (not just the state-transition side) — this is the actual source of the ~150-250% published-rate bug, not sigma scaling.process_factory::make_yield_curve_process()and everyores.syntheticcall site threaddtthrough explicitly (1/365 for the IR curve feed's day-per-tick convention), replacing the SQL- embeddedkappa/365=/=sigma/sqrt(365)scaling insynthetic_ir_curve_configs_basic_populate.sql=/ =_realistic_populate.sql, which revert to plain annualised parameters.- Root cause and the corrected recursion are documented in
doc/knowledge/domain/vasicek_process.org,hull_white_process.org,cir_process.org, including the QuantLib cross-check and attribution. - The published
synthetic.ir_curve_configs.basicdataset produces realistic rates end-to-end (verified via the same manual verification path that surfaced the bug) for at least a 2Y entry, for each process type the dataset exercises. - Comprehensive test coverage in
vasicek_process_tests.cpp=/=hull_white_process_tests.cpp=/ =cir_process_tests.cpp(see Plan): flat-curve reproduction across adtsweep (year/day/hour/minute), cross-dtconsistency, degenerate small-kappabranch sanity, and default-dtbackward-compatibility (existing assertions unchanged).
Plan
Cross-checked against QuantLib
(/mnt/development/Development/OreStudio/Engine.remote/QuantLib):
ql/processes/ornsteinuhlenbeckprocess.cpp,
ql/processes/coxingersollrossprocess.cpp,
ql/models/shortrate/onefactormodels/vasicek.cpp,
hullwhite.cpp. Confirms the fix direction and supersedes the
sigma*sqrt(dt) framing above — the actual numeric root cause is more
specific and doesn't need caller-side sigma pre-scaling at all once
dt is explicit (see the parent story's * Decisions for the full
writeup):
- QuantLib keeps
speed=(kappa) and =dtalways separate:exp(-speed*dt),variance = sigma^2*(1-exp(-2*speed*dt))/(2*speed)— never a pre-scaled composite kappa. - Applying that to
hull_white_process: the only change needed innext()isdecay = exp(-kappa_ * dt_)(wasexp(-kappa_)); the existingvar = (1-decay^2)/(2*kappa_)formula is already correct oncedecayitself is dt-aware (dividing by annualisedkappa_was never the bug — the missing*dtin the exponent was). - The actual, more damaging bug is in
discount_factor(): itsnext_b = 1.0 + b*decaytreats every tick as one full year of bond-time regardless of what a tick represents. Fix:next_b = dt_ + b*decay. For a 730-daily-tick 2Y entry this was integratingexp(-730*rate_)instead ofexp(-2*rate_)— the actual source of the ~150-250% published rates, not sigma at all. cir_process:next_stochastic()'sdecay=/=c=/=lambdaneed the sameexp(-kappa_*dt_)fix;discount_factor()'s closed form needstau = ticks_ahead * dt_(was rawticks_ahead) — CIR already uses a closed form (no iteration), so this is a one-line fix oncedtis threaded through, unlike Hull-White's recursion.- Degenerate small-
kappabranches (Hull-White'skappa <0=, both processes' driftless/ODE paths) get the same*dttreatment (b +dt_=,rate_ +sigma_*sqrt(dt_)*z=), following QuantLib'sspeed < sqrt(QL_EPSILON)algebraic-limit pattern rather than a bare<0= guard. dtdefaults to1.0(one tick per year) on every constructor — exactly today's behaviour, so this is purely additive for every existing caller/test that doesn't pass it.process_factory::make_yield_curve_process()gains the samedtparameter (default1.0);ores.synthetic's three call sites (ir_curve_feed.cpp,ir_curve_preview_handler.hppx2) pass1.0/365.0, matchingir_curve_template_resolver's already- documented "1 tick == 1 calendar day" convention.synthetic_ir_curve_configs_basic_populate.sql=/ =_realistic_populate.sqlrevert theirkappa/365=/=sigma/sqrt(365)SQL-embedded scaling back to plain, real annualised Vasicek parameters — the scaling now lives in tested C++, not SQL comments.
Notes
Manual scenario run surfaced a real gap this fix's original scope
missed: IrCurveEditor's (Qt) Process tab – the Simple-mode κ/σ
sliders, their tooltips, and a new curve's default κ/θ/σ values –
was never updated from the old day-scaled convention. Its slider
ranges were hardcoded to [0, 0.02]=/[0, 0.005]= (assuming
day-scaled kappa ~0.0007-0.0015, sigma ~0.0004-0.0007), so once the
SQL populate scripts reverted to real annualised values (kappa
~0.2-0.55, sigma ~0.006-0.022) the sliders pegged visually at their
max even though the underlying stored value was correct. Fixed:
slider ranges to [0, 2.0]=/[0, 0.05]=, tooltips to describe
annualised (not per-tick) magnitudes with the real seeded ranges, and
a new curve's defaults from kappa=0.001,sigma=0.0005 to
kappa=0.35,sigma=0.01.
Also flagged during the same run: published rates read as "a tad
small" in the Curve Snapshot chart (raw decimal, e.g. 0.03022
rather than 3.022% or 302.2bp). The rates themselves are correct
(EUR ESTR hovering ~2.98-3.04%, exactly theta≈3% plus a few bp of
realistic vol) – this is a display-scale question, not a
calibration bug, and is already the exact scope of the sibling
IR rate display/quoting
conventions task in this story, so it stays there rather than
expanding this task's scope.
Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario, run
through the QA Validation Runner panel) that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
| Verify day-scaled kappa/sigma calibration fix for IR curve feeds | PASSED | All 5 steps passed after the IrCurveEditor slider fix; AUD/AONIA curve snapshot showed plausible, moving rates (4.17-4.28%) across every tenor. |
PRs
| PR | Title |
|---|---|
| #1661 | [analytics.quant,synthetic,sql] Fix day-scaled kappa/sigma calibration across short-rate processes |
Review
Three independent @claude review passes on #1661, all issue-level (no line comments): all three found no correctness/security/ maintainability issues and marked the PR ready to merge. Non-blocking observations only:
| # | Comment summary | File | Decision | Notes |
|---|---|---|---|---|
| 1 | dt=1.0/365.0 duplicated as two separately-named constants (ir_curve_feed.cpp's dt, ir_curve_preview_handler.hpp's preview_dt) | ir_curve_feed.cpp, ir_curve_preview_handler.hpp | Accepted | Consolidated into one ir_curve_feed_dt constant in ir_curve_template_resolver.hpp, next to the "1 tick == 1 calendar day" convention it implements; both call sites now reference it. |
| 2 | dt isn't validated by the shared process_parameter_validation layer (Qt client's validation surface), only by each process constructor's own throw | process_parameter_validation.hpp/.cpp | Declined | dt is never user-facing input (fixed 1/365 convention set in code, not an editable config field), so there is nothing for the shared UI-facing validator to validate; the constructor throw is the only meaningful validation site. |
| 3 | preview_shape() passes a literal 42 seed rather than deriving one from req, unlike simulate_paths() | ir_curve_preview_handler.hpp | Declined | Not a regression from this PR – matches prior behaviour (the old 4-arg call relied on the constructor's implicit seed=42 default); out of scope for a dt-focused fix. |
Result
Root-caused and fixed the day-scaling bug shared by every discrete-
time short-rate process (Vasicek/Hull-White, CIR) plus OU: the actual
bug was discount_factor()'s bond-time accumulation treating every
tick as a full year regardless of what a tick represents (+1.0 per
tick instead of +dt), not the sigma scaling first suspected —
confirmed by cross-checking against QuantLib's own process sources,
which always keep speed=/=dt as separate arguments.
Shipped: an explicit dt (year-fraction per tick) constructor
parameter on all four processes, defaulting to 1.0 for full
backward compatibility; process_factory and every ores.synthetic
call site thread dt=1/365 through explicitly; the SQL populate
scripts reverted to plain annualised parameters; 191 new dt-tagged
Catch2 assertions across all four processes (flat-curve exactness,
discretisation convergence, statistical variance scaling, degenerate-
branch sanity); three knowledge docs updated with the QuantLib cross-
check; and a real regression the fix's own manual verification
surfaced — IrCurveEditor's κ/σ sliders/tooltips/defaults, still
calibrated for the old day-scaled magnitudes — fixed in the same PR.
All acceptance criteria met. Manual end-to-end verification (PR
#1661) PASSED: AUD/AONIA curve snapshot showed plausible, moving
rates (4.17-4.28%) across every tenor 1M-10Y, confirming the original
"~150-250% blown-up, frozen far point" symptom is gone. Full build
green, rat test suite green, schema validation clean, three
independent @claude review passes found no correctness issues.