Investigation: MSVC C1202 and rfl complexity

Table of Contents

Overview

ORE Studio uses reflect-cpp (rfl) for JSON serialisation and deserialisation. rfl's internal mechanism for field-uniqueness checking (rfl::internal::no_duplicate_field_names in rfl/Literal.hpp) creates a compile-time rfl::Literal<f1, f2, …, fN> holding every reflected field name and performs an O(N²) constexpr string-equality check across all N names.

MSVC imposes a hard internal limit on the "recursive type or function dependency context" — the width of the template instantiation graph. When N is large enough, this graph saturates and MSVC aborts with:

fatal error C1202: recursive type or function dependency context too complex

The /constexpr:depth and /constexpr:steps flags (already set to 100 000 and 10 000 000 respectively) have no effect on C1202: they govern constexpr evaluation budgets, not the template instantiation graph width.

Clang has a related but distinct failure: fold-expression nesting in rfl::AddTagsToVariants exceeds -fbracket-depth (default 256) when a variant has many alternatives.

This document is the canonical reference. Individual sprint tasks and stories link here rather than re-stating the analysis.

Mechanics of the failure

rfl::Literal and no_duplicate_field_names

When rfl reflects a struct S, it builds an rfl::Literal<f1, f2, …, fN> where the fi are the effective field names of S. For a plain struct, "effective" means the struct's own direct fields. For a struct containing rfl::Flatten<Sub>, "effective" means the union of the parent's non-Flatten fields plus all of Sub's effective fields — the flattening is recursive.

no_duplicate_field_names then checks every pair (fi, fj) with i≠j for equality, producing an O(N²) instantiation graph of width proportional to N².

Key fact: rfl::Flatten does not reduce the parent Literal size. A parent struct with three rfl::Flatten members whose sub-structs have 5, 9, and 5 fields respectively will produce a parent Literal with 5+9+5 = 19 entries — identical to the original flat 19-field struct. The sub-structs are also reflected independently (producing smaller Literals), but the parent Literal is not reduced.

Two distinct failure modes

Understanding the mode of failure is essential for choosing the right fix:

Mode 1 — Context accumulation (TU saturation)

C1202 fires when a TU has already accumulated a large template instantiation context from other headers and types, and then encounters one more large rfl Literal that tips it over the limit. The C1202 is not caused by any single struct alone — it requires the combination of a saturated context and an additional Literal.

Characteristics:

  • Fires late in the build (high step number, e.g. step 4651/4936).
  • Disappears when the offending TU is split or the context is reduced.
  • The same struct compiles fine in a clean TU.
  • Sensitive to compilation order.

Mode 2 — Single-struct saturation (clean TU)

C1202 fires on a specific struct even in a clean TU with minimal includes. The struct's Literal alone saturates the limit regardless of surrounding context.

Characteristics:

  • Fires early in the build (low step number, e.g. step 1140/4939).
  • Does not disappear with TU splitting or two-phase parse.
  • Repeatable regardless of compilation order.
  • The ONLY fix is to reduce the Literal size for that struct below the threshold.

The threshold for Mode 2 is somewhere below 19 fields. 9-field sub-structs compile without issue; 19 fields in a single effective Literal does not.

Chronological record of approaches

Sprint 16 — Initial TU split (workaround, not a fix)

ClientManager.cpp was split into focused per-function TUs (PRs #754, #757). This addressed Mode 1 failures by reducing context per TU. It is a maintenance burden but provided short-term relief.

Sprint 17 — Constexpr flag raises (ineffective for C1202)

Tasks: Raise MSVC constexpr limits (sprint 17)

Raised /constexpr:depth and /constexpr:steps to large values. Had no effect on C1202 (wrong category of limit). Reverted/superseded.

Sprint 17 — trade struct decomposition via rfl::Flatten (worked for Mode 1)

Task: Decompose trade into rfl::Flatten sub-structs (sprint 17) — PR #761

trade had 25 fields. Decomposed into 5 sub-structs (identity/parties/ classification/lifecycle/audit, max 7 fields each) composed via rfl::Flatten.

Result: C1202 on trade resolved. This worked because trade's C1202 was a Mode 1 failure — the 25-field parent Literal was not saturating the limit alone; it only fired in combination with other large types in the same TU. Reducing the accumulated instantiation context (by making each sub-struct available as a separate, smaller template instantiation) was sufficient.

What was NOT understood at the time: rfl::Flatten does NOT reduce the parent trade Literal — trade still has a 25-field effective Literal. The fix worked because trade was a Mode 1 case. Had trade been a Mode 2 case (firing alone in a clean TU), rfl::Flatten would not have helped.

Sprint 18 — Investigation and Triple Isolation (addressed trade_instrument)

Task: Investigate RFL complexity root cause (sprint 18) Story: Resolve RFL complexity issues (sprint 18)

Investigation identified rfl::AddTagsToVariants on trade_instrument (a variant of ~30 leaf types totalling 502 field names) as the primary driver of Clang failures. The Triple Isolation strategy was implemented: TU split + two-phase parse + global -fbracket-depth=1024.

swaption_instr_outer was introduced as a surrogate struct to isolate heavy rfl instantiation from caller TUs. This is a correct pattern for Mode 1.

Sprint 19 task 9 — swap_leg rfl::Flatten (applied to a Mode 2 case — did not work)

Task: Fix C1202 in ClientManagerExportPortfolio.cpp (task 9, sprint 19) — PR #1031

swap_leg has 19 fields. After PR #1010 isolated export_portfolio_response into ClientManagerExportPortfolio.cpp, MSVC C1202 fired at step 1140/4939 (early in the build, clean TU) — a Mode 2 failure.

Following the sprint 17 precedent, rfl::Flatten was applied: swap_leg was split into swap_leg_identity (5), swap_leg_terms (9), swap_leg_audit (5) composed via rfl::Flatten.

Result: C1202 persisted. The Windows CI error (step 1140/4939) after PR #1031 shows the identical 19-field Literal:

rfl::Literal<"version","tenant_id","id","instrument_id","leg_number",
"leg_type_code","day_count_fraction_code","business_day_convention_code",
"payment_frequency_code","floating_index_code","fixed_rate","spread",
"notional","currency","modified_by","performed_by","change_reason_code",
"change_commentary","recorded_at">

Why it failed: swap_leg is a Mode 2 case. Its 19-field effective Literal (which rfl::Flatten does not reduce) saturates the MSVC graph limit in a clean TU. The sprint 17 precedent was misapplied — that precedent was valid for Mode 1 but not Mode 2.

Side effects of task 9 that should be kept: The struct decomposition (swap_leg_identity / swap_leg_terms / swap_leg_audit) is architecturally sound and all callers have been updated. The decomposition should be retained; only the rfl::Flatten composition mechanism should change.

The correct fix for Mode 2

The only way to eliminate a Mode 2 C1202 is to ensure no single rfl Literal exceeds the MSVC threshold. Since the threshold is somewhere below 19, and sub-structs with ≤9 fields compile cleanly, the fix is:

Remove rfl::Flatten from swap_leg and use plain nested sub-structs.

// Before (rfl::Flatten — effective Literal still has 19 fields):
struct swap_leg {
    rfl::Flatten<swap_leg_identity> identity;  // contributes 5 names to parent Literal
    rfl::Flatten<swap_leg_terms>   terms;       // contributes 9 names to parent Literal
    rfl::Flatten<swap_leg_audit>   audit;       // contributes 5 names to parent Literal
    // → parent swap_leg Literal: 19 fields → C1202
};

// After (plain nested structs — each reflected independently):
struct swap_leg {
    swap_leg_identity identity;   // reflected as its own 5-field Literal
    swap_leg_terms    terms;      // reflected as its own 9-field Literal
    swap_leg_audit    audit;      // reflected as its own 5-field Literal
    // → parent swap_leg Literal: 3 fields → safe
};

With plain nested structs, rfl reflects each sub-struct independently. The parent swap_leg Literal has 3 fields; no Literal exceeds 9 fields.

Wire format consequence: the JSON representation changes from flat to nested:

// Before (flat):
{"version": 1, "tenant_id": "...", "id": "...", ...(19 fields at top level)...}

// After (nested):
{"identity": {"version": 1, "tenant_id": "...", ...},
 "terms":    {"leg_type_code": "Fixed", ...},
 "audit":    {"modified_by": "...", ...}}

Both the server (ores.trading.service) and the client (ores.qt) are in the same repository and deployed together. There are no external consumers of the swap_leg wire format. The change is safe.

Caller simplification: the .get() accessor (an rfl::Flatten artifact) is removed. sl.identity.get().version becomes sl.identity.version.

Architectural impact of the plain nested struct fix

The fix (remove rfl::Flatten, use plain nested sub-structs) has broader architectural consequences worth recording. On balance the change is positive — it removes a compiler workaround at the cost of a wire-format change, and leaves the codebase in better shape on every dimension except one.

Code clarity — improvement

rfl::Flatten::get() is a library artefact with no domain meaning. Callers had to write sl.identity.get().leg_number when they meant sl.identity.leg_number. The nested struct approach is idiomatic C++; a reader unfamiliar with rfl can understand the code immediately.

Before:

auto& tm = sl.terms.get();
tm.leg_type_code = "Fixed";

After:

sl.terms.leg_type_code = "Fixed";

Compiler dependency — improvement

With rfl::Flatten, the struct definition itself depends on the rfl library header. Without it, swap_leg is a plain C++ struct that can be included in any TU with no rfl dependency. rfl is now only needed at serialisation boundaries, which is the correct layering.

Portability — improvement

A plain nested struct can be serialised by any library (rfl, nlohmann::json, Boost.JSON, protobuf, etc.) without structural modification. The flat rfl::Flatten form ties the struct layout to rfl's specific Flatten semantics.

Wire format — trade-off

The JSON changes from flat (19 top-level fields) to nested (3 top-level keys). This is a breaking wire-format change. The risk is low in ORE Studio because:

  • The service and client are in the same repository, deployed together.
  • swap_leg is an internal RPC type; no external consumers exist.
  • The format change is consistent: the C++ nested structure now mirrors the JSON structure, eliminating a conceptual mismatch.

Recommendation: bump the export_portfolio_response protocol version to signal the format change clearly, and verify no persisted JSON (e.g. test fixtures or fixtures in the database) uses the flat format.

Structural coherence — improvement

The C++ structure now accurately reflects the logical grouping:

  • identity — who and when (version, tenant, id, instrument, leg number)
  • terms — the economic terms (rate type, rate, notional, currency, indices)
  • audit — the audit trail (who changed it, why, when)

This grouping makes the domain model self-documenting. It also aligns swap_leg with how trade was decomposed in sprint 17 (five sub-structs), creating a consistent pattern across the domain layer.

Performance — neutral

Both layouts are POD structs. Memory layout and cache behaviour are effectively identical. There is no runtime cost difference.

Maintenance — improvement

Flat 19-field structs are fragile: adding a 20th field (if the threshold is ~17-19) would silently re-trigger C1202. The nested layout makes adding fields safe: new fields go into the appropriate sub-struct (≤9 fields each) and C1202 cannot fire regardless of how many fields are added in future.

Summary

Dimension Before (rfl::Flatten) After (nested structs)
Code clarity Obscured by .get() Clean, idiomatic
Compiler dependency Struct requires rfl Struct is pure C++
Portability rfl-specific Library-agnostic
Wire format Flat JSON (19 fields) Nested JSON (3 keys)
Structural coherence Mismatch (flat JSON, nested C++) Aligned
Performance Baseline Identical
Future-proofing Fragile (field count) Robust
C1202 Fires (Mode 2) Eliminated

Decision table: which approach for which failure mode

Approach Mode 1 (TU saturation) Mode 2 (clean TU) Wire format
Raise constexpr flags No effect on C1202 No effect on C1202 Unchanged
TU split / isolation TU Effective Ineffective Unchanged
Two-phase parse / surrogate Effective Ineffective Unchanged
rfl::Flatten decomposition Reduces context, may help Ineffective (Literal unchanged) Unchanged (flat)
Plain nested structs (no Flatten) N/A Effective Changes to nested
Custom rfl reflector Possible but complex Possible but complex Configurable

Open questions and future work

Threshold

The exact field count at which MSVC fires C1202 in a clean TU is not precisely known. 9 fields is safe; 19 fields is not. Any new struct with more than ~12 fields should be evaluated before using rfl on it in isolation.

reflect-cpp issue #350

reflect-cpp Issue #350 was opened to request an upstream fix. If rfl eliminates the O(N²) compile-time uniqueness check (or provides a way to disable it per type), Mode 2 failures would disappear and the wire-format change would be unnecessary. Monitor this issue.

Other structs at risk

Any domain struct with >~12 fields that appears in an rfl serialisation chain in a relatively clean TU is at risk of Mode 2 C1202. A project-wide audit of field counts against the safety threshold would prevent future surprises.

Related artefacts

Artefact Link Notes
Sprint 16 story Windows portability fixes Initial TU split
Sprint 17 story MSVC C1202 constexpr depth fixes trade Flatten + constexpr flags
Sprint 17 task Decompose trade into rfl::Flatten sub-structs Mode 1 fix (correctly applied)
Sprint 17 plan MSVC C1202 — Decompose trade into Sub-Structs Design doc for trade fix
Sprint 18 story Resolve RFL complexity issues trade_instrument / Triple Isolation
Sprint 18 task Investigate RFL complexity root cause Produced original investigation report
Sprint 19 story Fix rfl complexity failure (Windows + macOS CI) Current story
Sprint 19 task 9 Fix C1202 in ClientManagerExportPortfolio.cpp (swap_leg Flatten) rfl::Flatten misapplied to Mode 2
Sprint 19 task 10 Replace rfl::Flatten with plain nested structs in swap_leg Correct Mode 2 fix
reflect-cpp upstream Issue #350 Upstream awareness

See also

Emacs 29.1 (Org mode 9.6.6)