Story: Generic entity-mirror cache primitive + codegen facet (party as first consumer)
Table of Contents
This page documents a story in Sprint 23. It captures the goal, current status, acceptance criteria, and the tasks that compose it.
Goal
Any entity that a consuming service needs to mirror in-process gets a thread-safe cache with almost no hand-written code, kept fresh via NATS eventing, with the subject binding checked at compile time (no hardcoded subject-name string that can drift from the publisher).
This story was originally scoped as a one-off fix to
ores.iam.core's party_cache (see Decisions for the investigation
that got us here: refdata's outbox eventing for party works fine —
ores.refdata.party_changed is published on every save — but
party_cache subscribed to a different, nonexistent subject string,
refdata.v1.parties.changed, that only ever existed in its own
docstring). That class of bug — bespoke subscribe-and-reload glue,
hand-typed subject strings, ad hoc concurrency — is worth solving
generically rather than patching once, so the story is now: build
the generic mechanism, with party as its first real consumer.
Two layers:
- Layer A — generic primitive (hand-written once, no codegen): a
partitioned_cache<PartitionKey, Key, Value>template (e.g. inores.eventing.core) backed byimmer::atom<immer::map<PartitionKey, immer::map<Key, Value>>>, following theimmer::atom-based idiom already established byores.analytics.quant'srate_engine=/=rate_snapshot. Reads are wait-free (load()on the atom, never blocked by a writer); partition replacement is a compare-and-swap loop (update(), not a blindstore()), so concurrent reloads of different partitions never race each other into a lost update. Supports an optional aux-index policy so entity-specific derived structure (e.g. party's parent/child hierarchy) stays generic rather than hardcoded into the primitive. - Layer B — a
nats-event-cachefacet: given an entity that already has a changed-event +event_traitsbinding and a read-for-cache RPC (today hand-written only for party asread_parties_for_cache; should become a standard generated archetype, likelist=/=save=/=removealready are), generates the thin per-entity cache class (apartitioned_cacheinstantiation) and the registrar subscribe/warm-up wiring. Per the MASD facet model, the declaration lives with the entity, where it is defined — e.g. acached_by: ores.iamannotation on party's own model entry inores.refdata, not a second, competing declaration authored insideores.iam. The facet's output path is the only thing that differs from every other existing facet: it resolves its\{component\}token from the named consumer incached_by, not from the entity's own component, because the generated cache is a translation unit that must physically compile into the consumer's binary.
Party becomes the first consumer: ores.refdata's party model gains
cached_by: ores.iam, the facet generates
ores.iam.core's cache class and registrar wiring, and the
hand-written prototype (already rewritten once during
investigation, on immer, correctly subject-bound) is retired in
favour of the generated version.
Status
| Field | Value |
|---|---|
| State | DONE |
| Parent sprint | Sprint 23 |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-07-14 |
Acceptance
partitioned_cache<PartitionKey, Key, Value>exists (Layer A),immer::atom-backed, with wait-free reads and CAS-safe partition replacement; unit-tested for concurrent-writer correctness (no lost updates across concurrently-reloaded partitions) and for readers never blocking on a concurrent write.- The read-for-cache RPC is generated as a standard archetype rather than hand-written per entity.
- A
nats-event-cachefacet exists infacet_catalogue.org, declared via acached_byannotation on the entity's own model, generating the consumer-side cache class and registrar wiring with the subject bound viaevent_traits(no hardcoded subject strings anywhere in generated or hand-written code). - Party is migrated to the generated cache:
ores.refdata's party model declarescached_by: ores.iam; the hand-written prototype inores.iam.core/service/party_cache.hppis retired in favour of the generated version. - Reproduction from the original capture no longer occurs: activating
a party via
save_party_request(e.g. throughbarclays_system_provision.ores) is visible to a subsequent IAM login without restarting the IAM service. - Regression/unit test coverage for cache reload-on-notification using the generated party cache.
Tasks
| Task | State | Start | End | Description |
|---|---|---|---|---|
| Implement generic partitioned_cache<PartitionKey,Key,Value> primitive | DONE | 2026-07-12 | 2026-07-13 | Layer A: an immer::atom<immer::map<PartitionKey,immer::map<Key,Value>>>-backed template, following the rate_engine/rate_snapshot idiom already in ores.analytics.quant. Wait-free reads, CAS-safe partition replacement, optional aux-index policy. |
| Generalize read-for-cache RPC into a standard archetype | DONE | 2026-07-13 | 2026-07-13 | Today read_parties_for_cache is hand-written, only for party (see nats_registrar_implementation doc, which documents it as a manual escape hatch). Turn it into a standard generated archetype, like list/save/remove already are, so any cacheable entity gets a read-for-cache RPC generated on its producer side without hand-written code. |
| Add nats-event-cache facet: cache class + registrar wiring archetypes | DONE | 2026-07-13 | 2026-07-13 | Add a nats-event-cache facet to facet_catalogue.org generating two archetypes: the consumer-side {entity}_cache.hpp (a partitioned_cache instantiation) and the registrar subscribe/warm-up wiring snippet, subject-bound via event_traits (no hardcoded strings). Declared via a cached_by annotation on the entity's own model (where the entity is defined); the facet's output component token resolves from cached_by, not the entity's own component, since the generated cache must compile into the consumer's binary. |
| Migrate party to the generated cache: first real nats-event-cache consumer | DONE | 2026-07-13 | 2026-07-14 | Add cached_by: ores.iam to party's model in ores.refdata; regenerate ores.iam.core's party cache via the new facet; retire the hand-written prototype (task: Implement generic partitioned_cache primitive) in favour of the generated version. Confirms the facet design end-to-end on a real entity. |
| Test party_cache reload-on-notification | DONE | 2026-07-14 | 2026-07-14 | Add regression/unit test coverage in ores.iam.core for the generated cache: publishing ores.refdata.party_changed for a tenant causes the cache to reload that tenant's partition, and lookups reflect the change without a restart. |
| Verify activation reproduction is fixed end-to-end | DONE | 2026-07-14 | 2026-07-14 | Re-run the capture's original reproduction: activate a party via save_party_request (e.g. barclays_system_provision.ores) against a running environment and confirm a subsequent IAM login reflects Active status without restarting the IAM service. |
| Add auth check to the read_for_cache archetype | DONE | 2026-07-14 | 2026-07-14 | The generated read_for_cache handler (unlike list/save/history/hierarchy) does not call make_request_context/has_permission — it trusts the caller-supplied tenant_id outright, inherited from party's hand-written read_parties_for_cache. Add the same auth check the other generated methods use, or explicitly design the intended trust model for this internal service-to-service RPC (e.g. system/service-level permission), before any consumer beyond the trusted party-cache warm-up case adopts the flag. |
| Find other hand-rolled entity caches to migrate onto nats-event-cache | DONE | 2026-07-14 | Search the codebase for other consumer-side, hand-written in-process caches of another service's entity data (the same pattern party_cache used before this story), and produce a list of candidate migrations onto the generic nats-event-cache facet, each requiring cached_by/read_for_cache on the producer's model. |
Decisions
- 2026-07-12: Original capture hypothesised the bug was on
ores.refdata's producer side (missing publish). Reading the code showed refdata's outbox eventing for party already works correctly and publishesores.refdata.party_changedon every save. - 2026-07-12: Second read of
registrar.cppfound a subscription already existed, just bound to the wrong, hardcoded subject string (refdata.v1.parties.changed, which appears nowhere exceptparty_cache's own docstring) instead of the real one (ores.refdata.party_changed, viaevent_traits). Root cause is a subject-name mismatch, not a missing subscription. - 2026-07-12: Decided to generalise rather than patch once, since
this class of bug (hand-typed subject strings, bespoke
subscribe-and-reload glue, ad hoc concurrency) will recur for any
future cross-service cache. Story rescoped from a one-off
party_cachefix to building a genericpartitioned_cache<PartitionKey,Key,Value>primitive (immer::atom-backed, following therate_engine=/=rate_snapshotidiom already inores.analytics.quant) plus anats-event-cachecodegen facet, with party migrated to be its first consumer. - 2026-07-13: The
nats-event-cachefacet needed a generic aux-index mechanism (not just entries) to fully retire party's hand-written prototype —compute_visible_party_idsdepends on a parent/child index the facet didn't originally generate. Added acache_aux_typeflag plus three paste points, mirroring the existingcustom_subscriptionspaste-point convention, rather than special-casing party. - 2026-07-14: Investigated for other hand-rolled, cross-service entity-mirror caches to migrate — none found; party's was the only instance. Two services (trading, reporting) make repeated cross-service NATS calls without caching, noted as future performance opportunities, not filed as tasks (no staleness bug to fix today).
- 2026-07-14: Closing the
read_for_cacheauth gap (nomake_request_contextcheck, unlike every other generated handler) surfaced a startup-ordering hazard: IAM authenticating against itself (its ownservice-login) to armparty_cache. Confirmed viaores.nats'sclient.cppthat NATS subscriptions dispatch on the NATS client's own thread, independent ofio_ctx— so it's an ordering bug, not a threading deadlock. Wrote Service Bootstrap Phases naming the general Phase 1 (wire everything) / Phase 2 (mint tokens, warm caches) pattern, and restructuredores.iam.core's registrar accordingly. - 2026-07-13: Corrected two design points before committing to the
facet shape: (1) terminology — the generated unit is a facet
(grouping archetypes), not an "archetype" as first stated; (2)
placement — every existing facet's output path is anchored to the
entity's own component (never a third party's), so the
nats-event-cachefacet's declaration must live on the entity's own model (cached_by: ores.iamon party's model inores.refdata), not in a separate, competing model authored insideores.iam. Only the facet's output path resolves against the named consumer; the single source of truth for "who caches this entity" stays with the entity, like every other fact about it. - 2026-07-13: Layer A (
partitioned_cache) implemented and merged ahead of Layers B/C so it can be reviewed as a self-contained, tested unit before the larger codegen-touching work starts. Its concurrency tests caught a genuine bug (retry-callback consuming its payload by move under CAS contention) present in both the new primitive and theparty_cacheprototype it was extracted from — confirms the value of writing the concurrency tests before, not after, generalising.
Out of scope
- General TTL/reload-on-miss fallback strategy for
partitioned_cache, unless the eventing approach proves unreliable during implementation. - Migrating any other existing ad hoc in-process cache in the
codebase onto
partitioned_cache— this story only dogfoods it on party. Follow-up entities adopt it opportunistically. - Broader ores.iam/ores.refdata service-boundary cleanup (tracked separately as "IAM/Refdata service boundary cleanup").
- A runtime variability flag to disable cross-service cache eventing and fall back to on-demand reads — a plausible future need, but not required by any known failure mode yet.