Cache-Aside
Table of Contents
This page defines the service architecture pattern Cache-Aside in the pattern language. A reader looks in a cache first and, on a miss, loads the value from its source and keeps it. In ORE Studio the run token cache of every step service works this way.
1. Also known as
- Lazy loading.
2. Context
A value is expensive to obtain and read often: a token from IAM, a grant, a configuration. It changes rarely, or has a known lifetime.
3. Problem
How does a reader avoid asking the source every time, without the cache becoming the source?
4. Forces
- Asking every time loads the source.
- A copy can go stale.
- The cache must be safe to lose.
5. Solution
Read the cache. On a hit that is still fresh, use it. On a miss, or near the end of the value's life, load from the source and store the value with its lifetime. Bound the cache and evict on expiry. The source remains the system of record.
A cache filled from change events instead of on a miss is a different technique; the platform's entity-mirror caches work that way, as Where should an entity-mirror cache live? describes.
Figure 1: Cache-Aside: a fresh hit is used; a miss loads from the source and fills the cache.
Source: [[./pattern_cache_aside.puml][pattern_cache_aside.puml]]. Regenerate with
plantuml -tpng doc/knowledge/service_architecture_patterns/pattern_cache_aside.puml.
6. Failure handling
| Concern | What the pattern does |
|---|---|
| Errors | A failed load fails the read; it never serves a value it does not have. |
| Timeouts | The load has the source's timeout. |
| Retries | Concurrent misses for one key share one load. |
| Idempotency | Does not apply: reads have no effect. |
| Observability | Hits, misses and load latency are measured. |
7. Pitfalls
- Serving past expiry. A token used after its lifetime fails downstream.
- An unbounded cache. Memory grows with every key ever seen.
- A thundering herd. Many misses at once all load; share one load per key.
- A cache that must survive a restart. Then it is a store, not a cache.
8. Security
The cache holds what the source issued, for no longer than its lifetime, and only in memory.
9. Resulting context
- Benefits
- Fewer loads, and lower latency.
- Liabilities
- Staleness within the lifetime, and memory.
10. In ORE Studio
The service token provider of Client Credentials keeps and renews its own token the same way.
- Run grants and run tokens — the run token cache exchanges on a miss and keeps the token until shortly before it expires.
- Where should an entity-mirror cache live? — the partition token cache
(
partition_token_cacheinores.eventing.core) holds one token per tenant partition of an entity-mirror cache. On a miss, near expiry, or when a target answerstoken_expired, it exchanges for a token that acts inside that tenant. The key is the tenant, so a token never serves another tenant. - Where should an entity-mirror cache live? — the partition token cache
(
partition_token_cacheinores.eventing.core) holds one token per tenant partition of an entity-mirror cache. On a miss, near expiry, or when a target answerstoken_expired, it exchanges for a token that acts inside that tenant. The key is the tenant, so a token never serves another tenant.
11. Related patterns
- Token Exchange — the source the run token cache loads from.
- Database per Service — the owner stays the source.
12. Sources
- Microsoft, Azure Architecture Center, Cloud design patterns, Cache-Aside pattern.
13. See also
- Service architecture patterns — the structure note that orders this pattern language, and where to read this page in it.