System design guide
Caching strategies for system design interviews
Almost every system design answer adds a cache. Interviewers then ask the follow-ups: where exactly, which write strategy, how do you invalidate, and what happens when it fails? Here is a compact reference with the trade-offs to say out loud.
Where caches go
| Layer | What it caches | Typical tool |
|---|---|---|
| Client / browser | Static assets, API responses with cache headers | HTTP cache, app memory |
| CDN / edge | Static files, media, sometimes whole API responses | Commercial CDN, edge workers |
| Application (in-process) | Tiny, very hot data (config, hottest keys) | In-memory LRU map |
| Distributed cache | Query results, objects, sessions, counters | Redis, Memcached |
| Database | Pages, query plans, result sets | Buffer pool, materialised views |
In an interview, name the layer and what it buys you: "a CDN for thumbnails cuts origin egress", "Redis in front of the user table absorbs 95% of profile reads".
Read strategies
Cache-aside (lazy loading)
The application reads from the cache; on a miss it reads the database and populates the cache. Simple, resilient (if the cache is down, reads still work, just slower), and only caches what's used. Downsides: the first read of every key is slow, and the cache can serve stale data until the entry expires or is invalidated.
Read-through
The cache itself loads missing data from the database (via a loader the cache library calls). Same behaviour as cache-aside, but the logic lives in one place. Common with caching libraries and some managed caches.
Refresh-ahead
Entries that are about to expire and are still being read get refreshed in the background. Good for very hot keys with expensive recomputation; wasted work for keys that turn cold.
Write strategies
| Strategy | How writes work | Pros | Cons |
|---|---|---|---|
| Write-around + invalidate | Write the database, delete the cache key | Simple; no unread data cached | Next read misses |
| Write-through | Write cache and database synchronously | Cache always fresh for written keys | Slower writes; caches data that may never be read |
| Write-back (write-behind) | Write the cache; flush to the database asynchronously | Very fast writes; batches database load | Data loss if the cache fails before flushing; complex |
A reasonable default to say: "cache-aside for reads, update the database then delete the cache key on writes, with a TTL as a safety net." Reach for write-back only for things like counters and metrics where losing a few seconds of updates is acceptable.
Eviction and expiry
- LRU (least recently used): the default; good for recency-driven access.
- LFU (least frequently used): protects consistently popular keys from being flushed by one-off scans.
- TTL: bounds staleness. Add random jitter to TTLs so keys written at the same time don't all expire together.
- Size-aware eviction matters when values vary a lot in size.
Invalidation and consistency
The classic race with cache-aside: reader A misses and reads the old value from the database; writer B updates the database and deletes the key; reader A then writes the old value into the cache, which now stays stale until its TTL. Mitigations, from simple to strong:
- Short TTLs on data where staleness hurts.
- Delayed double delete: delete again a short time after the write.
- Versioned values: store a version with each entry and refuse to overwrite a newer version (compare-and-set).
- Invalidate from the database's change stream (change data capture), so every committed write produces an invalidation in commit order.
Be explicit about what consistency the product needs. A like count can be seconds stale; an account balance shouldn't come from a cache at all. See consistency models and CAP.
Failure modes interviewers ask about
- Stampede / thundering herd: coalesce concurrent misses, use a lock key, or refresh early.
- Hot keys: one key overloads one cache node. Replicate it across nodes, or keep a small in-process cache for the hottest keys.
- Penetration: requests for keys that don't exist always miss. Cache negative results briefly, or put a Bloom filter of valid keys in front.
- Cold start: after a cache flush or new deployment, the database takes full load. Warm the cache from logs of recent hot keys, or ramp traffic gradually.
- Cache outage: make sure the database can survive at least a partial loss of the cache, or degrade features (serve stale or partial data) rather than fall over.
How to talk about caching in an interview
- Justify it with numbers from your estimates: read QPS, hot set size, hit ratio.
- Name the pattern (cache-aside, write strategy) and the eviction policy.
- State the staleness you'll tolerate and how you invalidate.
- Mention one failure mode and its mitigation (stampede is the usual one).
For how a cache cluster itself is built (partitioning, replication, failover), see design a distributed cache.
FAQ
What is the most common caching pattern?
Cache-aside (lazy loading): the application checks the cache, and on a miss reads the database and writes the result into the cache. On writes it updates the database and deletes the cache entry. It's simple, works with any cache, and only caches data that is actually read.
Should I update or delete the cache on writes?
Delete (invalidate) in most cases. Updating the cache from the writer can race with concurrent readers and leave stale data; deleting forces the next read to load the fresh value. Keep a TTL as a backstop.
What is a cache stampede?
When a popular key expires and many requests miss simultaneously, they all hit the database to recompute the same value. Prevent it with request coalescing (one recompute, others wait), a short lock, or refreshing the value shortly before it expires.