How does caching improve reliability, and what failure modes does it introduce?

Question

Accepted Answer

Caching helps reliability in two ways. It reduces load. On a read heavy service with a 10 to 1 read to write ratio, a cache at a 95 percent hit rate means the database sees roughly one twentieth of the reads, which protects it from overload and improves both latency and effective capacity. It also improves availability. With stale while revalidate, the cache keeps serving slightly old data even when the backing store is slow or briefly down, so a 30 second database blip never becomes a user facing outage. But caching is not free, and the failure modes are the interesting part. The classic one is a stampede, or thundering herd. A hot key expires and every request misses at once and slams the backend together, which can take down the very database the cache was protecting. I mitigate that with request coalescing or a lock so only one request recomputes the value, and with jittered TTLs, so a 5 minute TTL becomes 5 minutes plus or minus 30 seconds and the keys do not all expire on the same tick. There is also staleness, so I need a deliberate invalidation strategy. And the cache becomes a dependency itself. If it dies or comes back cold after a restart, 100 percent of traffic hits the database at full force. So I plan for cache redundancy and for warming or gracefully degrading with a cold cache. Caching is a powerful reliability tool that brings its own reliability concerns to design for.

Answer

What is a cache stampede and how do you prevent it?

A hot key expires and many concurrent requests all miss and hit the backend at once. Prevent with request coalescing/locking (one recompute, others wait) and jittered TTLs so keys do not expire simultaneously.

Answer

What happens when your cache restarts cold, and how do you handle it?

Every request misses and hits the database at full load (cold-cache thundering), which can overload it. Handle with cache warming, gradual rollout, request coalescing, and ensuring the DB can survive the surge or is shielded.

How does caching improve reliability, and what failure modes does it introduce?

How does caching improve reliability, and what failure modes does it introduce?

The follow-ups they ask next

What the interviewer is listening for

What sinks the answer

If you genuinely do not know

Keep going with reliability

Knowing the answer is not the same as recalling it under pressure