🔒 Paid

Cache Node Failure

One node in the distributed cache cluster has failed. Its keys are redistributing — some concentrate onto the surviving nodes, others fall through as misses that hit the primary database directly. The database must stay within its normal read-latency range: sustained unprotected read pressure against it is treated as a critical incident, not just a cache-layer inconvenience. Restore the cache layer's effective capacity and protect the database from the fallback-read spike before it compounds.

Difficulty
Intermediate
Expected Time
~10 min
Category
Caching

Skills

  • Diagnosing cache-node loss and key redistribution
  • Protecting a downstream database from fallback-read pressure
  • Recognizing that aggressive cache warming can self-inflict a load spike
  • Comparing capacity-restoration vs. database-side protection strategies

System Architecture

  • API Cluster
  • Distributed Cache
  • Primary Database

This is a Pro scenario. Upgrade to Pro to unlock it.

Upgrade to ProBack to Incidents