🔒 Paid

Storage IOPS Saturation

A nightly background compaction job is saturating the database's persistent storage volume. Customer-facing order and account data must remain available and fast at all times — the Database tier is shared by every customer-facing read and write, so storage contention here degrades the whole platform, not just one feature. The background compaction job has its own scheduled completion window and is not itself customer-facing. Diagnose why storage IOPS, query latency, and checkpoint queueing are climbing together, and bring the system back to a durable, sustainable state.

Difficulty
Senior
Expected Time
~12 min
Category
Databases & Storage

Skills

  • Diagnosing storage-layer IOPS contention from a background workload
  • Distinguishing a durable capacity fix from a workload-suppression tradeoff
  • Reasoning about checkpoint-frequency and crash-recovery tradeoffs
  • Prioritizing customer-facing health against non-critical background work

System Architecture

  • API Service
  • Database
  • Persistent Storage

This is a Pro scenario. Upgrade to Pro to unlock it.

Upgrade to ProBack to Incidents