🔒 Paid

Poison Message Loop

A malformed message from a buggy upstream producer has entered the queue and is failing to process. Because retries have no bound and the dead-letter queue is disabled, the consumer keeps re-delivering and re-failing the same message, starving every healthy message stuck behind it. Diagnose the head-of-line block and get the backlog flowing again before the queue falls critically behind.

Difficulty
Intermediate
Expected Time
~12 min
Category
Messaging & Streaming

Skills

  • Diagnosing head-of-line blocking in message queues
  • Retry bound and dead-letter queue design
  • Recognizing when added concurrency doesn't fix the problem
  • Recognizing plausible-but-harmful mitigations

System Architecture

  • Producer Service
  • Message Queue
  • Consumer Worker
  • Processing Service
  • Dead Letter Queue

This is a Pro scenario. Upgrade to Pro to unlock it.

Upgrade to ProBack to Incidents