Retry Budget Exhaustion
Recommendation Service calls Pricing Service synchronously on every request. Its retry logic has no retry budget and no circuit breaker: every failed call is automatically retried up to 3 times. Pricing Service's own database has just entered a maintenance-driven slowdown — a moderate degradation, not an outage, and not something you can fix directly. But because Recommendation Service retries every failure with no limit, it is silently multiplying the load Pricing Service receives far beyond real customer traffic, turning a moderate degradation into a severe one. Diagnose why Pricing Service's load looks so much worse than its real traffic would explain, and stop the amplification without assuming more Pricing Service capacity alone will catch up.
- Difficulty
- Intermediate
- Expected Time
- ~10 min
- Category
- Microservices & Dependencies
Skills
- Distinguishing a dependency's real degradation from self-inflicted amplified load
- Bounding a retry policy with a retry budget and a circuit breaker
- Recognizing that added downstream capacity can't outpace an unbounded retry multiplier
System Architecture
- Recommendation Service
- Pricing Service
This is a Pro scenario. Upgrade to Pro to unlock it.
Upgrade to ProBack to Incidents