Schema Migration Under Live Traffic
A database schema migration is in progress while Old App Fleet and New App Fleet both serve real production traffic against one Shared Database. The migration is only partially backward/forward compatible: New App Fleet depends on a new field that Old App Fleet cannot safely interpret, and rows already written with only the new field are starting to accumulate. Backfill/migration workers can repair this backlog, but running them aggressively competes with live traffic for the database's lock and write capacity. Business invariants: rows written during mixed-version operation must remain readable by the currently-serving application fleet. Acknowledged writes may not be discarded to make rollback easier. A schema rollout may be paused or slowed. Backfill completion is less important than customer correctness. The migration must not leave the database in a state that only one app version can interpret unless traffic has been safely fenced to that version. Diagnose whether the system is actually safe before deciding whether to roll back, continue, or hold.
- Difficulty
- Principal
- Expected Time
- ~20 min
- Category
- Databases & Storage
Skills
- Recognizing row-shape/schema incompatibility hiding behind a healthy availability dashboard
- Sequencing containment, rollout control, database protection, and backfill correctly
- Distinguishing a frozen-but-unrepaired backlog from a genuinely safe handback state
- Evaluating rollback safety as a function of history, not just the current configuration
System Architecture
- Clients
- Load Balancer
- Old App Fleet
- New App Fleet
- Shared Database
- Backfill / Migration Workers
This is a Pro scenario. Upgrade to Pro to unlock it.
Upgrade to ProBack to Incidents