Authorization Policy Rollout Failure

A policy rollout has introduced selective false-denies on a subset of resource/identity combinations, while some Policy Decision Point caches across the fleet still enforce the previous, correct policy version — a real policy-version-skew mechanism, not a uniform outage. Five business invariants govern every decision here: (1) unauthorized access must never be granted to reduce false-denies — availability loss is always preferable to unauthorized access for protected resources; (2) critical/protected resources default fail-closed, with no exception; (3) a rollback or pin to a known-good policy version is not complete until cache convergence across the fleet has been verified — fixing the source policy changes what NEW cache fetches receive, it does not retroactively change what already-cached Policy Decision Point instances are currently enforcing; (4) mixed policy versions across the fleet must be observable and bounded — exposure must be frozen before it is fixed, never left to keep expanding while you investigate; (5) availability loss is preferable to unauthorized access for protected resources, full stop. Decide how to stop the regression from spreading further, how to restore the known-good policy without assuming caches have already converged just because the source looks fixed, how to bound the blast radius for non-critical resources while convergence completes, and when fleet-wide convergence is genuinely verified rather than merely assumed.

Difficulty

Principal

Expected Time

~25 min

Category

Security & Identity