Rollback Strategies That Actually Work: Redeploys, Feature Flags, and Migrations

Blue-green and canary deploys — covered in more depth here — are how you ship forward carefully. Rolling back needs the same care, in the opposite direction. “Just roll back” is easy advice that assumes a rollback is always the same operation. It isn’t — a bad deploy of stateless code, a bad feature behind a flag, and a bad database migration each need a completely different recovery action, and the specific way an incident goes wrong is usually a team reaching for the wrong one of these three under pressure.

Redeploy: The Default, and Its Actual Limits

Redeploying the previous known-good artifact is the obvious answer and the correct one for a specific, narrower case than people usually apply it to: a bug in application code, with no accompanying schema change, where the previous version’s code is fully compatible with the current data:

$ kubectl rollout undo deployment/api
$ kubectl rollout status deployment/api

Clean, fast, and it fully solves the problem — provided the two things above hold. It stops being sufficient the instant a database migration ran as part of the same deploy: the previous code version wasn’t written against the current, already-migrated schema, and rolling back the application without also rolling back the data model reintroduces a different, often worse bug — the old code encountering columns or constraints it doesn’t understand. “Roll back the deploy” and “roll back the migration” are two separate decisions that get collapsed into one in most people’s mental model, and separating them explicitly is the actual fix.

Feature Flags: Instant, and Only for What’s Actually Behind One

A flag-gated rollback is the fastest recovery there is, because nothing redeploys at all — the code is already running, and flipping a flag just changes which path it executes:

if (flags.isEnabled('new-checkout-flow', userId)) {
  return newCheckoutFlow();
}
return legacyCheckoutFlow();

// incident response: flip the flag off, done in seconds, no deploy
flags.disable('new-checkout-flow');

This only works for exactly what was built behind a flag in the first place — it’s a mitigation available by design, not a general incident-response tool you can reach for after the fact. The trap that erodes trust in flags as a rollback mechanism: flag debt — flags that shipped, proved stable, and were never cleaned up, left permanently toggleable in code alongside dozens of others. Six months in, nobody’s confident which flags are load-bearing versus dead weight, and “just flip it off” stops being a safe reflex because nobody’s sure what else quietly depends on that flag’s current state. Flags earn their value as an incident tool only with a real deprecation process — remove the flag and the old code path once the new one’s been stable for an agreed period, rather than letting the flag list grow indefinitely.

Database Migrations: The One That Actually Requires Planning in Advance

This is the case that can’t be solved reactively during an incident — it has to be designed for before the migration ever runs, because most schema changes aren’t safely reversible after the fact once new data has been written against the new shape. The pattern that keeps rollback possible is the expand/contract migration, split into stages that are each independently safe:

-- Stage 1 (expand): add the new column, keep the old one, backfill
ALTER TABLE users ADD COLUMN email_verified_at TIMESTAMP;
UPDATE users SET email_verified_at = created_at WHERE is_verified = true;

-- Application deploys, dual-writing to both columns for a transition period

-- Stage 2 (contract): only after the new column is proven correct in production
ALTER TABLE users DROP COLUMN is_verified;

Splitting a single “rename this column” instinct into expand (add new, keep old, backfill), a transition period where the application writes to both, and only then contract (drop the old one) means a rollback at any point before the contract step is a simple application redeploy — the old code still works, because the old column is still there and still being kept in sync. Doing the rename as one migration collapses that entire safety window into a single irreversible step, and a bug discovered after that step has already run has no clean rollback path at all — only a forward fix, under worse time pressure than a rollback would have been.

Blue-Green as the General-Purpose Answer for Stateless Rollback

For anything stateless, a blue-green setup makes “rollback” close to instant regardless of what changed, because it’s not re-running a deploy pipeline at all — it’s switching which of two already-running, fully deployed environments receives traffic:

# router config — the entire "rollback" is this one line changing
upstream active_backend {
    server blue.internal:8080;   # was: green.internal:8080
}

“Green” stays fully deployed and warm even after traffic moves to “blue” for the duration of a rollback window, precisely so this switch has nothing left to build or start — just to redirect. The real cost that’s easy to underestimate: this only fixes stateless services outright. A shared database behind both environments is still shared, and a migration incompatibility isn’t fixed by switching which app version receives traffic — which is exactly why blue-green and the expand/contract migration pattern above aren’t alternatives to each other, they’re two answers to two different halves of the same deploy.

The Actual Decision Tree During an Incident

Bug in stateless application code only, no migration involved: redeploy the previous version, or switch environments if blue-green is set up — both are equally fast and either is correct. Bug in new functionality that shipped behind a flag: flip the flag, verify the flag itself isn’t stale or entangled with something else first. Bug that only appeared after a schema change: this is the case that needed the expand/contract plan made in advance — if it wasn’t, the incident is now a forward-fix situation, not a rollback situation, and pretending otherwise by attempting to force a schema rollback under pressure is usually how a bad deploy turns into data loss.

The Checklist

• Redeploy handles application-only bugs — verify no accompanying migration before assuming it’s sufficient.
• Flags give instant rollback only for what was deliberately built behind one — and only stay trustworthy with an actual deprecation process, not indefinite accumulation.
• Plan migrations as expand/contract from the start — this is the one category where “we’ll figure out rollback if we need it” doesn’t work, because the safe window closes the moment the contract step runs.
• Blue-green solves stateless rollback almost instantly, and solves nothing about a shared database — they’re complementary tools, not substitutes for each other.