Lesson 22 / 24

Rotating Secrets Safely

Rotate without downtime by overlapping the old and new values.

Overlap, switch, retire

A safe rotation has three steps: create the new secret while the old still works, update every consumer to the new one, then retire the old one. Switching in one step risks an outage if any consumer was missed.

Make rotation routine

Secrets that are rotated regularly make a leak far less harmful and a rotation far less scary.

Three stages: plan, rotate, verify.
Figure 8.1 — Plan, rotate and verify.

A rotation plan

Write the plan as steps so it can be repeated. Verify with a real request before retiring the old key.

1. Create new key in provider (keep the old one active)
2. Store the new key in the 1Password item
3. Redeploy services that read it
4. Check that requests succeed with the new key
5. Revoke the old key
6. Note the date for the next rotation

Prefer short-lived credentials

Credentials that expire on their own, such as cloud tokens issued per job, remove most rotation work and shrink the window of misuse. Use long-lived keys only where nothing better exists.

Quick check: Why create the new key before retiring the old one?

  • To avoid downtime while consumers switch
  • Providers require two keys
  • It makes keys longer
  • It hides the rotation date
Answer

To avoid downtime while consumers switch — Both values work during the switch, so no consumer is left with a dead key.