# Staged Rollouts, Shadow Mode and Rollback — LLM Engineering Foundations

Source: https://www.geekswithgeeks.com/en/llm-engineering/s-release

> Release prompt and model changes gradually and be able to undo them.

## Prove it on a little traffic first

Even after passing offline evaluation, a change may behave differently on real traffic. Release in stages: **shadow mode** (run the new version on live requests in parallel, log its outputs, show users the old version, compare offline); **canary** (route 1 to 5% of traffic to the new version and watch quality, latency, cost, error and feedback metrics); then **ramp up** (25%, 50%, 100%) if the numbers hold. Keep the **old version** deployed and a one-step **rollback**, controlled by configuration or a feature flag rather than a redeploy. For user-facing changes, run an **A/B test** with a clear success metric and enough traffic to detect the difference you care about; remember the noise lessons: small gaps need many samples. Define in advance what result triggers a rollback, and who decides. Communicate changes to support and product teams so they know when behaviour shifts.

## A rollout plan

Each stage has a duration, a watch list and a rollback rule. Numbers are examples.

```text
stage        traffic   duration   watch                                   rollback if
shadow       0% (logged) 2 days    judge score vs old, cost, latency        new version loses by > noise margin
canary       2%        1 day      validity rate, p95, thumbs-down, cost    p95 +30% or validity < 99%
ramp 1       25%       2 days     same + category breakdown                any category -5 points
ramp 2       50%       2 days     same                                     any critical safety event
full         100%      keep old   dashboards + alerts                      one-step switch to previous release
```

## Decide rollback rules in advance

Write down what result triggers a rollback and who decides.

**Quiz:** What does shadow mode do?

- [ ] Deletes the old version
- [ ] Shows the new version to everyone immediately
- [x] Runs the new version on live requests without showing its output to users, for comparison
- [ ] Disables logging

*Answer:* Runs the new version on live requests without showing its output to users, for comparison. It gives real-traffic evidence with no user risk.