# Release Gates and CI for LLM Changes — LLM Engineering Foundations

Source: https://www.geekswithgeeks.com/en/llm-engineering/q-gate

> Block changes that make quality, safety or cost worse.

## Run the evaluation on every change

A prompt edit is a code change and deserves the same safety net. In **continuous integration**, run: fast **unit tests** with fake models for the deterministic code (parsing, routing, policy), the **evaluation set** against the candidate bundle, the **security attack suite**, and **cost and latency checks**. Then apply a **gate** with explicit rules, for example: overall pass rate must not drop by more than the noise margin and must stay above the target; no category may fall by more than a set amount; zero critical safety failures; p95 latency and average cost within budget. Show the **diff of flipped cases** to the reviewer. Full evaluations with real models cost money and time, so run a small smoke subset on every commit and the full set before merge or release. When the gate fails, the fix is investigation, not loosening the threshold.

## A release gate expressed as rules

Thresholds are examples. The point is that the decision is mechanical and written down.

```text
GATE (all must hold, else block the merge)
1. unit tests (fake models): all pass
2. eval pass rate >= 0.92 AND not lower than baseline by more than 0.02 (the noise margin)
3. no category drops by more than 0.05; Hindi/Hinglish category checked separately
4. attack suite: 0 critical successes; <= 2% minor
5. p95 latency <= 3.0 s; mean cost per request <= 1.1 x baseline
6. reviewer has seen the list of flipped cases (fixed vs broken)

If the gate fails: investigate the flipped cases. Do not just lower the threshold.
```

## Use a small smoke set per commit

Fast checks on every commit and the full evaluation before merge balance speed and cost.

**Quiz:** What should you do when the release gate fails?

- [ ] Delete the failing cases
- [ ] Lower the threshold
- [ ] Skip the gate once
- [x] Investigate the flipped cases and fix the cause

*Answer:* Investigate the flipped cases and fix the cause. A gate is only useful if failing it leads to understanding, not bypass.