Lesson 25 / 29
Rolling Out Agents to a Team
Start small, set rules, measure and iterate.
A staged rollout
A sensible rollout: (1) pilot with a few volunteers on low-risk work (tests, docs, small bugs) in a sandboxed setup; (2) write the team conventions: a shared instruction file, which tasks are suitable, the permission policy, the review standard for agent-written code (the same as human code, plus extra scrutiny of tests and dependencies), and how to disclose AI assistance in PRs; (3) put guardrails on the server: protected branches, required reviews and CI, secret scanning; (4) train people on prompting, reviewing diffs and recognising failure modes; (5) measure with the internal benchmark and delivery metrics; (6) expand gradually, and keep a way to pause. Avoid mandates that count AI usage as a goal in itself; the goal is delivering good software, and sometimes the right call is not to use an agent.
Habits, ownership, learning
Shared conventions, clear ownership and deliberate learning keep AI-assisted work healthy.
A rollout checklist
Use as a starting point and adapt.
[ ] pilot group + low-risk tasks + sandboxed environment
[ ] shared AGENTS.md / instruction file reviewed like code
[ ] permission policy (allow / ask / deny) documented and enforced
[ ] server-side: protected main, required review, required CI, secret scanning
[ ] PR template: what the agent did, how it was verified, what a human checked
[ ] training: task writing, diff review checklist, failure modes
[ ] metrics: internal benchmark, lead time, rework, incidents
[ ] pause switch + incident process if an agent causes harmDisclose AI assistance in PRs
A line in the PR template about what the agent did and how it was verified helps reviewers focus.
Quick check: Where should the strongest controls on agent-written code live?
- Only in developers' memory
- Only in the agent's prompt
- On the server: protected branches, required reviews and CI
- Nowhere
Answer
On the server: protected branches, required reviews and CI — Server-side controls apply to everyone, including agents that ignore instructions.