Documentation, Roles and Team Practice

Write down what the system is and is not

LLM features outlive their authors, so document them. A short system card (or model/feature card) states: purpose and intended users, what it must not be used for, data used, the model and prompt versions, how it was evaluated (sets, metrics, results, known weaknesses), safety controls, human oversight, monitoring and who to contact. Keep a decision log of important choices and why (model selection, thresholds, trade-offs). Define roles: who owns prompts and releases, who reviews safety, who triages user reports, who is on call, who approves changes to the evaluation set. Run blameless reviews after incidents, hold regular quality reviews of sampled conversations, and share findings across teams. Train developers and support staff on the limits of LLMs so they can set accurate expectations with users. Good documentation also speeds onboarding and audits.

A one-page system card outline

Fill in for your feature and keep it next to the code.

Name / owner / contact:
Purpose and intended users:
Out of scope (do NOT use for):
Model, prompt and retrieval versions (release ID):
Data used (sources, personal data, retention):
Evaluation: sets, metrics, latest results, known weaknesses, languages covered
Safety controls: output handling, tool policy, approvals, rate/spend caps, red-team status
Human oversight and escalation path:
Monitoring: dashboards, alerts, feedback loop, review cadence
Change history / decision log:

Keep the system card next to the code

Documentation that lives with the code is more likely to stay current.

Quick check: Why include "out of scope" uses in a system card?

• There is no reason
• To shorten the document
• Because law forbids listing uses
• To prevent misuse of a feature in situations it was not tested or designed for