Documentation and Auditability

If it is not written down, it did not happen

Auditors, regulators, customers and your own future team will ask: what is this system for, what data trained or informed it, how was it tested, who approved it, what changed since, and what incidents occurred? Keep system cards, impact assessments, evaluation reports, approval records, change logs (prompt, model and data versions) and incident records in a place that is retrievable and access-controlled. Aim for evidence produced as a by-product of normal work, not a scramble before an audit.

A change-log entry

Small, consistent entries make it possible to trace a change in behaviour back to its cause.

{
  "date": "2026-09-18",
  "system": "support-assistant",
  "change": "prompt v14 -> v15; retrieval top_k 4 -> 6",
  "reason": "reduce unsupported answers on billing questions",
  "eval": {"cases": 250, "grounded_correct": "91% -> 93%", "hindi_slice": "84% -> 88%"},
  "approved_by": "AI review board (ticket GOV-212)",
  "rollback": "revert to prompt v14, top_k 4"
}

Link evidence to decisions

A pile of documents is not an audit trail. Link each approval to the evaluation results and assessment it relied on, so a reviewer can follow the reasoning in minutes.

Quick check: What makes documentation useful for audits?

• Deleting old records
• A very long single file
• Verbal agreements only
• Linked, versioned evidence of what was tested, approved and changed