Lesson 21 / 25
Audit Logging
Record every action, decision and reason in a form you can search later.
What happened, and why was it allowed?
Log every tool call as one structured line: time, session, tool, a redacted command or path, the decision (allow, ask, deny) and the reason. After an incident this answers "what exactly ran?" in minutes. Store logs where the agent cannot edit them, keep them for a sensible period and never write secrets into them.
See it, test it, recover
You need a record of what the agent did, tests that attack your guardrails, and a plan for when something slips.
One log line, run
I ran this. The token in the command was redacted before logging. Note the redaction pattern also removed the rest of that token, so the line looks truncated; that is the safe direction to be wrong in.
import json, re
def redact(t): return re.sub(r"(?i)(password|secret|token)\s*[=:]\s*\S+", "[REDACTED]", t)
line = json.dumps({
"tool": "bash",
"cmd": redact("curl -H 'token=abc123' x"),
"decision": "deny",
"reason": "network",
}, sort_keys=True)
print(line)
Output:
{"cmd": "curl -H '[REDACTED] x", "decision": "deny", "reason": "network", "tool": "bash"}Alert on patterns, not only on events
A single denied command is normal. Twenty denials in a minute, a denied attempt to read .env, or an attempt to reach an unknown host deserve an alert and a human look.
Quick check: What should each audit log line include besides the action?
- The agent's password
- The decision and the reason
- A joke
- Nothing else
Answer
The decision and the reason — Knowing why an action was allowed or denied is essential for improving the policy.