Lesson 22 / 27

Protecting Keys and Logs

Prevent accidental leaks through logs, errors and prompts.

Leaks happen in logs

Keys leak less often through hacking than through carelessness: printed in debug logs, included in error reports, committed to Git, pasted in tickets. Defences: load keys from the environment or a secrets manager; redact anything that looks like a key or token before logging (and redact personal data in prompts and replies); never put keys in prompts or tool results; use separate keys per environment and rotate them; run secret scanning in CI and pre-commit hooks; and restrict who can read production logs. Remember that logging full prompts and replies can capture customers' personal data, so decide what you need, how long to keep it and who can see it.

Protect keys, data and users; test without a model

Keep secrets out of logs, treat model output as untrusted, and test your integration offline.

Four habits: protect, validate, observe, test.
Figure 7.1 — Protect, validate, observe and test.

Redacting keys from log lines, run

I ran this plain-Python (standard library only) example. A regular expression replaces sk-... style keys and Bearer ... tokens with [REDACTED] before the line is logged. A pattern like this is a safety net, not a substitute for keeping keys out of logs in the first place.

import re

KEY = re.compile(r"(sk-[A-Za-z0-9_-]{6,}|Bearer\s+[A-Za-z0-9._-]+)")

def redact(text):
    return KEY.sub("[REDACTED]", text)

log_line = "calling api with Authorization: Bearer sk-live-abc123XYZ and key sk-test-123456"
print(redact(log_line))

Output:

calling api with Authorization: [REDACTED] and key [REDACTED]

Quick check: What is the best protection against logging a key?

  • Put it in the URL
  • Log everything and hope
  • Print it once for convenience
  • Never include it in anything you log, and redact as a safety net
Answer

Never include it in anything you log, and redact as a safety net — Prevent the leak at the source; redaction catches mistakes.