# Incident Response and Secure Development Habits — LLM Application Security

Source: https://www.geekswithgeeks.com/en/llm-security/o-response

> Prepare to contain, learn and improve.

## Plan the first hour before you need it

When an LLM incident happens (data leaked in an answer, a tool abused, a huge bill, a harmful output posted publicly), the response has the usual shape. **Contain**: have switches to disable a tool, a feature or the whole assistant, block a user, rotate exposed credentials and revert a prompt or model version. **Investigate**: use audit logs to see what was asked, what was retrieved, which tools ran and with whose permissions. **Notify**: follow your data-breach and customer-communication obligations. **Fix and test**: close the weakness (usually a missing control, not a better prompt), add the attack to the regression suite, and verify. **Learn**: run a blameless review and update the threat model. As habits: do a **threat model for each new capability**, review AI features in **code review** with a security checklist, keep **secrets out of prompts**, follow **least privilege**, keep **dependencies pinned**, and train the team on prompt injection. Align with the standards and laws that apply to you.

## An LLM incident runbook

Fill in owners and switches for your own system.

```text
1. CONTAIN   disable feature flag "assistant-tools"; block user/IP; rotate exposed keys; pin previous prompt+model
2. PRESERVE  snapshot audit logs, prompts, retrieved chunks, tool calls (access-controlled)
3. ASSESS    what data/actions were reachable? which users affected? was it exfiltrated or only exposed?
4. NOTIFY    security lead, legal/privacy, affected customers as required by contract and law
5. FIX       add the missing control (permission, filter in retrieval, approval, cap); not just a prompt tweak
6. TEST      add the attack to the regression suite; re-run the full attack suite
7. LEARN     blameless review; update threat model and runbook
```

## Practise the kill switch

Run a drill: disable a tool in staging and confirm the app degrades gracefully.

**Quiz:** After an incident, what is usually the right kind of fix?

- [x] Adding the missing control (permissions, validation, limits) and a regression test
- [ ] Only rewording the system prompt
- [ ] Hoping it does not recur
- [ ] Deleting the logs

*Answer:* Adding the missing control (permissions, validation, limits) and a regression test. Controls outside the model are more reliable than wording.