LLM Application Security

Secure applications built on language models: prompt injection, unsafe output handling, excessive agency, data leaks, RAG access control, supply chain, abuse and cost attacks, testing and incident response, with attacks and defences you can run.

Start course →

Syllabus

Why LLM Apps Need Their Own Security Thinking

  1. What Is Different About LLM Applications
  2. Assets, Attackers and Trust Boundaries
  3. Blast Radius: What Can a Compromised Model Do
  4. A Map of the Main Risk Families

Prompt Injection and Jailbreaks

  1. Direct and Indirect Prompt Injection
  2. Why Keyword Filters Are Not Enough
  3. System Prompt Leakage and Canary Tokens
  4. Defence in Depth Against Injection

Insecure Output Handling

  1. The Rule: Treat Model Output Like User Input
  2. SQL Injection Through Model Output
  3. Cross-Site Scripting and Markdown/Link Exfiltration
  4. Command Injection and Path Traversal
  5. Tools That Fetch URLs: SSRF and Allow-Lists

Excessive Agency and Tool Security

  1. Over-Broad Tools, Permissions and Autonomy
  2. A Policy Engine Outside the Model
  3. Human Approval That Cannot Be Tampered With
  4. Per-User Authorisation and Audit Logging

Protecting Data: Disclosure, RAG and Poisoning

  1. Sensitive Information Disclosure and Redaction
  2. RAG Access Control and Vector Store Security
  3. Data Poisoning and Untrusted Knowledge Sources

Supply Chain, Abuse and Cost Attacks

  1. Models, Packages and Plugins as Dependencies
  2. Plugins, MCP Servers and Agent Tool Trust
  3. Unbounded Consumption: Rate Limits and Denial of Wallet

Testing, Monitoring and Response

  1. Red-Teaming and Security Regression Tests
  2. Monitoring, Detection and Logging
  3. Incident Response and Secure Development Habits

Putting It Together

  1. Case Study: Securing a Customer-Support Assistant
  2. Revision: Cheat Sheet and Self-Check