What Is Different About LLM Applications

The model is a confused deputy by design

Traditional software separates code from data: a SQL driver knows which part is the query and which is the value. A language model has no such boundary. System instructions, the user's message, a retrieved document, a web page and a tool result all arrive as text in one prompt, and the model decides what to do from all of it. So any text the application feeds the model can act as an instruction. Add two more facts: the model's output is non-deterministic and attacker-influenced, and modern apps give the model power (tools, APIs, data access). Together these create new risks, on top of ordinary ones (authentication, injection, secrets, availability) that still apply. The practical stance: treat the model as an untrusted component that is often helpful, put hard controls outside it, and assume it can be tricked. This course covers the main risk families; see the OWASP Top 10 for LLM Applications for a maintained list, whose categories change over time.

Instructions and data in one channel

A language model reads instructions and data as one stream of text, so anything it reads can try to steer it.

A helpful assistant who reads every note

Imagine an eager assistant who follows any written instruction on any paper that crosses the desk, including a note slipped into a customer letter. You would never give that person the safe key. LLM apps need the same care about what the assistant can reach.

Quick check: What is the core reason prompt injection is possible?

• Prompts are too short
• Models run on weak hardware
• The model receives instructions and untrusted data as one stream of text
• Because of slow networks