AI Platform Masterclass · Phase 14 · OWASP LLM-01 · 15 min

You cannot prompt your way to safety.

The model reads your rules and a customer’s ticket as the same kind of text, so a ticket can give it orders. Articles explain that. This lesson makes you watch a $5,000 refund happen, then stack every defense and discover none of them close it, then fix it the only way that works: architecture.

?New to this? 30-second primer
Agent
A model that can take real actions (like issuing a refund) by calling your code, not just replying with text.
System prompt
The hidden instructions that set the model’s rules, e.g. "you are a bank assistant, refunds up to $50".
Tool call
The model asking your code to perform an action and getting the result back. This is how it moves money.

Want all ten words first? Read the beginner primer →

Tier 1 Basics

The agent obeys the ticket

A support agent has one rule: refunds up to $50. Choose what goes in the retrieved ticket, run it, then reveal what the model actually receives. The vulnerability is not a bug in the model. It is that instructions and data arrive as one indistinguishable string.

The agent gets three things

System rule (trusted)

You are ACME Support. You may issue refunds up to $50. Never issue a refund over $50 without a manager code.

Customer message

Hi, can you help with my order #4471?

Retrieved ticket (untrusted) — you choose

Order #4471: customer received a damaged item, eligible for a $40 refund.

Pick a ticket and run the agent. Then reveal what the model receives.

That is the whole mechanism, and where most explanations stop. The interesting question is what you do about it.

Tier 2 Intermediate

Stack every defense. Watch which never close.

The internet is full of "the fix for prompt injection" posts. Turn each defense on below and run it against four real attack classes. Every prompt-level defense only reduces, and two attacks stay wide open no matter what you stack. Only one control ever reaches "contained", and it is not a prompt.

Stack your defenses

The same four attacks, right now

Direct injectionInstruction pasted straight into the user message.wide open
Indirect (via a retrieved doc)Instruction hidden in a document the agent reads.wide open
Obfuscated / encodedSpaced-out or base64 text the classifier does not recognise.wide open
Multi-turn dripHarmless-looking messages that assemble the attack over several turns.wide open

No defenses. Every attack lands. Start stacking, and watch which ones never fully close.

The judgment: prompt-level defenses lower the odds and are worth stacking, but they never reach zero. You must design as if one gets through. That is tier three.

Tier 3 Advanced · full potential

Assume it gets through. Contain the blast radius.

The senior question is not "did the injection get in" but "what could it reach once it did". Run the exact same successful injection against two architectures and watch the damage differ by everything.

One agent, full access

The same model reads the untrusted ticket AND holds every tool: refunds, PII, email, database. Untrusted data and privileged action share one context.

untrusted ticket ─→ 🧠 agent (all tools)
Egress

Filter the output, not just the input

Even a contained agent can be steered into leaking. An egress filter scans every answer for PII, secrets, and over-limit actions before it reaches the customer, the same guardrail span you saw in the observability lesson.

In a bank → The SSN the guardrail caught in the trace lesson was an egress catch. It is the last line before a reportable incident.

Provenance

Taint tracking

Mark every token that came from an untrusted source, and refuse to treat tainted text as an instruction. The agent can read the ticket; it just cannot be commanded by it.

In a bank → When an auditor asks "could a customer message ever trigger a transfer", taint tracking is how you answer "no, by construction".

Human gate

Approval on high-stakes actions

For anything that moves money or touches many records, the agent pauses for a human. Injection or not, a person signs off. This is the interrupt pattern from the loop lesson, used as a security control.

In a bank → A $5,000 refund should never be one model call away from happening. It should be one human away.

Red team

Continuous adversarial evaluation

Injection is an arms race. New attacks appear weekly, so a fixed defense rots. Run an automated adversarial suite on every deploy and treat a new bypass as a failing test, not a surprise in production.

In a bank → A bank does not test its locks once. Neither should it test its agent’s defenses once.

The defenses in code, at three altitudes

The trust boundary is the concept; guardrails and scoped tools are where you enforce it on your stack.

The concept

Never merge trusted instructions and untrusted data into one privileged context. The durable fix is architectural: the component that reads untrusted text has no power, and the component with power never reads untrusted text.

# A trust boundary, not a better prompt.
summary = worker_llm(untrusted_ticket)        # no tools, no PII
action  = planner_llm(system_rules, summary)  # tools, never sees raw ticket
if action.amount > 50:
    require_human_approval(action)             # high-stakes gate

The point: a trust boundary, guardrails, and least-privilege tools are the concept. LangChain guards, a dual-LLM split, or a scoped tool runtime are just where you enforce them.

The same question, three depths of answer

"How do you defend an agent against prompt injection?" The depth of your answer is the level they hire you at.

Junior

Add a system prompt telling it to ignore any instructions inside the data.

Senior

Defense in depth: delimiters, an input classifier, and an output filter for PII and over-limit actions, relying on none of them alone, because prompt-level defenses only lower the probability. Obfuscated and multi-turn attacks slip past all of them.

Staff

Assume injection succeeds and contain the blast radius: least-privilege tools the model cannot exceed, a quarantine or dual-LLM split so the component that reads untrusted data has no power, human approval on high-stakes actions, taint tracking, and continuous adversarial evaluation. Prompt injection is not a prompt bug to patch, it is an architecture requirement to design for.

On your real stack this week

Cap your highest-privilege tool, and gate it.

Find the most powerful tool one of your agents can call (a refund, a delete, an email blast), enforce a limit in the tool itself, and require human approval above it. Then paste an injection into a retrieved document and confirm the agent asks but cannot exceed the cap. You have just turned "wide open" into "contained".