Prompt Injection Defenses to Require Before Signing an Agentic AI SOW

Most agentic AI pilots are pitched as productivity wins. The uncomfortable part shows up in the security review, not the demo: an LLM that can read email, call APIs, browse the web, and write to a database is a confused deputy waiting to happen. The recent survey of prompt injection against tool-using agents (arXiv 2508.16853) lays out why the usual guardrails are not enough, and why the interesting attacks arrive through data the agent was told to trust.

This piece is aimed at the person signing the SOW. It translates the research into concrete requirements a buyer should demand before greenlighting a build, whether the work is going to an outside vendor or an internal team. The threat model is not academic. Prompt injection sits at #1 on the OWASP Top 10 for LLM Applications for the second consecutive edition, precisely because LLMs process instructions and data in the same channel with no built-in separation.

What follows is the short list of defenses to require, and the sections of an SOW where each one belongs.

The Attack Surface Is the Tool Belt, Not the Prompt

Direct prompt injection ("ignore previous instructions") is the version most non-specialists picture. It is also the least of the problem in an agentic system. The higher-impact class is indirect prompt injection: adversarial instructions planted in a web page, a PDF, a support ticket, a calendar invite, an email signature, or the JSON returned by an external API. When the agent ingests that content as part of a task, the injected instructions become part of its working context and can trigger tool calls the user never asked for.

NIST recognized this shift explicitly. The March 2025 update to NIST AI 100-2 added dedicated coverage of autonomous agent vulnerabilities, including indirect injection through pages, documents, emails, and tool outputs, none of which appeared in the 2023 edition. The OWASP project followed with a separate Top 10 for Agentic Applications, announced at Black Hat Europe 2025, calling out goal hijacking, tool misuse, and memory or context poisoning as risks that come from autonomous decision-making, persistent memory, and tool access.

The practical implication for an SOW: the security section cannot only cover "the prompt." It has to cover every surface where untrusted bytes enter the agent's context window and every action the agent can take when they do.

Ask for an Explicit Trust Boundary Between Planner and Tools

The most durable architectural pattern in the survey is separation of the component that decides from the component that reads. Simon Willison's Dual LLM pattern, proposed in April 2023, pairs a privileged LLM that plans and calls tools with a quarantined LLM that processes untrusted content but has no tool access at all. Untrusted data never touches the planner directly. Google DeepMind's CaMeL extended this with capability-based information-flow tracking and, in initial AgentDojo evaluations, mitigated 67% of prompt injection attacks.

What to require in the SOW:

• An architecture diagram that names which model or process holds tool credentials and which one is allowed to read scraped or retrieved content.
• A written statement that raw untrusted text never enters the planner's context without a mediation layer (summarization by a quarantined model, structured extraction into typed fields, or capability tokens).
• A specific commitment that the planner does not use tool outputs as new instructions. Tool outputs are treated as data, not as continued dialog.

Vendors who cannot answer these questions with a diagram are proposing a monolithic agent. That is the design the research literature has spent two years documenting as brittle.

Least Privilege on Tools, Not Just on Users

Every tool the agent can call is a privilege the attacker inherits on a successful injection. The Microsoft Security Response Center's defense-in-depth writeup is blunt about this: combine fine-grained permissions with deterministic blocking of known exfiltration methods (markdown image injection is the canonical one) and human-in-the-loop consent for sensitive actions. Compliance frameworks are catching up. Article 15 of the EU AI Act requires high-risk AI systems to be resilient against unauthorized attempts to alter their outputs or performance by exploiting vulnerabilities, and that language will end up in procurement checklists whether the buyer is EU-based or not.

Concrete SOW requirements:

• A tool inventory with per-tool scopes, read vs. write classification, and the specific principal (service account, OAuth app) each tool authenticates as.
• Human-in-the-loop confirmation for any action with side effects a user would not undo casually: sending external email, moving money, deleting records, granting access, publishing content.
• Deterministic egress filters on any output rendered back to the user. Markdown image URLs, autolinks, and hidden Unicode characters should be stripped or normalized on the server, not the model.

The Unicode point is not theoretical. In August 2024, Johann Rehberger disclosed a Microsoft 365 Copilot vulnerability using ASCII smuggling: special Unicode characters invisible in the UI were embedded in clickable hyperlinks and used to exfiltrate personal data via prompt injection. If the render path does not sanitize, the model does not need to be jailbroken; it just needs to be asked politely by a document.

Require Provenance and Isolation for Retrieved Content

Retrieval-augmented generation and web browsing are the two most common vectors for indirect injection. The mitigation is to keep track of where every span of text in the context window came from and to treat non-first-party spans as data with limited authority. This is the same discipline that multi-tenant Postgres design demands: never let one tenant's payload cross into another tenant's execution path without an explicit check.

What that looks like in a build spec:

• Every chunk in a RAG index carries a provenance tag: source URI, ingestion time, trust class (first-party, contracted third-party, open web).
• The system prompt tells the planner to weight instructions found inside retrieved content at zero. Techniques like spotlighting (data marking with a rotating delimiter and metaprompt) belong here, not as the only line of defense but as one of several.
• Tool outputs are structured (JSON with a schema) wherever possible, so the parser can reject fields containing embedded instructions before they reach the model.

Buyers pitching an internal RAG build should also insist that the ingestion pipeline log every source, so a compromised document can be traced back and quarantined without rebuilding the entire index.

Test the System the Way an Adversary Would

Unit tests on the happy path prove nothing about agent security. The SOW should require adversarial evaluation on named benchmarks before shipping, and continuously after. AgentDojo, InjecAgent, and Agent Security Bench are the current touchstones in the literature; new work like the coding-assistant taxonomy catalogs 42 attack techniques and reports that most defenses achieve less than 50% mitigation against sophisticated adaptive attacks. That number is the reason no single control is sufficient, and the reason the SOW should list several.

Required deliverables:

• A red-team report against at least one public benchmark, with attack success rate broken out by category (direct injection, indirect injection, tool misuse, data exfiltration).
• A regression harness the buyer owns, not a service the vendor keeps behind a login. Injection tests should run in CI on every prompt or tool change.
• A named process for handling new attack disclosures. When the next technique gets published, who owns the patch and what is the SLA?

Observability, Audit, and Rollback

Agent traffic looks nothing like traditional app logs. A useful trace records the user's original goal, every tool call with arguments and return values, every retrieved document with its provenance tag, and the model's plan at each step. Without that, incident response after a suspected injection is guesswork.

An honest SOW should commit to:

• Structured trace logs retained for a defined window, with PII handling that matches the rest of the contract.
• Reversible actions where feasible: soft-delete instead of hard-delete, draft-then-send instead of immediate send, staged writes with a review queue for anything above a risk threshold.
• A kill switch that disables tool execution while keeping the read-only assistant available, so incidents do not force a full outage.

Teams that have shipped this pattern before will not blink at these requirements. Teams that treat agent security as a prompt-engineering exercise usually will. That reaction is itself signal. For a broader view of when to graduate from a demo to a hardened build, the tradeoffs in AI proof of concept vs production pilot are worth reading alongside this checklist, and the general limits of prompt-driven development are covered in vibe coding limitations.

What to Do With the SOW in Hand

Read the security section first. If it does not name the planner/tool trust boundary, list tool scopes with principals, describe egress sanitization, and commit to adversarial evaluation on a named benchmark, the document is not ready to sign. Send it back with these six requirements attached and ask for a revised architecture diagram, not a revised paragraph. Most competent teams will welcome the specificity; it lets them scope the work honestly instead of absorbing security debt into a fixed price. For buyers assembling the team side of this equation, the practical guide to hiring an AI development team and the broader AI development services overview cover what senior agent work actually costs.

Agentic AI is genuinely useful. It is also a new class of system with a new class of failure mode, and the contract is the last place a buyer gets to insist on the controls that make it safe to deploy.