Threat Modeling for AI Agent Systems

U.S. organizations paid an average of $10.22 million per data breach in 2025, yet only 37% maintain formal threat modeling processes. The gap between breach cost and adoption rate is not a mystery. Threat modeling earned a reputation for heavyweight workshops, dense documentation, and security-team exclusivity, and engineering organizations voted with their feet. If you want the companion offensive view — attacking the agent you just modeled — start with How to Red Team an AI Agent, which covers the four-layer attack surface and a six-step playbook.

That reputation is outdated, and the stakes just went up. When you add AI agents, the system you are modeling stops being a set of components and becomes a set of components that act on their own: call tools, read memory, move data, send messages. Every one of those actions is a data flow, and most of them are undocumented.

This guide merges the two classic levels of threat modeling, engineering-level and architecture-level, and extends both to agent systems. You get the framework comparison, the integration patterns, and a decision matrix for where to start, plus the agent-specific surfaces that scanners cannot see.

Four questions, not four months

Every practical threat model answers four questions, regardless of which framework the team selects:

• What are we building? A data flow diagram or architecture sketch that defines the boundary.
• What can go wrong? Threats identified through a structured methodology.
• What are we going to do about it? Mitigations, accepted risks, and countermeasures.
• Did we do a good enough job? Validation through review, testing, and iteration.

The attacker's perspective is what separates threat modeling from a generic risk register. Risk assessment asks "what could go wrong and how bad would it be?" Threat modeling asks "how could an attacker exploit this system?" That shift changes the output from a list of hypothetical worries to an architecture-level understanding of attack surface, something automated scanners cannot provide.

The starting point is always the same: trace how data moves through the system. Data flows turn vague worries into specific questions. "What happens if this API response is tampered with?" "What if this model input is poisoned?"

For an agent system, the same exercise exposes a different surface. The data flows are: user prompt in, tool call out, tool result back in, memory write, memory read on the next turn, MCP server connection, credential retrieval. Each of those crosses a trust boundary, and most teams have never drawn them.

Risk assessment asks what could go wrong and how bad it would be. Threat modeling asks how an attacker would actually exploit the system. The perspective shift is the whole game.

Framework comparison: STRIDE, PASTA, LINDDUN, VAST

Four methodologies dominate the field. No single framework wins across every dimension; each optimizes for a different constraint. The 2024-2025 State of Threat Modeling Report, the first community-driven study with input from over 60 organizations, found that STRIDE remains the most common approach at 88% adoption, but most companies blend elements from three or more methods.

Framework	Focus	Best for	Output	Effort
STRIDE	Threat categorization	Teams getting started; application-level analysis	Classified threat list per component	Low-Medium
PASTA	Risk-centric attack simulation	Organizations needing business impact alignment	Attack tree + risk-ranked mitigations	High
LINDDUN	Privacy threat modeling	Systems handling PII, regulated data	Privacy threat list + compliance mapping	Medium
VAST	Agile, visual, scalable	Large orgs with many product teams	Threat-per-epic cards in backlogs	Low

STRIDE: the default starting point

STRIDE categorizes threats into six types: Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege. Each type maps to a security property it violates: authentication, integrity, non-repudiation, confidentiality, availability, and authorization. Microsoft created STRIDE in the late 1990s, and its longevity comes from a specific design decision: structured enough to be systematic, simple enough to teach in a 30-minute session.

A team whiteboards the architecture, walks each data flow through the six categories, and produces a threat list. No scoring required, no attack tree construction. The trade-off is that STRIDE identifies threats without ranking them, so a separate risk assessment step handles prioritization.

PASTA: attack simulation for business alignment

Process for Attack Simulation and Threat Analysis is a seven-stage methodology: define objectives, define technical scope, decompose the application, analyze threats, analyze vulnerabilities, enumerate and model attacks, then analyze risk and impact. Where STRIDE asks "what could happen here?", PASTA asks "how would an attacker actually do this, and what does it cost the business?"

The output is attack trees that link specific techniques to business consequences. The cost is 4-8 hours per session and genuine security expertise. Poor fit for a lightweight introduction, excellent when a decision needs a business case.

LINDDUN: when privacy is the primary concern

LINDDUN extends threat modeling to privacy: Linkability, Identifiability, Non-repudiation, Detectability, Disclosure of information, Unawareness, and Non-compliance. For systems handling personal data, it catches what STRIDE misses: purpose limitation violations, consent gaps, and data minimization failures. It maps directly to GDPR principles, which makes it the default for compliance-driven environments.

VAST: threat modeling at agile scale

Visual, Agile, and Simple Threat modeling was built for organizations where many product teams operate independently. Instead of a single centralized model, VAST produces per-feature threat cards that drop straight into sprint backlogs. It trades depth for coverage: each model is shallower than a PASTA analysis, but the organization gets visibility across every feature rather than deep analysis of a few.

The architecture layer: where agents actually break

Application-level threat modeling classifies threats per component. Architecture-level threat modeling asks a different question: what failure modes are baked into the structure of the system?

The Capital One breach of 2019 exposed 106 million records because a misconfigured WAF role in a cloud architecture allowed an SSRF to escalate into full S3 access. The vulnerability was not in the code; it was in the architecture. A trust boundary that should have been isolated across availability zones was collapsed into a single IAM role. No amount of application-layer scanning catches a structural failure like that.

NIST SP 800-160 Vol. 1 Rev. 1 makes the principle explicit: security is an emergent property of the system, not a feature that can be bolted on. The architect who draws the initial data flow diagram is making security decisions, whether they acknowledge it or not.

For agent systems, the architecture level is where the new attack surface lives. An agent is not one component; it is a topology: model, tools, memory, MCP servers, credentials, and the human in the loop. The architectural threat categories apply directly:

Threat category	Architectural scope	Agent-system example
Trust boundary collapse	Service-to-service, zone-to-zone, tenant isolation	One shared API key for every tool the agent can call
Data-flow exposure	Sensitive data crossing untrusted zones	Conversation history with secrets flowing to a third-party model
Cascading failure mode	Single points of failure propagating across dependencies	One MCP server that, when compromised, feeds every downstream tool
Implicit trust assumption	Components trusting each other without verification	Agent treating tool output as ground truth, which enables prompt injection
Supply-chain attack surface	Third-party dependencies, CI/CD, build systems	An unpinned MCP package pulling attacker-controlled code

Browser-facing surfaces get the same treatment. If any component of the system is a web app the agent (or its user) drives, its Content-Security-Policy is a trust boundary drawn in a header — and it deserves the same read-the-actual-value scrutiny as any credential. The CSP Header Analyzer parses a pasted policy and explains every finding, which is the fastest way to check whether the boundary you think you shipped is the one the browser enforces.

These overlap with STRIDE: trust boundary collapse is a form of elevation of privilege, and implicit trust assumptions enable spoofing. The difference is granularity. STRIDE analyzes a component's data flows. Architectural categories analyze the relationships between components and the assumptions embedded in the topology. An agent topology carries more of those assumptions than any traditional service graph.

An agent's tool call is a trust boundary crossing. Most systems treat it like an ordinary function call.

The trust boundary map and the failure cascade

An architect's threat model produces two structural artifacts, and both apply directly to agent systems:

• A trust boundary map. A diagram showing every boundary where privilege levels change: network zones, tenant boundaries, and for agents, the model-to-tool boundary, the memory boundary, and the MCP server boundary. Each boundary is labeled with its trust level and the controls that enforce separation.
• A failure cascade diagram. A directed graph showing which services depend on which, and what happens when each fails. For agents: what happens when the tool call returns garbage, when memory returns poisoned context, when the MCP server dies mid-task.

The agent topology as a trust boundary map. Every edge is a data flow; every dashed ring is a place where privilege changes. The tool-call boundary is the one to model first.

These two documents do what a component-level threat list cannot: they show where a single architectural decision amplifies risk across the entire system. A trust boundary map reveals when two zones that should be isolated share a credential. A failure cascade diagram reveals when a non-critical service's outage blocks a critical path, or when a compromised tool poisons the whole agent loop.

Agent-specific threat surfaces

Standard threat modeling was designed when systems did not act. Agents change four surfaces, and these are the ones to port into any existing program:

Tool-call boundaries. Every tool call is a small trust decision: is this the right tool, with the right arguments, for the right reason? Threat modeling asks this per data flow rather than per implementation. The mechanics of validating calls before they execute are covered in Validate Agent Tool Calls Before They Execute; threat modeling is what finds those boundaries before they exist in code.

Memory as a data flow. Agent memory is a write path, a read path, and a persistence layer. Poisoned memory is a tampering threat. Memory exfiltration is information disclosure. Memory that outlives its purpose is a data minimization violation. STRIDE catches the first two; LINDDUN catches the last.

MCP and tool supply chains. Every MCP server is a third-party dependency with execution power. You cannot meaningfully threat-model a service you do not control, so model the integration surface: APIs, schemas, auth flows, and what the server can do with the credentials it receives.

The human in the loop. Agents introduce a new trust relationship: the human who approves an action rarely verifies it. Repudiation (who authorized this?) and elevation of privilege (can the agent escalate its own permissions?) get genuinely new answers in agent topologies.

The agile integration pattern

The biggest barrier to adoption is not framework complexity; it is the perception that threat modeling is a separate, pre-development activity. The Martin Fowler and Thoughtworks threat modeling guide articulates the alternative: design little and often. Instead of one workshop before a project begins, integrate threat modeling into the regular development rhythm.

Integration point	When	Activity	Duration
Sprint planning	At the start of each sprint	Review data flow changes in planned work; identify new trust boundaries	15-30 min
Architecture review	When new services or integrations are proposed	Full STRIDE walkthrough on the new component	60-90 min
Pre-deploy gate	Before merging significant changes	Verify threat model updated; new threats have mitigations or accepted-risk labels	5-10 min

For agent features, the sprint planning slot is where you catch the new stuff: a new MCP server, a new tool, a memory policy change, a new model. If the data flow diagram does not update when the agent's toolset grows, the threat model is stale the week it is written.

Automation: from workshops to workflows

Manual threat modeling scales poorly. The IriusRisk community report found that the average organization faces 10 threat-modeling-related challenges and produces 10-100 threat models per year. At that volume, tooling is not optional.

Tool	Type	Key capability	Best for
OWASP Threat Dragon	Diagramming + generation	Draw DFDs, auto-generate STRIDE/LINDDUN threats via rule engine	Visual, methodology-agnostic teams
pytm (OWASP)	Programmatic (Python)	Define application models as Python; generate DFDs and threat reports	Code-as-model teams
IriusRisk Community	Diagramming + knowledge base	Pattern library, guided workflow	Pattern-driven teams
Threagile	Architecture-as-code (YAML)	Declarative architecture modeling; auto risk reports and DFDs	Teams versioning architecture in CI/CD

The emerging pattern is architecture-as-code: trust boundaries, data classifications, and dependencies defined in YAML or Python that live beside the source. When a service is added, the model updates, the generator runs, and new threats land in the backlog. Threat models that live in a wiki are always stale; threat models that live in the repository are always current.

For agent systems, the same pattern applies to the agent topology: model the tool list, the MCP servers, the memory policy, and the credential bindings in the same file as the agent config. That is exactly the kind of file the AI Agent Config Checker audits when it looks for leaked secrets and unbounded tool permissions.

Exceptions and limits

Threat modeling is not a universal solution. Several scenarios limit its effectiveness or require adaptation:

• Legacy systems without documentation. When the architecture exists only in tribal knowledge, drawing a data flow diagram becomes a reverse-engineering exercise. The model will be incomplete, so flag confidence levels on each component.
• Rapid prototyping and throwaway experiments. A spike that will not reach production does not need a threat model. The threshold: if it touches real user data or network-exposed endpoints, model it. An agent prototype that can call real tools touches both.
• Third-party SaaS with no visibility. You cannot meaningfully threat-model a service you do not control. The productive approach is to model the integration surface (APIs, webhooks, data sharing) rather than the provider's internals. For MCP servers, model what the server can do with the credentials it receives.
• Over-modeling. Trying to apply threat modeling to every component at once is the most common failure mode. Teams spend weeks on edge cases while a basic authentication flaw sits unaddressed. Scope to the highest-risk boundary first: for agents, that is the tool-call boundary and the memory boundary.

The honest assessment

Dimension	Strength	Limitation
Threat discovery	Finds architecture-level flaws that scanners miss	Quality depends on participants' security experience
Cost efficiency	Finding a design flaw costs 30-100x less than in production	Up-front time delays features in the short term
Scalability	Automated tools reduce per-model effort	Automation covers known patterns; novel threats require human analysis
Organizational buy-in	Quantified threat lists enable data-driven security investment	52% of organizations lack management reporting channels for threat models
Agent coverage	Catches boundaries scanners cannot see	Frameworks need adapting; no off-the-shelf "agent STRIDE" exists yet

The reporting gap deserves emphasis. Fifty-two percent of organizations doing threat modeling have no regular reporting to management, and only 25% have any kind of threat model dashboard. Threat modeling happens in isolation from the decision-makers who allocate budget and set priorities. The fix is structural: accepted risks and unmitigated threats must surface in the same artifacts where business decisions are made. Risk registers, board reports, and sprint retrospectives all qualify as integration points.

Actionable takeaways

• Start with data flows, not frameworks. Before choosing STRIDE or anything else, map how data enters, moves through, and exits the system, including every agent tool call and memory read. The diagram is the foundation; the framework is the lens.
• Pick STRIDE for the first model. It is the most teachable, the most widely adopted (88%), and the fastest to a useful output. Add PASTA for business impact or LINDDUN for privacy only when a specific need appears.
• Model in sprints, not separate workshops. A 15-minute threat review at sprint planning produces more value than a 4-hour annual workshop the team forgets by the next quarter.
• Draw the trust boundary map for the agent topology. Model-to-tool, agent-to-MCP, memory boundaries, credential bindings. A complete map reveals threats no methodology would have surfaced on its own.
• Automate the diagram, not the judgment. Tools generate threats from data flow diagrams, but deciding which threats matter is irreducibly human.
• Close the reporting loop. Attach accepted-risk items to the risk register and publish dashboards. Threat visibility is a leadership concern, not a security-team secret.
• Validate against production incidents. After every incident, check whether the failure mode was already in the threat model. If not, the boundary was missed. If yes, the mitigation was insufficient. Either way, the model improves.

The threat model does not prescribe the fix. It makes the need for the fix visible.

When you are ready for the first pass, the STRIDE Threat Model Worksheet walks a component list through the six threat categories and produces a per-element checklist to start from. That is a 10-minute version of the 30-minute session. Use it to find the first boundary worth drawing properly.