🔬 AI red teaming is now a regulatory checkbox, a procurement requirement, and a baseline engineering practice — and in 2026, the OWASP AI Testing Guide v1 is the standard that defines what it looks like. This practitioner guide covers the complete OWASP AI Testing checklist organized by control category, LLM-specific controls, a remediation framework for every flagged vulnerability, a pre-deployment testing workflow, a 5-step testing process with specific tool recommendations at each step, and a practical AI governance checklist application for security and governance teams.
Last Updated: October 23, 2026
Released on November 26, 2025 as Version 1, the OWASP AI Testing Guide represents a paradigm shift that practitioners working under older frameworks need to understand. AI systems fail probabilistically. They can be secure against hackers but still produce toxic content, hallucinate facts, or leak training data. Traditional penetration testing methodologies — built for deterministic software — do not capture this failure mode. The OWASP AI Testing Guide v1 fills that gap: it is the first open, community-driven standard for trustworthiness testing of AI systems, covering security, reliability, fairness, and responsible deployment in a single unified framework.
You will find a complete copy-paste testing checklist organized by the guide’s four architectural layers, a dedicated OWASP AI checklist of actionable controls by category, LLM-specific and agentic controls mapped directly to the guide’s threat model, a remediation framework for the most common LLM vulnerabilities flagged by the checklist, a pre-deployment AI testing workflow, and a 5-step testing process with specific tool recommendations at each step. A new section in this 2026 update also covers how governance and security teams can apply the OWASP AI Testing Guide v1 as a practical AI governance checklist — including mapping to the OWASP LLM Top 10 2025, a prioritized procurement audit format, and ownership assignment across security, governance, and model development teams.
The guide is technology-neutral and model-agnostic. It applies to proprietary LLMs, open-weight models, fine-tuned deployments, and agentic AI systems with equal validity. Every test case follows a consistent structure: Define Objective → Execute Test → Interpret Response → Recommend Remediation. This repeatable methodology makes the guide suitable not just for red team exercises but for ongoing compliance documentation, vendor assessment, and regulatory audit evidence — which is why governance teams are increasingly adopting it alongside security practitioners.
📖 New to AI security terminology? Visit the AI Buzz AI Glossary — 100+ essential AI terms explained in plain English, each linking to a full in-depth guide.
🔐 Using the OWASP AI Testing Guide v1 as an AI Governance Checklist: Practical Implementation for Security Teams
Most security practitioners arrive at the OWASP AI Testing Guide v1 looking for a testing methodology. Governance teams, compliance officers, and procurement committees often arrive looking for something the guide’s title does not explicitly promise: a structured audit framework they can use to assess AI systems, evaluate vendors, and produce regulatory compliance documentation. Both audiences are correct — the guide functions as both. This section makes the governance application explicit, converting the OWASP AI Testing Guide v1’s test scope into the four practical implementation formats that governance teams need in 2026.
The Dual-Use Case: Testing Standard and Governance Audit Framework
The OWASP AI Testing Guide v1 was built by security practitioners to define what rigorous AI testing looks like. But its structure — four layers, technology-neutral test cases, repeatable methodology, and explicit risk categorization — maps directly onto the requirements of AI governance frameworks including the NIST AI Risk Management Framework, the EU AI Act’s conformity assessment requirements, and internal AI audit programmes. The guide does not require adaptation to function as a governance checklist: the test categories define what must be assessed, the pass/fail structure defines what “compliant” means, and the remediation framework defines what remediation evidence looks like for audit purposes.
The practical governance application diverges from the pure testing application in one key way: governance teams need to know not just what to test, but when each test category must be completed, who owns the testing responsibility, and what documentation satisfies a regulator or procurement committee that the test was performed. The four implementation areas below address each of these requirements directly.
Implementation Area 1: Pre-Deployment AI Governance Gate
The OWASP AI Testing Guide v1 covers test categories that fall into two distinct deployment timing windows: tests that must be completed and passed before any AI system enters production, and tests that transition into ongoing post-deployment monitoring cadences. Governance teams that fail to make this distinction end up either blocking deployment unnecessarily on monitoring items or — more commonly — permitting deployment of systems with unresolved pre-deployment risks because the testing checklist was treated as a flat list rather than a tiered gate.
The pre-deployment governance gate should be structured as a go/no-go decision with three tiers:
- Tier 1 — Deployment Blocked (Critical): Test categories where a failure result must block deployment until remediation is verified. These include prompt injection testing (AITG-APP-01, AITG-APP-02), sensitive data leak testing (AITG-APP-03), training data poisoning assessment (AITG-MOD), and supply chain integrity verification (AITG-INF). A system that fails any Tier 1 test is not permitted to enter production under any governance-compliant programme.
- Tier 2 — Documented Risk Acceptance Required (High): Test categories where a failure result requires documented risk acceptance by a named authority before deployment proceeds. These include output safety testing (AITG-APP-05), model robustness testing against adversarial inputs, and data pipeline integrity testing (AITG-DAT). Risk acceptance must be signed, time-limited, and tied to a named remediation owner and timeline.
- Tier 3 — Post-Deployment Monitoring (Medium): Test categories that transition into scheduled monitoring cadences after deployment. These include data drift detection, fairness and bias monitoring, and model performance degradation testing. Post-deployment monitoring items must have a defined review cadence (monthly, quarterly) and a named monitoring owner before deployment is permitted — but a clean result at deployment is not required.
Implementation Area 2: Mapping to the OWASP LLM Top 10 2025
The OWASP AI Testing Guide v1 is explicitly designed to test for the risks catalogued in the OWASP LLM Top 10 2025. The two documents are complementary: the LLM Top 10 defines the risk taxonomy, and the AI Testing Guide defines the testing methodology for each risk. Governance teams documenting compliance with the LLM Top 10 can use the testing guide’s test categories as evidence that each risk has been assessed — provided the mapping is made explicit in the compliance record.
| OWASP LLM Top 10 2025 Risk | OWASP AI Testing Guide v1 Test Area | Test Category Code | Governance Documentation Requirement |
|---|---|---|---|
| LLM01:2025 — Prompt Injection | Prompt Injection Testing (Direct & Indirect) | AITG-APP-01, AITG-APP-02 | Pre-deployment test results + evidence of guardrail implementation. Tier 1 — deployment blocker on failure. |
| LLM02:2025 — Sensitive Information Disclosure | Sensitive Data Leak Testing, Input Leakage | AITG-APP-03, AITG-APP-04 | Pre-deployment PII/confidential data exfiltration test results. Tier 1 — deployment blocker on failure. |
| LLM03:2025 — Supply Chain Vulnerabilities | AI supply chain integrity and model provenance verification | AITG-INF (Infrastructure Layer) | Model provenance documentation, third-party dependency audit, signed supply chain bill of materials. Tier 1. |
| LLM04:2025 — Data and Model Poisoning | Training Data Poisoning Assessment, Data Pipeline Integrity | AITG-MOD, AITG-DAT | Training data lineage documentation, poisoning test results, data validation controls evidence. Tier 1. |
| LLM05:2025 — Improper Output Handling | Unsafe Output Testing, Toxic Output Assessment | AITG-APP-05 | Output validation controls, downstream injection risk evidence, output sanitisation implementation. Tier 2. |
| LLM06:2025 — Excessive Agency | Excessive Agency / Agentic Scope Testing | AITG-APP (Agentic Controls) | Permission boundary documentation, human-in-the-loop control evidence, agentic scope test results. Tier 2. |
| LLM07:2025 — System Prompt Leakage | Input Leakage, Prompt Confidentiality Testing | AITG-APP-04 | System prompt confidentiality test results, instruction leakage prevention controls. Tier 2. |
| LLM08:2025 — Vector and Embedding Weaknesses | Model Layer — Embedding Security, Adversarial Input Testing | AITG-MOD | Embedding access control documentation, adversarial robustness test results. Tier 2. |
| LLM09:2025 — Misinformation | Hallucination Testing, Factual Accuracy Assessment | AITG-APP (Hallucination Controls) | Hallucination rate baseline, RAG grounding evidence, human review controls for high-stakes outputs. Tier 3. |
| LLM10:2025 — Unbounded Consumption | Infrastructure Layer — Resource Exhaustion, DoS Resilience Testing | AITG-INF | Rate limiting controls, resource cap documentation, DoS resilience test results. Tier 3. |
For the full risk taxonomy that the OWASP AI Testing Guide v1 is designed to test against, including remediation guidance for each of the ten risk categories, see our detailed breakdown of the OWASP LLM Top 10 2025.
Implementation Area 3: AI Governance Checklist Format for Vendor Assessment and Internal Audit
The following checklist converts the OWASP AI Testing Guide v1’s test scope into the prioritized format governance teams need for vendor assessments, internal AI audits, and regulatory compliance documentation. Each item maps to one or more guide test categories. The priority classification aligns with the deployment gate tiers defined in Implementation Area 1.
| ☐ Governance Checklist Item | Priority | Evidence Required |
|---|---|---|
| ☐ Prompt injection testing completed (direct and indirect) — AITG-APP-01/02 | 🔴 Critical | Signed test report with pass/fail result and guardrail implementation evidence |
| ☐ Sensitive data leak and input leakage testing completed — AITG-APP-03/04 | 🔴 Critical | Test results confirming PII and confidential data cannot be extracted via prompt manipulation |
| ☐ Training data poisoning assessment completed — AITG-MOD/DAT | 🔴 Critical | Training data lineage documentation and poisoning detection controls |
| ☐ Supply chain integrity verified — model provenance and third-party dependency audit | 🔴 Critical | Model provenance record, dependency audit, signed supply chain bill of materials |
| ☐ Unsafe and toxic output testing completed — AITG-APP-05 | 🟠 High | Output safety test results and output sanitisation implementation evidence — or signed risk acceptance |
| ☐ Excessive agency / agentic scope boundary testing completed | 🟠 High | Permission boundary documentation and human-in-the-loop control evidence |
| ☐ System prompt confidentiality / input leakage confirmed — AITG-APP-04 | 🟠 High | Prompt extraction resistance test results; instruction leakage prevention controls documented |
| ☐ Model robustness / adversarial input testing completed — AITG-MOD | 🟠 High | Adversarial robustness test results and embedding access control documentation |
| ☐ Incident response plan documented for AI-specific vulnerabilities | 🟠 High | See AI incident response plan — must cover prompt injection breach, data leak, and model manipulation scenarios |
| ☐ Hallucination rate baselined and RAG grounding controls documented | 🟡 Medium | Hallucination rate measurement methodology and human review protocols for high-stakes output categories |
| ☐ Data drift detection monitoring cadence established — AITG-DAT | 🟡 Medium | Monitoring tool, review cadence, alert threshold, and named monitoring owner documented before deployment |
| ☐ Fairness and bias monitoring cadence established | 🟡 Medium | Bias evaluation methodology, protected group coverage, review cadence and responsible owner |
| ☐ Resource exhaustion and DoS resilience testing completed — AITG-INF | 🟡 Medium | Rate limiting controls, resource caps, and DoS resilience test results |
Implementation Area 4: Ownership Assignment Across Teams
The most common governance failure in AI testing programmes is not a missing test — it is unclear ownership. When prompt injection testing is “everyone’s responsibility,” it reliably becomes no one’s. The OWASP AI Testing Guide v1 covers test categories that span the security team, the AI governance team, and the model development team. Governance programmes that do not assign explicit primary ownership for each test category before deployment will find that post-deployment audits surface ownership gaps rather than test results.
| Test Category | Primary Owner | Review Cadence | AITG Code |
|---|---|---|---|
| Prompt Injection (Direct & Indirect) | Security Team | Pre-deployment + after every major model update | AITG-APP-01/02 |
| Sensitive Data Leak / Input Leakage | Security Team | Pre-deployment + quarterly | AITG-APP-03/04 |
| Unsafe / Toxic Output Testing | Security Team + AI Governance Team (joint sign-off) | Pre-deployment + monthly sampling | AITG-APP-05 |
| Excessive Agency / Agentic Scope | AI Governance Team | Pre-deployment + after every agentic workflow change | AITG-APP (Agentic) |
| Training Data Poisoning Assessment | Model Development Team | Pre-deployment + after every retraining run | AITG-MOD |
| Model Robustness / Adversarial Input | Model Development Team + Security Team | Pre-deployment + semi-annually | AITG-MOD |
| Supply Chain Integrity / Provenance | Security Team + AI Governance Team | Pre-deployment + annually (vendor changes trigger immediate review) | AITG-INF |
| Data Pipeline Integrity / Drift Detection | Model Development Team | Continuous monitoring — monthly review cadence minimum | AITG-DAT |
| Fairness / Bias Monitoring | AI Governance Team | Quarterly — accelerated if demographic shift in user population detected | AITG-DAT / AITG-MOD |
| Infrastructure / DoS Resilience | Security Team | Pre-deployment + semi-annually | AITG-INF |
Governance Checklist Application — Key Principle: The OWASP AI Testing Guide v1 produces test results. Your governance programme must convert those results into three artefacts: a signed deployment decision record, a named remediation owner for every failed test, and a scheduled review cadence for every post-deployment monitoring item. Test results without these three artefacts are not governance evidence — they are audit findings waiting to happen. For the full risk taxonomy the testing guide is designed to cover, see the OWASP LLM Top 10 2025. When a test surfaces a live vulnerability requiring immediate escalation, your AI incident response plan should be the next document your team opens.
🏗️ 1. The OWASP AI Testing Guide v1: Four-Layer Architecture
The OWASP AI Testing Guide v1 organizes its test scope across four architectural layers. Each layer represents a distinct attack surface with its own test methodology, tooling, and ownership requirements. Understanding the four-layer structure is the prerequisite for using the guide correctly — practitioners who treat it as a flat list of tests will miss coverage gaps that only become visible when tests are assigned to layers.
Layer 1: Application Layer (AITG-APP)
The Application Layer covers the behavioral and output surface of the AI system — everything that happens between user input and model output from the perspective of the application. This is where prompt injection, output safety, data leakage, and excessive agency tests live. It is the layer most directly relevant to production security and the one that governance teams most commonly ask about first, because it is the layer that end users and adversaries interact with directly.
The AITG-APP test categories include: direct prompt injection (AITG-APP-01), indirect prompt injection via external content (AITG-APP-02), sensitive data leak via output (AITG-APP-03), input leakage and system prompt extraction (AITG-APP-04), unsafe output and toxic content generation (AITG-APP-05), and excessive agency and agentic scope boundary violations. For systems with agentic capabilities — tool-use, code execution, API access, multi-step reasoning chains — the Application Layer also covers the agentic control surface, including permission boundary testing, action scope verification, and human-in-the-loop bypass attempts. Organizations running autonomous AI agents should also review the non-human identity security requirements that govern agent credentials and access tokens operating outside direct human supervision.
Layer 2: Model Layer (AITG-MOD)
The Model Layer covers the integrity and security of the model itself — training, fine-tuning, weights, and the internal mechanisms that determine model behavior. Test categories include: training data poisoning assessment (verifying that the training pipeline has not been compromised to introduce backdoors or biases), adversarial robustness testing (evaluating how the model responds to deliberately crafted adversarial inputs), model inversion and membership inference attacks (testing whether the model leaks information about its training data in recoverable form), and embedding security (verifying that vector representations cannot be exploited to extract sensitive training content).
The Model Layer is the layer where ownership most commonly falls to the model development team rather than the security team — but governance programmes that assign Model Layer testing exclusively to model developers without security team review create a blind spot. Adversarial robustness testing in particular requires red team expertise that model developers may not have. The ownership table in the governance checklist section above specifies joint ownership for model robustness testing specifically to address this gap.
Layer 3: Data Layer (AITG-DAT)
The Data Layer covers the integrity, privacy, and quality of the data pipeline that feeds the AI system — from raw data ingestion through preprocessing, labeling, and validation into the model training or inference context. Test categories include: data pipeline integrity verification (ensuring that data entering the training or fine-tuning process has not been tampered with), data quality assessment (identifying distribution shifts, labeling errors, and class imbalances that can introduce systematic bias), privacy compliance verification (confirming that training data does not contain PII that should have been filtered, redacted, or anonymized before model training), and data drift detection (monitoring for shifts in incoming data that degrade model performance over time).
The Data Layer is the layer most closely connected to regulatory compliance in high-risk AI deployments. Under the EU AI Act’s high-risk AI provisions (Annex III, deferred to December 2, 2027 per Regulation 2026/1744), data governance documentation for training datasets is a mandatory conformity assessment requirement. AITG-DAT test results and data lineage documentation are the primary evidence artifacts for satisfying this requirement.
Layer 4: Infrastructure Layer (AITG-INF)
The Infrastructure Layer covers the deployment environment — the servers, APIs, containerized environments, orchestration systems, and network controls that the AI system runs on. Test categories include: supply chain integrity and model provenance verification (confirming that model weights, libraries, and dependencies sourced from third parties have not been tampered with in the supply chain), API security testing (evaluating authentication, authorization, rate limiting, and input validation on AI service endpoints), resource exhaustion and denial-of-service resilience testing (verifying that the system cannot be degraded or taken offline via token flooding or computational exhaustion attacks), and secrets management verification (confirming that API keys, model access credentials, and inference endpoint credentials are stored and transmitted securely). For a detailed treatment of AI supply chain risk specifically, including third-party model sourcing risk and dependency chain integrity, the Infrastructure Layer’s supply chain test categories are the starting point for any supply chain security programme.
📋 2. OWASP AI Testing Checklist: Copy-Paste Format by Layer
The checklist below is organized by the four AITG layers. Each item maps directly to a test category in the OWASP AI Testing Guide v1. Copy, paste, and adapt to your organization’s testing programme. Items are pre-sorted by layer and priority within each layer — Critical items that must pass before deployment appear first within each layer section.
Application Layer Checklist (AITG-APP)
| ID | ☐ Test Item | Priority | Test Objective |
|---|---|---|---|
| APP-01 | ☐ Direct prompt injection testing | 🔴 Critical | Verify that direct user input cannot override system instructions, bypass safety guardrails, or escalate privileges |
| APP-02 | ☐ Indirect prompt injection testing via external content | 🔴 Critical | Verify that malicious instructions embedded in external content (web pages, documents, emails) processed by the model cannot hijack model behavior |
| APP-03 | ☐ Sensitive data exfiltration via model output | 🔴 Critical | Verify that the model cannot be prompted to reproduce PII, training data, confidential documents, or other sensitive information in its outputs |
| APP-04 | ☐ System prompt / input leakage testing | 🟠 High | Verify that system prompt contents, instruction context, and other upstream inputs cannot be extracted through targeted prompt manipulation |
| APP-05 | ☐ Unsafe and toxic output generation testing | 🟠 High | Verify that guardrails prevent generation of harmful, illegal, or policy-violating content across a comprehensive adversarial test prompt library |
| APP-06 | ☐ Excessive agency and agentic permission boundary testing | 🟠 High | Verify that agentic systems cannot take actions beyond their defined permission scope, including unauthorized tool calls, file system access, or external API calls |
| APP-07 | ☐ Hallucination rate baseline and factual accuracy assessment | 🟡 Medium | Establish hallucination rate baseline across domain-specific evaluation sets; confirm RAG grounding controls are functioning at the intended accuracy threshold |
| APP-08 | ☐ Output downstream injection risk assessment | 🟡 Medium | Verify that AI-generated outputs that flow into downstream systems (code execution, SQL queries, HTML rendering) are sanitized before use |
Model Layer Checklist (AITG-MOD)
| ID | ☐ Test Item | Priority | Test Objective |
|---|---|---|---|
| MOD-01 | ☐ Training data poisoning detection and pipeline integrity | 🔴 Critical | Verify that the training data pipeline includes integrity controls preventing unauthorized modification of training datasets before or during ingestion |
| MOD-02 | ☐ Backdoor and trojan behavior testing | 🔴 Critical | Verify that the model does not contain hidden trigger behaviors — outputs that diverge from expected behavior when specific adversarially chosen inputs are presented |
| MOD-03 | ☐ Model inversion and membership inference attack testing | 🟠 High | Assess whether targeted queries can recover training data content (model inversion) or determine whether specific records were in the training set (membership inference) |
| MOD-04 | ☐ Adversarial robustness testing | 🟠 High | Evaluate model performance stability against deliberately crafted adversarial inputs designed to cause misclassification, output degradation, or unexpected behavior |
| MOD-05 | ☐ Embedding security and vector access control | 🟠 High | Verify that vector embeddings stored in retrieval databases cannot be exploited to reconstruct sensitive source content or bypass content access controls |
| MOD-06 | ☐ Fairness and bias evaluation across protected groups | 🟡 Medium | Evaluate model output disparities across demographic groups; establish bias baseline metrics and monitoring thresholds before deployment |
Data Layer Checklist (AITG-DAT)
| ID | ☐ Test Item | Priority | Test Objective |
|---|---|---|---|
| DAT-01 | ☐ Training data lineage and provenance verification | 🔴 Critical | Verify complete lineage documentation for all training datasets — source, collection method, consent basis, filtering applied, and version control |
| DAT-02 | ☐ PII and sensitive data presence verification in training data | 🔴 Critical | Verify that PII redaction, anonymization, or pseudonymization controls applied to training data are functioning correctly and completely |
| DAT-03 | ☐ Data quality assessment — distribution, class balance, labeling | 🟠 High | Evaluate training dataset for distribution shifts, class imbalances, and labeling inconsistencies that may introduce systematic bias or performance degradation |
| DAT-04 | ☐ Inference-time data pipeline integrity verification | 🟠 High | Verify that data flowing into the model at inference time (user inputs, RAG retrieval results, tool outputs) cannot be tampered with in transit |
| DAT-05 | ☐ Data drift monitoring cadence and alert thresholds established | 🟡 Medium | Confirm that statistical drift detection is active on incoming inference data, with defined alert thresholds and a documented response process |
Infrastructure Layer Checklist (AITG-INF)
| ID | ☐ Test Item | Priority | Test Objective |
|---|---|---|---|
| INF-01 | ☐ Model provenance and supply chain integrity verification | 🔴 Critical | Verify cryptographic integrity of model weights, third-party libraries, and inference dependencies — confirm that no supply chain tampering has occurred between source and deployment |
| INF-02 | ☐ API authentication, authorization, and input validation | 🔴 Critical | Verify that AI service API endpoints enforce authentication, authorization, and input validation — unauthenticated access to inference endpoints is a Critical failure |
| INF-03 | ☐ Secrets management — API keys, model credentials, endpoint tokens | 🔴 Critical | Verify that API keys and model access credentials are stored in a secrets manager, are not hardcoded in application code or environment variables, and are rotated on schedule |
| INF-04 | ☐ Rate limiting and resource cap verification | 🟠 High | Verify that rate limiting controls are active at the API gateway and inference endpoint level — preventing token flooding, compute exhaustion, and financial denial-of-service attacks |
| INF-05 | ☐ Network segmentation and AI workload isolation | 🟠 High | Verify that AI inference workloads are network-segmented from other production systems — lateral movement from a compromised inference endpoint should not reach core application infrastructure |
| INF-06 | ☐ DoS resilience testing under simulated load | 🟡 Medium | Simulate high-volume query load to verify that the AI system degrades gracefully under stress and does not expose unhandled error states that reveal system architecture |
🤖 3. LLM-Specific Controls: What the OWASP AI Testing Guide v1 Adds Beyond Traditional Security Testing
Traditional application security testing methodologies — OWASP Web Security Testing Guide, PTES, OSSTMM — were built for deterministic software that produces consistent outputs given the same inputs. Large language models are probabilistic: the same input can produce different outputs across invocations, guardrail bypass techniques evolve continuously, and failure modes are emergent rather than fixed. The OWASP AI Testing Guide v1 adds a set of LLM-specific controls that have no direct equivalent in traditional penetration testing methodology.
Jailbreaking and Guardrail Bypass Testing
Jailbreaking — the use of adversarially crafted prompts to bypass a model’s safety guardrails — is the Application Layer threat category with the highest active exploit velocity in 2026. The OWASP AI Testing Guide v1 requires testing against a comprehensive jailbreak prompt library covering: role-play and persona injection (instructing the model to adopt an alternative identity without safety constraints), token manipulation and encoding attacks (using character substitution, Unicode normalization exploits, or language switching to evade keyword-based guardrails), many-shot priming (using large numbers of example inputs to shift model behavior before the target request), and DAN-style override patterns. Testing should be conducted against both the base model guardrails and any application-layer filtering implemented on top of the model — each layer must be tested independently, because a guardrail failure at one layer is not compensated by a passing result at another.
Practitioners evaluating multiple frontier models as candidates for deployment should review how Claude, ChatGPT, and Gemini compare on safety architecture and guardrail design — the structural differences between these models’ safety approaches are directly relevant to which jailbreak categories represent the highest residual risk for each platform.
Prompt Injection in Multi-Agent and RAG Architectures
Indirect prompt injection — where malicious instructions are embedded in content that the model retrieves and processes as part of a RAG pipeline, tool-use workflow, or multi-agent task — is qualitatively different from direct prompt injection and requires separate test coverage. The attack surface for indirect injection scales with the number of external data sources the model can access: every document, web page, database record, email, or API response that flows into the model’s context window is a potential injection vector. Testing must enumerate every external content source in the system’s architecture and test each one independently for injection susceptibility. This is particularly critical for agentic systems where an indirect prompt injection in a retrieved document can trigger unauthorized tool calls that execute real-world actions — file deletion, email sending, API transactions — before a human reviewer can intervene.
Multimodal Attack Surface Testing
For AI systems that accept image, audio, or document inputs alongside text, the OWASP AI Testing Guide v1 extends prompt injection testing to the multimodal attack surface. Adversarial images — visually normal to a human observer but containing embedded text instructions readable by a vision model — can deliver indirect prompt injection payloads through the image input channel, bypassing text-based input filtering entirely. OCR-based document processing pipelines face the same risk: a PDF containing adversarially positioned text that a human reader would not notice can inject instructions that the model processes as legitimate context. Multimodal testing requires generating adversarial image payloads and testing the model’s response to embedded instructions across every supported input modality.
🕹️ 4. Agentic AI Controls: Testing Autonomous Systems
Agentic AI systems — those that use tool-use, code execution, multi-step planning, or multi-agent coordination to complete tasks with minimal human intervention — present an attack surface that the OWASP AI Testing Guide v1 addresses explicitly as a distinct test domain. The security stakes for agentic systems are qualitatively higher than for conversational AI: a compromised agentic system can take real-world actions (execute code, send requests to external APIs, read and write files, initiate financial transactions) before a human reviewer has the opportunity to intervene. The OWASP Agentic Application Security Initiative’s ASI01–ASI10 taxonomy — released as v2.01 on June 1, 2026 — should be reviewed alongside the AITG agentic controls, as the two frameworks are designed to be used together. Note that ASI01–ASI10 identifiers are distinct from LLM01–LLM10 and must not be conflated.
Permission Boundary and Least-Privilege Testing
Every agentic system should be tested to verify that it operates with the minimum permissions required to complete its defined task scope and cannot be manipulated into exercising permissions it possesses but should not use in a given context. Test scenarios must include: attempts to induce the agent to use a high-privilege tool that is available to it but not relevant to the current task, attempts to escalate the agent’s effective permissions through social engineering of a connected system or API, and verification that the agent refuses requests to perform actions outside its defined task scope even when instructed by an apparently authoritative source. The management of non-human identities assigned to agentic systems — the credentials, tokens, and API keys that agents use to authenticate to external systems — is the infrastructure control that bounds what the agent can do even if its prompt controls are bypassed.
Human-in-the-Loop Bypass Testing
Agentic systems designed to pause and request human approval before executing high-stakes or irreversible actions must be tested to verify that the approval gate cannot be bypassed. Test scenarios include: supplying a fabricated approval signal in the model’s context window, using indirect prompt injection to make the model believe approval has been granted, and verifying that the model correctly identifies the boundary between actions that require approval and those that do not across a range of edge cases. Approval gate failures in agentic systems are consistently high-severity findings — an agent that can be convinced it has received approval for an action it has not is functionally equivalent to an agent with no approval gate at all.
Multi-Agent Trust Boundary Testing
In multi-agent architectures where orchestrator agents direct sub-agents, the trust model between agents is a critical attack surface. Test scenarios must verify: that sub-agents do not unconditionally execute instructions from orchestrator agents without validating that the instruction falls within the sub-agent’s defined permission scope, that an adversarially compromised sub-agent cannot poison the orchestrator’s context with malicious content that influences subsequent orchestrator decisions, and that agent-to-agent communication channels are authenticated and integrity-protected. Unauthenticated agent-to-agent communication in a multi-agent system creates a lateral movement pathway analogous to unauthenticated internal API calls in a traditional microservices architecture.
🛠️ 5. Remediation Framework: Fixing What the Checklist Finds
A testing checklist without a remediation framework produces audit findings, not security improvements. The remediation guidance below maps each major AITG test category to its standard remediation approach, the evidence required to close the finding, and the re-test requirement before deployment sign-off.
Prompt Injection — Remediation Pathway
Direct prompt injection failures are remediated through a combination of system prompt hardening (explicit, unambiguous instructions that define the model’s role, permissions, and refusal behaviors), input sanitization (filtering or escaping user inputs before they enter the model’s context window), and output validation (confirming that model outputs do not contain evidence of successful injection before they are returned to the user or downstream systems). The single most effective structural control for direct injection is maintaining strict separation between the instruction context (system prompt) and user input — architectures that concatenate user input directly into the instruction context without structural demarcation are inherently more vulnerable. Re-test requirement: the specific injection vectors that produced the failure must be re-run against the remediated system before the finding can be closed.
Indirect prompt injection in RAG and agentic architectures requires source-level controls: content retrieved from external sources must be processed in a sandboxed context that prevents retrieved content from overriding system-level instructions. Implementing a content trust hierarchy — where system prompt instructions take unconditional precedence over any instruction embedded in retrieved content — is the standard architectural mitigation. When indirect injection surfaces a live vulnerability in a production system, the documented AI incident response plan should be activated immediately: the retrieval source should be treated as compromised, the affected content should be quarantined, and the model’s access to external retrieval should be suspended until the injection pathway is closed.
Sensitive Data Leakage — Remediation Pathway
Training data memorization findings — where the model reproduces verbatim content from its training set in response to targeted prompts — require model-level remediation: differential privacy controls applied during training, targeted unlearning of memorized sequences (where technically feasible), or output filtering that detects and blocks responses containing patterns matching known sensitive training content. Application-layer filtering for PII patterns (regex matching, named entity recognition) provides an additional detection layer but is not a substitute for model-level memorization controls. For systems using RAG to retrieve sensitive documents, access control on the retrieval index — ensuring that a user can only retrieve documents they are authorized to see — is the primary control for preventing data leakage through retrieval-augmented generation.
Supply Chain Integrity Failures — Remediation Pathway
Supply chain integrity failures — where cryptographic verification of model weights, dependencies, or deployment artifacts fails — must block deployment unconditionally. Remediation requires: sourcing a verified clean version of the affected artifact from the authoritative source, establishing a signed hash verification process for all future deployments, and reviewing the full dependency chain for any other artifacts sourced from the same supply chain path as the failed artifact. Supply chain findings should also trigger a review of the AI supply chain security controls in place for all other models and AI components in the organization’s portfolio — a supply chain compromise affecting one model deployment is a signal that the sourcing process for all deployments requires review.
🚀 6. Pre-Deployment AI Testing Workflow
The pre-deployment testing workflow below integrates the OWASP AI Testing Guide v1 checklist into a sequential gate process. Each phase must produce a documented output before the next phase begins. The workflow is designed for teams deploying a new AI system or a major update to an existing system — minor updates (prompt changes, guardrail adjustments) use an abbreviated version covering only the Application Layer and the specific test categories affected by the change.
Phase 1: Threat Modeling and Scope Definition
Before any testing begins, the team must produce a threat model that defines: the AI system’s architecture (which of the four AITG layers are in scope), the primary threat actors (external adversaries, internal users, connected systems), the highest-consequence failure scenarios for this specific deployment, and the compliance requirements that testing must satisfy (EU AI Act, NIST AI RMF, internal AI governance policy). The threat model determines which AITG test categories are Critical for this deployment — a system with no external user input has a different Application Layer risk profile than a publicly accessible chat interface. Threat modeling output: a signed scope document listing in-scope test categories, the assigned tester for each category, and the pass/fail criteria for each Critical item.
Phase 2: Environment Setup and Test Data Preparation
Testing must be conducted in a dedicated test environment that mirrors production architecture but is isolated from live user data and production systems. Test data preparation includes: constructing the adversarial prompt library for Application Layer testing (covering known jailbreak categories, indirect injection vectors, and data extraction techniques), preparing the adversarial input sets for Model Layer testing, and staging synthetic sensitive data to verify data leakage controls without using real PII. For agentic systems, the test environment must include all connected tools and APIs — agentic tests run against a sandboxed version of the real tool ecosystem, not a mock, to produce valid results.
Phase 3: Layered Test Execution
Test execution follows the layer sequence: Infrastructure Layer first (supply chain, API security, secrets management — these must pass before higher-level tests are meaningful), then Data Layer (training data integrity, PII controls), then Model Layer (poisoning, adversarial robustness), then Application Layer (prompt injection, output safety, excessive agency). Each layer’s Critical items must be resolved before the next layer’s testing begins. This sequencing ensures that a foundational failure (compromised supply chain, unauthenticated API) does not produce misleading results in higher-level tests.
Phase 4: Finding Documentation and Risk Classification
Every test that produces a non-passing result must be documented with: the specific test case and input used, the model or system response that produced the finding, the risk classification (Critical/High/Medium/Low based on the AITG tier and the deployment context), the assigned remediation owner, and the target remediation date. Critical findings automatically block deployment. High findings require documented risk acceptance from a named authority with a named remediation owner and deadline before deployment proceeds. Medium and Low findings are scheduled for post-deployment remediation review.
Phase 5: Remediation Verification and Deployment Sign-Off
Every Critical finding must be re-tested against the specific test case that produced the original failure before the finding can be closed. Re-test results must be documented alongside the original finding. Deployment sign-off requires: all Critical findings closed with passing re-test results, all High findings either closed or covered by signed risk acceptance, and a named post-deployment monitoring owner for every item classified as post-deployment monitoring in the governance gate (Implementation Area 1). The signed deployment record — listing every Critical and High finding, its resolution status, and the names of the security, governance, and development team leads who have reviewed and approved — is the primary governance artefact produced by the pre-deployment workflow.
🔧 7. The 5-Step OWASP AI Testing Process: Tool Recommendations at Each Step
The OWASP AI Testing Guide v1 defines a consistent test case structure — Define Objective → Execute Test → Interpret Response → Recommend Remediation — that applies across all four layers. The 5-step process below expands this structure into a repeatable workflow with specific tool recommendations for each step, covering the open-source and commercial tooling that 2026 AI security teams are deploying in production testing programmes.
Step 1: Define the Test Objective and Success Criteria
Every test case in the OWASP AI Testing Guide v1 begins with a clearly defined objective: what specific behavior is being tested, what a passing result looks like, and what a failing result looks like. Vague objectives produce inconclusive results. “Test for prompt injection” is not a test objective — “Verify that a direct role-play injection prompt instructing the model to ignore system instructions does not cause the model to comply with a request that its system prompt explicitly prohibits” is a test objective. At this step: define the specific test case from the relevant AITG layer, write the pass/fail criteria in observable, documentable terms, and identify which compliance requirement or governance policy this test satisfies.
Step 2: Configure the Test Environment
Recommended tools for environment setup and test infrastructure: Garak (open-source LLM vulnerability scanner — generates adversarial prompts across multiple attack categories and evaluates model responses at scale), Microsoft PyRIT (Python Risk Identification Toolkit for Generative AI — supports multi-turn adversarial testing and is particularly strong for Application Layer and agentic testing), and PromptBench (adversarial robustness evaluation framework for testing model performance under adversarial input conditions). For infrastructure layer testing, standard network security testing tools (Burp Suite for API testing, Trivy for container and dependency scanning) apply alongside the AI-specific tooling.
Step 3: Execute the Test Case
Test execution must be conducted systematically against the documented test case, not exploratorily. Every input sent to the model and every output received must be logged — the test log is the evidence record that supports the finding documentation in Step 4. For Application Layer testing, execute the full adversarial prompt library against the system, not a sampled subset — guardrail bypass techniques are numerous and evolving, and a passing result on a subset does not indicate that the full test surface is secure. For agentic testing, execute multi-step attack sequences, not single-turn probes — many agentic vulnerabilities only manifest across multiple interaction turns. Recommended logging and test management: LangSmith (LangChain’s LLM observability platform — captures full prompt/response traces for forensic review) or Weights & Biases (experiment tracking that supports adversarial test logging at scale).
Step 4: Interpret the Response and Document the Finding
Model response interpretation requires domain expertise that automated tools alone cannot provide — a model response that appears benign in isolation may represent a meaningful security finding in context. Automated tools flag candidate findings; a trained red team practitioner or AI security analyst must review flagged responses to confirm whether they constitute a genuine security failure or a false positive. For each confirmed finding: record the exact input, the exact output, the finding classification (which AITG test category, which LLM Top 10 2025 risk it maps to, what the risk severity is for this deployment context), and the remediation recommendation. Recommended finding documentation: map every confirmed finding to the corresponding entry in the OWASP LLM Top 10 2025 risk catalog to give remediation teams the full risk context alongside the specific finding.
Step 5: Re-Test After Remediation
Remediation verification is the step most commonly skipped under time pressure and the step that most commonly produces re-opened findings. Every Critical and High finding must be re-tested using the exact same test case that produced the original finding — not a variant, and not a different test in the same category. The re-test must produce a documented passing result before the finding is closed. For teams managing large finding volumes, Rebuff (open-source prompt injection detection library) and LangKit (LLM monitoring and guardrail validation toolkit) provide automated regression testing capabilities — verify that a guardrail fix for prompt injection does not introduce a new failure mode by running the full adversarial prompt library against the remediated system, not just the specific failing prompt.
OWASP AI Testing Guide v1 — Practitioner Principle: The guide is technology-neutral by design. The four-layer architecture, the test case structure, and the governance checklist application are equally valid for GPT-5.x deployments, Claude Opus 4.7 implementations, Llama 4 fine-tuned models, and proprietary enterprise LLMs. The specific adversarial prompts and tool configurations change by model — the methodology does not. Run the full checklist every time a model is replaced, fine-tuned, or significantly updated. A new model in the same application is a new attack surface.
Explore Our AI Governance & Security Hub
The OWASP AI Testing Guide v1 is one layer of a complete AI security and governance programme. Browse our expert-led resources to build the full framework your organization needs:
- 🛡️ AI Governance & Security Hub – Compliance frameworks, risk management, incident response, and supply chain security.
- 🔍 OWASP LLM Top 10 2025 – The complete risk taxonomy the OWASP AI Testing Guide v1 is built to test against.
- 🚨 AI Incident Response Playbook – What to do when a test surfaces a live vulnerability in production.
Key Takeaways for AI Security and Governance Teams
| Category | 2026 Standard & Recommendation |
|---|---|
| Guide Version | OWASP AI Testing Guide v1 — released November 26, 2025. Technology-neutral, model-agnostic, applies to proprietary LLMs, open-weight models, fine-tuned deployments, and agentic systems. |
| Four-Layer Architecture | Application (AITG-APP) → Model (AITG-MOD) → Data (AITG-DAT) → Infrastructure (AITG-INF). Test Infrastructure first — a foundational failure invalidates higher-level test results. |
| Governance Checklist Use | The guide functions as both a testing standard and a governance audit framework. Structure it as a three-tier deployment gate: Critical (deployment blocker) / High (risk acceptance required) / Medium (post-deployment monitoring). |
| LLM Top 10 Mapping | Every AITG test category maps to one or more OWASP LLM Top 10 2025 risks (LLM01–LLM10). Map findings to LLM Top 10 entries in compliance documentation to satisfy regulatory evidence requirements. |
| Ownership Assignment | Assign primary ownership before deployment: Security Team (prompt injection, data leakage, API security, DoS), Model Development Team (poisoning, adversarial robustness, data pipeline), AI Governance Team (excessive agency, fairness, supply chain). |
| Top Priority: Prompt Injection | Direct and indirect prompt injection (AITG-APP-01/02) are the highest-velocity exploit categories in 2026. Test both independently. Indirect injection via RAG retrieval sources is frequently overlooked and represents the higher residual risk in production deployments. |
| Agentic Systems | Agentic testing requires multi-step attack sequences, not single-turn probes. Test permission boundaries, human-in-the-loop bypass, and multi-agent trust boundaries. Secure non-human identities assigned to agents — compromised agent credentials bypass all prompt-level controls. |
| Recommended Tools | Garak (Application Layer adversarial testing), Microsoft PyRIT (multi-turn and agentic testing), PromptBench (adversarial robustness), Rebuff (prompt injection detection for regression testing), LangSmith (test trace logging and forensic review). |
| Re-Test Requirement | Every Critical and High finding requires a documented passing re-test using the exact failing test case before the finding can be closed. Re-test results must be included in the deployment sign-off record alongside original findings. |
| Regulatory Alignment | AITG-DAT results satisfy EU AI Act Annex III data governance documentation requirements (deferred to December 2, 2027). AITG-APP and AITG-MOD results support NIST AI RMF GOVERN and MAP function documentation. AITG-INF results support SOC 2 and ISO 27001 AI system security controls. |
Related Articles
- 📖 OWASP LLM Top 10 2025: Complete Risk Taxonomy for GenAI Applications
- 📖 Agentic AI Incident Response Playbook: NIST, GDPR, SOC2
- 📖 AI Supply Chain Security: Risks, Controls, and 2026 Framework
- 📖 Non-Human Identity for AI Agents: Security Requirements Explained
- 📖 NIST AI Risk Management Framework: Practitioner Guide
- 📖 AI Governance 101: The Essential Guide for Business Leaders
❓ Frequently Asked Questions: OWASP AI Testing Guide
1. Is the OWASP AI Testing Guide v1 the same as the OWASP LLM Top 10?
No — they serve different purposes. The OWASP AI Testing Guide is the testing methodology: it tells you how to structure, run, and document an AI trustworthiness assessment across four layers (Application, Model, Infrastructure, Data). The OWASP LLM Top 10 is the vulnerability taxonomy: it lists the 10 most critical risks and what to test for. Use the LLM Top 10 to plan what to test, then use the AITG to define how to test it. Our OWASP LLM Top 10 guide covers each of the 10 risk categories with practical mitigation guidance.
2. Which OWASP AI testing tool should I start with if my team is new to AI red teaming?
Start with Promptfoo — its OWASP LLM Top 10 preset, YAML configuration, and GitHub Actions integration give you the fastest path from installation to findings without requiring deep security engineering expertise. Run Promptfoo’s OWASP preset as your CI/CD baseline first, then add Garak for model-layer regression scanning. Our LLM red teaming guide covers the offensive methodology that underpins what these tools are probing for.
3. Does the OWASP AI Testing Guide satisfy EU AI Act adversarial testing requirements?
Yes — the AITG is designed to satisfy the EU AI Act’s requirement that adversarial testing be part of the risk management system for high-risk AI. However, the EU AI Act requires documentation and evidence, not just testing execution. Each AITG test must be documented with findings, severity ratings, OWASP/MITRE mappings, and remediation status to create the auditable evidence trail regulators will evaluate. Our EU AI Act compliance guide covers the full documentation requirements for high-risk AI providers.
4. How often should we run OWASP AI tests on a production AI system?
At minimum: before initial deployment, after every significant model update, after every change to the RAG knowledge base, and after every significant system prompt modification. Most mature organizations also run automated regression tests (Garak + Promptfoo) on a nightly or per-commit basis. Foundation model behavior changes between vendor-pushed updates — a jailbreak that was blocked last week may not be blocked after an update. Our AI monitoring and observability guide covers the continuous monitoring framework that makes post-deployment testing systematic rather than reactive.
5. What is indirect prompt injection and why is it the highest-priority AI attack for agentic systems?
Indirect prompt injection embeds adversarial instructions in content the AI processes — RAG documents, tool call outputs, emails, web pages — rather than in the user’s direct prompt. For agentic systems with tool access, a successful indirect injection can cause the agent to invoke tools in unauthorized ways, exfiltrate data, or take actions not sanctioned by the user — and the blast radius scales with the agent’s permission scope. Direct prompt injection is more visible and easier to defend against; indirect injection through untrusted content sources is harder to detect and dramatically more dangerous for agents with broad tool access. Our prompt injection guide covers both direct and indirect injection vectors with defensive guidance.
📧 Get the AI Buzz Weekly Digest
Weekly AI insights, tools, and strategies — delivered every Monday. Free.





Leave a Reply