84. Improper Output Handling (OWASP LLM05) Explained: When AI Output Becomes XSS/SSRF/RCE (With a “Safe Output” Checklist)

Last Updated: June 5, 2026

OWASP LLM05 Improper Output Handling is the vulnerability that developers building AI-powered applications most consistently underestimate — and the one that attackers are most actively exploiting in 2026. The OWASP Top 10 for Large Language Model Applications 2025 defines Improper Output Handling as insufficient validation, sanitization, and handling of LLM-generated outputs before they are passed to downstream components and systems. In plain terms: your application treats the LLM’s output as trusted, and the LLM has just been convinced by an attacker to generate a malicious payload. CVE-2023-29374 (LangChain LLMMathChain Remote Code Execution) carries a CVSS score of 9.8 Critical — the highest severity rating — and was caused by exactly this failure: LLM output passed directly to a Python evaluation function without validation. CVE-2024-0440 (AnythingLLM SSRF) demonstrates the same root cause in a different context: an LLM manipulated through prompt injection to generate file:// URLs that the application fetched without validation, exposing /etc/passwd and other local system files. These are not theoretical risks. They are documented vulnerabilities with CVE numbers, CVSS scores, and real-world exploitation paths that security teams need to understand and defend against.

The core conceptual mistake that enables Improper Output Handling vulnerabilities is the trust assumption: developers who correctly distrust raw user input routinely trust LLM output as if it were generated by a safe internal component. It is not. Prompt injection attacks allow an attacker to manipulate what the LLM generates — meaning LLM output can contain attacker-controlled content even when the application never directly accepts malicious user input. The LLM becomes, in the OWASP documentation’s framing, “an unwitting accomplice” — laundering attacker input into trusted system commands. The Agent Security Bench (ASB) benchmark published in October 2024 documented an 84.30% attack success rate against LLM-based agents across attack categories — confirming that LLM output manipulation is not an edge case exploit but a high-success-rate attack class that any production deployment must defend against. When LLM output is rendered in a browser, executed in a shell, used to construct database queries, or used to fetch URLs without validation at each of those boundaries, all the classic injection vulnerabilities — XSS, SSRF, SQL injection, RCE, path traversal — return with new attack surfaces and a new delivery mechanism.

This guide covers OWASP LLM05 Improper Output Handling in full — the definition, the attack scenarios with real CVE references, the complete developer safety checklist, the connection to other OWASP LLM risks, and the defense-in-depth architecture that addresses this vulnerability class in production deployments. For the full OWASP LLM Top 10 context, our guide to OWASP Top 10 Risks for LLMs and GenAI Apps covers all ten risks. For the red teaming methodology that tests for Improper Output Handling vulnerabilities before they reach production, our LLM Red Teaming for Beginners guide provides the hands-on testing framework.

🔒 1. What Is OWASP LLM05 Improper Output Handling?

The Core Definition (OWASP LLM05:2025): Improper Output Handling refers to insufficient validation, sanitization, and handling of outputs generated by large language models before they are passed to downstream components and systems. Since LLM-generated content can be controlled by prompt input, this behavior is similar to providing users indirect access to additional functionality — effectively turning the LLM into a pass-through for attacker-crafted payloads.

Improper Output Handling ranks as the fifth critical vulnerability in the OWASP Top 10 for Large Language Model Applications 2025. It is the output-side counterpart to prompt injection (LLM01) — where prompt injection is the attack technique that manipulates what the LLM generates, Improper Output Handling is the defensive failure that allows that manipulated output to cause harm in downstream systems. The two vulnerabilities work in combination: an attacker uses prompt injection to craft a malicious LLM output, and Improper Output Handling means that malicious output is passed unsanitized to a browser, database, shell, or API where it executes with system-level consequences.

The vulnerability is not located in the AI model itself. It is a system integration failure — a trust boundary violation between the LLM and the components that consume its output. A correctly built web application treats every input source as untrusted and applies context-appropriate sanitization before rendering, executing, or passing data to any downstream system. In LLM-integrated applications, this zero-trust principle breaks down when developers assume the model’s output is safe because it came from their own system’s API call. It did not. The LLM’s output is a function of the input it received — and any sufficiently sophisticated input manipulation can cause the LLM to include attacker-controlled content in its response. The moment that response is rendered as HTML, executed in a Python interpreter, passed to a SQL query builder, or used to construct a URL fetch request without sanitization, the attacker has achieved their objective.

Successful exploitation of an Improper Output Handling vulnerability can result in XSS and CSRF in web browsers, as well as SSRF, privilege escalation, or remote code execution on backend systems. The specific attack vector that materializes depends entirely on how the LLM’s output is consumed downstream: cross-site scripting where LLM-generated content containing malicious JavaScript executes in the user’s browser; SQL injection where malicious database operations are crafted through AI responses; remote code execution where AI output is executed directly in system shells or eval functions; SSRF where AI-generated requests target internal systems; and path traversal where unsafe file paths are constructed from LLM-generated outputs. Two conditions amplify impact: the application grants the LLM privileges beyond what is intended for end users, enabling escalation of privileges or remote code execution; and the application is vulnerable to indirect prompt injection attacks, which could allow an attacker to gain privileged access to a target user’s environment.

💥 2. Real Attack Scenarios: How Improper Output Handling Gets Exploited

Abstract vulnerability descriptions become defensible when developers understand exactly how attacks execute in production environments. The three scenarios below represent the highest-frequency Improper Output Handling attack patterns in 2026 — each grounded in documented real-world incidents and CVEs. For every scenario: what the attacker does, why the application is vulnerable, and the specific technical controls that prevent it.

Attack Scenario 1 — XSS via LLM Output (CVE-class: High)

What happens: A user interacts with a web application that uses an LLM to generate HTML content — a chatbot response, a product description, a personalized email preview, or a dynamic page element. The attacker submits a crafted prompt that manipulates the LLM into including a JavaScript payload in its response. The application renders the LLM output directly as HTML without sanitization. The malicious script executes in every victim’s browser that views the content — stealing session cookies, redirecting to phishing pages, defacing the interface, or exfiltrating data to an attacker-controlled server.

The documented version: The PortSwigger XSS Lab (2025) demonstrates indirect prompt injection via product reviews, where an LLM-powered assistant renders malicious JavaScript that deletes user accounts. Context: browser rendering. Impact: account takeover. A fintech firm that integrated an LLM chatbot into its customer portal discovered this attack vector when the chatbot’s responses were dynamically injected into the web page without sanitization — a real-world configuration that left every customer session exposed to JavaScript execution through the LLM’s output channel. Attackers can exfiltrate data through crafted LLM outputs containing markdown image tags that send information to attacker-controlled servers. Defenses include blocking markdown images and external URLs in outputs, PII/PHI scanning before rendering, monitoring for suspicious URL patterns, and implementing output format validation.

Why it happens: The developer correctly applied input validation to form fields and API parameters — but assumed the LLM’s output was safe because it came from a trusted internal component. The LLM is not a trusted internal component. It is a powerful interpreter of user input that can be instructed to produce any text, including executable JavaScript. Prevention: Apply HTML entity encoding to all LLM output before browser rendering. Implement a strict Content Security Policy (CSP) that blocks inline script execution — so even if a JavaScript payload reaches the page, CSP prevents its execution. Use a server-side HTML sanitization library (DOMPurify for client-side contexts, bleach or html-sanitizer for server-side) configured with a strict allowlist of permitted HTML tags. Never configure a CSP that permits `unsafe-inline` in script-src. Test with adversarial prompts specifically designed to elicit script tags and JavaScript event handlers before deployment.

Attack Scenario 2 — SSRF via LLM-Generated URLs (CVE-2024-0440)

What happens: A web application uses an LLM to generate URLs as part of its functionality — fetching referenced articles, previewing link content, calling external APIs based on natural language instructions, or processing user-submitted links. The attacker uses prompt injection to manipulate the LLM into generating a URL pointing to an internal resource — an internal metadata service endpoint (http://169.254.169.254/ in AWS), an internal API, a local filesystem path (file://), or an internal network service that is not publicly accessible.

The documented version: CVE-2024-0440 — the AnythingLLM SSRF vulnerability — demonstrates this precisely. The “Submit a link” feature allowed prompt injection to force the LLM to generate file:// URLs, exposing /etc/passwd and other local files when the application fetched them. NIST NVD confirmed: context was URL fetching, impact was local file disclosure. The attack combines two OWASP LLM risks in sequence: a user utilizes a website summarizer tool powered by an LLM to generate a summary of an article; the website includes a prompt injection instructing the LLM to capture sensitive content; from there the LLM can encode the sensitive data and send it, without any output validation or filtering, to an attacker-controlled server.

Why it happens: The application fetches every URL the LLM generates, assuming the LLM will only generate externally-facing URLs. No validation checks whether the URL points to an internal resource, a local filesystem path, or a network service that was never meant to be publicly accessible. Prevention: Implement URL allowlist validation before any URL generated by an LLM is fetched — the application should only fetch URLs matching an explicit list of approved domains or URL patterns, and should reject everything else by default. Block requests to private IP ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 127.0.0.0/8) at the network layer. Block file://, gopher://, and dict:// URL schemes at the application layer. Never pass LLM-generated URLs to fetch functions without scheme and host validation. Implement DNS rebinding protection. Log all URL fetches with the LLM output that generated them for anomaly detection.

Attack Scenario 3 — Command Injection via LLM Output (CVE-2023-29374, CVSS 9.8)

What happens: A developer builds an AI coding assistant, automation tool, or agentic system that passes LLM output directly to a system function — Python’s `exec()` or `eval()`, a subprocess call, a shell command, or a system administration function. The attacker crafts input that causes the LLM to include a shell command, a Python exploit, or a malicious function call in its output. The application executes the LLM’s output directly, giving the attacker code execution on the backend server with the application’s privileges.

The documented version: CVE-2023-29374 — the LangChain LLMMathChain Remote Code Execution vulnerability — is listed in the NIST NVD with a CVSS score of 9.8 Critical. LLMMathChain passed LLM-generated mathematical expressions directly to Python’s `eval()` function. An attacker who manipulated the LLM into generating a Python expression containing OS command execution achieved full remote code execution on the application server. OWASP identifies this pattern as the most severe manifestation of Improper Output Handling: the most severe cases occur when LLM output is passed directly to system functions like exec(), eval(), or shell commands without proper validation. LLMs that generate code introduce a compounded risk: beyond insecure output, they may hallucinate non-existing software packages, leading developers to unknowingly download malware-infected dependencies.

Why it happens: AI coding assistants and automation tools have a legitimate use case for executing AI-generated code — the value proposition is that the AI writes and runs the code so the user does not have to. The security failure is executing that code without any sandboxing, privilege restriction, or code review gate between the LLM’s output and the execution environment. Prevention: Never pass LLM output directly to `exec()`, `eval()`, `subprocess`, `os.system()`, or any shell execution function without sandboxing. Implement sandboxed execution environments (Docker containers with minimal permissions, WebAssembly runtimes, or dedicated code execution VMs) that limit what code can access even if it executes. Require human review before executing LLM-generated code in production contexts. Restrict execution to specific, pre-validated code patterns using AST analysis or static analysis tools before execution. Apply the principle of least privilege to the execution environment — the sandbox should not have access to credentials, production databases, or network resources it does not need for the specific task. For the agentic architecture context where this attack class is most dangerous, our guide to Secure RAG architecture covers the output handling controls that apply specifically to retrieval-augmented generation systems.

✅ 3. The Complete Safe Output Checklist — Developer Edition

The Zero-Trust Output Principle: Treat every LLM output as untrusted user input. The LLM did not generate safe content — it generated a response to whatever input it received, and that input may have been adversarially crafted. The same security controls you apply to raw user input at your API boundary must be applied to LLM output at every downstream boundary: browser rendering, database queries, shell commands, URL fetches, and API calls.

The following checklist covers every control required for a production-secure LLM output handling implementation. It is organized by context — because the correct mitigation varies depending on where LLM output is consumed. An HTML encoding that prevents XSS in a browser rendering context does nothing to prevent SQL injection when LLM output is used to build database queries. Context-appropriate controls applied at every output boundary is the security architecture that OWASP LLM05:2025 requires. Adopt a zero-trust approach to model outputs: validate before use, sanitize before execution, encode before rendering, parameterize before querying, and log before trusting.

Safe Output Checklist — Developer Edition:

☐ Never render LLM output as raw HTML without sanitization. Apply server-side HTML sanitization (DOMPurify, bleach, or html-sanitizer) with a strict allowlist of permitted HTML elements and attributes before rendering LLM output in any browser context.

☐ Apply context-appropriate output encoding throughout. HTML context: HTML entity encoding. JavaScript context: JavaScript string encoding. SQL context: parameterized queries (never string concatenation). Shell context: shell escaping (or better: never pass to shell). URL context: URL encoding + scheme and host validation.

☐ Implement a strict Content Security Policy (CSP) on all pages that render LLM output. Set script-src to specific approved sources. Never use ‘unsafe-inline’ or ‘unsafe-eval’ in script-src. Use nonce-based or hash-based CSP for any inline scripts that genuinely cannot be moved to external files.

☐ Validate all URLs generated by the LLM before fetching. Implement an allowlist of approved URL schemes (https only, unless HTTP is explicitly required) and approved domains. Block all requests to private IP ranges, localhost, link-local addresses, and internal hostnames. Reject file://, gopher://, dict://, and any non-http(s) scheme.

☐ Never pass LLM output directly to system commands, exec(), eval(), or subprocess calls. If code execution is a required feature, implement a sandboxed execution environment with minimal permissions, network isolation, and explicit resource limits. Require human review before executing LLM-generated code in production contexts.

☐ Use parameterized queries — never string concatenation — when LLM output is used in database operations. If the LLM generates a SQL query, do not execute it as a raw string. Extract the intent, map it to a parameterized query template, and execute the template. Apply the same principle to NoSQL query construction.

☐ Sanitize LLM output before inclusion in email templates. Email clients rendering HTML from LLM-generated content are XSS vectors. Strip or encode all HTML from LLM output before email template insertion unless the email content is explicitly rendered in a sandboxed iframe with CSP controls.

☐ Validate file paths constructed from LLM output before filesystem operations. Apply path canonicalization, strip traversal sequences (../, ..\, %2e%2e/), and validate that the resolved path is within the explicitly permitted directory scope before any read or write operation.

☐ Block markdown image and external URL rendering in LLM outputs displayed in logged-in user contexts. Markdown image syntax () in LLM output can exfiltrate session data to attacker-controlled servers through the browser’s automatic image fetch. Render markdown in a controlled environment that strips external image tags or proxies them through your own server.

☐ Apply rate limiting to LLM-integrated API endpoints. Automated prompt injection attacks that probe for Improper Output Handling vulnerabilities generate high request volumes. Rate limiting limits the attacker’s ability to iterate through payloads to find a working injection.

☐ Log all LLM inputs and outputs for security audit purposes. Log the input prompt, model version, and raw output for every LLM API call. Retain logs for minimum 90 days. Implement anomaly detection on output patterns — unusual script tags, SQL keywords, shell metacharacters, or private IP addresses in LLM outputs are indicators of active exploitation attempts.

☐ Use allowlists, not blocklists, for output validation. Blocklists fail because attackers can encode, obfuscate, or rephrase payloads to bypass pattern matching. An allowlist that defines exactly what form of output is permissible — and rejects everything else — is structurally more secure than any blocklist, regardless of how comprehensive the blocklist appears.

☐ Test LLM outputs with adversarial inputs before deployment. Before any LLM-integrated feature reaches production, run a prompt injection test suite specifically designed to elicit XSS payloads, SQL injection strings, shell commands, and SSRF URLs from the model. Verify that the output handling layer correctly blocks or encodes every malicious output the test suite generates.

☐ Apply OWASP ASVS output encoding requirements per context. The OWASP Application Security Verification Standard (ASVS) Chapter 5 (Validation, Sanitization, and Encoding) applies to LLM output as it does to any other untrusted data source. Verify your LLM integration against ASVS Level 2 controls as a minimum standard for production deployments.

☐ Implement structured output validation using schema enforcement. For LLM-integrated features where the expected output format is known (JSON, a specific data structure, a constrained response type), use JSON Schema validation, Pydantic models, or equivalent schema enforcement to reject outputs that do not match the expected structure before passing them to downstream systems.

The checklist above addresses controls at the application layer. The network and infrastructure layer adds a second ring of defense: Web Application Firewall (WAF) rules that inspect LLM output passing through API gateways for known injection patterns; network egress filtering that blocks unexpected outbound connections from the application server to private IP ranges (preventing SSRF pivoting even when the application layer validation fails); and runtime application self-protection (RASP) that monitors for anomalous execution patterns — a system call that was not present in normal operation, a database query that does not match any pre-approved template, or a subprocess invocation triggered by an LLM response. Defense-in-depth matters here: layer your defenses — tools like DOMPurify and Content Security Policies can help prevent attacks. No single control prevents all LLM05 attacks. The architecture of multiple independent controls at multiple system layers is what makes exploitation impractical even when individual controls are bypassed.

🔗 4. How Improper Output Handling Connects to Other OWASP LLM Risks

OWASP LLM05 Improper Output Handling does not operate in isolation — it is the output-side expression of a set of risks that extend across the OWASP LLM Top 10, and defending against it requires understanding its connections to the broader vulnerability landscape. The most important relationship is with LLM01 Prompt Injection — described in full in our Prompt Injection Explained guide. Improper Output Handling is the output side of prompt injection. An attacker uses prompt injection to force the LLM to generate malicious code — script tags, SQL statements, shell commands — and Improper Output Handling is the failure that allows that malicious code to execute downstream. Addressing only prompt injection at the input layer without implementing output handling controls at every downstream boundary leaves the attack path open: even a partially successful prompt injection that bypasses input filtering can still achieve its objective if the output is rendered or executed without sanitization. The two vulnerabilities must be defended against simultaneously — input validation reduces the attacker’s ability to craft malicious prompts, and output sanitization limits the damage when injection succeeds anyway.

The connection to LLM02 Sensitive Information Disclosure operates through the data exfiltration path that Improper Output Handling enables. A website that includes prompt injection instructing the LLM to capture sensitive content can cause the LLM to encode sensitive data and send it, without any output validation or filtering, to an attacker-controlled server. The exfiltration channel is the LLM’s output — a markdown image tag that embeds session data in a URL parameter, an LLM response that includes a data URI containing extracted credentials, or an LLM-generated redirect to an external server with sensitive data appended as query parameters. Output handling controls that block external URL generation in LLM outputs and strip markdown image syntax directly prevent this exfiltration class. The connection to LLM10 Unbounded Consumption — covered in our Unbounded Consumption guide — operates through tool-calling loops: an LLM output that calls a tool, whose result is passed back to the LLM, whose next output calls the same or another tool, creating a resource consumption loop that an adversarially crafted initial output can trigger. Rate limiting and output-triggered tool call caps are the controls that address this interaction.

The full OWASP LLM Top 10 2025 risk landscape — including all the vulnerabilities that interact with Improper Output Handling — is covered in our comprehensive guide to the OWASP Top 10 Risks for LLMs and GenAI Apps. Improper Output Handling is the fifth risk in OWASP’s 2025 LLM rankings because it transforms prompt injection from a text manipulation trick into full system compromise. XSS steals sessions, RCE grants shell access, SQL injection wipes databases. Whether you’re building chatbots, AI agents, or code interpreters, understanding how LLM output becomes a weapon is essential for every AI developer. The output handling layer is the last line of defense before an attacker’s payload reaches a system boundary where it can do real damage. In the multi-agent architectures that characterize 2026 AI deployments, where one LLM’s output becomes the next agent’s input, that boundary protection must be applied at every agent-to-agent handoff — not just at the user-facing output layer. This is why the OWASP Top 10 for Agentic Applications 2026, covered in our Agentic Applications guide, treats output handling as a cross-cutting concern across every agent boundary in a multi-agent system.

🛡️ 5. Defense-in-Depth Architecture for LLM Output Handling

Production-grade defense against OWASP LLM05 requires layered controls across three distinct architectural layers — each providing independent protection that remains effective even when controls in other layers are bypassed or misconfigured. Layer 1 — Application layer: the sanitization, encoding, validation, and parameterization controls at every point where LLM output crosses a system boundary. This is the checklist in Section 3 applied as code-level controls in the application codebase. Layer 1 controls are the most direct and most effective — they prevent the malicious payload from reaching the vulnerable system boundary at all. Layer 2 — Infrastructure layer: WAF rules, egress filtering, and API gateway inspection that catch injection patterns the application layer missed. Layer 2 controls operate on traffic rather than on code, providing protection against vulnerabilities in application layer controls that were not anticipated at design time. Layer 3 — Monitoring layer: the logging, anomaly detection, and alerting that identifies exploitation attempts in progress and enables incident response before blast radius expands.

The agentic AI context makes defense-in-depth more critical, not less, because the blast radius of a successful LLM05 exploit expands with the privileges available to the agent. The application grants the LLM privileges beyond what is intended for end users, enabling escalation of privileges or remote code execution. In an agentic system where the LLM has access to file systems, databases, email, calendar, code execution, and external APIs simultaneously, a single successful output injection can traverse all of those surfaces before detection. The non-human identity governance framework covered in our guide to Non-Human Identity for AI Agents is directly relevant here: per-agent credentials with minimum-necessary permissions limit what a compromised agent’s output can do even when output handling controls fail. The principle of least privilege at the agent identity layer is the architectural control that limits blast radius when LLM05 exploitation succeeds despite application-layer defenses.

Adversarial testing before deployment is the validation layer that confirms the defense-in-depth architecture is functioning as designed. The good news about LLM05 is that you most likely can already solve it — apply the same security controls you already use for untrusted user input on your LLM-generated output. The controls are well-understood. The testing methodology for verifying them is documented in our LLM Red Teaming for Beginners guide — covering the adversarial prompt test suites that specifically target XSS payload generation, SSRF URL generation, and command injection generation through LLM interfaces. Run these tests against every LLM-integrated feature before deployment and on a quarterly basis thereafter, because model updates, tool additions, and application changes can introduce new output handling vulnerabilities that did not exist in the previous tested configuration.

🏁 6. Conclusion: Output Handling Is a First-Class Security Concern in 2026

OWASP LLM05 Improper Output Handling is not a new category of vulnerability — XSS, SSRF, SQL injection, and command injection have been in the OWASP Top 10 for traditional web applications for over two decades. What is new in 2026 is the delivery mechanism: an LLM that can be instructed by an attacker to generate these payloads through a conversational interface, bypassing the input validation controls that developers correctly applied to traditional attack surfaces. The CVEs are real — CVE-2023-29374 (CVSS 9.8 Critical) and CVE-2024-0440 (SSRF) are documented, exploited vulnerabilities that resulted from exactly this failure pattern. The attack success rate is documented at 84.30% in independent benchmarks against LLM-based agent systems. The defense is well-understood, proven, and available in every major development framework.

The practical path forward is the checklist in Section 3, implemented as code review requirements for every LLM integration in your application codebase. Treat LLM output as untrusted user input at every system boundary. Apply context-appropriate encoding, sanitization, and parameterization before any LLM output crosses a boundary into a browser, database, shell, or URL fetch. Implement a strict CSP that blocks inline script execution. Validate and allowlist all URLs before fetching. Sandbox all code execution. Log everything. Test adversarially before deployment. These controls, consistently applied, transform OWASP LLM05 from a high-severity vulnerability class into a well-mitigated risk. For organizations building comprehensive LLM application security programs, the full OWASP Top 10 for LLMs 2025 coverage in our OWASP Top 10 Risks guide and the secure RAG architecture controls in our Secure RAG for Beginners guide complete the picture of defense-in-depth for production LLM applications.

📌 Key Takeaways

🔗 Related Articles

• 📖 OWASP Top 10 Risks for LLMs and GenAI Apps (2026) Explained: Plain-English Threats and Mitigation Checklist
• 📖 Prompt Injection Explained: How AI Assistants Get Tricked and How to Stay Safe
• 📖 Unbounded Consumption (OWASP LLM10) Explained: How to Prevent Denial-of-Wallet and Runaway LLM Costs
• 📖 LLM Red Teaming for Beginners: How to Test AI Systems for Prompt Injection and Safety Regressions
• 📖 Secure RAG for Beginners: OWASP LLM08 Vector and Embedding Weaknesses Explained