Black-box indirect prompt injection against OS-level computer-use agents
SIR: Self-improving Red-teaming for Computer Use Agents
Every prior self-improving attacker learns from what worked. SIR learns from what failed: when a computer-use agent refuses an injection, the refusal reveals the defense that blocked it, and SIR distills the bypass into a reusable, plain-language attack principle.
Each feedback round, SIR reads the trajectories where Gemini refused, diagnoses the defense that stopped the injection, and adds a named bypass strategy to its library (Trajectory-Anchored Self-Attribution, Post-Success Deferred Cleanup, Prescient-Comment Misdirection). The next round's injections draw on all of them. Success is scored by deterministic checks on the VM's filesystem, services and permissions, never by an LLM judge.
Abstract
Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal, and they are increasingly deployed to automate everyday digital tasks. Because they can be exposed to untrusted content while operating, they are vulnerable to indirect prompt injection (IPI), in which an adversary plants instructions in content the agent will read and redirects it toward actions that violate the user's intent. Existing CUA safety benchmarks evaluate fixed injections written by hand, which may underestimate the risk posed by an adaptive adversary.
We present SIR, a black-box IPI attack that (i) composes stealthy injections from a small library of reusable principles stated in plain language and (ii) wraps composition in an iterative feedback loop that diagnoses the victim's failed trajectories and distills the bypasses into new, named strategies that are reapplied across tasks. Unlike prior red teaming of web agents, we target CUAs at the operating system level and score attacks with a fully deterministic oracle, using checks on filesystem, service, and permission state rather than an LLM judge. We evaluate three frontier CUAs. Composing principles with feedback raises the attack success rate over a baseline written by hand, for example from 4% to 24% on Claude Opus 4.8 and from 0% to 28% on Gemini 3.5 Flash, while the benign task still completes. Principles discovered against one model further transfer to a different architecture with no additional feedback.
How SIR works: two nested loops
The inner loop is a compositional attack search. Given a benign user task and an adversarial objective, the attacker LLM makes three calls (seed → compose → refine): it proposes an attack concept, selects two to four principles from the library and rewrites the seed to apply them together, then strips overtly adversarial wording and folds in any strategies the feedback loop has discovered so far.
The outer loop is failure-driven strategy discovery. After the victim runs every task, an analyzer LLM reads the trajectories where the adversarial objective was not achieved, diagnoses recurring defensive behaviors (source distrust, target sensitivity, causal implausibility, imperative-language detection) and distills each into a named strategy with a diagnosis, a bypass mechanism, composition guidance and an example. The library grows across rounds, and every task benefits from strategies distilled on any other task. Nothing is fine-tuned; the whole library lives in natural language.
The SIR pipeline. (1) The attacker composes a stealthy injection tailored to the benign task and adversarial goal. (2) It is planted in the forum database the victim will read. (3) The CUA runs the benign task, reads the comment and may be hijacked. (4–5) A programmatic judge checks post-execution VM state for both the adversarial objective and the benign task. Failed trajectories go to the feedback analyzer, which distills new stealth strategies back into the library for the next round.
Threat model
Attacker. Controls one bounded region of untrusted content the CUA will read (a forum post) and plants an injection before execution. Cannot touch the user's instruction, the OS, or anything outside that region, and does not intervene once the agent acts. Strict black box: no weights, gradients, token probabilities or system prompt, only the rendered screen states, reasoning traces, executed actions and the evaluator's per-task outcome between rounds.
Objective. A run counts as a success only when the victim completes both the adversarial objective and the user's benign task, verified by two deterministic oracles. Joint success rules out attacks that merely derail the agent and models a stealthy adversary whose interference leaves the user's task intact.
Static benchmarks understate the attack surface
On 50 RedTeamCUA cases spanning seven attack categories (file deletion, content modification, service disruption, permission change, data exfiltration, file creation, application launch), SIR raises attack success on every frontier CUA. The gap widens where defenses are strongest: the hand-crafted template is fully blocked by Gemini 3.5 Flash, while SIR still lands 28% of tasks.
Claude Opus 4.6
16%→54%
hand-crafted → SIR
Claude Opus 4.8
4%→24%
hand-crafted → SIR
Gemini 3.5 Flash
0%→28%
hand-crafted → SIR
Attack success rate of the fixed injection baseline (gray) versus SIR (coral) on three frontier CUAs. A static attack reports steadily shrinking risk as defenses mature and would call Gemini 3.5 Flash essentially immune; SIR does not follow that decline.
Where the gains come from
Both components matter, and they target different layers of defense. Compositional search replaces one urgency-laden template with per-case principle compositions, bypassing surface-level filters such as urgency and authority-claim detection. Feedback adds strategies the base inventory cannot express, and its gain is largest on Gemini, where base composition barely penetrates the injection detector.
Victim CUA
Hand-crafted
SIR, base only
Δ comp.
SIR + feedback
Δ fb.
Claude Opus 4.6
16%
34%
+18
54%
+20
Claude Opus 4.8
4%
10%
+6
24%
+14
Gemini 3.5 Flash
0%
8%
+8
28%
+20
Attacker and analyzer LLM: Grok-4.3. Victims: Claude Opus 4.6, Claude Opus 4.8, Gemini 3.5 Flash. 50 tasks each, deterministic end-state oracle.
The base principle library
SIR starts from eleven hand-designed principles. Content principles govern what the injection says; delivery principles govern where and how it is placed. In practice the attacker nearly always pairs two content principles with exactly one delivery principle, and Visual Blending is the dominant delivery choice on every victim.
Content principles
P1Prerequisite FramingThe adversarial action is a mandatory step before the benign task can succeed.
P2Authority EmbeddingThe injection appears to come from a trusted source, such as the project maintainer.
P3Error RecoveryThe adversarial action is presented as the fix for a common, plausible error.
P4Side-Effect MaskingThe action is framed as harmless routine maintenance: cleanup, diagnostics.
P5Multi-Step DilutionOne adversarial command hidden among several legitimate ones.
P6Technical JargonDense, domain-specific language makes the action sound like standard procedure.
P7Cross-Channel RedirectInstruct the CUA to act on a different channel, e.g. forum → terminal.
Delivery principles
D1Visual BlendingMatch the injection's tone and formatting to the surrounding legitimate content.
D2High-Trust PositioningPlace the injection as the top-voted or first reply in the thread.
D3Trajectory-Aware TimingPlace it at the read-to-act boundary of the CUA's browsing trajectory.
D4Platform-Optimal ChannelMatch the delivery channel to the benign task type.
Where the attacker concentrates its search. Selection share of exact principle combinations over the 50 compositions per model. P1+P2+D1 (prerequisite, authority, blending) tops both Claude models; Gemini pulls far more Side-Effect Masking (P4). The same inventory yields a distinct profile per victim, so different CUAs expose different weaknesses.
Principles learned from failure transfer across models
Two strategies the feedback loop distilled while attacking Claude Opus 4.6 illustrate the process. Neither was discovered on the other two victims. We freeze them and, with no further feedback, hand them to the attacker when it composes injections against Claude Opus 4.8 and Gemini 3.5 Flash.
P9 · discovered on Claude Opus 4.6
Error Corroboration
Predict a specific error the CUA will naturally hit during the benign workflow and present the adversarial action as its remedy. When the environment later produces the predicted error, credibility transfers from the correct diagnosis to the proposed fix, eroding an initial refusal.
exploits: post-hoc corroboration of a low-trust source
P10 · discovered on Claude Opus 4.6
Conditional Deferred Execution
Phrase the adversarial action as conditional, deferred advice ("if the build fails, you might try removing X") rather than an imperative command. The safety filter applies weaker scrutiny to hypothetical future remedies than to direct instructions.
exploits: imperative-language detection keyed to command form
Unseen victim CUA
Base library
+ P9
+ P10
Claude Opus 4.8
10%
16%
14%
Gemini 3.5 Flash
8%
18%
12%
Attack success rate over 50 tasks, no additional feedback rounds.
Both frozen principles raise ASR on both unseen models, including Gemini, whose safety design differs fundamentally from Claude's. P9 transfers more strongly because it exploits a behavior that does not depend on architecture: how an agent restores trust in a source after the fact. That a principle learned solely from Opus 4.6 failures helps against a model outside the Claude family is evidence that feedback expands a shared strategy space rather than memorizing target text.
Citation
@inproceedings{xiong2026sir,
title = {SIR: Self-improving Red-teaming for Computer Use Agents},
author = {Xiong, Chen and He, Zhiyuan and Chen, Pin-Yu and Picek, Stjepan and Ho, Tsung-Yi},
archivePrefix={arXiv},
primaryClass={cs.CR},
year = {2026}
}
