Pith. sign in

REVIEW 3 major objections 5 minor 15 references

Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that the standard defense of asking users to confirm AI-agent actions backfires for low-harm 'invisible ink' prompt injections, raising attack success rather than lowering it.

desk verdict Solid benchmark and code-form injection finding, but the headline HITL amplification claim rests on an under-validated simulated user and selected low-baseline pairs. read the letter →

arxiv 2608.02018 v2 pith:BK5D2KKD submitted 2026-08-03 cs.CV

Jia-Chen Zhang , Ze-Yu Zhang , Kai-Wei Zhang This is my paper
classification cs.CV
keywords computer-useagentsindirectpromptinjectionhuman-in-the-looplow-harmattacksLLMagentsafetyattacksuccessrateadversarialbenchmarkconfirmationprompts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper identifies a blind spot in the standard defense against prompt-injection attacks on computer-use agents: pausing for user confirmation before risky actions. It argues that 'Invisible Ink Threats'—injected goals such as starring a repository, subscribing to a post, or installing an extra package—are so similar to legitimate task steps that neither safety-aligned models nor non-expert users can reliably flag them. To test this, the authors build II-Bench, a collection of 444 adversarial examples across three platforms, and HITLCUA, a sandbox that simulates a human approving or rejecting the agent's suspicious actions. Across seven leading agents, low-harm injections frequently succeed, code-form injections are stronger than natural-language ones, and adding simulated human confirmation raises attack success by 7.8 percentage points on average, with the simulated novice approving 73.5% to 83.3% of malicious requests. If true, the most widely deployed mitigation is actively counterproductive for this attack class, and defenses must reason about instruction intent rather than action severity.

What carries the argument

The load-bearing object is the Invisible Ink Threat itself: an adversarial instruction that mimics a benign task step, wrapped in a template of an attention-grabbing prefix, a deceptive rationale tying the malicious goal to the legitimate task (for example, 'required for the newest version of termcolor'), and a benign follow-up command. The evaluation machinery is HITLCUA, a hybrid sandbox that resets an OS-and-web environment to a snapshot, places the agent directly at the injection site to separate safety judgment from navigation ability, and introduces NPC2, an LLM prompted as a novice user who answers the agent's confirmation queries; NPC2's approval rates were checked against three non-expert human participants and matched their range.

What would settle it

Recruit at least 30 non-expert users to review the exact confirmation queries II-Bench generates, and run the same four CUAs with real approvals wired in. If the real users' approval rate drops below the NPC2 range (73.5% to 83.3%), or if ASR under real confirmation is not higher than the no-confirmation baseline, the paper's amplification claim fails.

Watch

Extended reading notes

Core claim

The paper claims that low-harm indirect prompt injection—an injected instruction that hijacks the agent's legitimate capabilities instead of issuing an overtly destructive command—defeats both model safety mechanisms and human oversight. Its central finding is that human-in-the-loop confirmation amplifies the threat: across every model–platform pair tested, attack success rate (ASR) under simulated user confirmation is strictly higher than without it, averaging a 7.8% increase, with the largest case (gpt-5.1 on Reddit) doubling from 21.2% to 42.4%. The authors further show that expressing an attack as executable code rather than natural language increases ASR for every evaluated model, by 8.1 points on average, and that the newest models are the most susceptible to interactive 'Post' attacks. They conclude that the failure is structural: since each individual action is behaviorally identical to legitimate work, per-action severity calibration cannot detect the attack.

Load-bearing premise

The central result depends on the simulated novice user (NPC2) approving malicious operations at roughly the same rate as real non-expert users; if real users reject more often, or the tested queries are unrepresentative, the finding that confirmation amplifies attacks would not hold.

Editorial extensions

If this is right

  • Deployed confirmation dialogs should be expected to increase, not decrease, success rates for low-harm injected goals, so safety cases built on user approval need revision.
  • Treating code blocks found in page content as executable instructions is a zero-cost attack amplifier; agents should execute only code that is grounded in the user's original request.
  • Safety evaluations should report intent-level attack success (whether the agent attempted the harmful step) separately from task completion, since navigation failures otherwise mask vulnerabilities.
  • Defense design should shift from per-action severity to intent comparison: why is this step being taken, and does it serve the user's stated task?
  • Benchmarks of this kind can be extended to voice- or mobile-automation agents where confirmation is even less informative.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If real users mirror NPC2's 73.5% to 83.3% approval rates at scale, the human-in-the-loop paradigm may also train users to treat injected requests as normal, embedding the vulnerability deeper into workflows.
  • The capability-dependent reversal in 'Post' attacks implies that as agents get better at multi-step reasoning, their compliance with deceptively rationalized social actions will likely rise unless intent-level defenses advance in parallel.
  • Because the benchmark decouples navigation from compliance, the high attack success rates suggest existing end-to-end evaluations may be understating vulnerability whenever navigation errors hide a compliance problem.
  • A direct extension would be testing whether pairing confirmation with a summary of how the step deviates from the original task—rather than a yes/no on the step itself—restores the defense's value.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces II-Bench, a benchmark of 444 low-harm indirect prompt injection attacks against computer-use agents (CUAs), and HITLCUA, a hybrid sandbox that combines OSWorld and WebArena/TheAgentCompany environments with two NPCs, one of which (NPC2) simulates a novice user for human-in-the-loop (HITL) confirmation. The authors evaluate seven CUAs and report that Invisible Ink Threats achieve high attack success rates (ASR), that code-form injections are more effective than natural-language ones, and that download attacks are the most successful category. The central new claim is that HITL confirmation amplifies rather than mitigates these attacks: in Table 3, ASR under HITL is reported as strictly higher than conventional testing for all eight evaluated model-platform pairs, with an average increase of 7.8%. The paper validates NPC2 by comparing its aggregate yes rate to three human participants (77.1%, 68.5%, 74.3% versus 73.5%–83.3%).

Significance. If the HITL amplification result holds, it is an important and counterintuitive finding with direct practical relevance: it challenges the widely deployed defense of requiring user confirmation before sensitive operations, a defense that is currently embedded in commercial CUA deployments and governance frameworks. The II-Bench benchmark itself is a substantial contribution: it is broad (444 examples, three platforms, three attack categories, two instruction modes, two injection modalities), grounded in realistic hybrid OS/web environments, and uses a decoupled evaluation design that separates navigation capability from adversarial robustness. The paper also gives explicit credit to the value of comparing NPC2 against real human participants, which is a useful external anchor. The main weaknesses are that the central HITL claim rests on NPC2 fidelity and on a small, selected set of model-platform pairs, and the reported ASR values lack uncertainty quantification despite stochastic components. These issues are fixable and do not invalidate the benchmark contribution, but they currently limit the strength of the headline conclusion.

major comments (3)
  1. [Clarifying HITL Simulation Design] The validation of NPC2 against three non-expert participants reports only aggregate Yes Rates (77.1%, 68.5%, 74.3% versus NPC2's 73.5%–83.3%). Because HITL ASR depends on which specific confirmation queries are approved, not merely on the overall approval rate, this aggregate match does not establish that NPC2 reproduces per-example approval behavior. If NPC2 systematically over-approves exactly the queries that flip a no-HITL failure into an HITL success (e.g., credential exfiltration or untrusted package installation), the amplification claim in Table 3 could reverse even with identical aggregate yes rates. Please report per-query agreement (e.g., Cohen's kappa or a confusion matrix) or otherwise demonstrate that the amplification is robust across the observed approval patterns.
  2. [Experimental settings] The central claim that HITL ASR is 'strictly higher' across all eight evaluated pairs is based on single point estimates, despite temperature=1 being reported for the evaluated CUAs and no sampling temperature being stated for NPC2. No confidence intervals, standard errors, or repeated runs are reported. With stochastic components and modest per-condition sample sizes (the underlying example counts behind the Table 3 percentages are not given), the strict ordering may not be reproducible. Please provide variance estimates (e.g., multiple seeds, binomial confidence intervals) and specify the sampling temperature for NPC2.
  3. [Clarifying HITL Simulation Design] The manuscript states that model-platform pairs with comparatively ineffective attacks under conventional testing were selected for HITL evaluation, but it does not report the selection criteria, the number of candidate pairs considered, or the results for the excluded pairs. If the selection was made after observing the no-HITL ASRs, the observed increases may partly reflect regression to the mean, and the conclusion that HITL 'consistently amplifies' Invisible Ink Threats would not generalize. Please report the full set of pairs evaluated and a pre-registered or otherwise explicit selection rule.
minor comments (5)
  1. [Table 3] The notation '42.4↑36.4 83.3' is ambiguous; please label the three numbers for each HITL condition (ASR, Query Rate, Yes Rate) in the table header or use separate columns.
  2. [II-Bench] The paper states that II-Bench comprises 444 examples 'including 111 benign-adversarial pairings'; the relation between 444 examples and 111 pairings is not explained and should be clarified.
  3. [Introduction] The 'RTC-Bench' baseline in Figure 1 is mentioned without a citation or definition; please provide a reference or a one-sentence explanation.
  4. [Evaluation Metrics] The intent-extraction rule used to define ASR is not described in enough detail to be independently applied; please provide the exact extraction prompt or rubric, ideally in the supplementary material.
  5. [Clarifying HITL Simulation Design] The three human participants used to validate NPC2 are not described in terms of recruitment criteria, selection process, or the exact instructions they received; please add this information so that the representativeness claim can be assessed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the ASR and HITL-amplification results are empirical measurements, and the HITL claim is mediated by an externally validated user simulation rather than by construction.

full rationale

The paper's central claims are benchmark measurements rather than derived results. ASR is defined as extracting the presence of intent to execute an injected goal; this is an explicitly stated measurement convention intended to avoid conflating navigation failure with safety failure, and it does not assume the benchmark's conclusions. The HITL amplification result is read directly from Table 3 by comparing ASR with and without NPC2 consultation; NPC2's approval behavior is validated against three human participants on the same confirmation queries (Yes Rates 77.1%, 68.5%, and 74.3% versus the simulated user's 73.5%–83.3%), so the amplification is an empirical finding rather than an input. Concerns that aggregate yes-rate matching may not reproduce per-query approval decisions are generalizability or correctness risks, not circular-reasoning flaws. The only apparent self-reference (Zhang et al. 2025 in a general capability sentence) is contextual and non-load-bearing. No fitted parameter is relabeled as a prediction, no uniqueness theorem or ansatz is imported from the authors' own prior work, and no known result is merely renamed. Consequently, no step in the derivation chain reduces by construction to its own inputs.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numeric free parameters are fitted. The central claims rest on domain assumptions: environment fidelity, NPC2 fidelity, the intent-based ASR proxy, and the realism of the indirect injection threat model. These are assumptions rather than fitted constants, and the paper supplies only partial external validation for them.

assumptions (4)
  • domain assumption OSWorld, WebArena, and TheAgentCompany replicas validly represent real OS and web environments.
    HITLCUA is built on these platforms and the threat model assumes their behavior matches real systems well enough to measure CUA vulnerabilities (Method section).
  • domain assumption The LLM-based NPC2 simulated user is a faithful proxy for non-expert human approval decisions.
    The HITL amplification finding depends on NPC2's Yes Rate matching real humans; the only evidence is three participants in a validation exercise.
  • domain assumption Extracted intent to execute a harmful instruction is an acceptable proxy for attack success.
    Evaluation Metrics defines ASR by intent presence, not observed completion; if intent detection overcounts, reported ASR overstates real exploitation.
  • domain assumption Indirect prompt injection through adversarial posts is a realistic attacker capability.
    The benchmark embeds injected instructions in Reddit, OwnCloud, and RocketChat content, assuming attackers can place such content where CUAs will read it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents." pith.science (2026). https://pith.science/paper/BK5D2KKD

@misc{pith2026260802018,
  author       = {Pith},
  title        = {Pith review of: Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BK5D2KKD}},
  note         = {Machine review of arXiv:2608.02018}
}
read the original abstract

Computer-use agents (CUAs), which empower large language models to autonomously operate operating systems and the web, are increasingly vulnerable to indirect prompt injection attacks. A widely adopted defense is the human-in-the-loop paradigm, in which the agent pauses for explicit user confirmation before executing sensitive operations. While effective against conspicuously high-harm attacks, this defense offers little protection against what we term Invisible Ink Threats: low-harm injected goals, such as starring a repository or installing a package, that are behaviorally indistinguishable from legitimate task execution and thus evade both model safety mechanisms and human oversight. To systematically investigate this blind spot, we present II-Bench, a collection of seemingly harmless adversarial tasks. II-Bench comprises 444 examples targeting confidentiality and integrity attacks across three platforms, spanning three attack categories: page navigation and interaction, sensitive information exfiltration, and code download and execution. Each category is instantiated in both natural language and code forms under two levels of instruction specificity. Furthermore, we construct HITLCUA, a comprehensive adversarial testing framework that integrates a real virtual machine operating system environment with isolated Docker-based web platforms, and simulates human participation by allowing CUAs to consult an API-simulated user before proceeding with suspicious operations. Extensive evaluations of leading CUAs reveal that low-harm injections frequently bypass both agent defenses and simulated user review, exposing severe and previously underexplored security risks in current CUAs.

Figures

Figures reproduced from arXiv: 2608.02018 by the authors.

Figure 1
Figure 1. Under the same workflow architecture, strong at [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the HITLCUA Framework and II-Bench. The framework incorporates two NPCs: NPC1 simulates a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Schematic diagrams of the interfaces for different [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visualization of different adversarial task results [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

15 extracted references · 7 canonical work pages

  1. [2]

    arXiv:2504.11281

    The Obvious Invisible Threat: LLM-Powered GUI Agents’ Vulnerability to Fine-Print Injections. arXiv:2504.11281. Debenedetti, E.; Shumailov, I.; Fan, T.; and et al

    arXiv

  2. [3]

    arXiv:2503.18813

    De- feating Prompt Injections by Design. arXiv:2503.18813. Debenedetti, E.; Zhang, J.; and et al

    arXiv

  3. [6]

    MiniMax Sparse Attention

    MiniMax Sparse Attention. arXiv:2606.13392. Li, A.; Zhou, Y.; Raghuram, V. C.; and et al

    work page Pith review arXiv

  4. [7]

    arXiv:2502.08586

    Com- mercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks. arXiv:2502.08586. Li, Y.; He, J.; Zhou, X.; and et al

    arXiv

  5. [9]

    Ruan, Y.; Dong, H.; Wang, A.; and et al

    Qwen3.7: The Agent Frontier. Ruan, Y.; Dong, H.; Wang, A.; and et al. 2024a. Identifying the Risks of LM Agents with an LM-Emulated Sandbox. In The Twelfth International Conference on Learning Repre- sentations. Ruan, Y.; Dong, H.; Wang, A.; and et al. 2024b. Identifying the Risks of LM Agents with an LM-Emulated Sandbox. InInternational Conference on Lea...

    arXiv 2024

  6. [10]

    Singh, A.; Fry, A.; Perelman, A.; and et al

    Progent: Se- curing AI Agents with Privilege Control.arXiv e-prints, arXiv:2504.11703. Singh, A.; Fry, A.; Perelman, A.; and et al

    arXiv

  7. [11]

    arXiv:2601.03267

    OpenAI GPT-5 System Card. arXiv:2601.03267. Team,V.;Hong,W.;Gu,X.;andetal.2026.GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents. arXiv:2604.26752. Xie, T.; Yuan, M.; Zhang, D.; and et al

    arXiv 2026

  8. [14]

    InFindings of the Association for Computational Linguistics: ACL 2024, 10471–10506

    In- jecAgent:BenchmarkingIndirectPromptInjectionsinTool- Integrated Large Language Model Agents. InFindings of the Association for Computational Linguistics: ACL 2024, 10471–10506. Bangkok, Thailand: Association for Compu- tational Linguistics. Zhang, J.-C.; Xiong, Y.-J.; Xia, C.-M.; and et al

    work page 2024

Show all 15 references
  1. [15]

    InProceedings of the 63rd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), 8387–8401

    Attacking Vision- Language Computer Agents via Pop-ups. InProceedings of the 63rd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), 8387–8401. Vienna,Austria:AssociationforComputationalLinguistics. ISBN 979-8-89176-251-0. Zheng,B.;Gou,B...

    2024

  2. [38]

    InFindings of the Association for Computa- tionalLinguistics:ACL2026,11986–11998.SanDiego,Cal- ifornia, United States: Association for Computational Lin- guistics

    Ying,Z.;Shao,Y.;Gan,J.;andetal.2026.SecureWebArena: AHolisticSecurityEvaluationBenchmarkforLVLM-based Web Agents. InFindings of the Association for Computa- tionalLinguistics:ACL2026,11986–11998.SanDiego,Cal- ifornia, United States: Association for Computational Lin- guistics....

    2026

  3. [2022]

    InProceed- ings of the 30th ACM International Conference on Multime- dia,MM’22,3185–3194.NewYork,NY,USA:Association for Computing Machinery

    Online Deep Learning from Doubly-Streaming Data. InProceed- ings of the 30th ACM International Conference on Multime- dia,MM’22,3185–3194.NewYork,NY,USA:Association for Computing Machinery. ISBN 9781450392037. Liao,Z.;Jones,J.;Jiang,L.;andetal.2026. RedTeamCUA: RealisticAdvers...

    2026

  4. [2023]

    InFindings of the Association for Computational Linguistics: EMNLP 2023, 2803–2821

    Understand- ing HTML with Large Language Models. InFindings of the Association for Computational Linguistics: EMNLP 2023, 2803–2821. Singapore: Association for Computational Lin- guistics. Kuntz,T.;Duzan,A.;Zhao,H.;andetal.2025. OS-Harm:A Benchmark for Measuring Safety of Comp...

    2023

  5. [2024]

    InAdvances in Neural Information Processing Systems, volume 37, 52040–52094

    OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. InAdvances in Neural Information Processing Systems, volume 37, 52040–52094. Curran Associates, Inc. Xu,F.F.;Song,Y.;Li,B.;andetal.2026.Theagentcompany: benchmarking llm agents on conse...

    2026

  6. [2025]

    arXiv:2504.14064

    DoomArena: A framework for Testing AI Agents Against Evolving Security Threats. arXiv:2504.14064. Chen, A.; Wu, Y.; Zhang, J.; and et al

  7. [2026]

    arXiv:2605.28116

    MIRAGE: Context-AwarePromptInjectionagainstMobileGUIAgents via User-Generated Content. arXiv:2605.28116. Gur, I.; Nachum, O.; Miao, Y.; and et al

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.