REVIEW 4 major objections 4 minor 25 references
This paper claims that prompt injection attacks can be systematically optimized against LLM-driven navigation agents, achieving an average attack success rate of 87.5% across indoor and outdoor agents.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 09:26 UTC pith:GESJJOLE
load-bearing objection A timely first stab at prompt-injection attacks on navigation agents, but the headline ASR is inflated by a permissive 'any deviation is failure' metric and is measured on the same model used for optimization. the 4 major comments →
PINA: Prompt Injection Attack against Navigation Agents
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On its own terms, the paper establishes that prompt injection attacks can be optimized under realistic black-box, long-context, and action-executable constraints to reliably disrupt LLM navigation agents. The key empirical claim is an average attack success rate of 87.5% across an indoor NavGPT agent and an outdoor aerial agent, with the outdoor agent failing in 100% of attacks because any deviation from its perfect clean trajectory is counted as failure. The paper also reports that the optimized prompts transfer to a different LLM backend (GPT-4) and degrade navigation quality metrics (SPL, nDTW, CLS) well beyond baseline injections.
What carries the argument
The central mechanism is PINA's feedback loop combining two analysis modules. The Attack Evaluator turns multiple navigation metrics (trajectory length, navigation error, success rate, SPL, nDTW, CLS) into a single scalar score. The Distribution Analyzer uses a surrogate LLM to measure global output shifts via average KL divergence and local token-level influence via changes in maximum probability and entropy, identifying 'key tokens'. Adaptive Prompt Refinement translates these scores and tokens into natural-language feedback that iteratively rewrites the injection prompt until the attack score exceeds a threshold.
Load-bearing premise
The attack's success rests on the surrogate simulator used to optimize prompts being behaviorally faithful to the real, closed target agent; the indoor 75% result is produced on the very model used for optimization, and the outdoor 100% result follows from counting any deviation from a perfect path as mission failure.
What would settle it
Run PINA optimization on NavGPT/GPT-3.5 as the surrogate, then evaluate the optimized prompts on a navigation agent with different architecture (e.g., Balcı et al.) or a different LLM backend never used in optimization, using the strict criterion that the agent must arrive within a fixed distance of the goal to count as success; if ASR on this held-out target falls to baseline levels, the claimed black-box transferability fails.
If this is right
- Attackers who can inject text into any input channel of an LLM navigation agent can systematically craft prompts that cause mission failure, with success rates far above naive injection.
- Optimized prompts transfer across LLM backends (from GPT-3.5 to GPT-4) and across navigation agent architectures, widening the attack surface.
- Lightweight defenses such as self-reminder prefixes reduce but do not neutralize the attack (ASR drops from 75% to 68.8%), so stronger defenses are needed.
- Navigation-quality metrics degrade sharply under attack, meaning even 'successful' runs become inefficient and unsafe.
Where Pith is reading between the lines
- The 75% indoor ASR is measured on NavGPT with GPT-3.5-turbo, the same model that served as the Attack Evaluator during prompt optimization; treating this as a black-box result may overstate transferability, whereas the outdoor 100% ASR partly reflects a permissive mission-failure criterion (any NE deviation).
- The strongest unstated assumption is that the surrogate simulator is behaviorally faithful to the closed target; if the target differs in architecture, prompt format, or post-processing, the optimized prompt may lose effectiveness.
- The method is a natural testbed for evaluating prompt-injection defenses for embodied agents: the same Attack Evaluator metrics could be reused to score defensive wrapper prompts, instruction hierarchies, or output validation modules.
- A sharper evaluation design would hold out the target agent from the optimization loop entirely and use a strict 'reached the designated goal location' criterion rather than deviation-based failure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PINA, an adaptive prompt optimization framework for prompt injection attacks against LLM-based navigation agents. PINA combines an Attack Evaluator that aggregates navigation metrics into a scalar score, a Distribution Analyzer that measures KL divergence and identifies key tokens using a surrogate LLM, and an Adaptive Prompt Refinement loop that iteratively rewrites injection prompts. Experiments on an indoor NavGPT agent and an outdoor Balcı et al. agent report ASRs of 75% and 100%, an average ASR of 87.5%, superiority over four baselines, and robustness under ablation and a self-reminder defense. The paper claims to be the first systematic prompt-injection study for navigation agents.
Significance. If the quantitative claims were substantiated, this would be a meaningful contribution to the security of embodied LLM agents: it would show that black-box attackers can systematically craft prompts that misguide physical navigation, with implications for safety and for the design of defenses. The paper also ships code, includes an ablation study, and evaluates an adaptive defense, which are useful steps. However, the central empirical claim currently rests on an ASR definition that counts any statistically detectable deviation from clean behavior as a mission failure, and on an evaluation setting where the model used to optimize prompts is also the primary victim. These issues are load-bearing: until they are fixed, the reported 87.5% average ASR and the claimed superiority over baselines are not established.
major comments (4)
- [§4.1, §4.2, Table 1] The ASR definition is problematic. Section 4.1 states that 'five times the standard deviation (5δ) of SPL' is used as the threshold, without specifying the distribution from which δ is computed, the number of clean trials, or why a statistical deviation corresponds to mission failure. Section 4.2 then explicitly says that for the outdoor agent, because the clean system always has NE=0, 'any deviation caused by injection is counted as a complete failure.' This makes ASR a measure of output perturbation, not of task success or dangerous behavior. A prompt that nudges the drone a few centimeters off the ideal path is counted as a successful attack, inflating the outdoor ASR to 100% and the headline average to 87.5%. The authors should redefine ASR using task-level criteria such as failure to reach the goal (SR=0) or a clearly specified, pre-registered performance threshold, and report the c
- [§4.1, §4.2, Algorithm 1] The black-box claim is weakened by circularity. In Section 4.1 the Attack Evaluator is NavGPT with GPT-3.5-turbo, and Algorithm 1 selects the prompt T* with the best attack score S from that same evaluator. In Section 4.2, the first victim is also NavGPT with GPT-3.5-turbo. Thus the reported 75% indoor ASR is measured on the exact model used to optimize the prompt, not on a held-out black-box target. The transferability to GPT-4 and to the outdoor agent is the only genuinely held-out evidence, but the outdoor result is confounded by the ASR definition. The authors should evaluate PINA on a navigation agent (or at least a model) that was not used in the optimization loop, and report ASR separately for in-distribution and held-out targets.
- [§4.1, Table 1] The definition of Navigation Error (NE) is nonstandard and appears internally inconsistent with the reported numbers. Section 4.1 defines NE as 'the maximum Euclidean distance between the actual flight path and the reference path,' whereas the navigation literature typically uses NE for final-position error. Under this nonstandard definition, a successful clean trajectory could have a large maximum deviation, yet Section 4.2 reports clean outdoor NE=0. Moreover, in the indoor results, PINA increases NE only from 8.49 to 8.76 while ASR jumps to 75%; if ASR reflects mission failure, one would expect a much larger effect on terminal error. The authors should clarify which NE definition is used, align it with the standard metric, and report the goal-reaching success rate (SR) alongside ASR so that mission failure and path perturbation are not conflated.
- [§4.2, Tables 1–3] No sample sizes, confidence intervals, or significance tests are reported for any ASR value or navigation metric. The ASR percentages (e.g., 25.00%, 37.50%, 75.00%) appear to be based on small samples, and the outdoor claim 'clean system always succeeds (NE=0)' suggests a small or trivial task set. Without sample sizes or uncertainty bounds, the 'surpasses all baselines' conclusion and the ablation differences (69.25% vs. 72.90% vs. 75.00%) are not statistically supported. Please report the number of test episodes for each agent, per-condition sample sizes, and confidence intervals or a significance test.
minor comments (4)
- [§4.1] The notation '5δ of SPL' is ambiguous: δ should be defined as the standard deviation of what distribution, measured over how many clean runs. Please define it explicitly.
- [Table 1 and Table 2] Abbreviations such as 'KLM' and 'KTI' are not defined in the table captions. Readers must infer them from the text. Please add a note or expand the abbreviations.
- [§3.3, Algorithm 1] The termination threshold τ and the attack-score weights w in S = wᵀ M_nav are never specified. Please state how w is chosen and provide a sensitivity analysis, since the reported ASR depends on this weighted combination.
- [References] Reference [2] (Balcı et al.) lacks a venue and publication year. Please complete the bibliographic information.
Circularity Check
The headline indoor 75% ASR is measured on the same NavGPT/GPT-3.5 model used as the optimization evaluator, and the outdoor 100% ASR is forced by defining any deviation as mission failure; the 87.5% average therefore partly reduces to in-sample selection and metric construction.
specific steps
-
fitted input called prediction
[Section 4.1 (Implementation; Algorithm 1) and Section 4.2 (Table 1)]
"In the optimization process, we utilize NavGPT [1] (with LLM GPT-3.5-turbo) as our Attack Evaluator and random 100 examples from R2R [16] as our instruction set for training. ... As shown in Tab. 1, PINA consistently achieves the highest ASR, reaching 75% on indoor agents and 100% on the outdoor agent, surpassing all the baselines by a clear margin."
Algorithm 1 selects the best prompt T* by running S <- A(T) on the AttackEvaluator and keeping the prompt with highest attack score, where S = w^T M_nav. The AttackEvaluator is NavGPT with GPT-3.5-turbo, and Table 1 reports the first victim as NavGPT with GPT-3.5-turbo using the same R2R instruction family. Thus the 75% indoor ASR is a selection-optimized quantity measured on the exact model and task used to pick the prompt, not a held-out black-box prediction. The framing calls this black-box because gradients and parameters are hidden, but the evaluation target is identical to the optimization oracle, so the headline number is in-sample by construction.
-
self definitional
[Section 4.1 (Evaluation Metrics) and Section 4.2 (Attack Effectiveness)]
"Attack Success Rate (ASR) is the percentage of prompt injection attack samples leading to the victim agent's mission failure. We use five times the standard deviation (5δ) of SPL as the threshold in ASR. ... Since the clean system always succeeds (NE=0), any deviation caused by injection is counted as a complete failure, leading to an ASR of 100% and SPL, nDTW collapse to near zero."
ASR is operationally defined as any SPL deviation beyond a 5δ threshold, not as demonstrated mission failure. For the outdoor agent the paper explicitly equates 'any deviation caused by injection' with 'complete failure,' so the 100% outdoor ASR follows arithmetically from the metric definition rather than from evidence of route failure or dangerous maneuvers. The same criterion also inflates the 87.5% average and makes baseline comparisons reflect output perturbation rather than task-breaking, since even a small NE or SPL change counts as a successful attack.
full rationale
The paper is not wholly circular: it contains a genuine transfer experiment to GPT-4, an ablation study, and an adaptive-defense comparison, and it does not rely on a load-bearing self-citation or an imported uniqueness theorem. However, the two numbers that produce the central average ASR are not independent. The indoor 75% is obtained by optimizing prompts against NavGPT/GPT-3.5 and then evaluating ASR on that same model, so the reported effectiveness is partly a fitted maximum rather than a prediction. The outdoor 100% is forced by the paper's own statement that any deviation counts as complete failure when clean NE=0, making the metric self-definitional. These two issues together make the headline claim '87.5% average ASR' partially circular, warranting a score of 6 rather than a lower score. The remaining ablation and transfer results provide some independent content, but they do not rescue the headline from the in-sample evaluation and the permissive ASR criterion.
Axiom & Free-Parameter Ledger
free parameters (5)
- Attack score weights w =
unspecified (sum to 1)
- Distribution score blend α =
0.5
- Key-token threshold τ_token
- Termination threshold τ
- ASR threshold 5δ of SPL =
five times SPL standard deviation
axioms (4)
- domain assumption The surrogate simulator matches the target in architecture, I/O formats, and high-level behavior (Sec. 2).
- domain assumption Token-level probabilities from Llama2-7b reveal the distributional shift relevant to the target agent (Sec. 3.2).
- domain assumption Navigation metrics can be aggregated into a system-independent attack score with a single weight vector (Sec. 3.1).
- ad hoc to paper Mission failure can be defined uniformly as SPL below five standard deviations of clean SPL (Sec. 4.1).
read the original abstract
Navigation agents powered by large language models (LLMs) convert natural language instructions into executable plans and actions. Compared to text-based applications, their security is far more critical: a successful prompt injection attack does not just alter outputs but can directly misguide physical navigation, leading to unsafe routes, mission failure, or real-world harm. Despite this high-stakes setting, the vulnerability of navigation agents to prompt injection remains largely unexplored. In this paper, we propose PINA, an adaptive prompt optimization framework tailored to navigation agents under black-box, long-context, and action-executable constraints. Experiments on indoor and outdoor navigation agents show that PINA achieves high attack success rates with an average ASR of 87.5%, surpasses all baselines, and remains robust under ablation and adaptive-attack conditions. This work provides the first systematic investigation of prompt injection attacks in navigation and highlights their urgent security implications for embodied LLM agents.
Reference graph
Works this paper leans on
-
[1]
INTRODUCTION Large language models (LLMs) are increasingly adopted as natural interfaces for robot navigation, enabling users to issue natural language instructions for route planning and control [1–3]. While this paradigm improves accessibility, it also introduces a critical security risk: prompt injection attacks [4, 5]. By embedding malicious instructi...
Pith/arXiv arXiv 2025
-
[2]
We adopt a black-box assumption: the attacker cannot access or modify the target system’s internal parameters or low-level controllers
THREAT MODEL In this paper, we consider a prompt injection attack that aims to (1) prevent a navigation agent from reaching its designated target (i.e., reduce task success rate) and (2) degrade trajec- tory quality (e.g., increased path length or larger deviation). We adopt a black-box assumption: the attacker cannot access or modify the target system’s ...
-
[3]
As shown in Fig
METHODOLOGY We proposePINA, a prompt optimization framework that im- proves the effectiveness of injection attacks against naviga- tion agents. As shown in Fig. 2,PINAconsists of three com- ponents: (1) theAttack Evaluator, which quantifies attack impact using multiple navigation metrics; (2) theDistribu- tion Analyzer, which captures global distributiona...
-
[4]
EV ALUATION 4.1. Experimental Setup Victim Agent(s).We consider one indoor (NavGPT [1]) and one outdoor (Balcı et al. [2]) navigation agent as victims, where [1] is fine-tuned by the R2R dataset [16], and [2] was prompt-tuned. Attack Baseline(s).Following [5] setups, we select four prompt injection attacks as baselines, which consist of Naive Attack, Esca...
arXiv 2025
-
[5]
CONCLUSION In this work, we presentedPINA, the first framework that systematically adapts prompt injection attacks to LLM-based navigation agents. Unlike prior studies focusing on text or web applications, our work highlights the unique risks of nav- igation settings, where adversarial manipulations can directly misguide physical actions. Looking forward,...
-
[6]
Benchmarking and defending against indirect prompt injection attacks on large language models,
Jingwei Yi, Yueqi Xie, Bin Zhu, Emre Kiciman, Guangzhong Sun, Xing Xie, and Fangzhao Wu, “Benchmarking and defending against indirect prompt injection attacks on large language models,” inProceed- ings of the 31st ACM SIGKDD Conference on Knowl- edge Discovery and Data Mining V . 1, 2025, pp. 1809– 1820
2025
-
[7]
Navgpt: Explicit reasoning in vision-and-language navigation with large language models,
Gengze Zhou et al., “Navgpt: Explicit reasoning in vision-and-language navigation with large language models,” inProceedings of the AAAI Conference on Ar- tificial Intelligence, 2024
2024
-
[8]
Prompting large language models for aerial navigation,
Emirhan Balcı et al., “Prompting large language models for aerial navigation,”
-
[9]
Chatgpt for robotics: Design prin- ciples and model abilities,
Sai H Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor, “Chatgpt for robotics: Design prin- ciples and model abilities,”Ieee Access, vol. 12, pp. 55682–55696, 2024
2024
-
[10]
Prompt injection at- tack against llm-integrated applications,
Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, et al., “Prompt injection at- tack against llm-integrated applications,”arXiv preprint arXiv:2306.05499, 2023
Pith/arXiv arXiv 2023
-
[11]
Formalizing and benchmarking prompt injection attacks and defenses,
Yupei Liu et al., “Formalizing and benchmarking prompt injection attacks and defenses,” in33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 1831–1847
2024
-
[12]
Uavs meet llms: Overviews and perspectives towards agentic low- altitude mobility,
Yonglin Tian, Fei Lin, Yiduo Li, Tengchao Zhang, Qiyao Zhang, Xuan Fu, Jun Huang, Xingyuan Dai, Yutong Wang, Chunwei Tian, et al., “Uavs meet llms: Overviews and perspectives towards agentic low- altitude mobility,”Information Fusion, vol. 122, pp. 103158, 2025
2025
-
[13]
Badrobot: Jailbreaking embodied llms in the physical world,
Hangtao Zhang, Chenyu Zhu, Xianlong Wang, Ziqi Zhou, Changgan Yin, Minghui Li, Lulu Xue, Yichen Wang, Shengshan Hu, Aishan Liu, et al., “Badrobot: Jailbreaking embodied llms in the physical world,” arXiv preprint arXiv:2407.20242, 2024
Pith/arXiv arXiv 2024
-
[14]
Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injec- tion,
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz, “Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injec- tion,” inProceedings of the 16th ACM workshop on artificial intelligence and security, 2023, pp. 79–90
2023
-
[15]
Yunpeng Gao, Zhigang Wang, Linglin Jing, Dong Wang, Xuelong Li, and Bin Zhao, “Aerial vision- and-language navigation via semantic-topo-metric rep- resentation guided llm reasoning,”arXiv preprint arXiv:2410.08500, 2024
Pith/arXiv arXiv 2024
-
[16]
Mapgpt: an au- tonomous framework for mapping by integrating large language model and cartographic tools,
Yifan Zhang, Zhengting He, Jingxuan Li, Jianfeng Lin, Qingfeng Guan, and Wenhao Yu, “Mapgpt: an au- tonomous framework for mapping by integrating large language model and cartographic tools,”Cartography and Geographic Information Science, vol. 51, no. 6, pp. 717–743, 2024
2024
-
[17]
Towards learning a generalist model for embodied navigation,
Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang, “Towards learning a generalist model for embodied navigation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2024, pp. 13624–13634
2024
-
[18]
Llama 2: Open foundation and fine-tuned chat mod- els,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al., “Llama 2: Open foundation and fine-tuned chat mod- els,”arXiv preprint arXiv:2307.09288, 2023
Pith/arXiv arXiv 2023
-
[19]
You should remind
68.80% 6.13 8.59 5.49 36.08 38.04 to the full design. These results demonstrate that KL diver- gence and key-token analysis are complementary, and their combination is essential for maximizing attack success while imposing the strongest disruption on trajectory efficiency and fidelity. 4.4. Adaptive Defense We further evaluate our method against target sy...
-
[20]
Typefly: Flying drones with large language model,
Guojun Chen, Xiaojing Yu, Neiwen Ling, and Lin Zhong, “Typefly: Flying drones with large language model,”arXiv preprint arXiv:2312.14950, 2023
Pith/arXiv arXiv 2023
-
[21]
Wasp: Benchmarking web agent security against prompt injection attacks,
Ivan Evtimov et al., “Wasp: Benchmarking web agent security against prompt injection attacks,”arXiv preprint arXiv:2504.18575, 2025
Pith/arXiv arXiv 2025
-
[22]
Optimizing generative ai by backpropagating language model feedback,
Mert Yuksekgonul et al., “Optimizing generative ai by backpropagating language model feedback,”Nature, vol. 639, no. 8055, pp. 609–616, 2025
2025
-
[23]
Vision-and-language naviga- tion: Interpreting visually-grounded navigation instruc- tions in real environments,
Peter Anderson et al., “Vision-and-language naviga- tion: Interpreting visually-grounded navigation instruc- tions in real environments,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3674–3683
2018
-
[24]
Recent advances in robot navigation via large language models: A review,
Haotian Pan et al., “Recent advances in robot navigation via large language models: A review,” 2024
2024
-
[25]
Defending chatgpt against jailbreak attack via self-reminders,
Yueqi Xie et al., “Defending chatgpt against jailbreak attack via self-reminders,”Nature Machine Intelligence, 2023
2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.