{"id":"3e79aae9-19f3-4470-b2bf-6656f6a2e2b2","arxiv_id":"2412.04415","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Adding the prefix 'Ignore the document' to adversarial prompts increases measured attack success rates against several RAG-based LLM agents, though the evaluation is underspecified.","lead":"This preprint tests whether adding the phrase 'Ignore the document' to prompts makes RAG-based LLM agents override retrieved context and produce harmful outputs. The authors report high attack success rates across several LLMs and argue that current defenses are insufficient.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2 never reports the promised 'ASR with Prefix' condition from §2.3, so the central claim that 'Ignore the document' yields high ASR is not backed by the only quantitative results.","rationale":"The reader's weakest assumption was the unstated ASR scoring rubric. I agree that the rubric is undefined, but the more load-bearing problem is that Table 2 does not contain the central comparison at all. Section 2.3 promises a Baseline ASR and an 'ASR with Prefix' metric; Table 2 instead shows Baseline, Adaptive Attack Prompt, and ArtPrompt. No defined row or column isolates the 'Ignore the document' prefix, and the 'w/o' row labels are unexplained. Because the abstract's entire claim is that this specific simple prefix achieves high ASR, the quantitative support for that claim is missing from the paper as written. This concern is internal to the manuscript: it does not depend on external consensus, code availability, or scoring leniency. It could be resolved by a one-column addition; if the prefix-only ASR is high and well above baseline, the central claim would gain direct support. The paper's own Limitations section already concedes limited generalizability and no real-world validation, but those are secondary. My recommendation is unchanged from the reader's REJECT because the evidence bar for the stated claim is not met; the missing Table 2 condition strengthens that rejection rather than moving it to a different category. I mark agreement as partial because the reader focused on rubric ambiguity, whereas I see a prior, more fundamental gap: the key condition is not reported.","tokens_in":5719,"tokens_out":6857,"duration_ms":65063,"concrete_test":"Request that the authors add a dedicated 'Prefix-only ASR' column to Table 2, reusing the same EPASS prompts with the only modification being the 'Ignore the document' prefix, alongside the exact scoring rubric and per-prompt classifications. If the prefix-only ASR is not clearly above the baseline ASR for each model—or if it turns out that the Adaptive Attack Prompt or ArtPrompt columns already include the prefix—then the high ASRs cannot be attributed to the prefix, and the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that adding the prefix 'Ignore the document' yields a high attack success rate. The only results table (Table 2) reports three conditions: Baseline, Adaptive Attack Prompt, and ArtPrompt, while §2.3 defines exactly two metrics: Baseline ASR and 'ASR with Prefix.' No column or row reports the with-prefix condition, and the row labels ('Gemma2 w/o' vs 'Gemma2') are never defined. Unless the prefix was silently folded into the Adaptive Attack Prompt or ArtPrompt condition—which the text does not state—the headline result is absent from the reported evidence. The abstract's conclusion is therefore unsupported by the only quantitative table. Moreover, no scoring rubric defines what counts as a successful manipulation, so even the reported columns are not interpretable. This is more basic than the scoring-rubric concern: even with a perfect rubric, the paper does not display the effect of the prefix.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims that a simple adversarial prefix, \"Ignore the document\", can make LLM-based RAG agents ignore retrieved document context and produce dangerous or unintended outputs, thereby bypassing safety mechanisms. The authors test 1,134 prompts from the EPASS dataset against several instruction-tuned models, comparing three conditions (baseline, adaptive attack prompt, ArtPrompt). They report attack success rates and attribute the vulnerability to a lack of hierarchical instruction prioritization and to the inadequacy of agent-level defenses. The central hypothesis is that the prefix alone yields a high ASR.","tokens_in":5927,"tokens_out":7378,"duration_ms":125232,"significance":"The question of whether a short natural-language prefix can override RAG context is practically important for the safety of RAG-based agents. The paper uses a reasonably sized public dataset and covers several model families. However, the current manuscript does not provide the evidence needed for the claim: the 'ASR with Prefix' condition defined in Section 2.3 is never reported in Table 2, and there is no scoring rubric or statistical detail. Therefore the significance of the finding cannot be assessed from the submitted text; the contribution is potentially relevant but unverified.","major_comments":[{"comment":"The paper defines 'ASR with Prefix' as a primary metric, but Table 2 contains no column or row labelled 'with prefix' or 'Prefix'. The row labels 'w/o' versus unlabeled rows are not defined anywhere, and the sentence after Table 2 ('Models with pre-trained defense mechanisms (w/) ...') is inconsistent with the table. As a result, the abstract's claim that 'Ignore the document' yields a high ASR is not supported by the only quantitative table. A revision must explicitly report the with-prefix condition for each attack type and define the row labels.","section":"Section 2.3, Table 2"},{"comment":"No scoring rubric is provided. The paper does not state what counts as a successful manipulation, how refusals or hedges are classified, or whether outputs are scored by a human, a classifier, or the model itself. Without this, the numerical ASR values in Table 2 (e.g., 0.189 vs 0.327 for Gemma2 baseline) are not interpretable. Please define the rubric, provide example prompt/output pairs for each model, and release the evaluation prompts and outputs to make the results reproducible.","section":"Section 2.3"},{"comment":"No trial counts, seeds, or confidence intervals are reported. ASR is stochastic; for small sample sizes the differences between conditions (e.g., GPT-4o baseline 0.072 vs 0.073, or Llama3.1 0.054 vs 0.034) are within sampling noise. The paper should report the number of prompts used per condition, the number of independent runs, and a measure of variance such as 95% confidence intervals or standard error.","section":"Section 3"}],"minor_comments":[{"comment":"The sentence after Table 2 states that models with pre-trained defense mechanisms (w/) generally perform better, but the table uses 'w/o' and blank rows without defining them. Please use explicit labels such as 'w/o prefix' and 'w/ prefix' or provide a clearly defined legend.","section":"Section 4 / Table 2"},{"comment":"The exact prefix string should be given in a monospaced form with capitalization and punctuation specified, since variations in whitespace or case might affect the attack's success.","section":"Section 1"},{"comment":"The sentence 'highlighted these concerns in multi-agent systems' is missing a subject or citation; it should be completed or removed.","section":"Section 6.4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the headline result isn't in the table. Section 2.3 defines two metrics — baseline ASR and ASR with Prefix — but Table 2 reports baseline, Adaptive Attack Prompt, and ArtPrompt columns only. No column or row shows the effect of adding 'Ignore the document.' The abstract's main conclusion is therefore unsupported by the only quantitative evidence in the paper. That's the thing to know.\n\nWhat the paper does well: it assembles a reasonable testbed, using 1,134 EPASS prompts across several models in a RAG setup, and it compares against two published attack families. The observation that a simple instruction-override can beat retrieved context is plausible and worth a short empirical note. The limitations section is candid about generalizability and the lack of real-world validation.\n\nBut the missing with-prefix column is load-bearing. Even the rows labeled 'w/o' are never defined, so the reported numbers are ambiguous. There is no scoring rubric, no trial counts, no error bars, and no released prompts or outputs. The conclusion that 'existing LLM defenses' are inadequate overshoots, because no defense was actually tested. And the novelty is incremental: adding 'Ignore the document' to known jailbreak prompts is a canonical prompt-injection variant, not a new attack class.\n\nThe paper's own Section 8 admits limited generalizability, but that doesn't fix the central problem: the key evidence is absent. As submitted, this is not ready for peer review. It could become a useful minor contribution if the authors report the actual with-prefix ASR, define the scoring rubric, clarify the row labels, and release the artifacts. As it stands, I'd desk reject, with an invitation to resubmit with the missing evidence.","headline":"The paper's central claim about 'Ignore the document' is not backed by the only results table, which never reports the promised with-prefix condition.","tokens_in":6398,"tokens_out":2924,"would_cite":false,"duration_ms":31557,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a short 'Ignore the document' prefix can override RAG-retrieved context in LLM agents, sharply raising attack success rates and exposing a missing instruction hierarchy.","keywords":["adversarial attacks","retrieval-augmented generation","prompt injection","jailbreak","instruction hierarchy","AI agent safety","large language models","attack success rate"],"falsifier":"Re-run the same 1,134-prompt suite with a pre-registered scoring rule that counts an attack as successful only when the model explicitly produces the harmful content requested (with an explicit refusal scored as failure), then compare the resulting ASR with Table 2; if the recalculated rates drop toward baseline, the central claim fails.","tokens_in":5448,"feed_emoji":"⚠️","tokens_out":9787,"duration_ms":86012,"temperature":0.7,"pith_summary":"The paper sets out to test a simple hypothesis: that telling a large language model to 'Ignore the document'—a plain-English prefix, not a crafted exploit—can make a RAG-based agent discard its retrieved context and comply with harmful requests. Across 1,134 adversarial prompts in ten categories, the authors report attack success rates that often jump far above baseline when the prefix is used, with the largest effects on open models such as Llama3.1 and Mistral-7B. The point of the exercise is to show that the weak point is the LLM core's instruction-prioritization logic, not the retrieval pipeline or agent-level safety layers, and that agent-level defenses cannot catch an attack that succeeds inside the LLM. A sympathetic reader would take this as evidence that RAG pipelines inherit rather than solve LLM vulnerability, and that defense needs to be built into instruction hierarchy.","feed_headline":"'Ignore the document' slips past LLM safety guards in tests","feed_subtitle":"A five-word prefix overrides retrieved context in RAG-based agents, raising attack success rates across leading models.","key_machinery":"The central object is the literal prefix 'Ignore the document'—a short instruction that tells the model to drop the retrieved RAG context. The paper's claim is that this prefix exploits a missing instruction hierarchy: immediate user instructions outrank previously supplied document context, so a few words can overturn the safeguard. Around it sits the EPASS dataset of 1,134 adversarial prompts and the attack success rate (ASR) metric that measures how often the model complies.","core_discovery":"The central discovery, stated in the paper's own terms, is that a deceptively simple adversarial prefix—'Ignore the document'—can compel LLMs to produce dangerous or unintended outputs by bypassing their contextual safeguards. In experiments across GPT-4o, GPT-4o Mini, Llama3.1, Llama3.2, Mistral-7B, and Gemma2, the authors report high attack success rates, with the largest jumps on Gemma2, Llama3.1, and Mistral-7B when adaptive and ArtPrompt attacks are applied. They interpret this as evidence that the LLM core, not the retrieval layer, is the fragile point: the model's instruction-processing logic lets an immediate prompt override retrieved context, and agent-level defenses cannot catch what happens inside the LLM.","pith_inferences":["A straightforward follow-up would measure the prefix's marginal contribution by comparing each attack condition with and without 'Ignore the document' under otherwise identical prompts; the paper reports attack-type ASRs but not this isolated delta.","The same instruction-override logic may transfer to other context channels—tool outputs, memory blocks, or system-prompt summaries—suggesting a general 'context can be vetoed by an instruction' failure mode in agent architectures.","The qualitative conclusion could be stress-tested by re-scoring the same outputs under a stricter, pre-registered definition of success that requires the model to actually emit the harmful content, then checking whether the reported pattern survives."],"forward_implications":["A RAG-based agent cannot rely on retrieved documents as a safety anchor: a few words in the user prompt can discard the context the pipeline fetched to constrain the answer.","Agent-level monitoring, output filters, and safety wrappers are not sufficient on their own, because the compromise happens inside the LLM before those layers act.","In a multi-agent setup that shares an LLM core, a single successful instruction-override can cascade into harmful outputs across many interconnected agents.","Defense work should focus on hierarchical instruction processing that keeps system-level and retrieved context above user-level instructions, and on context-aware instruction evaluation rather than high-level agent safeguards."],"supporting_citations":[{"why":"Supplies the Adaptive Attack Prompt method that the paper combines with the 'Ignore the document' prefix, and gives the jailbreaking baseline the experiments build on.","marker":"[Andriushchenko et al., 2024]"},{"why":"Supplies ArtPrompt, the ASCII-art attack method used as one of the experimental attack conditions.","marker":"[Jiang et al., 2024]"},{"why":"Provides the multi-turn jailbreak precedent showing that missing instruction hierarchies lead to harmful outputs, which the paper extends to RAG context-override.","marker":"[Russinovich et al., 2024]"},{"why":"Supplies AutoDAN, cited as evidence that gradient-based attacks find vulnerabilities even in robust architectures, supporting the paper's claim that current defenses are insufficient.","marker":"[Zhu et al., 2024]"},{"why":"Establishes universal and transferable adversarial attacks on aligned LLMs, the foundation for the paper's universal-prefix argument and for universal adversarial training as a defense.","marker":"[Zou et al., 2023]"}],"fun_headline_variants":["Simple five-word prefix dominates RAG agent safety tests","Ignore the document: One line breaks LLM-guarded RAG agents","RAG agents fall to bare LLM prompt, not retrieval tricks","LLM core vulnerability topples agent defenses in new study","Five-word attack defeats all leading RAG agent models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported attack success rates depend on an unstated rubric for what counts as a successful attack; if that rubric treats any non-refusal as success, the high ASR numbers could overstate how often harmful outputs actually occur.","fun_headline_variants_meta":{"raw":{"variants":["Simple five-word prefix dominates RAG agent safety tests","Ignore the document: One line breaks LLM-guarded RAG agents","RAG agents fall to bare LLM prompt, not retrieval tricks","LLM core vulnerability topples agent defenses in new study","Five-word attack defeats all leading RAG agent models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000437,"raw_usage":{"total_tokens":2182,"prompt_tokens":863,"completion_tokens":1319,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":479,"completion_tokens_details":{"reasoning_tokens":1235}},"tokens_in":479,"tokens_out":1319,"duration_ms":11344,"temperature":1.0,"reasoning_tokens":1235,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:23:14.671349+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same 1,134-prompt suite with a pre-registered scoring rule that counts an attack as successful only when the model explicitly produces the harmful content requested (with an explicit refusal scored as failure), then compare the resulting ASR with Table 2; if the recalculated rates drop toward baseline, the central claim fails.","supporting_citations":[{"cited_title":"Autodan: Interpretable gradient-based adversarial attacks on large language models","cited_arxiv_id":null,"evidence_quote":"Supplies AutoDAN, cited as evidence that gradient-based attacks find vulnerabilities even in robust architectures, supporting the paper's claim that current defenses are insufficient."}],"review_version":1}