{"id":"0da9a647-4869-4fcf-b2db-9658e1456f44","arxiv_id":"2508.17674","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper defines 'Advertisement Embedding Attacks' as prompt injection at service proxies plus weight poisoning of open-source models, and demonstrates both on current LLMs.","lead":"Researchers describe advertisement embedding attacks that inject promotional or malicious content into LLM outputs by hijacking service proxies or by uploading backdoored open-source model weights. The paper maps five victim groups and proposes a prompt-based self-inspection defense, warning that this threat is under-addressed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The evidence for the central attack claim is a single unquantified Gemini demo; the fine-tuned open-source path lacks any configuration details, making both headline claims unverifiable as stated.","rationale":"The reader's weakest_assumption focuses on the real-world reach of the SDP man-in-the-middle path; that is a legitimate threat-model concern about deployment realism, but it is not strictly necessary for the paper's central technical claim, which is that the injection works against a production model and that fine-tuning works against open models. My stress-test concentrates on the evidential basis for that technical claim: the SDP demonstration is a single unquantified transcript, the fine-tuning result is stated with no configuration or measurement details, the defense is a single anecdote, and no comparison against cited prior work (PoisonGPT, PoisonPrompt) is provided. These are internal-evidence weaknesses rather than reliance on a contested consensus; they are addressable by repeating the experiments with stated parameters and reporting rates. The paper could be repaired: a systematic evaluation with N trials, model versions, hyperparameters, poisoning fractions, and defense success rates would directly support or falsify the headline. I therefore partially agree with the reader: both the threat-model reach and the evidence quality are real concerns, but the more load-bearing issue for the central empirical claim is the missing quantification and missing experimental configuration, since those are what would let a reviewer verify that the attack is 'easy' and 'effective' rather than a curated example. The suggested verdict remains CONDITIONAL because the paper's contribution, as a threat report and taxonomy, is plausible and worth conditional acceptance, but the empirical backbone must be supplied before the strong claims in Section V can be treated as established.","tokens_in":9299,"tokens_out":1802,"duration_ms":16786,"concrete_test":"Re-run the SDP attack against Gemini 2.5 (and at least one other production model) for N=100 trials over the six questions in Figure 2, with temperature fixed, model version recorded, and the attack-prompt plus attacker-data configuration exactly as specified in Section IV-B1; report the fraction of responses containing the attacker's predefined answer or promotional link, together with 95% confidence intervals. Independently, rerun the LoRA fine-tuning on one open-source model (e.g., LLaMA 3 8B) with a published dataset size, LoRA rank/alpha, epoch count, and evaluation split, and measure exact-match reproduction of attacker answers on held-out questions; if the reproduction rate drops well below 'almost 100%' or requires an unrealistically large poisoning fraction, the central claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that AEA can easily mislead Gemini 2.5 into prioritizing attacker data (Section V), yet Section IV-B1 reports only a single illustrative exchange with no iteration count, no pass-rate measurement, no model version pinning, no temperature/sampling settings, and no data on how often the injected promotional answer actually appears across repeated trials. A one-off cherry-picked transcript cannot establish that the attack works reliably; the wording 'can be easily misled' requires a demonstrated success rate. The open-source path is in worse shape: Section IV-B2 states LLaMA 3, Mistral 7B, and Phi-3 were fine-tuned with attacker data and reproduced 'almost 100%' of preset responses, but gives no dataset size, no LoRA rank/alpha/target modules, no training hyperparameters, no base-model versions, no evaluation protocol, and no distinction between memorization of the poisoning set and generalization. These omissions matter because the advertisement-embedding threat model depends on the attack being reproducible at low cost by real adversaries; without the recipe or measurements, the reader cannot distinguish a systematic attack class from a handful of favorable demonstrations. The paper also does not compare against the explicitly cited PoisonGPT and PoisonPrompt baselines, so it cannot substantiate novelty or relative effectiveness. The defense claim in Section IV-C ('This prompt can effectively defend against attacks based on API Providers') is supported only by an unspecified single example, with no attack-success-rate comparison before and after the self-inspection prompt.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 'Advertisement Embedding Attacks' (AEA), a class of attacks that stealthily inject promotional or malicious content into LLM outputs via two vectors: (i) hijacking or masquerading as Service Distribution Platforms (SDPs) to prepend adversarial prompts and data to user queries sent to API providers such as Gemini; and (ii) fine-tuning open-source checkpoints (LLaMA 3, Mistral 7B, Phi-3) with attacker data and redistributing them through Model Distribution Platforms. It maps five victim groups, provides one qualitative demonstration against Gemini 2.5 showing that a game-style prompt can make the model answer from a supplied list of false or biased answers, claims near-100% reproduction of planted responses after LoRA fine-tuning on consumer hardware, and proposes a prompt-based self-inspection defense. The paper's central claim is that these low-cost attacks are a realistic and under-addressed threat.","tokens_in":9558,"tokens_out":4344,"duration_ms":45723,"significance":"If the empirical claims were established, the paper would draw timely attention to a practical threat: covert ad and promotion injection through both service-proxy prompt nesting and redistributed poisoned checkpoints. The dual-vector threat model, the stakeholder enumeration, and the proposed lightweight self-inspection defense are useful starting points for a research program on LLM output integrity. The paper, however, contains no quantified experiments: the Gemini demonstration is a single unlabeled exchange, the fine-tuning results are stated without any configuration or measurement, and the defense is asserted without evaluation. The conceptual novelty is also partially overlapping with existing prompt injection and backdoor literature, so the contribution at present is more a well-framed position piece than a demonstrated attack-class result. The paper ships no code, data, or models.","major_comments":[{"comment":"The central claim that Gemini 2.5 'can be easily misled' by AEA prompts (Section V) rests solely on the single qualitative exchange in Figure 2. There is no success rate, no number of trials, no model version string, no sampling or temperature settings, and no specification of how many repeated queries produced the shown behavior. As written, the demonstration is consistent with a one-off cherry-picked response, so it cannot support the 'easily' and 'prioritize' language. Please provide a systematic evaluation with at least N trials, a per-query success rate (e.g., fraction of responses containing the injected list content or otherwise following the attack instruction), model version pinning, access date, and decoding parameters.","section":"IV-B1, V"},{"comment":"The LoRA fine-tuning claim that LLaMA 3, Mistral 7B, and Phi-3 reproduced 'almost 100%' of preset responses from the attacker dataset is not verifiable because no experimental details are given: no dataset size or composition, no LoRA rank/alpha/target modules, no training hyperparameters, no base-model versions, and no evaluation protocol. Moreover, the paper does not distinguish memorization of the poisoning set from generalization to prompts that are not in the training list, which is essential for assessing the real-world threat of redistributed checkpoints. Since the threat model depends on the attack being cheap and reproducible by actual adversaries, a complete recipe and per-model reproduction rates with error bars are needed.","section":"IV-B2"},{"comment":"The defense claim 'This prompt can effectively defend against attacks based on API Providers' is unsupported: Section IV-C contains no experiments, no attack prompts tested, no ablation over the defense prompt, and no baseline for comparison. The next sentence correctly concedes that the prompt cannot defend against parameter-level attacks, so either the claim should be narrowed to the specific prompt-injection scenarios actually tested (with evidence), or the section should present a small evaluation across the attack prompts used in Section IV-B.","section":"IV-C"},{"comment":"The SDP attack path assumes that the attacker controls or has hijacked the service distribution platform, giving the attacker the ability to edit user requests and returned responses at will. This is a full man-in-the-middle capability and should be stated as such in the threat model. The paper should also discuss how prevalent such a capability is in practice (e.g., TLS endpoint authentication, user-supplied API keys, or consent requirements) and how the attack changes if the attacker can only edit requests but not responses. Without this, the 'advertisement embedding' framing suggests a weaker adversarial capability than the model actually assumes.","section":"III-C1"},{"comment":"The paper cites PoisonGPT [16], PoisonPrompt [17], and the Sleeper Agents line [18] but does not compare AEA against these baselines either conceptually or empirically. Since Section III-A defines AEA as a union of prompt-nesting and weight-poisoning mechanisms that largely coincide with those earlier works, the paper needs to articulate precisely what is genuinely new (e.g., the advertisement-integrity goal, the two-vector pipeline, or the self-inspection defense) and to locate the contribution with respect to the cited baselines. As written, the novelty claim is asserted rather than demonstrated.","section":"II-C, III-A"}],"minor_comments":[{"comment":"Figure 1 contains a typo ('Retuen' for 'Return'); the labels would also benefit from consistent capitalization and from arrows that clearly distinguish attacker actions from user flows.","section":"Figure 1"},{"comment":"The attack prompt is presented as one long inline paragraph; consider placing it in a verbatim block and explicitly marking the placeholders for 'attacker data' and 'user input' to improve readability and reproducibility.","section":"IV-B1"},{"comment":"The phrase 'almost 100%' is ambiguous; report exact reproduction rates with confidence intervals and specify which subset of prompts was evaluated (in-distribution versus out-of-distribution).","section":"IV-B2"},{"comment":"The conclusion calls Gemini 2.5 'state-of-the-art' without giving the exact model identifier or access date; include these details so the result can be reproduced and so the claim remains checkable.","section":"V"},{"comment":"The title and abstract promise a treatment of 'AI Agents', but the paper contains no agent-specific attack scenario or evaluation; either add an agent example (e.g., a tool-calling or ReAct agent) or revise the title and abstract to scope the contribution to standalone LLMs.","section":"Title and Abstract"},{"comment":"References [5]-[8] are self-citations to mental-health applications that do not support the claims about traffic prediction, weather forecasting, and other cited domains; please replace them with technical references for those application areas or delete them.","section":"References [5]-[8]"},{"comment":"The paper describes concrete attack prompts and fine-tuning recipes with no discussion of ethics or responsible disclosure; please add a brief statement covering institutional review, reporting to the affected model provider (Gemini), and safeguards against misuse of the published details.","section":"IV-B"}],"recommendation":"major_revision","confidential_remarks":"The paper is best understood as an extended abstract or position paper rather than a completed empirical study. The core idea—that free/low-cost LLM proxies and redistributed checkpoints can be used to covertly deliver promotional or biased content—is timely and worth publishing in a venue that values early threat identification, but the current lack of any quantitative evaluation is a serious gap relative to the claims. I would also flag that the self-citation cluster [5]-[8] is unrelated to the security content and does not support the introductory claims, which may raise concerns about citation padding. The definitional overlap with PoisonGPT and PoisonPrompt should be addressed head-on in revisions; if the authors can provide the missing measurements and a clear novelty statement, the paper could become a useful contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. The paper names a real attack pattern and gives it a clear victim taxonomy, but the two vectors it describes are indirect prompt injection and open-source weight backdooring, both well established in the work it cites. The new part is the framing, not the mechanism.\n\nWhat it does well: it ties the academic discussion to an observable phenomenon (underground markets pushing gambling links via compromised models), lays out the two attack paths in enough detail to reproduce the prompt-level trick, and identifies five victim groups that are genuinely different. I also credit the defense section for stating its own limit: the prompt-based defense does not address parameter-level attacks. That is honest scope control.\n\nThe soft spots are where the stress-test lands. The central claim—that Gemini 2.5 is 'easily misled'—rests on a single screenshot-style example. No success rate, no trial count, no temperature or model-version pinning. That cannot support 'easily.' The open-source experiment is worse: 'almost 100%' reproduction is stated with no dataset size, no LoRA rank or hyperparameters, no base-model versions, and no distinction between memorization and generalization. A reader cannot run the attack or evaluate the claim. The defense evaluation is similarly a single example with no before/after numbers. The paper also calls this a 'new class' while citing Greshake et al. and PoisonGPT as prior art; the novelty is the packaging and the victim analysis, not the mechanism.\n\nThe SDP threat model assumes the attacker controls or hijacks the service distribution platform. That is a strong assumption, but not a broken one—many users do route through third-party proxies. The paper should state it up front as the boundary of the threat.\n\nBottom line: this is a position paper and early warning, not a measured attack study. It is valuable as a taxonomy and a call for defenses, and the victim framing will be useful to practitioners. But the evidence is too thin to support the strong language. I would send it to peer review, not desk reject, because the threat is plausible and the area is underserved. A competent referee should push for real measurements and full experimental details.","headline":"A useful early-warning taxonomy, but the headline attack claims rest on anecdote, not measurement.","tokens_in":10112,"tokens_out":3610,"would_cite":false,"duration_ms":35405,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single prepended prompt can make Gemini 2.5 return attacker-chosen answers.","keywords":["Advertisement Embedding Attacks","prompt injection","LLM security","backdoor attacks","service distribution platforms","model distribution platforms","LoRA fine-tuning","information integrity"],"falsifier":"Send the paper's exact attacker prompt and list to a fresh Gemini 2.5 session through the official API with no proxy, ask the same user questions, and check whether the model still answers from the attacker's list; if it answers correctly every time, the claim that a prepended prompt alone works against the production service is refuted.","tokens_in":9098,"feed_emoji":"🕵️","tokens_out":9510,"duration_ms":80716,"temperature":0.7,"pith_summary":"This paper introduces and names a class of LLM security attacks — Advertisement Embedding Attacks (AEA) — whose goal is to covertly replace or color model outputs with attacker-selected ads, propaganda, or false information instead of degrading accuracy. The authors argue that two low-cost vectors are sufficient: prepending a crafted attacker prompt and data list to user requests inside a hijacked service-distribution platform, and publishing open-source checkpoints fine-tuned with attacker data. The evidence is a simple prompt that makes the Gemini 2.5 model return attacker-prescribed answers, and LoRA fine-tuning of LLaMA 3, Mistral 7B, and Phi-3 that reproduces planted responses at nearly 100% in about an hour on one consumer GPU. The paper also maps five victim groups and sketches a prompt-based self-inspection defense for the prompt-level vector. If the central claim is right, information integrity — not just task performance — is an exposed, under-defended attack surface for LLM services and AI agents.","feed_headline":"One prompt can make Gemini return attacker-chosen answers","feed_subtitle":"A hijacked platform or poisoned open-source model can quietly slip ads and lies into LLM responses.","key_machinery":"The central mechanism is the attacker's prepended instruction block — a prompt that frames the conversation as a 'game' plus a JSON list of question–answer pairs — which is inserted at the same prompt layer as the model's predefined system prompt, so the model treats injected instructions as legitimate. For the model-distribution path, the mechanism is LoRA fine-tuning: attacker data is folded into the model weights with low-rank updates, and the modified checkpoint is redistributed through open model platforms. The prompt block works by redefining the task and forbidding disclosure; the fine-tuned weights work by making planted responses the most likely next tokens.","core_discovery":"The paper's central discovery is a low-cost attack class, named AEA, where the attacker's goal is to steer what a model says rather than to make it fail. The demonstration shows that prepending a short attacker prompt and a key-content list to a user request on Gemini 2.5 makes the model supply the attacker's wrong answers — Tokyo for Australia's capital, Picasso for the Mona Lisa, a gambling site for investment advice — while the rest of its responses look normal. On the open-weights side, fine-tuning LLaMA 3, Mistral 7B, and Phi-3 with attacker data through LoRA made the models reproduce nearly 100% of the preset responses, with the whole tuning run taking about an hour on a local RTX 4070. The paper treats this as evidence that information integrity is an attack surface that API providers and open-source model platforms are not yet defending.","pith_inferences":["If AEA generalizes beyond the demonstrated models, any downstream AI-agent pipeline that prepends tool descriptions, RAG context, or conversation history to user input inherits the same exposure, because the injected data occupies the same instruction layer that the attack exploits.","Because the model-distribution variant needs only one consumer GPU and an hour to achieve near-100% response reproduction, the more durable risk may be the open-weights ecosystem rather than hijacked proxies; platform-side provenance checks, not prompt filters, are the likely limiting defense.","The demonstrated attack changes answer content without degrading fluency, so standard quality and user-satisfaction metrics would probably miss it; detecting AEA in the wild likely requires auditing outputs for repeated branded content or out-of-context links across users."],"forward_implications":["A hijacked third-party service-distribution platform can make a production model return attacker-selected promotional content while still answering other questions normally, so users cannot tell that the response has been tampered with.","Open-source checkpoints fine-tuned with attacker data through LoRA reproduce the planted responses at near 100% using a single RTX 4070 in about one hour, making the model-distribution vector cheap to repeat.","Because the attack does not degrade general accuracy, conventional performance-based defenses and monitoring will not flag it.","A prompt-based self-inspection defense can mitigate the prompt-level vector without retraining, but the same defense does not stop parameter-level attacks on modified weights.","Five stakeholder groups — end users, inference providers, open-source model owners, model platforms, and service platforms — each bear distinct reputational, financial, or decision-making losses if this attack is used."],"supporting_citations":[{"why":"First observation of compromised models distributed to push traffic to gambling websites; motivates the attack class.","marker":"[16]"},{"why":"Shows backdoor attacks can target prompt-based LLMs, supporting the prompt-level vector.","marker":"[17]"},{"why":"Demonstrates that deceptive behaviors planted in model weights persist through safety training, supporting the backdoored-checkpoint vector.","marker":"[18]"},{"why":"Establishes indirect prompt injection against LLM-integrated applications, supporting the prepended-prompt threat model.","marker":"[35]"},{"why":"Introduces supply-chain backdooring of machine-learning models, the lineage for the fine-tuned checkpoint attack.","marker":"[12]"}],"fun_headline_variants":["Stealth ad attack: LLMs tricked into serving attacker content","Backdoored LLMs: cheap fine-tuning slips in hidden ads","One prompt: Gemini returns attacker-chosen wrong answers","LLM integrity attack: hidden ads via platform or backdoor","Stealth ad embedding: LLMs silently pushed to promote"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The service-distribution attack only works if the attacker controls or masquerades as the platform that relays every user request to the model provider, so each prompt can be silently edited before it reaches the model.","fun_headline_variants_meta":{"raw":{"variants":["Stealth ad attack: LLMs tricked into serving attacker content","Backdoored LLMs: cheap fine-tuning slips in hidden ads","One prompt: Gemini returns attacker-chosen wrong answers","LLM integrity attack: hidden ads via platform or backdoor","Stealth ad embedding: LLMs silently pushed to promote"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000809,"raw_usage":{"total_tokens":3513,"prompt_tokens":869,"completion_tokens":2644,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":2559}},"tokens_in":485,"tokens_out":2644,"duration_ms":19518,"temperature":1.0,"reasoning_tokens":2559,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:00:59.658277+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Send the paper's exact attacker prompt and list to a fresh Gemini 2.5 session through the official API with no proxy, ask the same user questions, and check whether the model still answers from the attacker's list; if it answers correctly every time, the claim that a prepended prompt alone works against the production service is refuted.","supporting_citations":[],"review_version":2}