MAFIA: Query-Only Memory Attacks via Probing and Factual Injection against Audited LLM Agents

Jiaming Chen; Jun Zhang; Yanping Li; Yisen Gao; Yumeng Zhang; Zifan Liu

REVIEW 4 major objections 4 minor 25 references

MAFIA: Query-Only Memory Attacks via Probing and Factual Injection against Audited LLM Agents

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A parenthetical 'fact' — 'patient V (→ patient T in this DB)' — lets a query-only attacker poison audited agent memory, redirecting later queries while single-record audits detect at most 7.4%.

desk verdict Solid incremental attack paper with one unaddressed load-bearing gap: probes are never shown to pass the same auditor the threat model applies to every query. read the letter →

arxiv 2608.03844 v1 pith:35W7KVHQ submitted 2026-08-04 cs.AI

Jiaming Chen , Yisen Gao , Yanping Li , Zifan Liu , Yumeng Zhang , Jun Zhang This is my paper · ORCID

classification cs.AI

keywords memorypoisoningquery-onlyattackretrieval-augmentedagentsLLMsecurityinputauditevasioncompactfactualcloakretrievalcompetitionentityredirect

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MAFIA claims that a query-only attacker — someone who can only chat with an agent, never touch its memory store — can durably poison a retrieval-augmented agent even when the agent sits behind an input auditor and its memory holds thousands of benign records. The paper's two-stage design makes this plausible: a probing stage learns where future victim queries will land by reading the question fields the agent's own responses surface, and a payload stage hides the poison in a short parenthetical 'fact' that reads as a schema note rather than an instruction. The injected records stay near victim queries in embedding space, win top-K retrieval slots, and steer later behaviour toward a target entity. Across four agent–dataset settings the attack reaches 58.5–92.6% macro attack success, essentially unchanged after audit filtering, while audited detection stays between 0 and 7.4%; the leading query-only baseline is flagged at up to 83.3%. If correct, record-level text audits cannot protect long-lived agent memory.

What carries the argument

The load-bearing object is the compact factual cloak: a short parenthetical of the form 'V (→ T in this DB)' inserted into an otherwise benign base query. Each component carries part of the argument — the parentheses frame the insertion as a supplementary schema annotation so LLM-based auditors classify it as benign rather than a behavioural directive; the arrow encodes the entity substitution without the action verbs auditors are tuned to flag; and 'in this DB' casts the mapping as a local database fact that the agent's write-back step accepts as task evidence. The supporting mechanism is probing-based placement: with the agent asked to surface the question fields of its retrieved demonstra

What would settle it

Run MAFIA against the same four agents with a response interface that returns only the final answer and never echoes the retrieved question fields, holding the benign pool, retriever, and auditor fixed; if attack success collapses, the 'query-only' claim holds only for agents whose scaffolding leaks retrieval context, and if it does not, the surfacing assumption is not load-bearing. A second check: equip the auditor with write provenance — the originating interaction behind each candidate record — and test whether detection of cloaked records rises without the 44.9% false-positive rate that ma

Watch

Extended reading notes

Core claim

MAFIA establishes that memory poisoning survives the two constraints that defeat prior query-only attacks: a dense benign memory pool and an LLM-based write-time audit. The placement stage converts scale into an asset: schema-derived probe queries elicit the question fields of retrieved demonstrations, aggregating into a surrogate of the memory distribution; size-ranked round-robin allocation spends the fixed injection budget (30–90 queries per victim–target pair) across dense regions, and injecting sparse regions first lets early target-directed records reinforce later ones. The payload stage resolves the conflict between audit stealth and retrieval proximity with the compact factual cloak,

Load-bearing premise

The attack's aiming mechanism depends on the agent's responses revealing the question fields of the memory records it retrieved; if an agent answers without surfacing those fields, the attacker cannot build the distribution map and the budget allocation goes blind — and for one of the four agents (Data Interpreter) the paper asserts, rather than validates, that such surfacing happens.

Editorial extensions

If this is right

Record-level text audits are not a sufficient defense for long-lived agent memory: cloaked records pass the evaluated single-record auditors while still redirecting behaviour, and an auditor sharpened to catch the cloak also flags legitimate entity-merge records.
The V→T redirect recipe is general: the same pipeline redirects patient lookups, drug substitutions, product-category brand searches, and model-dependency selection, suggesting a transferable attack class rather than a per-agent gadget.
A small write budget suffices against a dense pool: 30 cloaked writes per pair beat retrieval competition in a ~5,800-record benign memory, and the attack survives changing to other dense encoders, BM25, entity-aware, and hybrid retrieval, with pure Levenshtein matching as the shared boundary case.
Memory-side consistency checking flips the failure mode rather than closing it: the tested post-retrieval defense flags 75.6% of poisoned appearances but also 44.9% of benign ones, so purely text-side defenses sit between high false positives and missed poison.
Adaptive auditing has a structural limit: a judge that reliably detects every cloak variant also flags every legitimate patient-merge record, so reliable defence needs signals outside the record text, such as provenance or write-authorization policy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the surfacing assumption holds, the placement machinery transfers to any shared retrieval store whose top-K results leak through the response surface — tool-selection caches, demonstration banks, or memory panels in an agent UI — making probing-and-placement a general budget-constrained poisoning recipe rather than a memory-specific trick.
The paper's own adaptive-audit result implies a design principle it does not spell out: the line between 'malicious remapping' and 'legitimate merge' is not a property of the text but of authorship, so the durable fix is provenance- and authorization-based write enforcement, not sharper classifiers.
A testable hardening the paper does not evaluate: agents that separate retrieved demonstrations from the written record, or that require explicit user confirmation before persisting memory derived from a query, would break the write-back chain MAFIA depends on; adding such confirmation is a cheap experiment a memory operator could run.
The probing economics (300 probes, roughly $1.50, reused across all nine pairs) suggest per-deployment attack cost is dominated by the poison writes, so rate-based write monitoring could cap the attack even when the audit cannot — a direction the paper names but does not quantify.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What to know: MAFIA is a competent, incremental extension of MINJA, not a reworking of the field. The two mechanisms are genuine: probing-based placement uses surfaced question fields to allocate a fixed injection budget, and the compact factual cloak stays close to victim queries in embedding space while avoiding instruction-like cues. The empirical work is the strongest part—four agent settings, nine pairs each, ablations, distribution drift, retriever generalization, and an honest A-MemGuard evaluation that reports its 44.9% FPR rather than hiding it. The central effectiveness claim, that MAFIA achieves high ASR in large benign pools under active auditing, is plausible and supported by the main experiments. This is a carefully measured attack paper with real engineering behind it.

The soft spots, in order of severity. First, the probe-audit gap is load-bearing and unanswered. The threat model says every query is routed through the LLM auditor, but Algorithm 1 gates only attack queries and never probes. The paper never reports a probe rejection rate. Probes explicitly ask the agent to surface retrieved demonstrations' question fields—exactly the kind of request an auditor could flag. If a meaningful fraction of probes are rejected, D cannot be built, the budget allocation goes blind, and the reported ASR is unreachable. This is not a question about the payload; the cloak is tested. It is a question about the first stage operating outside the paper's own threat model. The authors need either an experiment showing probes pass the same auditor or a clear argument that probes are exempt.

Second, the abstract's "suppressing audit detection... at most 7.4%" is selective. That number comes from the main auditor in Table 1 and the three low-FPR auditors in Table 2. Table 2 itself shows GuardReasoner-3B detects 94.4% of MAFIA records and PromptGuard 100%, both at high benign FPR. The paper honestly reports these, and high FPR limits deployment, but the headline overstates a clean evasion result.

Third, reproducibility: code is promised but not released, and MINJA is a reimplementation. Neither is fatal, but both should be cleaned up before publication. Minor: no error bars on the main ASR/DR macro-averages, and DataInterpreter's surfacing behavior is asserted via analogy rather than validated.

Bottom line: this paper deserves a serious referee. The attack design is coherent, the placement ablation convincingly attributes the gains, and the drift tests are an honest check. But the probe-audit gap must be addressed before the central claim is clean. I would condition acceptance on that experiment plus code release, and I would cite it if I were working on memory-poisoning defenses.

Referee Report

4 major / 4 minor

Summary. The paper proposes MAFIA, a query-only memory-poisoning attack against RAG-based LLM agents under two realistic pressures: large-scale benign memory pools and LLM-based input auditing. The method has two stages: (1) placement, which probes the retriever by having the agent surface question fields of retrieved demonstrations, clusters the surfaced questions, allocates a fixed injection budget via size-ranked round-robin selection, and schedules injections by ascending cluster size; and (2) payload design, which uses a compact factual cloak of the form "patient V (→ patient T in this DB)" to assert the V→T mapping as a local database fact, preserving retrieval similarity while evading semantic auditors. Experiments across EHRAgent/MIMIC-III, EHRAgent/eICU, RAP/WebShop, and DataInterpreter/HF Hub report that MAFIA outperforms the MINJA baseline on ASR, DR, Post-ISR, and Post-ASR under the authors' audit configuration, with ablations, retriever-generalization tests, distribution-drift tests, and a memory-side defense analysis.

Significance. If the results hold, MAFIA is a meaningful advance over prior query-only memory attacks: it demonstrates that a bounded query-only attacker can durably poison a dense RAG memory pool while remaining undetected by low-FPR semantic input auditors, and it provides a concrete mechanism (compact factual cloaks) that is both simple and plausible. The empirical coverage is a real strength: four agent–dataset settings, nine (V,T) pairs per setting, per-pair appendix tables, retriever-paradigm generalization, distribution-drift stress tests, adaptive-auditor experiments, and an evaluation of a memory-side defense. The authors also state that code will be released. However, the paper's central audit-evasion claim is currently broader than what the experiments support, and the probing stage is not validated under the paper's own 'every query is audited' threat model. These issues are fixable but require additional experiments and more careful claims.

major comments (4)

[§3.1, §4.2, Algorithm 1 (lines 3–6 vs. line 16)] The threat model states that "Every query q is first routed through an LLM-based input auditor" (§3.1), yet Algorithm 1 applies the audit gate only to attack queries (line 16) and never to probe queries (lines 3–6). The probing stage is load-bearing: D, the surrogate question set that drives clustering, budget allocation, and scheduling, is constructed entirely from responses to probes. Probe queries explicitly ask the agent to surface question fields of retrieved demonstrations, which is precisely the kind of retrieval-internal disclosure an input auditor is designed to block. The paper reports no probe rejection rate and offers no argument that probes are exempt from the audit. If a material fraction of probes are rejected, D cannot be constructed and the reported ASR is unattainable. This is not a presentation issue: the manuscript's own Limitations section does not acknowledge the ga
[Abstract; Table 2; §5.3] The headline claim that MAFIA suppresses audit detection "from a peak of 83.3% to at most 7.4%" is unqualified and is contradicted by the paper's own Table 2. GuardReasoner-3B detects 94.4% of MAFIA records, GuardReasoner-8B detects 88.9%, Llama Guard detects 40.0%, and the perplexity filter detects 43.3% — the last being higher than MINJA's 21.1%. The "at most 7.4%" figure holds only for the favorable low-FPR auditors (General Prompt, Targeted Prompt, ProtectAI). The main-experiment DR in Table 1 uses only the gpt-5.4-mini General Prompt auditor. The abstract and §5.3 should explicitly state that audit evasion is demonstrated only for low-FPR semantic auditors and that high-FPR auditors can detect MAFIA, sometimes better than they detect MINJA.
[§4.2; Table 1, DataInterpreter row] The placement stage assumes that the agent's responses surface the question fields of retrieved demonstrations. The paper notes that MEXTRA validates this interaction setting on EHRAgent and RAP, but for DataInterpreter it merely states that the agent "provides an analogous user-observable response interface" without experimental validation. DataInterpreter is one of the four headline settings, and its ASR/Post-ASR numbers depend on the probing stage functioning. The authors should either demonstrate that DataInterpreter actually surfaces question fields in response to the composed probes, or reduce the strength of the claims for that setting.
[Table 1; §5.2] The main macro-averaged metrics (ISR, ASR, DR, Post-ISR, Post-ASR) are reported without error bars or significance tests. Each eICU, WebShop, and DataInterpreter pair is evaluated on only 30 victim queries, and the paper reports a single run. Given the stochasticity of LLM-based agents and the small per-pair sample, the reported macro-averages could shift nontrivially under different seeds or query samples. At a minimum, the authors should provide bootstrap confidence intervals or standard deviations across repeated runs for the headline numbers in Table 1.

minor comments (4)

[§5.1, Input Audit] The text says "Before write-back, each candidate record is screened," but Algorithm 1 applies the audit to q_attack before the agent executes and before write-back. Clarify whether the audit is an input-query audit (as in §3.1) or a write-time record audit. This distinction matters for the threat model.
[§4.2] The notation "D= S q S(q)" is garbled; it should be a union over q, e.g., D = ⋃_q S(q).
[Table 2, §5.3] The sentence "Low-FPR semantic audits miss MAFIA, while high-FPR audits are over-sensitive" should be refined: on the Perplexity Filter and Llama Guard rows, MAFIA has higher detection than MINJA, so the statement that these auditors are merely over-sensitive is not always accurate.
[Throughout] The name "DataInterpreter" is written both as one word and as "Data Interpreter" (e.g., Table 1 vs. §5.1). Please standardize. Also, Algorithm 1's "AGGLOMERATIVECLUSTER" and "ROUNDROBINSELECT" lack spaces; align with the prose notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MAFIA's attack success and audit-evasion numbers are empirically measured outcomes, not quantities derived from their own inputs.

full rationale

The paper makes no derivation of a predicted quantity from a fitted parameter. MAFIA's placement stage builds a surrogate D from surfaced question fields and uses it to allocate the injection budget, but the central metric ASR is then evaluated on held-out clean victim queries that were not used to select the bases; the attack success is measured, not implied by construction. The audit-suppression claim (DR from 83.3% down to at most 7.4%) is likewise an empirical measurement under the specified single-record auditors, and the FPR/DR trade-offs across auditors are reported as data rather than derived. The cited prior work is not load-bearing in a self-referential way: MINJA is a baseline being compared against, and MEXTRA is an independent prior validation of the surfacing setting; neither is a self-citation chain that forces the result. The compact factual cloak is an empirically evaluated payload design, not a renaming of a known result. The strongest caveat in the manuscript is a threat-model consistency issue, not circularity: Algorithm 1 applies the audit gate only to attack queries and never to probe queries, and no probe rejection rate is reported, so the placement stage's feasibility under the 'every query is audited' rule is unverified. That is an unvalidated load-bearing condition and a correctness/robustness risk, but it is not an instance of an equation reducing to its own inputs or of a fitted parameter being renamed as a prediction. Accordingly, no circular steps are identified and the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The attack is empirical; the free parameters are hyperparameters, not fitted constants. The load-bearing assumptions are about the observability of the agent's retrieval (surfacing) and the embedding-space behavior of the cloaked records.

free parameters (4)

Clustering threshold tau = 0.96
Cosine-similarity threshold for agglomerative clustering with average linkage; hand-chosen, controls granularity of the surrogate clusters and therefore base-template selection.
Probe budget K_p = 300
Number of seed probe queries per dataset, chosen to balance cost ($1.5) and coverage; not fitted to the success metric.
Per-pair injection budget b = 90 (MIMIC-III), 30 (other settings)
Attack budget per (V,T) pair; sets the number of poisoned write-backs. Tuned per dataset to match semantic diversity.
Retrieval K = 4 (EHR), 3 (WebShop)
Number of retrieved demonstrations, following MINJA; not a free adaptation.

assumptions (4)

domain assumption Retrieved memory records are scored primarily by question fields under cosine similarity.
The placement and payload designs assume the retriever ranks records by the question text, so the parenthetical cloak minimally displaces the record in embedding space. Invoked in Section 4.2.
domain assumption The agent's response interface exposes question fields of retrieved demonstrations to the attacker (MEXTRA setting).
Probing collects S(q) from surfaced question fields to construct surrogate D; if surfacing fails, placement is blind. Stated in Section 4.2 with reliance on Wang et al. 2025.
domain assumption The input audit inspects each record in isolation and has no access to provenance or cross-record context.
The threat model defines the audit as an LLM-based input filter on single queries (Section 3.1); the lock-step finding in Appendix C.1 that a bracket-aware judge catches all cloaks but also all legitimate merges relies on this isolation assumption.
standard math Standard agglomerative clustering and embedding methods are used without modification.
The pipeline uses cosine similarity, average linkage agglomerative clustering, and off-the-shelf embeddings (e5-small-v2, etc.).

how reviews work

0 comments

Cite this review

Pith. "Pith review of MAFIA: Query-Only Memory Attacks via Probing and Factual Injection against Audited LLM Agents." pith.science (2026). https://pith.science/paper/35W7KVHQ

@misc{pith2026260803844,
  author       = {Pith},
  title        = {Pith review of: MAFIA: Query-Only Memory Attacks via Probing and Factual Injection against Audited LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/35W7KVHQ}},
  note         = {Machine review of arXiv:2608.03844}
}

read the original abstract

Memory-augmented LLM agents rely on rich context for long-horizon reasoning and acting, yet their memory modules expose a persistent attack surface for malicious records, making the study of memory poisoning threats imperative. However, existing query-only attacks often fail to remain effective in two realistic and prevalent settings: large-scale benign memory pools and active input auditing. Consequently, current approaches fall short when facing the dual challenges of high retrieval competitiveness and rigorous semantic checks. To overcome these limitations, we propose MAFIA, a query-only Memory Attack framework via probing and Factual Injection against Audit, tailored to this extended threat model. Specifically, MAFIA introduces: (1) a placement strategy that ensures retrieval-competitive injection via memory probing, budget allocation, and scheduling; and (2) a payload design that bypasses audits using compact factual cloaks, preserving malicious effects while maintaining high semantic similarity. Extensive evaluations reveal that MAFIA achieves up to a 90.7% attack success rate while suppressing audit detection from a peak of 83.3% to at most 7.4%, exposing critical vulnerabilities across agentic memory systems. Code will be made publicly available at https://github.com/JiamingChen1234/MAFIA.

Figures

Figures reproduced from arXiv: 2608.03844 by the authors.

**Figure 1.** Two deployment pressures we evaluate. (a) LLM audit detection rate (DR) on MAFIA(Ours) and MINJA memory entries. (b) ASR of both methods across the evaluated benign-memory sizes on eICU Pair 1. Each sweep point evaluates 30 victim queries; lines connect observed points, and crosses mark the fullpool results at size b=5,842. are increasingly augmented with external memory banks to store retrievable experiences for l… view at source ↗

**Figure 2.** MAFIA pipeline. Probe queries enter through the ordinary interaction interface; the attacker records only historical question fields surfaced in the agent’s answer or action, not internal top-K records or retrieval outputs. These surfaced questions guide clustering, budget allocation, and scheduling. The compact factual cloak then enters through the same interface and is stored through the normal write path; a later… view at source ↗

**Figure 3.** Agent-backbone generalization of MAFIA on eICU Pair 1. Hatched bars report ISR and solid bars report ASR under the same strict success predicate. anisms [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Payload-form ablation on MIMIC-III Pair 1 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Case study (MIMIC-III Pair 1, V =71558→T=18866). For the majority of victim queries (60/103 q-tags), at least three of the four cosine-top-4 slots are MAFIA’s rewrites, all asserting the same false schema fact. The agent’s first thought quotes that fact as a database p…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 10 canonical work pages

[1]

denotes mean cosine similarity to the nearest victim query

Cos. denotes mean cosine similarity to the nearest victim query. Payload Added chars RIR@4↑Cos.↑ CFC parenthetical+2990.29 0.932 Declarative sentence+4972.82 0.918 Hard imperative+2968.93 0.917 MINJA shortened+3357.28 0.915 MINJA full+16033.01 0.904 E.2 Placement Design Diagnostics Table 11 compares the proposed probing, alloca- tion, and selection strate...

work page arXiv
[2]

arXiv preprint arXiv:2107.03374

Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li

arXiv
[3]

Table 6: Retriever-paradigm generalization on MIMIC- III.(a)Macro RIR@ 4 across nine pairs.(b)ASR under the symmetricall-MiniLM-L6-v2retriever

We hypothesize that contrastive training places greater weight on salient entity tokens, helping the entity-matching cloak outrank benign distractors, whereas symmetric models emphasize holistic semantic alignment and penalize differences in the surrounding context. Table 6: Retriever-paradigm generalization on MIMIC- III.(a)Macro RIR@ 4 across nine pairs...

work page arXiv 2018
[5]

In Findings of the Association for Computational Linguistics: ACL 2025, pages 19796–19821

Data inter- preter: An llm agent for data science. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19796–19821. Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, and 1 others

work page 2025
[6]

arXiv preprint arXiv:2512.13564

Memory in the age of ai agents. arXiv preprint arXiv:2512.13564. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and 1 others

arXiv
[7]

arXiv preprint arXiv:2312.06674

Llama guard: Llm-based input- output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674. Tomoyuki Kagaya, Thong Jing Yuan, Yuxuan Lou, Jayashree Karlekar, Sugiri Pranata, Akira Kinose, Koki Oguri, Felix Wick, and Yang You

arXiv
[10]

arXiv preprint arXiv:2501.18492

Guardreasoner: Towards reasoning-based llm safe- guards. arXiv preprint arXiv:2501.18492. Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez

arXiv
[11]

In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22

Generative agents: Interactive simu- lacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22. Atharv Singh Patlan, Ashwin Hebbar, Pramod Viswanath, and Prateek Mittal. 2025a. Context ma- nipulation attacks: Web agents are susceptible to cor- rupted memory. arXiv preprint arXiv:2506.173...

arXiv

Show all 25 references

[12]

arXiv preprint arXiv:2602.15344

Er-mia: Black-box adversar- ial memory injection attacks on long-term memory- augmented large language models. arXiv preprint arXiv:2602.15344. Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce C Ho, Carl Yang, and May Dongmei Wang
[13]

In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22315–22339

Ehragent: Code empowers large language models for few- shot complex tabular reasoning on electronic health records. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22315–22339. Noah Shinn, Federico Cassano, Ashwin Gopinath, Kart...

2024
[14]

arXiv preprint arXiv:2512.16962

Mem- orygraft: Persistent compromise of llm agents via poisoned experience retrieval. arXiv preprint arXiv:2512.16962. Bo Wang, Weiyi He, Shenglai Zeng, Zhen Xiang, Yue Xing, Jiliang Tang, and Pengfei He
[15]

arXiv preprint arXiv:2305.16291

V oyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291. Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, and Heng Ji

arXiv
[16]

In International Conference on Learning Representations, volume 2024, pages 32593–32627

Mint: Evaluating llms in multi-turn interaction with tools and language feedback. In International Conference on Learning Representations, volume 2024, pages 32593–32627. Qianshan Wei, Tengchao Yang, Yaochen Wang, Xin- feng Li, Lijun Li, Zhenfei Yin, Yi Zhan, Thorsten Holz, Zh...

2024
[17]

arXiv preprint arXiv:2510.02373

A-memguard: A proactive defense framework for llm-based agent memory. arXiv preprint arXiv:2510.02373. Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang
[18]

In Findings of the Association for Computational Linguistics: ACL 2024, pages 3053–3077

Agenttun- ing: Enabling generalized agent abilities for llms. In Findings of the Association for Computational Linguistics: ACL 2024, pages 3053–3077. Longtao Zheng, Rundong Wang, Xinrun Wang, and Bo An

2024
[19]

In International Conference on Learning Representations, volume 2024, pages 19036–19066

Synapse: Trajectory-as- exemplar prompting with memory for computer control. In International Conference on Learning Representations, volume 2024, pages 19036–19066. Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, a...

2024
[20]

(→ Tin this DB)

Mem1: Learning to synergize memory and reason- ing for efficient long-horizon agents. arXiv preprint arXiv:2506.15841. A Algorithm Pseudocode Algorithm 1 gives the end-to-end MAFIA attack pipeline corresponding to §4. COMPOSEPROBEcombines a schema-derived seed with the agent-s...

arXiv
[24]

The HF Hub (DataInterpreter) row group targets HuggingFace model-identifier redirects in the agent’s from_pretrained call; individual pair definitions appear in Appendix F

For each pair Pi, sub-columnsO(Our MAFIA) and M(MINJA) sit side by side; bold marks the winning method per cell (higher for ISR / ASR / Post-ISR / Post-ASR, lower for DR). The HF Hub (DataInterpreter) row group targets HuggingFace model-identifier redirects in the agent’s from...

1973
[25]

The agent operates with frozen memory, uti- lizing the same gpt-5.4-mini backbone and re- triever configuration as the main experiments. Task accuracy is evaluated based on task-specific met- rics: EHR settings use the gold-answer match from EHRSQL, WebShop uses the environmen...

2026
[2021]

arXiv preprint arXiv:2108.07732

Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others

arXiv
[2022]

out of stock

by execut- ing each candidate’s reference SQL together with the EHRAgent code generated for it. We remove examples that are marked impossible or lack re- quired query metadata, deduplicate the remaining records, and exclude evaluated identifiers: all evalu- ated patient IDs fo...

2024
[2023]

arXiv preprint arXiv:2312.10997, 2(1):32

Retrieval- augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2(1):32. Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Bin- hao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, and 1 others

arXiv
[2024]

arXiv preprint arXiv:2402.03610

Rap: Retrieval-augmented planning with contextual mem- ory for multimodal llm agents. arXiv preprint arXiv:2402.03610. Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu

arXiv
[2025]

arXiv preprint arXiv:2504.19413

Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413. Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang

arXiv
[2026]

Yue Liu, Hongcheng Gao, Shengfang Zhai, Yufei He, Jun Xia, Zhengyu Hu, Yulin Chen, Xihong Yang, Jiaheng Zhang, Stan Z Li, and 1 others

Dive into claude code: The design space of today’s and future ai agent systems.arXiv preprint arXiv:2604.14228. Yue Liu, Hongcheng Gao, Shengfang Zhai, Yufei He, Jun Xia, Zhengyu Hu, Yulin Chen, Xihong Yang, Jiaheng Zhang, Stan Z Li, and 1 others

arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

denotes mean cosine similarity to the nearest victim query

Cos. denotes mean cosine similarity to the nearest victim query. Payload Added chars RIR@4↑Cos.↑ CFC parenthetical+2990.29 0.932 Declarative sentence+4972.82 0.918 Hard imperative+2968.93 0.917 MINJA shortened+3357.28 0.915 MINJA full+16033.01 0.904 E.2 Placement Design Diagnostics Table 11 compares the proposed probing, alloca- tion, and selection strate...

work page arXiv

[2] [2]

arXiv preprint arXiv:2107.03374

Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li

arXiv

[3] [3]

Table 6: Retriever-paradigm generalization on MIMIC- III.(a)Macro RIR@ 4 across nine pairs.(b)ASR under the symmetricall-MiniLM-L6-v2retriever

We hypothesize that contrastive training places greater weight on salient entity tokens, helping the entity-matching cloak outrank benign distractors, whereas symmetric models emphasize holistic semantic alignment and penalize differences in the surrounding context. Table 6: Retriever-paradigm generalization on MIMIC- III.(a)Macro RIR@ 4 across nine pairs...

work page arXiv 2018

[4] [5]

In Findings of the Association for Computational Linguistics: ACL 2025, pages 19796–19821

Data inter- preter: An llm agent for data science. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19796–19821. Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, and 1 others

work page 2025

[5] [6]

arXiv preprint arXiv:2512.13564

Memory in the age of ai agents. arXiv preprint arXiv:2512.13564. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and 1 others

arXiv

[6] [7]

arXiv preprint arXiv:2312.06674

Llama guard: Llm-based input- output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674. Tomoyuki Kagaya, Thong Jing Yuan, Yuxuan Lou, Jayashree Karlekar, Sugiri Pranata, Akira Kinose, Koki Oguri, Felix Wick, and Yang You

arXiv

[7] [10]

arXiv preprint arXiv:2501.18492

Guardreasoner: Towards reasoning-based llm safe- guards. arXiv preprint arXiv:2501.18492. Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez

arXiv

[8] [11]

In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22

Generative agents: Interactive simu- lacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22. Atharv Singh Patlan, Ashwin Hebbar, Pramod Viswanath, and Prateek Mittal. 2025a. Context ma- nipulation attacks: Web agents are susceptible to cor- rupted memory. arXiv preprint arXiv:2506.173...

arXiv