Pith. sign in

REVIEW 3 major objections 4 minor 66 references

WeClawArena: An Auditable Sandbox and Benchmark for Cross-User Agents Collaboration and Security in Human-Centered Agent Networks

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In human-centered agent networks, finishing a task and being attacked are independent failures, and WeClawArena measures both from bounded runtime evidence.

desk verdict A genuinely useful new benchmark for cross-user owned-agent collaboration, with a real question about whether the ASR judge measures harm or author hints — worth refereeing, and the authors are honest about most of it. read the letter →

arxiv 2608.03499 v1 pith:PD4QE7XQ submitted 2026-08-04 cs.AI

Prince Zizhuang Wang , Aojie Yuan , Haiyue Zhang , Xiyang Hu , Yue Zhao , Shuli Jiang This is my paper · ORCID
classification cs.AI
keywords human-centeredagentnetworkspersonalworkspacesmulti-agentcollaborationattacksuccessrateauditablesandboxgovernanceharmLLM-as-a-judgeprivacyleakage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces WeClawArena, a benchmark and runtime sandbox for situations in which several human owners each delegate an AI agent, and those agents must complete one collaborative tool-use task while each sees only its own private workspace. The central claim is that task utility and attack harm are separate outcomes: an agent team can finish the task while leaking private information, trusting poisoned evidence, or accepting an invalid approval, and it can also fail the task without any attack having caused final harm. To test this, WeClawArena contributes 124 base tasks across six domains, expanded into 620 matched scenarios with a benign control and four attack-vector variants each. The sandbox records messages, tool calls, resource operations, governance decisions, and final workspace states, and the evaluation reports deterministic task success and LLM-judged attack success separately. The empirical split motivates the design: of 3,743 judged attack rows, 1,152 reached attack success, but only 117 of those also completed the task.

What carries the argument

The load-bearing mechanism is the division of labor between a deterministic verifier and an offline attack judge. The verifier scores task utility from final workspace states and task contracts; the judge scores attack success from a bounded evidence packet assembled after each run, containing scenario metadata, attack metadata, transcript messages, tool calls and observations, task-score fields, and owner or governance context. An attack overlay may add messages, files, or database rows, but cannot directly edit the final score state, so any judged harm must have traveled through ordinary agent actions. This separation is what allows the paper to conclude that harm and utility are independe

What would settle it

An audit study that removes the 'expected exposure path' and 'evidence hints' from the evidence packets and has a judge or blinded human annotators re-label the 200-row pilot: if ASR drops substantially or agreement collapses, the original judge was leaning on the attack description rather than on recorded evidence of final harm.

Watch

Extended reading notes

Core claim

The paper's central claim is that collaborative tool use among agents who each serve a different human owner creates a distinct failure surface: a team can complete the visible task while leaking a private budget, trusting poisoned evidence, or accepting an invalid approval. WeClawArena operationalizes this claim as a benchmark and sandbox, with 124 base tasks across bargaining, bidding, travel, software-engineering workspaces, clinical coordination, and trading, each expanded into one benign control and four attack-vector variants, for 620 scenarios. Utility is scored deterministically from final workspace states and contract predicates, while attack success is judged offline from bounded e

Load-bearing premise

The headline attack-success numbers rest on a single LLM judge reading evidence packets that contain the attack's 'expected exposure path' and 'evidence hints'; if the judge is tracking those hints rather than independently establishing final harm, the ASR values and the model-resistance ranking could shift.

Editorial extensions

If this is right

  • Benchmarks for multi-agent collaboration should report utility and attack success separately; a utility-drop number is a symptom, not a substitute for an audit of final harm.
  • Attack harm is not one thing: in the sweep, governance and security attacks succeed far more often than collaboration attacks, and the same harm surface lands very differently across domains, so safety evaluation should be per-surface rather than a single scalar.
  • The 117 runs that both completed the task and reached attack success are deployment-relevant: completion signals alone would miss privacy leakage, poisoned evidence, and invalid authority paths.
  • Judged attack-resistance rank can be stable across judge choices, with the overall model ordering unchanged when a second judge re-reads the same evidence packets, which supports bounded-evidence ASR as a reproducible audit layer.
  • The benign-control false-positive rate stays below one percent for both judges, meaning the audit layer could serve as a low-noise filtering signal, not only as a headline metric.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I would expect the strongest practical leverage to be structural: because invalid-authority paths were the largest judged harm category, an agent-network gateway could enforce approval and ownership checks at the tool layer rather than relying on the model to refuse; the governance rows give a direct test of such enforcement.
  • The evidence-packet design generalizes beyond attacks: the same bounded records could support counterfactual auditing (what would have happened without the attack) and post-hoc explanation of ordinary task failures, which the paper leaves implicit.
  • A cheap deployment test suggested by the data: monitor not just final task outcomes but evidence links that indicate poisoned inputs or authority-path breaks, since many successful attacks preserved utility.
  • The gap between frontier and open-weight models in judged attack resistance points to attack pressure as a training axis rather than purely a prompting-time property; this is my inference, not the paper's claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. WeClawArena introduces a benchmark and Docker-based sandbox for multi-party owned-agent collaboration over personal workspaces. It contains 124 base tasks expanded into 620 scenario variants across six domains, with each base task paired with one benign control and four attack-vector variants (collaboration, security, privacy, governance). The paper's central methodological claim is that task utility and final attack harm are separable and measurable from bounded runtime evidence: utility is scored by deterministic task verifiers, while attack success rate (ASR) is judged post-hoc by an LLM from bounded evidence packets. Experiments across eight models report TSR and 1−ASR separately, with Claude Opus 4.7 as the most attack-resistant model. The paper includes a second-judge sensitivity pass, a 200-row human-annotated pilot, Wilson confidence intervals, and a detailed artifact-release plan.

Significance. This is a timely and potentially valuable benchmark: it targets a realistic deployment setting—persistent personal agents collaborating across owner-scoped workspaces—that is not covered by existing tool-use, multi-agent, or privacy benchmarks. The design has real strengths: utility and harm are explicitly separated; the runtime records gateway-mediated evidence; attack variants are matched to benign controls; and the paper is unusually transparent, with exact denominator tables, Wilson intervals, an independent LLM-judge sensitivity pass, and an explicit audit-claim boundary in Appendix A. The 117 task-success-plus-attack-success rows and the per-surface heterogeneity results, if reproducible, support the conceptual claim that utility and harm must be measured separately. However, the headline ASR numbers—and therefore the model-resistance ranking and the utility/harm separation evidence—rest on a single LLM judge whose evidence packets contain the attack designers' own 'expected exposure path' and 'evidence hints'. The current validation does not yet rule out hint-driven over-attribution, and there is a direct numerical inconsistency between the reported ASR pool sizes and the ca

major comments (3)
  1. [Appendix G / Table 3 / Table 11] The ASR denominator arithmetic is internally inconsistent. Appendix G states that ASR-MAIN-SIX-DOMAIN contains 3,743 judged attack-vector rows, with per-vector denominators 939 (collaboration), 922 (security), 943 (privacy), and 939 (governance). Table 11, described as the canonical denominator table for that pool, sums to 3,302 rows (collaboration 930, security 778, privacy 799, governance 795), and Table 3's per-model ASR denominators also sum to 3,302. The ASR-success numerators sum to 1,152 in both places. The numbers only close if the 441 rows present in Appendix G but absent from Table 11 are all non-successes, which would change the overall row-micro ASR from the reported 30.8% to 34.9%. Please reconcile these counts and state exactly which rows belong to ASR-MAIN-SIX-DOMAIN, ASR-SENSITIVITY-SIX-DOMAIN, and Table 11. This is load-bearing: every ASR percentage, the failure-analysis
  2. [Appendix E / Appendix H / Figure 8] The ASR judge is the load-bearing measurement for the separation claim and the model-resistance ranking, but the validation does not rule out hint-driven over-attribution. Figure 8 shows that the judge's evidence packet includes the attack recipe's 'expected exposure path' and 'evidence hints' for every attacked row, and Appendix H states that the two human annotators saw the same packets. Inter-judge κ=0.70 and human-judge κ=0.65–0.68 therefore share the same hint structure, and the 0.3–0.7% benign FPR is not an adequate control because benign rows contain no attack metadata or hints. I recommend a hint-redaction rerun: judge the same attacked rows with 'expected exposure path' and 'evidence hints' removed, and compare per-cell ASR; alternatively, have experts label a redacted subset. If ASR falls substantially, the headline 1−ASR values and the 117 task-success-plus-harm rows would shi
  3. [Table 3 / Appendix I.1 / Appendix A] Even after the denominator reconciliation, the model-level comparison must address non-random missingness. Several models have large gaps between nominal attack rows (496 per model) and judged rows: Claude Sonnet 4.5 has 293/496, and Qwen3 235B has 392/496, with other models in the 434–440 range. If unjudged rows correlate with task difficulty or judge failure, macro-vector resistance is biased. Please report the reason each row is absent (e.g., missing evidence, judge error, runtime failure) and add a sensitivity analysis that imputes missing rows as all-success/all-failure or restricts the ranking to the fully covered common subset. Relatedly, Appendix A excludes runs with gateway bypasses from valid evidence; the paper should report how many runs were excluded on that basis so readers can assess selection effects.
minor comments (4)
  1. [Table 1 caption] The caption should state explicitly that TSR is computed over all five variants, including attacked variants. As written, readers may misread the clinical and trading columns as benign-capability scores rather than all-variant resilience scores.
  2. [Figure 1] The 'A2A' label in the left panel is not defined in the caption. The relationship between the Agent2Agent protocol cited in Related Work and the gateway used in this paper should be clarified.
  3. [Appendix H] The statement that GPT-5.2's 'consistently lower ASR' means the headline numbers are a 'lower-bound on attack success' assumes that the lower-ASR judge is closer to truth. This is not established by the validation; if hint-driven over-attribution inflates ASR, the direction of the bias is the opposite. Please qualify or justify.
  4. [Section 6] 'WeClawArena establishes multi-party tool-use collaboration...' overstates the contribution; consider 'introduces' or 'provides' to match the benchmark framing used elsewhere.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; central claims rest on independent runtime measurement, with only non-load-bearing self-citations.

full rationale

WeClawArena is a benchmark and measurement study rather than a derivation, and I found no step where a purported prediction reduces by construction to its inputs. Task utility (TSR) is computed deterministically from final workspace state and domain task predicates, separate from the LLM-judged ASR. The central empirical claim—that utility and final harm are separable—is an observed contingency supported by counts such as 117 rows that both completed the task and reached judged attack success, not an identity forced by definitions. The most plausible circularity concern is that the ASR judge's evidence packet (Figure 8, Appendix E) includes the attack authors' 'expected exposure path' and 'evidence hints,' so the judge could be confirming the attack recipe rather than independently detecting harm. This is a genuine measurement-validity risk, but it is not circularity in the logical sense: the judge still requires actual run evidence (transcript, tool calls, final score fields) and a clear evidence link, and the verdicts are not derived from the hints alone. The paper also provides independent validation in Appendix H: a second LLM judge over the same packets (inter-judge κ=0.70, ranking Spearman=1.00), benign false-positive rates below 1%, and a 200-row human pilot (κ=0.65–0.68 against GPT-5.2). These checks do not eliminate the hint-contamination concern, but they show the ASR layer is not equivalent to its metadata by construction. The self-citations to Wang and Jiang (2026a, 2026b) are used only for framing and gap motivation in Sections 2 and 5, not as load-bearing evidence, uniqueness theorems, or smuggled ansatze. There is no fitted parameter renamed as a prediction, and no known result is repackaged under new coordinates. The score of 2 reflects the presence of minor, non-load-bearing self-citations; the core benchmark results are self-contained empirical measurements.

Assumptions & free parameters 3 free parameters · 5 assumptions · 3 invented entities

No numbers are fitted to data in this paper: the benchmark constructs tasks and evaluates models. The hand-chosen elements are runtime budgets, the judge model, and the rubric thresholds behind 'clear evidence link' and 'final harm.' The ASR layer is the least externally anchored component: its definitions are authored by the same team that built the attacks, with independent grounding limited to a second LLM judge and a small human pilot.

free parameters (3)
  • per-domain turn caps = 36 turns (bargaining, bidding, travel); 90 turns (SWE-Workspace)
    Hand-chosen caps determine which runs become turn-cap failures, affecting TSR and ASR denominators; no sensitivity analysis of the caps is reported.
  • ASR evidence-link strictness = rubric-defined
    The 'clear evidence link' and 'final harm' thresholds are author-written rubric criteria applied by the LLM judge; changing the wording would move ASR.
  • headline ASR judge model = GPT-5.2 (default settings)
    ASR values depend on the judge choice; the paper's own sensitivity pass with Claude Opus 4.7 shifts ASR by +1.8 to +4.1 points per surface, so headline numbers are lower-bound estimates.
assumptions (5)
  • domain assumption LLM-as-a-judge verdicts on bounded evidence packets track true final harm.
    Sections 3.2 and H: ASR is LLM-judged; validated against a second judge (kappa 0.70) and a 200-row two-author human pilot (kappa 0.65-0.68), not against an independent external ground truth.
  • domain assumption The Docker gateway captures all scoreable actions; no undetectable side channels exist.
    Appendix A 'Audit claim boundary': the attack-success claim is only valid if peer messages, tool calls, resource operations, and final artifacts cannot bypass the gateway.
  • domain assumption Attack overlays never directly edit final score state (no-direct-score-edit rule).
    Appendix E: attack material must reach harm through agent behavior; violation would make ASR measure the overlay rather than the agent.
  • domain assumption Task verifiers encode contracts correctly and deterministically.
    Section 3.2: utility is assigned by Vz from final state and evidence; correctness of the 124 contracts is asserted by two-author review, not independently audited.
  • domain assumption Privacy harm follows contextual integrity (Nissenbaum 2004).
    Section 2.3: disclosure is judged against social context, recipient, and contract; adopted from the cited literature rather than derived.
invented entities (3)
  • Four-harm-surface taxonomy (collaboration, security, privacy, governance)
    purpose: Organizes attack vectors and defines what ASR judges; the benchmark's central measurement construct.
    The taxonomy is author-defined and internal to the benchmark; it has no external falsifiable handle outside the paper.
  • Bounded evidence packet
    purpose: The standardized post-run input consumed by task verifiers and ASR judges; the basis of the audit claim.
    Packet composition is author-defined; the paper does not show that packets suffice for independent auditors to reach the same verdicts.
  • Owned-agent personal workspace node (nu = (u, Au, Wu))
    purpose: Formal unit of ownership that splits task state across users in the formal model.
    A formalization choice, not an empirically verified object; it serves as the substrate for all tasks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of WeClawArena: An Auditable Sandbox and Benchmark for Cross-User Agents Collaboration and Security in Human-Centered Agent Networks." pith.science (2026). https://pith.science/paper/PD4QE7XQ

@misc{pith2026260803499,
  author       = {Pith},
  title        = {Pith review of: WeClawArena: An Auditable Sandbox and Benchmark for Cross-User Agents Collaboration and Security in Human-Centered Agent Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PD4QE7XQ}},
  note         = {Machine review of arXiv:2608.03499}
}
read the original abstract

Recent advances in persistent personal-agent frameworks are making human-centered agent networks realistic deployment targets: each user can be served by an AI agent that acts on the user's behalf, maintains state, and communicates with other agents through social and task relations. In these networks, everyday tool use becomes multi-party owned-agent collaboration over personal workspaces, where files, records, tools, and policies are not directly visible across owners. Existing agent benchmarks study tool use and collaboration, but they do not provide an end-to-end sandbox for verifiable cross-user agent collaboration with realistic user digital workspaces or test how harmful actions can travel through the human-centered agent network. We introduce WeClawArena, an auditable benchmark and runtime sandbox for multi-party owned-agent collaboration over personal workspaces. WeClawArena targets collaborative tool-use tasks in which personal workspaces serve as both operational tools and personal constraints. The benchmark contains 124 base tasks across six cross-user task domains and expands them into 620 scenario variants, with one benign control and four attack-vector variants per base task. The sandbox records peer messages, tool calls, resource operations, governed decisions, and final workspace states. WeClawArena reports utility and attack success rate separately and audits attack success from bounded runtime evidence, supporting diagnosis of task breakdown, privacy leakage, poisoned evidence, and invalid authority paths.

Figures

Figures reproduced from arXiv: 2608.03499 by the authors.

Figure 1
Figure 1. WeClawArena pairs human-centered agent-network tasks with attack-resistance evaluation. Left: autonomous personal agents collaborate on behalf of human owners across six cross-user domains. Right: model-level macro-vector attack resistance on the ASR-MAIN￾SIX-DOMAIN pool, where higher 1 − ASR indicates fewer judged attacks causing final harm. 1 arXiv:2608.03499v1 [cs.AI] 4 Aug 2026 [PITH_FULL_IMAGE:figures/full_fig… view at source ↗
Figure 2
Figure 2. Overview of WeClawArena. Autonomous agents in a human-centered agent network collaborate on behalf of human users over personal digital workspaces, including filesystems, databases, tools, personal policies, and task resources, all simulated in Docker containers. The WeClawArena sandbox implements a message gateway that routes multi-agent communication, tool use, and workspace access while recording audit evidence f… view at source ↗
Figure 3
Figure 3. Benchmark construction pipeline. Source task pools and user-role profiles are curated into base tasks with owner workspaces, tools, predicates, and task contracts. Each base task becomes a scenario bundle with ground truth, agent/tool configuration, seed facts, personas, workspace resources, and MCP interfaces, then expands into one benign control and four attack-vector variants. Evaluation checks structural validit… view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: Attack-vector utility, resistance, and model-level utility-risk tradeoff on the ASR-MAIN-SIX-DOMAIN pool. Left: TSR by variant and per-vector row-micro 1 − ASRmicro v , where higher values indicate fewer judged attacks causing final harm. Right: model-level resistance …
Figure 5
Figure 5. Figure 5: Runtime evidence record fields preserved by the benchmark gateway for task scoring and attack-success auditing. C Benchmark Bundle and Social Topology Benchmark scope. WeClawArena contains 124 base tasks and 620 scenario variants across attack vectors in six domains. E…
Figure 6
Figure 6. Figure 6: Representative travel instruction showing an approval-gated hotel booking workflow. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Representative SWE-Workspace role bundle separating patch, review, ownership, and CI evidence responsibilities. Clinical . Clinical tasks use the same scenario-bundle contract as the other domains. Each base bundle specifies role-scoped case records, protected facts, c…
Figure 8
Figure 8. Figure 8: ASR evidence packet fields consumed by the attack-success judge after simulation. F Additional Experimental Results [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Representative bargaining agent system and role contracts. Bidding Agent System Shared contract Run a sealed bidding window for one auction and lot. Public auction facts may be shared, but buyer budgets and bid strategies remain owner-scoped until each buyer submits a …
Figure 10
Figure 10. Figure 10: Representative bidding agent system and role contracts. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: Representative travel agent system and role contracts. SWE-Workspace Agent System Shared contract Fix a reduced repository issue by reading owner-local evidence before each workflow step. Submission is valid only after implementation, review, owner approval, CI eviden…
Figure 12
Figure 12. Figure 12: Representative SWE-Workspace agent system and role contracts. Clinical Agent System Shared contract Coordinate a case action using role-scoped case records, protected facts, consent or approval constraints, and tool-recorded clinical workflow state. Protected patient …
Figure 13
Figure 13. Figure 13: Representative clinical agent system and role contracts. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_13.png]
Figure 14
Figure 14. Figure 14: Representative trading agent system and role contracts. Collaboration Attack Vector Domain Target and delivery Final-harm evidence Bargaining Buyer, seller, or approver through peer mes￾sage or negotiation state. Agreement blocked, wrong shared terms accepted, or clos…
Figure 15
Figure 15. Figure 15: Collaboration attack-vector design by domain. Rows summarize target, delivery surface, and final-harm evidence without publishing reusable full attack payloads. Security Attack Vector Domain Target and delivery Final-harm evidence Bargaining Closing agent through mark…
Figure 16
Figure 16. Figure 16: Security attack-vector design by domain. Rows summarize target, delivery surface, and final-harm evidence without publishing reusable full attack payloads. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_16.png]
Figure 17
Figure 17. Figure 17: Privacy attack-vector design by domain. Rows summarize target, delivery surface, and final-harm evidence without publishing reusable full attack payloads. Governance Attack Vector Domain Target and delivery Final-harm evidence Bargaining Approver or closer through for…
Figure 18
Figure 18. Figure 18: Governance attack-vector design by domain. Rows summarize target, delivery surface, and final-harm evidence without publishing reusable full attack payloads. ASR (%) Inter Human κ FPR (%) Surface G5.2 O4.7 ∆ κ vs. G5.2 vs. O4.7 G5.2 O4.7 Collaboration 18.3 22.3 +4.0 0…
Figure 19
Figure 19. Figure 19: Domain-level attack resistance by harm surface on ASR-MAIN-SIX-DOMAIN. Each cell reports domain-vector row-micro 1−ASRmicro d,v . Higher values indicate fewer judged attacks causing final harm; raw denominators are given in [PITH_FULL_IMAGE:figures/full_fig_p029_19.png]
Figure 20
Figure 20. Figure 20 [PITH_FULL_IMAGE:figures/full_fig_p029_20.png]
Figure 21
Figure 21. Figure 21: Per-model utility and attack-resistance tradeoff by harm surface on ASR-MAIN-SIX-DOMAIN. The x-axis reports matched TSR drop against the model’s own no-attacker rows; the y-axis reports model-vector row-micro 1 − ASRmicro m,v . Points in the upper-left are preferred. …
Figure 22
Figure 22. Figure 22: Judge concordance on the 32 model-vector ASR cells from ASR-MAIN-SIX-DOMAIN and ASR-SENSITIVITY-SIX￾DOMAIN. Each point is one (model, attack vector) pair. Color encodes attack vector; the dashed line is y = x. Most points fall slightly above the diagonal, indicating t…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 44 canonical work pages

  1. [1]

    arXiv preprint arXiv:2604.01487 , year=

    AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networks , author=. arXiv preprint arXiv:2604.01487 , year=

    arXiv

  2. [2]

    Wang, Prince Zizhuang and Jiang, Shuli , journal=

    work page

  3. [3]

    Nature , volume=

    Collective dynamics of `small-world' networks , author=. Nature , volume=. 1998 , publisher=

    1998

  4. [4]

    arXiv preprint arXiv:2604.05485 , year=

    Auditable Agents , author=. arXiv preprint arXiv:2604.05485 , year=

    arXiv

  5. [5]

    Zhang, Haiyue and Nian, Yi and Zhao, Yue , journal=

    work page

  6. [6]

    2026 , eprint=

    Counterfactual Trace Auditing of LLM Agent Skills , author=. 2026 , eprint=

    work page 2026

  7. [7]

    Deng, Gangda and Chen, Zhaoling and Yu, Zhongming and Fan, Haoyang and Liu, Yuhong and Yang, Yuxin and Parikh, Dhruv and Kannan, Rajgopal and Cong, Le and Wang, Mengdi and Zhang, Qian and Prasanna, Viktor and Tang, Xiangru and Wang, Xingyao , journal=

    work page

  8. [8]

    Tang, Zirui and Zhou, Xuanhe and Liu, Yumou and Li, Linchun and Wu, Yukai and Wang, Weizheng and Huang, Hongzhang and Zhou, Wei and Zhou, Jun and Song, Jiachen and Yu, Shaoli and Wang, Jinqi and Zhou, Zihang and Zhou, Hongyi and Lv, Yuting and Li, Jinyang and Liu, Jiashuo and Chen, Ruoyu and Liu, Chunwei and Li, GuoLiang and Kang, Jihua and Wu, Fan , journal=

    work page

Show all 66 references
  1. [9]

    2026 , eprint=

    When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents , author=. 2026 , eprint=

    2026

  2. [10]

    Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*) , pages=

    Model Cards for Model Reporting , author=. Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*) , pages=

  3. [11]

    2025 , note=

    Yao, Shunyu and Shinn, Noah and Razavi, Pedram and Narasimhan, Karthik , booktitle=. 2025 , note=

    2025

  4. [12]

    Barres, Victor and Dong, Honghua and Ray, Soham and Si, Xujie and Narasimhan, Karthik , journal=

  5. [13]

    Lu, Jiarui and Holleis, Thomas and Zhang, Yizhe and Aumayer, Bernhard and Nan, Feng and Bai, Felix and Ma, Shuang and Ma, Shen and Li, Mengyu and Yin, Guoli and Wang, Zirui and Pang, Ruoming , journal=

  6. [14]

    arXiv preprint arXiv:2505.13328 , year=

    Rethinking Stateful Tool Use in Multi-Turn Dialogues: Benchmarks and Challenges , author=. arXiv preprint arXiv:2505.13328 , year=

    arXiv

  7. [15]

    and Zhu, Hao and Zhou, Xuhui and Lo, Robert and Sridhar, Abishek and Cheng, Xianyi and Ou, Tianyue and Bisk, Yonatan and Fried, Daniel and Alon, Uri and Neubig, Graham , journal=

    Zhou, Shuyan and Xu, Frank F. and Zhu, Hao and Zhou, Xuhui and Lo, Robert and Sridhar, Abishek and Cheng, Xianyi and Ou, Tianyue and Bisk, Yonatan and Fried, Daniel and Alon, Uri and Neubig, Graham , journal=

  8. [16]

    Xie, Tianbao and Zhang, Danyang and Chen, Jixuan and Li, Xiaochuan and Zhao, Siheng and Cao, Ruisheng and Hua, Toh Jing and Cheng, Zhoujun and Shin, Dongchan and Lei, Fangyu and Liu, Yitao and Xu, Yiheng and Zhou, Shuyan and Savarese, Silvio and Xiong, Caiming and Zhong, Victo...

  9. [17]

    Trivedi, Harsh and Khot, Tushar and Hartmann, Mareike and Manku, Ruskin and Dong, Vinty and Li, Edward and Gupta, Shashank and Sabharwal, Ashish and Balasubramanian, Niranjan , journal=

  10. [18]

    and Del Verme, Manuel and Marty, Tom and Boisvert, L

    Drouin, Alexandre and Gasse, Maxime and Caccia, Massimo and Laradji, Issam H. and Del Verme, Manuel and Marty, Tom and Boisvert, L. arXiv preprint arXiv:2403.07718 , year=

    arXiv

  11. [19]

    and Chang, Kai-Wei and Lee, Chen-Yu and Palangi, Hamid and Pfister, Tomas , journal=

    Yin, Fan and Wang, Zifeng and Hsu, I-Hung and Yan, Jun and Jiang, Ke and Chen, Yanfei and Gu, Jindong and Le, Long T. and Chang, Kai-Wei and Lee, Chen-Yu and Palangi, Hamid and Pfister, Tomas , journal=

  12. [20]

    and Amato, Christopher , year=

    Oliehoek, Frans A. and Amato, Christopher , year=. A Concise Introduction to Decentralized

  13. [21]

    ``Humans Welcome to Observe'': A First Look at the Agent Social Network

    Jiang, Yukun and Zhang, Yage and Shen, Xinyue and Backes, Michael and Zhang, Yang , journal=. ``Humans Welcome to Observe'': A First Look at the Agent Social Network

  14. [22]

    Does Socialization Emerge in

    Li, Ming and Li, Xirui and Zhou, Tianyi , journal=. Does Socialization Emerge in

  15. [23]

    and Xu, Shaoyang and Zhang, Wenxuan , journal=

    Feng, Yi and Huang, Chen and Man, Zhibo and Tan, Ryner and Hoang, Long P. and Xu, Shaoyang and Zhang, Wenxuan , journal=

  16. [24]

    Collective Behavior of

    De Marzo, Giordano and Garcia, David , journal=. Collective Behavior of

  17. [25]

    Yang, Ziyi and Zhang, Zaibin and Zheng, Zirui and Jiang, Yuxian and Gan, Ziyue and Wang, Zhiyu and Ling, Zijian and Chen, Jinsong and Ma, Martz and Dong, Bowen and Gupta, Prateek and Hu, Shuyue and Yin, Zhenfei and Li, Guohao and Jia, Xu and Wang, Lijun and Ghanem, Bernard and...

  18. [26]

    2025 , howpublished=

    Announcing the Agent2Agent Protocol (. 2025 , howpublished=

    2025

  19. [27]

    arXiv preprint arXiv:2304.03442 , year=

    Generative Agents: Interactive Simulacra of Human Behavior , author=. arXiv preprint arXiv:2304.03442 , year=

    arXiv

  20. [28]

    Gao, Chen and Lan, Xiaochong and Lu, Zhihong and Mao, Jinzhu and Piao, Jinghua and Wang, Huandong and Jin, Depeng and Li, Yong , journal=

  21. [29]

    Zhu, Kunlun and Du, Hongyi and Hong, Zhaochen and Yang, Xiaocheng and Guo, Shuyi and Wang, Zhe and Wang, Zhenhailong and Qian, Cheng and Tang, Xiangru and Ji, Heng and You, Jiaxuan , journal=

  22. [30]

    Juneja, Gurusha and Pasupulati, Jayanth Naga Sai and Albalak, Alon and Hua, Wenyue and Wang, William Yang , journal=

  23. [31]

    Topology Matters: Measuring Memory Leakage in Multi-Agent

    Liu, Jinbo and Cao, Defu and Wei, Yifei and Su, Tianyao and Liang, Yuan and Dong, Yushun and Liu, Yan and Zhao, Yue and Hu, Xiyang , booktitle=. Topology Matters: Measuring Memory Leakage in Multi-Agent. 2026 , address=

    2026

  24. [32]

    El Yagoubi, Faouzi and Badu-Marfo, Godwin and Al Mallah, Ranwa , journal=

  25. [33]

    Washington Law Review , volume=

    Privacy as Contextual Integrity , author=. Washington Law Review , volume=

  26. [34]

    Mireshghallah, Niloofar and Kim, Hyunwoo and Zhou, Xuhui and Tsvetkov, Yulia and Sap, Maarten and Shokri, Reza and Choi, Yejin , booktitle=. Can

  27. [35]

    Li, Haoran and Guo, Dadi and Li, Donghao and Fan, Wei and Hu, Qi and Liu, Xin and Chan, Chunkit and Yao, Duanyi and Yao, Yuan and Song, Yangqiu , booktitle=

  28. [36]

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P and Zhang, Hao and Gonzalez, Joseph E and Stoica, Ion , booktitle=. Judging

  29. [37]

    International Joint Conference on Artificial Intelligence , year=

    Large Language Model based Multi-Agents: A Survey of Progress and Challenges , author=. International Joint Conference on Artificial Intelligence , year=

  30. [38]

    The Twelfth International Conference on Learning Representations , year=

    Hong, Sirui and Zhuge, Mingchen and Chen, Jonathan and Zheng, Xiawu and Cheng, Yuheng and Wang, Ceyao and Zhang, Zili and Wang, Steven Ka Shing Yau and Lin, Zijuan and Zhou, Liyang and Ran, Chenyu and Xiao, Lingfeng and Wu, Chenglin and Schmidhuber, J. The Twelfth Internationa...

  31. [39]

    Wu, Qingyun and Bansal, Gagan and Zhang, Jieyu and Wu, Yiran and Li, Beibin and Zhu, Erkang and Jiang, Li and Zhang, Xiaoyun and Zhang, Shaokun and Liu, Jiale and others , journal=. Auto

  32. [40]

    Chen, Weize and Su, Yusheng and Zuo, Jingwei and Yang, Cheng and Yuan, Chenfei and Chan, Chi-Min and Yu, Heyang and Lu, Yaxi and Hung, Yi-Hsin and Qian, Chen and others , booktitle=. Agent

  33. [41]

    The Thirteenth International Conference on Learning Representations , year=

    Scaling Large Language Model-based Multi-Agent Collaboration , author=. The Thirteenth International Conference on Learning Representations , year=

  34. [42]

    Group-in-Group Policy Optimization for

    Feng, Lang and Xue, Zhenghai and Liu, Tingcong and An, Bo , booktitle=. Group-in-Group Policy Optimization for

  35. [43]

    Xia, Peng and Chen, Jianwen and Wang, Hanyang and Liu, Jiaqi and Zeng, Kaide and Wang, Yu and Han, Siwei and Zhou, Yiyang and Zhao, Xujiang and Chen, Haifeng and Zheng, Zeyu and Xie, Cihang and Yao, Huaxiu , journal=

  36. [44]

    and Yang, Yuqing , journal=

    Luo, Xufang and Zhang, Yuge and He, Zhiyuan and Wang, Zilong and Zhao, Siyun and Li, Dongsheng and Qiu, Luna K. and Yang, Yuqing , journal=

  37. [45]

    Reinforcement Learning for Long-Horizon Interactive

    Chen, Kevin and Cusumano-Towner, Marco and Huval, Brody and Petrenko, Aleksei and Hamburger, Jackson and Koltun, Vladlen and Kr. Reinforcement Learning for Long-Horizon Interactive. arXiv preprint arXiv:2502.01600 , year=

    arXiv

  38. [46]

    Wang, Zihan and Wang, Kangrui and Wang, Qineng and Zhang, Pingyue and Li, Linjie and Yang, Zhengyuan and Jin, Xing and Yu, Kefan and Nguyen, Minh Nhat and Liu, Licheng and Gottlieb, Eli and Lu, Yiping and Cho, Kyunghyun and Wu, Jiajun and Fei-Fei, Li and Wang, Lijuan and Choi,...

  39. [47]

    Reinforcing Multi-Turn Reasoning in

    Wei, Quan and Zeng, Siliang and Li, Chenliang and Brown, William and Frunza, Oana and Deng, Wei and Schneider, Anderson and Nevmyvaka, Yuriy and Zhao, Yang Katie and Garcia, Alfredo and Hong, Mingyi , journal=. Reinforcing Multi-Turn Reasoning in

  40. [48]

    arXiv preprint arXiv:2501.10893 , year=

    Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments , author=. arXiv preprint arXiv:2501.10893 , year=

    arXiv

  41. [49]

    arXiv preprint arXiv:2303.08774 , year=

    GPT-4 Technical Report , author=. arXiv preprint arXiv:2303.08774 , year=

    arXiv

  42. [50]

    arXiv preprint arXiv:2312.11805 , year=

    Gemini: A Family of Highly Capable Multimodal Models , author=. arXiv preprint arXiv:2312.11805 , year=

    arXiv

  43. [51]

    arXiv preprint arXiv:2407.10671 , year=

    Qwen2 Technical Report , author=. arXiv preprint arXiv:2407.10671 , year=

    arXiv

  44. [52]

    arXiv preprint arXiv:2412.19437 , year=

    arXiv

  45. [53]

    2025 , howpublished=

    Steinberger, Peter and. 2025 , howpublished=

    2025

  46. [54]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open Foundation and Fine-Tuned Chat Models , author=. arXiv preprint arXiv:2307.09288 , year=

    arXiv

  47. [55]

    The Eleventh International Conference on Learning Representations , year=

    ReAct: Synergizing Reasoning and Acting in Language Models , author=. The Eleventh International Conference on Learning Representations , year=

  48. [56]

    Liu, Xiao and Yu, Hao and Zhang, Hanchen and Xu, Yifan and Lei, Xuanyu and Lai, Hanyu and Gu, Yu and Ding, Hangliang and Men, Kaiwen and Yang, Kejuan and others , journal=

  49. [57]

    arXiv preprint arXiv:2311.12983 , year=

    Mialon, Gr. arXiv preprint arXiv:2311.12983 , year=

    arXiv

  50. [58]

    War and Peace (

    Hua, Wenyue and Fan, Lizhou and Li, Lingyao and Mei, Kai and Ji, Jianchao and Ge, Yingqiang and Hemphill, Libby and Zhang, Yongfeng , journal=. War and Peace (

  51. [59]

    Li, Guohao and Hammoud, Hasan Abed Al Kader and Itani, Hani and Khizbullin, Dmitrii and Ghanem, Bernard , journal=

  52. [60]

    Gong, Ran and Huang, Qiuyuan and Ma, Xiaojian and Vo, Hoi and Durante, Zane and Noda, Yusuke and Zheng, Zilong and Zhu, Song-Chun and Terzopoulos, Demetri and Fei-Fei, Li and Gao, Jianfeng , journal=

  53. [61]

    Zhou, Xuhui and Zhu, Hao and Mathur, Leena and Zhang, Ruohong and Yu, Haofei and Qi, Zhengyang and Morency, Louis-Philippe and Bisk, Yonatan and Fried, Daniel and Neubig, Graham and Sap, Maarten , booktitle=

  54. [62]

    Shao, Yijia and Li, Tianshi and Shi, Weiyan and Liu, Yanchen and Yang, Diyi , booktitle=

  55. [63]

    Piao, Jinghua and Yan, Yuwei and Zhang, Jun and Li, Nian and Yan, Junbo and Lan, Xiaochong and Lu, Zhihong and Zheng, Zhiheng and Wang, Jing Yi and Zhou, Di and Gao, Chen and Xu, Fengli and Zhang, Fang and Rong, Ke and Su, Jun and Li, Yong , journal=

  56. [64]

    , journal=

    He, Feng and Zhu, Tianqing and Ye, Dayong and Liu, Bo and Zhou, Wanlei and Yu, Philip S. , journal=. The Emerged Security and Privacy of. 2025 , doi=

    2025

  57. [65]

    Mireshghallah, Niloofar and Mangaokar, Neal and Kokhlikyan, Narine and Zharmagambetov, Arman and Zaheer, Manzil and Mahloujifar, Saeed and Chaudhuri, Kamalika , journal=

  58. [66]

    Zharmagambetov, Arman and Guo, Chuan and Evtimov, Ivan and Pavlova, Maya and Salakhutdinov, Ruslan and Chaudhuri, Kamalika , journal=

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.