Pith. sign in

REVIEW 4 major objections 5 minor 57 references

Alternating joint tuning of planning and grounding agents closes the capability gap and improves held-out task performance by 4.4% on average.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

MOAT alternately aligns a planning and a grounding LLM agent via perplexity-based DPO and self-generated SFT, reporting small but consistent gains over independently tuned baselines.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A plausible empirical recipe for joint agent tuning, but the convergence proof is decorative; send it to review with the expectation of major revision. the 4 major comments →

arxiv 2509.09629 v1 pith:HMMKZB3P submitted 2025-09-11 cs.CL

Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems

Minghang Zhu , Zhengliang Shi , Zhiwei Xu , Shiguang Wu , Lingjie Wang , Pengjie Ren , Zhaochun Ren , Zhumin Chen This is my paper
classification cs.CL
keywords multi-agent systemsjoint alignment tuningdirect preference optimizationperplexity rewardtool-use agentscapability gapplan-ground-executegeneralization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that in a plan-ground-execute multi-agent system, independently fine-tuning the planner and the tool-using grounding agent leaves a capability gap that hurts end-to-end performance. It proposes MOAT, a two-stage loop: first, the planner is aligned to the grounding agent via direct preference optimization (DPO) using the grounding agent's perplexity on correct tool calls as a reward; second, the grounding agent is fine-tuned on diverse, critic-corrected subgoal-action pairs generated by the planner. The paper claims this alternation guarantees non-decreasing expected reward and convergence, and reports consistent gains on six benchmarks, with 3.1% average improvement on held-in tasks and 4.4% on held-out tasks across Llama, Mistral, and Qwen backbones.

Core claim

MOAT's central claim is that joint, iterative alignment beats independent tuning: optimizing the planner to produce subgoals the currently trained grounding agent can follow with low perplexity, and then fine-tuning the grounding agent on those planner-produced subgoals, yields a self-reinforcing loop that improves both coordination and generalization. The paper formalizes the loop as alternating optimization of two objectives—DPO on the planner with perplexity reward, and SFT on the grounding agent—and proves via the monotone convergence theorem that the expected reward sequence is non-decreasing and converges. Empirically, the method outperforms AgentTuning, Agent-FLAN, and Agent Lumos on

What carries the argument

The crux is a closed feedback loop between two agents. The grounding agent's perplexity PPL(a|x,I,s) over the ground-truth tool-call sequence given a candidate subgoal sequence serves as a reward for the planner: lower perplexity is taken to mean the subgoal better guides the grounding agent. DPO then shifts the planner toward low-PPL subgoals. In the second stage, the planner's sampled subgoals are fed back to the grounding agent, which generates actions that are filtered and corrected by a critic model; the resulting pairs fine-tune the grounding agent. This loop is the mechanism that is claimed to bridge the capability gap.

Load-bearing premise

The result rests on the assertion that lower perplexity of the grounding agent on ground-truth tool actions is a valid proxy for higher subgoal quality—an assumption stated in Lemma 5.1 but not empirically validated apart from final task scores.

What would settle it

Sample a set of subgoal sequences for held-out tasks, compute grounding-agent perplexity on the correct action sequence, and also measure end-to-end task success. If the rank correlation between lower perplexity and higher success is weak or negative, the DPO reward is misaligned and the claimed monotone improvement would not hold. A Spearman correlation below roughly 0.3 on held-out data would be direct counter-evidence.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If correct, a scalar perplexity signal from one agent can substitute for hand-crafted reward functions when aligning another agent in a multi-agent system.
  • The two-stage loop should transfer to other plan-ground-execute pipelines, since it operates on subgoal sequences and tool-call actions rather than on task-specific features.
  • The equal-training-time comparison indicates gains come from co-adaptation, not simply from more epochs; this suggests iterative joint tuning is a distinct and useful training signal.
  • The reported held-out improvements imply that alignment induces robust subgoal-action mappings rather than overfitting to training tasks.
  • Stronger critic models improve results, but the framework remains competitive with a smaller open-source critic, making it viable in settings without proprietary API access.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The convergence proof hinges on an assumption left unverified: that grounding-agent perplexity is monotonically negatively correlated with true task reward. A direct rank-correlation test on held-out subgoal-action pairs would either substantiate or undermine the theoretical guarantee.
  • The same alternating loop could be generalized to more than two agents by assigning a 'understanding score' between each producer and consumer; the paper does not explore this multi-hop extension.
  • Because the paper explicitly limits its evaluation to text-based scenarios, the claimed generalization should not be assumed for multimodal or embodied tasks until the framework is extended.
  • A practical byproduct worth testing: perplexity spikes in the grounding agent could serve as a lightweight runtime diagnostic for capability mismatch, flagging subgoals the system is ill-equipped to execute.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MOAT, an iterative joint alignment tuning framework for LLM-based multi-agent systems in a plan-ground-execute pipeline. In each iteration, the planning agent is trained with DPO using the grounding agent's perplexity on ground-truth action sequences as a reward signal, and the grounding agent is fine-tuned on critic-corrected subgoal-action pairs sampled from the planner. The authors claim a theoretical guarantee of non-decreasing expected reward and convergence, and support the method with experiments on six benchmarks across Llama, Mistral, and Qwen backbones, reporting average gains over Lumos on held-in and held-out tasks.

Significance. If the empirical results are robust, MOAT is a useful contribution: it demonstrates a concrete way to co-adapt a planner and a grounding agent, reports held-out generalization across three model families, and ships open code. The idea of using one agent's predictive confidence as a training signal for another agent is interesting and may inspire further work on joint multi-agent tuning. However, the paper's headline theoretical claim—non-decreasing and convergent joint optimization—is not established by the provided lemmas, and the empirical evidence is weakened by small test sets and the absence of uncertainty quantification. The practical contribution is plausible, but the theory needs substantial revision or removal.

major comments (4)
  1. [Section 5, Lemma 5.1 and Eq. (4)-(6)] The proof of non-decreasing expected reward rests entirely on the assertion that PPL is 'negatively correlated with the true reward R(s,a)'. This is not demonstrated. PPL is computed on the ground-truth action sequence under the current grounding model; it measures how easily that model can condition on the subgoal, not whether the subgoal leads to successful task completion. Because the grounding model changes every iteration, low PPL for pi_g^t does not imply high reward for pi_g^{t+1}. Moreover, DPO is a finite-sample, KL-regularized preference update; it does not by itself guarantee per-iteration monotone expected-reward improvement even with a perfect reward. The Monotone Convergence Theorem therefore cannot be invoked. The authors should either prove Lemma 5.1 under explicit assumptions and validate the PPL-reward correlation empirically (e.g., Spearman correlation with execution s
  2. [Section 5, Lemma 5.2 and Eq. (7)-(9)] The analogous step for the grounding agent is also unsupported. SFT on critic-corrected self-generated data maximizes likelihood on that filtered dataset, but this does not imply a non-decreasing expected task reward. The sentence 'improved log-likelihood corresponds to reduced PPL and, consequently, higher reward' conflates likelihood on the training distribution with task reward. The critic model modifies the actions using ground-truth references, so the training labels are not simply the grounding agent's own outputs; the distribution shift breaks the link between Eq. (8) and Eq. (9). As with Lemma 5.1, the monotonicity claim does not follow. A bounded-reward argument alone cannot rescue the proof if the sequence E[R] is not shown to be non-decreasing.
  3. [Section 7.1, Table 2] The experimental comparisons lack error bars, confidence intervals, or significance tests. Several reported gaps are within 1-3 points on small test sets: HotpotQA has n=100, StrategyQA n=300, Mind2Web n=200. For example, Qwen2.5-14B HotpotQA is 28.0 vs. 27.0 (one additional success), Mistral WebShop is 59.0 vs. 58.7, and Qwen GSM8K is 82.4 vs. 81.7. These differences may be sampling noise. The paper should report confidence intervals or paired significance tests, and the 'consistently outperforms' claim should be calibrated to the actual uncertainty.
  4. [Section 7.3, Table 4] The reported perplexity reduction is the exact training objective used to align the planning agent, so observing lower PPL after MOAT is not independent evidence that the capability gap is bridged. The claim that 'the reduction in perplexity demonstrates that our joint alignment process effectively harmonizes the agents' is circular unless PPL is shown to correlate with end-to-end task success on held-out data. The authors should validate this correlation, or reframe Table 4 as a sanity check rather than evidence of effectiveness.
minor comments (5)
  1. [Section 4.1, PPL formula] The perplexity formula contains a typo: it reads P_{pi_g}(s | a_{<i}, x, I, s), but the variable being scored should be a_i, not s. It should be P_{pi_g}(a_i | a_{<i}, x, I, s).
  2. [Abstract and Section 7.1] The abstract claims average improvements of 3.1% on held-in and 4.4% on held-out tasks, but these numbers are not directly reproducible from Table 2. The 4.4% figure matches the Mistral held-out comparison only, and the held-in average across model families is closer to 2.5-3.0%. Please clarify the comparison base.
  3. [Table 2] The column header 'GSK8K' is a typo for 'GSM8K'.
  4. [References] Yin et al. 2024a and 2024b are the same paper; use a single reference. Also fix 'baselins' in Section 6.1.
  5. [Section 7.6, Figure 5] The 'training iteration control analysis' is shown only on WebShop. State explicitly that this is a single-benchmark analysis, or add similar results for other tasks to support the claim that gains are not due to extra epochs.

Circularity Check

2 steps flagged

The convergence proof is anchored by a self-cited PPL-reward equivalence, and the reported PPL reduction is the optimized training objective itself; task-performance gains provide independent support.

specific steps
  1. self citation load bearing [Section 4.1, Planning Agent Alignment, first paragraph]
    "Perplexity reflects how well the grounding agent can follow a subgoal, where lower perplexity indicates higher suitability (Gao et al., 2024)."

    This sentence supplies the load-bearing premise for the entire Planning Agent Alignment stage: PPL of the grounding agent on gold actions is used as the DPO reward. The cited Gao et al. 2024 includes overlapping authors (Zhengliang Shi, Minghang Zhu, Pengjie Ren, Zhaochun Ren, Zhumin Chen) with the present paper. The same premise is restated without proof in Lemma 5.1 as 'PPL is negatively correlated with the true reward R(s,a)' and is then used to conclude non-decreasing expected reward. Since the cited work is not machine-checked, externally replicated, or parameter-free with stated assumptions independent of this result, the central reward-signal justification is imported from the authors' own prior work rather than independently established.

  2. other [Section 7.3, Analysis of the Capability Gap, Table 4]
    "To quantitatively validate our core hypothesis of a capability gap, we measured the perplexity of the grounding agent when generating correct actions. ... The reduction in perplexity demonstrates that our joint alignment process effectively harmonizes the agents, directly bridging this capability gap."

    The MOAT Stage 1 objective explicitly trains the planner to minimize this exact perplexity: Equation (4) maximizes E[-PPL(a; πg)] over the planner. Thus the observed PPL reduction in Table 4 is a direct consequence of the optimization target, not an independent validation that the capability gap has been bridged. The paper presents this self-referential metric as evidence of the framework's effectiveness. The end-task metrics (accuracy, step success rate, average reward) are external and provide the real empirical support, so this circularity is limited to the interpretation of Table 4 rather than the headline results.

full rationale

The paper's central empirical claims are not circular: the held-in and held-out improvements (3.1% / 4.4%) are measured with task success metrics that are external to the training loop, and the grounding-agent SFT data is filtered and corrected by a critic against ground-truth references. However, the theoretical convergence proof is not a self-contained derivation. Lemma 5.1 assumes that grounding-agent PPL on gold actions is negatively correlated with the true reward R(s,a), an equivalence that is asserted and, in the text, justified by a self-citation (Gao et al., 2024) rather than proven. Lemma 5.2 similarly assumes that improving log-likelihood on the generated training set implies higher expected task reward. These are correctness gaps, and they also create a partial circularity because the training loop optimizes exactly the quantity (PPL) that is later reported as validation. The Monotone Convergence Theorem application would be valid only if the lemmas held; since the crucial equivalence is unverified and the DPO/SFT updates do not by themselves guarantee monotone expected-reward improvement, the 'theoretical guarantee' reduces to an assumption rather than a derivation. Overall, the paper has independent experimental content, but the convergence claim is weakened by self-citation and by-construction metric reporting, yielding a score of 4.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The paper introduces no new physical or conceptual entity requiring independent evidence; 'capability gap' is a descriptive label, not a postulated mechanism. The main burden is carried by the unproved monotonicity axioms in Lemmas 5.1 and 5.2, the PPL-reward correlation assumption, and the reliability of the critic model.

free parameters (5)
  • K (number of sampled subgoal sequences) = 15
    Selected by scanning K=5/10/15 on held-in tasks (Figure 3); larger K improves results, so K=15 is a tuned choice.
  • Number of MOAT iterations = 2
    Chosen from held-in performance trends (Figure 4); iteration 3 gives marginal gains. This early stopping is based on test performance.
  • DPO reward-difference filter threshold = 0.1
    Appendix A.1: 'we filter out data samples where the reward difference between s_w and s_l is less than 0.1 for stability during DPO training.' Chosen ad hoc.
  • Sampling temperature for subgoal generation = 1.0
    Section 6.3: set to 1.0 to encourage diversity in generated subgoals; hand-chosen.
  • DPO temperature beta = not reported
    Hyperparameter in the DPO loss (Section 4.1); no value is given in the paper, so the strength of the preference update is unspecified.
axioms (6)
  • ad hoc to paper DPO updates give non-decreasing expected reward when the grounding agent is fixed (Lemma 5.1).
    This is the key premise of the convergence proof; no derivation is supplied and DPO does not in general provide a monotone reward guarantee.
  • ad hoc to paper SFT on critic-corrected generated pairs gives non-decreasing expected reward when the planning agent is fixed (Lemma 5.2).
    Assumes likelihood gains on the training distribution transfer to reward on the planner's output distribution; asserted without proof.
  • domain assumption Perplexity of the grounding agent on ground-truth actions is negatively correlated with true reward R(s,a).
    Used in Section 4.1 and Lemma 5.1; perplexity may reflect fluency rather than task success, and the correlation is not validated.
  • domain assumption LLM sampling produces a diverse and meaningful set of subgoal sequences, so the best of K samples is a useful positive example.
    Section 4.1 K-times sampling; if all K samples are poor, DPO will prefer the least-bad option and training can be misled (the paper itself notes this for K=5).
  • domain assumption The critic model's verification and corrections are reliable enough to avoid training collapse.
    Section 4.2 and Algorithm 1; grounding-agent SFT uses critic-corrected outputs, so critic errors directly corrupt the training signal.
  • standard math Monotone Convergence Theorem for bounded sequences.
    Section 5; the theorem is valid, but applying it requires the monotonicity that Lemmas 5.1 and 5.2 assert without proof.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems." pith.science (2026). https://pith.science/paper/HMMKZB3P

@misc{pith2026250909629,
  author       = {Pith},
  title        = {Pith review of: Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HMMKZB3P}},
  note         = {Machine review of arXiv:2509.09629}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The advancement of large language models (LLMs) has enabled the construction of multi-agent systems to solve complex tasks by dividing responsibilities among specialized agents, such as a planning agent for subgoal generation and a grounding agent for executing tool-use actions. Most existing methods typically fine-tune these agents independently, leading to capability gaps among them with poor coordination. To address this, we propose MOAT, a Multi-Agent Joint Alignment Tuning framework that improves agents collaboration through iterative alignment. MOAT alternates between two key stages: (1) Planning Agent Alignment, which optimizes the planning agent to generate subgoal sequences that better guide the grounding agent; and (2) Grounding Agent Improving, which fine-tunes the grounding agent using diverse subgoal-action pairs generated by the agent itself to enhance its generalization capablity. Theoretical analysis proves that MOAT ensures a non-decreasing and progressively convergent training process. Experiments across six benchmarks demonstrate that MOAT outperforms state-of-the-art baselines, achieving average improvements of 3.1% on held-in tasks and 4.4% on held-out tasks.

Figures

Figures reproduced from arXiv: 2509.09629 by Lingjie Wang, Minghang Zhu, Pengjie Ren, Shiguang Wu, Zhaochun Ren, Zhengliang Shi, Zhiwei Xu, Zhumin Chen.

Figure 1
Figure 1. Figure 1: Comparison between (a) previous indepen [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The proposed MOAT framework iteratively alternates between two stages: (1) Planning Agent Alignment: [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Results of MOAT on three held-in tasks under [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance trends of MOAT (K=15) on held [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance comparison on WebShop be￾tween MOAT and Lumos under equal training time. We use the Avg. Reward as evaluation metric. 7.6 Training Iteration Control Analysis A potential concern is that the observed perfor￾mance gains from our iterative training strategy may stem merely from additional training epochs, rather than from the collaborative optimization of planning and grounding agents. To investig… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 16 linked inside Pith

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    Pith/arXiv arXiv 2023

  2. [2]

    John Bibby. 1974. Axiomatisations of the average and a further generalisation of monotonic sequences. Glasgow Mathematical Journal

    1974

  3. [3]

    Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2024. Chateval: Towards better LLM -based evaluators through multi-agent debate. In International Conference on Learning Representations: ICLR

    2024

  4. [4]

    Harrison Chase. 2022. https://github.com/langchain-ai/langchain LangChain

    2022

  5. [5]

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. 2023. Fireact: Toward language agent fine-tuning. arXiv preprint arXiv:2310.05915

    Pith/arXiv arXiv 2023

  6. [6]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

    Pith/arXiv arXiv 2021

  7. [7]

    Nuo Chen, Yicheng Tong, Jiaying Wu, Minh Duc Duong, Qian Wang, Qingyun Zou, Bryan Hooi, and Bingsheng He. 2025 a . https://arxiv.org/abs/2508.04575 Beyond brainstorming: What drives high-quality scientific ideas? lessons from multi-agent collaboration . Preprint, arXiv:2508.04575

    Pith/arXiv arXiv 2025

  8. [8]

    Nuo Chen, GUOJUN XIONG, and Bingsheng He. 2025 b . MPAW : Multi-preference alignment through weak model collaboration for efficient and flexible LLM decoding. In Scaling Self-Improving Foundation Models without Human Supervision: SSI-FM

    2025

  9. [9]

    Zehui Chen, Kuikun Liu, Qiuchen Wang, Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao. 2024. Agent- FLAN : Designing data and methods of effective agent tuning for large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9354--9366

    2024

  10. [10]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    Pith/arXiv arXiv 2021

  11. [11]

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems: NeurIPS, 36:28091--28114

    2023

  12. [12]

    Elvis Dohmatob, Yunzhen Feng, and Julia Kempe. 2024. Model collapse demystified: The case of regression. In Advances in Neural Information Processing Systems: NeurIPS

    2024

  13. [13]

    Shen Gao, Zhengliang Shi, Minghang Zhu, Bowen Fang, Xin Xin, Pengjie Ren, Zhumin Chen, Jun Ma, and Zhaochun Ren. 2024. Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum. In Proceedings of the AAAI Conference on Artificial Intelligence: AAAI, volume 38, pages 18030--18038

    2024

  14. [14]

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies . Transactions of the Association for Computational Linguistics: TACL

    2021

  15. [15]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. 2024. Metagpt: Meta programming for a multi-agent collaborative framework. In International Conference on Learning Representations: ICLR

    2024

  16. [16]

    Zhenyu Hou, Xin Lv, Rui Lu, Jiajie Zhang, Yujiang Li, Zijun Yao, Juanzi Li, Jie Tang, and Yuxiao Dong. 2025. Advancing language model reasoning through reinforcement learning and inference scaling. arXiv preprint arXiv:2501.11651

    Pith/arXiv arXiv 2025

  17. [17]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: EMNLP, pages 6769--6781

    2020

  18. [18]

    Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems: NeurIPS, 36:51991--52008

    2023

  19. [19]

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. 2024. Agentbench: Evaluating llms as agents. In International Conference on Learning Representations: ICLR

    2024

  20. [20]

    Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten de Rijke, and Zhaochun Ren. 2024 a . Knowtuning: Knowledge-aware fine-tuning for large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: EMNLP, pages 14535--14556

    2024

  21. [21]

    Yougang Lyu, Lingyong Yan, Zihan Wang, Dawei Yin, Pengjie Ren, Maarten de Rijke, and Zhaochun Ren. 2024 b . Macpo: weak-to-strong alignment via multi-agent contrastive preference optimization. arXiv preprint arXiv:2410.07672

    Pith/arXiv arXiv 2024

  22. [22]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems: NeurIPS, 36

    2024

  23. [23]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. Are nlp models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: NAACL, pages 2080--2094

    2021

  24. [24]

    Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Huajun Chen, et al. 2024. Autoact: Automatic agent learning from scratch for qa via self-planning. In ICLR 2024 Workshop on Large Language Model (LLM) Agents

    2024

  25. [25]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, dahai li, Zhiyuan Liu, and Maosong Sun. 2024. Tool LLM : Facilitating large language models to master 16000+ real-world API s. In International Conference on Learning R...

    2024

  26. [26]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems: NeurIPS, 36

    2024

  27. [27]

    Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, and Fei Huang. 2024 a . Small LLM s are weak tool learners: A multi- LLM agent. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: EMNLP, pages 16658--16680

    2024

  28. [28]

    Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, and Fei Huang. 2024 b . Small llms are weak tool learners: A multi-llm agent. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: EMNLP, pages 16658--16680

    2024

  29. [29]

    Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, and Zhaochun Ren. 2024 a . Learning to use tools via cooperative and interactive agents. In Findings of the Association for Computational Linguistics: EMNLP 2024

    2024

  30. [30]

    Zhengliang Shi, Shuo Zhang, Weiwei Sun, Shen Gao, Pengjie Ren, Zhumin Chen, and Zhaochun Ren. 2024 b . Generate-then-ground in retrieval-augmented generation for multi-hop question answering. In Proceedings of the 62st Annual Meeting of the Association for Computational Linguistics: ACL, pages 7339--7353

    2024

  31. [31]

    Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. Ai models collapse when trained on recursively generated data. Nature, 631(8022):755--759

    2024

  32. [32]

    Avi Singh, John D Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J Liu, James Harrison, Jaehoon Lee, Kelvin Xu, et al. 2023. Beyond human data: Scaling self-training for problem-solving with language models. arXiv preprint arXiv:2312.06585

    Pith/arXiv arXiv 2023

  33. [33]

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. 2023 a . Llm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF international conference on computer vision: ICCV, pages 2998--3009

    2023

  34. [34]

    Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. 2024. Agentbank: Towards generalized llm agents via fine-tuning on 50000+ interaction trajectories. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 2124--2141

    2024

  35. [35]

    Yifan Song, Weimin Xiong, Dawei Zhu, Wenhao Wu, Han Qian, Mingbo Song, Hailiang Huang, Cheng Li, Ke Wang, Rong Yao, et al. 2023 b . Restgpt: Connecting large language models with real-world restful apis. arXiv preprint arXiv:2306.06624

    Pith/arXiv arXiv 2023

  36. [36]

    Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. 2025. Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617

    Pith/arXiv arXiv 2025

  37. [37]

    Theodore Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas Griffiths. 2024. Cognitive architectures for language agents. Transactions on Machine Learning Research: TMLR

    2024

  38. [38]

    Weiwei Sun, Zhengliang Shi, Shen Gao, Pengjie Ren, Maarten de Rijke, and Zhaochun Ren. 2023. Contrastive learning reduces hallucination in conversations. In Proceedings of the AAAI Conference on Artificial Intelligence: AAAI, volume 37, pages 13618--13626

    2023

  39. [39]

    XAgent Team. 2023. Xagent: An autonomous agent for complex task solving. XAgent blog

    2023

  40. [40]

    Yiying Wang, Xiaojing Li, Binzhu Wang, Yueyang Zhou, Yingru Lin, Han Ji, Hong Chen, Jinshi Zhang, Fei Yu, Zewei Zhao, et al. 2024. Peer: Expertizing domain-specific tasks with a multi-agent framework and tuning methods. arXiv preprint arXiv:2407.06985

    Pith/arXiv arXiv 2024

  41. [41]

    Zihan Wang, Ziqi Zhao, Yougang Lyu, Zhumin Chen, Maarten de Rijke, and Zhaochun Ren. 2025. A cooperative multi-agent framework for zero-shot named entity recognition. In Proceedings of the ACM on Web Conference: WWW, pages 4183--4195

    2025

  42. [42]

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. 2024. Autogen: Enabling next-gen LLM applications via multi-agent conversation. In ICLR 2024 Workshop on Large Language Model (LLM) Agents

    2024

  43. [43]

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024. Travelplanner: A benchmark for real-world planning with language agents. arXiv preprint arXiv:2402.01622

    Pith/arXiv arXiv 2024

  44. [44]

    Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, and Dongkuan Xu. 2023. Rewoo: Decoupling reasoning from observations for efficient augmented language models. arXiv preprint arXiv:2305.18323

    Pith/arXiv arXiv 2023

  45. [45]

    Hui Yang, Sifu Yue, and Yunzhong He. 2023. Auto-gpt for online decision making: Benchmarks and additional opinions. arXiv preprint arXiv:2306.02224

    Pith/arXiv arXiv 2023

  46. [46]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: EMNLP, pages 2369--2380

    2018

  47. [47]

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems: NeurIPS, 35:20744--20757

    2022

  48. [48]

    Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. 2024 a . Agent lumos: Unified and modular training for open-source language agents. In Proceedings of the 62st Annual Meeting of the Association for Computational Linguistics: ACL, pages 12380--12403

    2024

  49. [49]

    Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. 2024 b . Agent lumos: Unified and modular training for open-source language agents. In Proceedings of the 62st Annual Meeting of the Association for Computational Linguistics: ACL, pages 12380--12403

    2024

  50. [50]

    Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. 2024. Self-rewarding language models. In International Conference on Machine Learning: ICML, pages 57905--57923

    2024

  51. [51]

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. 2024. A gent T uning: Enabling generalized agent abilities for LLM s. In Findings of the Association for Computational Linguistics: ACL 2024, pages 3053--3077

    2024

  52. [52]

    Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Ming Zhu, Juntao Tan, Thai Hoang, Zuxin Liu, Liangwei Yang, et al. 2024 a . Agentohana: Design unified data and training pipeline for effective agent learning. arXiv preprint arXiv:2402.15506

    Pith/arXiv arXiv 2024

  53. [53]

    Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. 2024 b . Exploring collaboration mechanisms for LLM agents: A social psychology view. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics: ACL, pages 14544--14607

    2024

  54. [54]

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. 2023. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854

    Pith/arXiv arXiv 2023

  55. [55]

    Junda Zhu, Lingyong Yan, Haibo Shi, Dawei Yin, and Lei Sha. 2024. Atm: Adversarial tuning multi-agent system makes a robust retrieval-augmented generator. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: EMNLP, pages 10902--10919

    2024

  56. [56]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  57. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.