Pith. sign in

REVIEW 5 major objections 6 minor 46 references

Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Masking

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that training a small open-source LLM agent on its own mistakes, annotated by a stronger teacher, beats training on expert trajectories alone.

desk verdict A plausible method for training agents from teacher corrections, but the main comparison is confounded by data filtering and the evidence is too weak to support the headline numbers. read the letter →

arxiv 2505.20023 v1 pith:PYZTE7IJ submitted 2025-05-26 cs.CL

Yihan Chen , Benfeng Xu , Xiaorui Wang , Yongdong Zhang , Zhendong Mao This is my paper
classification cs.CL
keywords self-reflectedtrajectoriespartialmaskingLLMagentssupervisedfine-tuningerrorreflectionopen-sourceLLMsagentbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

STeP is a training recipe that turns a small open-source LLM into an agent that can spot and fix its own mistakes. The core claim is that augmenting training data with synthetic self-reflected trajectories—rollouts where a teacher LLM pauses at the student's error, explains the mistake, and prescribes a corrective action—teaches the student to solve tasks better than training on flawless expert trajectories alone. Across ALFWorld, WebShop, and SciWorld, a LLaMA2-7B-chat agent trained this way beats a golden-trajectory baseline by 9.2% on average while using fewer training examples. The paper also introduces partial masking, a loss-level mechanism that stops the student from imitating the erroneous thought/action pair while still learning from the reflection, the correction, and the surrounding correct steps.

What carries the argument

The load-bearing object is the self-reflected trajectory: a ReAct-format rollout in which erroneous (thought, action) pairs are flagged by a teacher, and immediately followed by a synthetic reflection step (the error, its reason, and a first-person lesson) and a correction action produced by the teacher. The companion mechanism is partial masking, which modifies the SFT loss so that designated error tokens (with indicator $\delta_i = 0$) do not contribute to the gradient. Together they allow the model to learn from failed interaction data at the substep level rather than discarding the whole trajectory or blindly imitating wrong steps.

What would settle it

Run STeP twice with the same teacher, once with intact reflections and corrections and once with the reflection sentences replaced by generic filler while keeping the correction actions unchanged; if average reward does not drop relative to the intact run, the reflective content is not what drives the gain.

Watch

Extended reading notes

Core claim

STeP's central discovery is that a small LLM agent learns more from its own near-misses, annotated by a stronger teacher, than from pristine demonstrations. The method first fine-tunes the base model on a subset of golden trajectories, then lets that base agent roll out on remaining tasks while a teacher (Qwen1.5-110B-Chat) judges each step in real time. When a step is wrong, the teacher writes a reflection and a corrected action in ReAct format, and successful trajectories containing such reflection-correction pairs are kept. During a final fine-tuning pass, partial masking sets the loss for the original erroneous thought/action to zero, so the model absorbs the reflective reasoning and the corrected behavior without internalizing the mistake. The measured outcome is a 9.2% average reward gain over training on golden trajectories alone, with smaller datasets, across three benchmark environments.

Load-bearing premise

The method's benefit depends on the teacher model accurately judging which actions are wrong and supplying correct, stylistically consistent reflections and corrections; if the teacher errs often, the masked data still contains misleading correction pairs.

Editorial extensions

If this is right

  • The trained agent achieves average rewards of 0.618 on WebShop, 0.716/0.719 on ALFWorld seen/unseen, and 0.664/0.660 on SciWorld seen/unseen, outperforming the golden-trajectory baseline on every test set.
  • The gains are largest on unseen splits (+10.3% on ALFWorld unseen, +8% on SciWorld unseen), indicating better generalization rather than simple memorization.
  • Both components are necessary: ablating partial masking drops average reward from 0.672 to 0.665, and ablating self-reflected trajectories drops it to 0.624.
  • STeP is not tied to one base model: Mistral-7B and LLaMA3-8B also show overall improvements when trained with self-reflected trajectories and partial masking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to apply the same reflection-and-mask recipe to non-text settings, such as robot policies or code repair, where a simulator or verifier can supply corrections and the erroneous segment can be masked.
  • The data-efficiency result suggests that exploration rollouts with corrective annotations may be a cheaper substitute for large expert corpora; one could test this by scaling the number of self-reflected trajectories and measuring the learning curve relative to golden data.
  • An open question the paper does not settle is whether the student becomes genuinely self-reflecting at inference time or simply learns shallow cues from the reflection tokens; this could be probed by ablating the reflection text during evaluation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The manuscript proposes STeP, a training method for small open-source LLM agents. Stage 1 fine-tunes a base agent on half of the golden expert trajectories (D1). Stage 2 runs the base agent on the remaining tasks while a larger teacher LLM, Qwen1.5-110B-Chat after screening, flags erroneous actions and supplies reflection and correction steps; trajectories that end with reward 1 and contain at most two flagged errors (one for WebShop) are kept as Dr (708 trajectories). Stage 3 retrains the original base model on D1 ∪ Dr, masking the loss on flagged erroneous thoughts and actions (partial masking). Experiments on ALFWorld, WebShop, and SciWorld report that LLaMA2-7B-chat + STeP outperforms LLaMA2-7B-chat + Golden Trajs by 9.2% average reward while using fewer training trajectories, and ablations report that both self-reflected trajectories and partial masking contribute to the gains.

Significance. The claimed finding, that small models can learn to self-reflect and self-correct from synthetically augmented trajectories more effectively and efficiently than from expert trajectories alone, is potentially valuable for open-source agent training. The paper has several strengths: it evaluates on three established benchmarks, uses automatic environment rewards, proposes a concrete partial-masking mechanism, and reports experiments with multiple base LLMs and teacher models. However, the central quantitative claim rests on a training-data comparison that is confounded by curation, on single-run numbers without variance reporting, and on a teacher-model selection procedure that may have used test-set feedback. These issues must be resolved before the empirical headline can be regarded as established.

major comments (5)
  1. [§4.2, §5.1, Table 2, Appendix B.3] The headline comparison is confounded by data curation. STeP is trained on D1 ∪ Dr, where Dr is filtered to trajectories with reward=1 and at most two erroneous steps (one for WebShop), while the Golden Trajs baseline is trained on the full expert set D. The reported 9.2% average gain and the 'fewer trajectories' claim in Figure 4 could therefore be produced by the filtering/curation step rather than by the self-reflected content or by partial masking. Please add matched baselines: train Golden Trajs on D1 plus an equal-size data-matched subset, and train STeP on Dr without the error-count filtering, or report results separately for the trajectories that were filtered out.
  2. [§5.2, Table 3] The ablation labeled '−self-reflected trajectories' trains only on D1, so it does not control for the amount of training data, the task subset, or the filtering criterion. It therefore cannot distinguish the contribution of reflection/correction content from the contribution of having more, or more selectively curated, training data. Add a control trained on D1 plus a data-matched set of additional golden trajectories or of successful self-generated trajectories without reflection/correction insertions.
  3. [§4.2, §5.3, Figure 5] The teacher model is selected after testing several candidates, and Figure 5 reports the final average reward of the agent trained on each teacher's self-reflected trajectories. If that selection was made using the test sets reported in Table 2, the main numbers partly reflect model selection on the evaluation split. Please state explicitly which data split was used for teacher selection; ideally, hold out a teacher-selection set or otherwise account for the selection process in the reported results.
  4. [§5.1, §5.2, Tables 2 and 3] All reported numbers appear to come from a single run per condition. Temperature 0 at inference does not remove run-to-run variation from SFT initialization, data ordering, or teacher sampling. The central claims include small differences such as 2.2% (SciWorld seen) and 1.8% (WebShop), and the partial-masking ablation goes in the opposite direction on ALFWorld seen (0.816 without masking vs. 0.716 with masking). Please provide means and standard deviations over at least three training seeds, or otherwise quantify the run-to-run variability, so that the reported differences can be assessed.
  5. [Limitations, §4.2] The Limitations section acknowledges that reflections and corrections of erroneous steps may not always be appropriate. This caveat interacts directly with the data filtering: if Dr is selected for success and for low teacher-error counts, the method's gain may consist largely of discarding exactly the trajectories in which the teacher is unreliable. The comparison with the unfiltered golden baseline then says little about the value of reflection per se. The matched-data controls requested in the first major comment are necessary to rule out this alternative explanation of the reported gains.
minor comments (6)
  1. [Figure 3, Section 1] The phrase 'Parital Masking' appears in Figure 3 and in the Section 1 introduction; it should read 'Partial Masking.'
  2. [Figure 4] The figure legend and caption are unclear: the caption lists three task names and two trajectory types but appears to show one curve, and it is not stated whether the plotted quantity is a task-wise average or a single-task result. Please clarify what 'Reducing 33%' and 'Up 10.3%' refer to.
  3. [Table 2] The column header 'Complate Rate' contains a typo, and the completion-rate column is not defined with respect to the max-step and max-context rules described in Appendix B.4. Please define the completion rate and report it per task.
  4. [Equation (3), §3.4] The definition of δ_i states that δ_i = 0 indicates an error step, but the loss equation uses δ_i as a coefficient without restating which tokens (erroneous thought, erroneous action, or both) are excluded. Please specify the masking convention for the self-reflected step tokens precisely.
  5. [Appendix B.3] The appendix says that self-reflected trajectories with a high number of erroneous steps are excluded, but it does not report how many trajectories per task were removed at each filtering threshold. Please provide the retention numbers for each teacher model and each threshold.
  6. [Table 5] The first data row contains the values '0.6180.7190.660' without separators, which is likely a formatting error; please ensure all table entries are visually separated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the claimed gains are direct empirical comparisons between trained models, not consequences of the method's definitions or of a self-citation chain.

full rationale

The paper's central claim is an empirical comparison: 'LLaMA2-7B-chat + STeP outperforms LLaMA2-7B-chat + Golden Trajs by an average of 9.2% on all tasks' (Section 5.1, Table 2). That number is a measured difference in environment reward between two independently trained models, not a quantity that the STeP loss defines or forces. No equation in Sections 3.3-3.4 makes the reported test reward equal to the training objective by construction, and no parameter is fitted to the test labels. The self-reflected trajectories are generated by an external teacher model (Qwen1.5-110B-Chat) and then converted into SFT data; the method does not define the eval metric in terms of that data. The only potentially self-referential citation is the Qwen technical report, which lists one of this paper's co-authors, but the citation merely identifies the teacher model used; the model's outputs are externally generated and the paper's results do not rest on an unverified theorem from that report. The paper's own Limitations statement that 'reflections and corrections of erroneous steps in self-reflected trajectories may not always be appropriate' is a quality caveat, not evidence of circularity. Two experimental-design concerns are real but distinct from circularity: the teacher model was selected after testing multiple teachers, and the Dr set is filtered to successful trajectories with few erroneous steps (Appendix B.3), so the STeP-versus-Golden comparison is partly confounded by data curation and teacher selection. Those concerns affect internal validity and overfitting risk, but they do not make the reported result equivalent to its inputs by definition. Accordingly, no circular step is present.

Assumptions & free parameters 3 free parameters · 2 assumptions · 0 invented entities

The method's empirical gains depend on several hand-chosen thresholds and the selected teacher model. None of these are fitted parameters in a derivation, but they introduce degrees of freedom that could inflate results if not carefully controlled.

free parameters (3)
  • Filtering thresholds for erroneous steps = ALFWorld/SciWorld: max 2; WebShop: max 1
    Hand-set in Section B.3. Trajectories with more error steps are excluded, which affects the composition of the self-reflected training set and could bias results.
  • Data split ratio for D1 = 50% of golden trajectories
    Randomly split in Section 4.1. The base agent is trained on D1; results may depend on the specific split even though it is random.
  • Teacher model choice = Qwen1.5-110B-Chat
    Selected after testing several open-source models (Section 5.2, Figure 5). If selection was based on test-set performance, it overfits the reported averages.
assumptions (2)
  • domain assumption The teacher model's real-time judgments of action correctness are accurate enough to generate useful training signal.
    Section 3.3 assumes the teacher can identify errors and produce corrections. The Limitations section admits this may not always be true.
  • domain assumption Masking error steps during SFT prevents the model from learning incorrect actions while still allowing it to learn reflection and correction from surrounding context.
    Section 3.4 introduces this assumption. The ablation in Section 5.2 partially supports it, but it remains an unproven modeling choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Masking." pith.science (2026). https://pith.science/paper/PYZTE7IJ

@misc{pith2026250520023,
  author       = {Pith},
  title        = {Pith review of: Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Masking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PYZTE7IJ}},
  note         = {Machine review of arXiv:2505.20023}
}
read the original abstract

Autonomous agents, which perceive environments and take actions to achieve goals, have become increasingly feasible with the advancements in large language models (LLMs). However, current powerful agents often depend on sophisticated prompt engineering combined with closed-source LLMs like GPT-4. Although training open-source LLMs using expert trajectories from teacher models has yielded some improvements in agent capabilities, this approach still faces limitations such as performance plateauing and error propagation. To mitigate these challenges, we propose STeP, a novel method for improving LLM-based agent training. We synthesize self-reflected trajectories that include reflections and corrections of error steps, which enhance the effectiveness of LLM agents in learning from teacher models, enabling them to become agents capable of self-reflecting and correcting. We also introduce partial masking strategy that prevents the LLM from internalizing incorrect or suboptimal steps. Experiments demonstrate that our method improves agent performance across three representative tasks: ALFWorld, WebShop, and SciWorld. For the open-source model LLaMA2-7B-Chat, when trained using self-reflected trajectories constructed with Qwen1.5-110B-Chat as the teacher model, it achieves comprehensive improvements with less training data compared to agents trained exclusively on expert trajectories.

Figures

Figures reproduced from arXiv: 2505.20023 by the authors.

Figure 1
Figure 1. A self-reflected agent could autonomously [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Self-Reflected Trajectories on WebShop. hancing generalization. Second, LLM agents are prone to cascading errors: a single mistake can propagate further failures, trapping the agent in er￾ror loops (Qin et al., 2024). Moreover, it remains unclear how to design an agent with self-reflective capabilities akin to proprietary models like GPT-4. Inspired by this, we propose STeP (Self￾Reflected Trajectories and Parital M… view at source ↗
Figure 3
Figure 3. STeP utilizes golden trajectories and corresponding instructions to train a Self-reflected LLM-based agent [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Compared to golden only, self-reflected trajec [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: The number of Self-Reflected Trajectories [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The prompt of ALFWorld that contains task requirements. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: The prompt of SciWorld that contains task requirements. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: The prompt of WebShop that contains task requirements. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: The prompt for teacher models to reflect and correct in real-time. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 14 canonical work pages

  1. [1]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    arXiv 2023

  2. [2]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  3. [3]

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. 2023. https://arxiv.org/abs/2310.05915 Fireact: Toward language agent fine-tuning . Preprint, arXiv:2310.05915

    arXiv 2023

  4. [4]

    STAN FRANKLIN. 1997. https://doi.org/10.1080/019697297126029 Autonomous agents as embodied ai . Cybernetics and Systems, 28(6):499--520

    work page doi:10.1080/019697297126029 1997

  5. [5]

    Priyanshu Gupta, Shashank Kirtania, Ananya Singha, Sumit Gulwani, Arjun Radhakrishna, Sherry Shi, and Gustavo Soares. 2024. https://arxiv.org/abs/2405.13009 Metareflection: Learning instructions for language agents using past reflections . Preprint, arXiv:2405.13009

    work page Pith review arXiv 2024

  6. [6]

    Izzeddin Gur, Hiroki Furuta, Austin V Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. 2024. https://openreview.net/forum?id=9JQtrumvg8 A real-world webagent with planning, long context understanding, and program synthesis . In The Twelfth International Conference on Learning Representations

    work page 2024

  7. [7]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. https://arxiv.org/abs/2310.0...

    arXiv 2023

  8. [8]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles

    2023

Show all 46 references
  1. [9]

    Bill Yuchen Lin, Yicheng Fu, Karina Yang, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Prithviraj Ammanabrolu, Yejin Choi, and Xiang Ren. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/4b0eea69deea512c9e2c469187643dc2-Paper-Conference.pdf Swiftsage: A gen...

    2023

  2. [10]

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. 2024. https...

    2024

  3. [11]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. https://openrev...

    2023

  4. [12]

    Meta. 2024. Introducing meta llama 3: The most capable openly available llm to date. https://ai.meta.com/blog/meta-llama-3/

    2024

  5. [13]

    Gr \'e goire Mialon, Cl \'e mentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. 2024. https://openreview.net/forum?id=fibxvahvs3 GAIA : a benchmark for general AI assistants . In The Twelfth International Conference on Learning Representations

    2024

  6. [14]

    OpenAI. 2024. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2024 arXiv

  7. [15]

    OpenDevin Team . 2024. OpenDevin: An Open Platform for AI Software Developers as Generalist Agents . https://github.com/OpenDevin/OpenDevin

    2024

  8. [16]

    Patil, Tianjun Zhang, Xin Wang, and Joseph E

    Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. https://arxiv.org/abs/2305.15334 Gorilla: Large language model connected with massive apis . Preprint, arXiv:2305.15334

    2023 arXiv

  9. [17]

    Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, and Jianfeng Gao. 2023. https://arxiv.org/abs/2302.12813 Check your facts and try again: Improving large language models with external knowledge and automate...

    2023 arXiv

  10. [18]

    Pomerleau

    Dean A. Pomerleau. 1991. https://doi.org/10.1162/neco.1991.3.1.88 Efficient Training of Artificial Neural Networks for Autonomous Navigation . Neural Computation, 3(1):88--97

    1991 doi

  11. [19]

    Cheng Qian, Shihao Liang, Yujia Qin, Yining Ye, Xin Cong, Yankai Lin, Yesai Wu, Zhiyuan Liu, and Maosong Sun. 2024. https://arxiv.org/abs/2401.13996 Investigate-consolidate-exploit: A general strategy for inter-task agent self-evolution . Preprint, arXiv:2401.13996

    2024 arXiv

  12. [20]

    Shuofei Qiao, Runnan Fang, Ningyu Zhang, Yuqi Zhu, Xiang Chen, Shumin Deng, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2024 a . https://arxiv.org/abs/2405.14205 Agent planning with world knowledge model . Preprint, arXiv:2405.14205

    2024 arXiv

  13. [21]

    Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, chengfei lv, and Huajun Chen. 2024 b . https://openreview.net/forum?id=StWjRTl8L1 Autoact: Automatic agent learning from scratch via self-planning . In ICLR 2024 Workshop on Large Langu...

    2024

  14. [22]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, dahai li, Zhiyuan Liu, and Maosong Sun. 2024. https://openreview.net/forum?id=dHng2O0Jjr...

    2024

  15. [23]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. 2023. https://openreview.net/forum?id=vAElhFcKW6 Reflexion: language agents with verbal reinforcement learning . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  16. [24]

    Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2020

  17. [25]

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2021. https://arxiv.org/abs/2010.03768 Alfworld: Aligning text and embodied environments for interactive learning . Preprint, arXiv:2010.03768

    2021 arXiv

  18. [26]

    Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. 2024. https://arxiv.org/abs/2403.02502 Trial and error: Exploration-based trajectory optimization for llm agents . Preprint, arXiv:2403.02502

    2024 arXiv

  19. [27]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  20. [28]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. 2024 a . https://doi.org/10.1007/s11704-024-40231-1 A survey on large language model based autonomous agents . Fron...

    2024 doi

  21. [29]

    Renxi Wang, Haonan Li, Xudong Han, Yixuan Zhang, and Timothy Baldwin. 2024 b . https://arxiv.org/abs/2402.11651 Learning from failure: Integrating negative examples when fine-tuning large language models as agents . Preprint, arXiv:2402.11651

    2024 arXiv

  22. [30]

    Ruoyao Wang, Peter Jansen, Marc-Alexandre C \^o t \'e , and Prithviraj Ammanabrolu. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.775 S cience W orld: Is your agent smarter than a 5th grader? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Pr...

    2022 doi

  23. [31]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824--24837

    2022

  24. [32]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, W...

    2023 arXiv

  25. [33]

    Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Dingwen Yang, Chenyang Liao, Xin Guo, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, and Yu-Gang Jiang. 2024. https://arxiv.org/abs/2...

    2024 arXiv

  26. [34]

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024 a . https://openreview.net/forum?id=l5XQzNkAOe Travelplanner: A benchmark for real-world planning with language agents . In Forty-first International Conference on Machine Learning

    2024

  27. [35]

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. 2024 b . https://arxiv.org/abs/2404.07972 Oswo...

    2024 arXiv

  28. [36]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024 arXiv

  29. [37]

    Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024 b . https://arxiv.org/abs/2405.15793 Swe-agent: Agent-computer interfaces enable automated software engineering . Preprint, arXiv:2405.15793

    2024 arXiv

  30. [38]

    Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. 2024 c . https://arxiv.org/abs/2402.13669 Self-distillation bridges distribution gap in language model fine-tuning . Preprint, arXiv:2402.13669

    2024 arXiv

  31. [39]

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/82ad13ec01f9fe44c01cb91814fd7b8c-Paper-Conference.pdf Webshop: Towards scalable real-world web interaction with grounded language agents . In Advances i...

    2022

  32. [40]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. https://openreview.net/forum?id=WE_vluYUL-X React: Synergizing reasoning and acting in language models . In The Eleventh International Conference on Learning Representations

    2023

  33. [41]

    Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh R N, Zeyuan Chen, Jianguo Zhang, Devansh Arpit, Ran Xu, Phil L Mui, Huan Wang, Caiming Xiong, and Silvio Savarese. 2024. https://openreview.net/forum?id=KOZu91CzbK Retroformer: Retrospect...

    2024

  34. [42]

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. 2023. https://arxiv.org/abs/2310.12823 Agenttuning: Enabling generalized agent abilities for llms . Preprint, arXiv:2310.12823

    2023 arXiv

  35. [43]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. http://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

    2024 arXiv

  36. [44]

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. https://arxiv.org/abs/2311.07911 Instruction-following evaluation for large language models . Preprint, arXiv:2311.07911

    2023 arXiv

  37. [45]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.