Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that pure reinforcement learning with a binary task-completion reward turns a 7B language model into a strong embodied planner, reaching 97.78% on ALFWorld and 79.92% on ScienceWorld without expert demonstrations.

desk verdict Outcome-only RL for embodied planning is a real contribution; the numbers are plausible but need error bars and a GRPO ablation before I'd trust the margins. read the letter →

arxiv 2506.23127 v1 pith:DNBDDENJ submitted 2025-06-29 cs.CL cs.AI

Zhaoye Fei , Li Ji , Siyin Wang , Junhao Shi , Jingjing Gong , Xipeng Qiu This is my paper
classification cs.CLcs.AI
keywords embodiedtaskplanningreinforcementlearningLLMagentssparserewardgrouprolloutinteractivepolicyoptimizationALFWorldScienceWorld
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that interactive planning ability in a language model can be induced by outcome-driven reinforcement learning alone, with no human demonstrations and no dense reward shaping. The agent, a 7B instruction-tuned model, learns by rolling out multiple trajectories in parallel inside text-based embodied environments, receiving a reward of 1 only when the task is completed, and updating its policy with a group-normalized advantage estimator the paper calls Interactive Policy Optimization (IPO). On ALFWorld the trained model completes 97.78% of tasks on average, and on ScienceWorld 79.92%, well above supervision-based baselines; the drop on unseen environments is only about three percentage points. If correct, this shows that the bottleneck in LLM planning is not missing expert knowledge but the absence of direct interaction data, and that binary task-completion feedback suffices to generate such data.

What carries the argument

The load-bearing mechanism is the group-relative policy update, Interactive Policy Optimization (IPO), applied to ReAct-style trajectories: $n$ parallel rollouts per task share a mean and standard deviation, and each trajectory's binary completion reward is normalized into an advantage (Eq. 8) that is attached to every token prefix of every step in that trajectory. The probability ratio in Eq. (6) is computed over the thought-plus-action prefix up to step $t$, and a clipped surrogate objective with a KL penalty prevents the policy from drifting from the reference. A parallel group-rollout sampler supplies diverse trajectories, and a completion-driven sparse reward (Eq. 5) is the only learning signal; the paper's stated purpose is to show that this minimal setup can absorb the credit-assignment demands of multi-turn interaction.

What would settle it

Run the same training with group size 1 (no comparison group) and with step-level credit for individual actions, holding compute constant; if the method's advantage vanishes or the step-level variant does not change performance, the paper's explanation of why IPO works is called into question. A concrete signature to look for: under group size 1, the completion rate on ALFWorld should drop substantially from 97.78% if group normalization is the operative mechanism.

Watch

Extended reading notes

Core claim

The central discovery claimed is that a policy gradient method over grouped, in-environment trajectories can turn a 7B LLM into an embodied planner without any expert trajectories. The reward is $1$ if the final state satisfies the instruction and $0$ otherwise; the advantage of trajectory $i$ is $(r_i - \mu_r)/\sigma_r$, assigned identically to every step, and the policy objective clips probability ratios relative to a frozen reference policy with a KL penalty. The paper reports that this training raises Qwen2.5-7B-Instruct from roughly 31% to 97.78% average completion on ALFWorld and from roughly 22% to 79.92% on ScienceWorld, beating SFT, DPO-style ETO, and negative-trajectory training (NAT) baselines that rely on human-curated data, while keeping a generalization gap of about three percentage points (stated as -3.33% in the main text and -3.66% in the abstract) on unseen scenes.

Load-bearing premise

The load-bearing premise is that awarding every step of a finished task the same group-comparison score correctly strengthens the steps that really mattered, even though the reward is a simple pass/fail and most steps in a long successful run did not individually cause the success.

Editorial extensions

If this is right

  • The reported gains imply that outcome-only reinforcement can replace expert demonstrations in text-based embodied planning, removing the human-annotation bottleneck for similar tasks.
  • A single frozen reward rule (task done or not) may suffice to train agents that adapt to unseen rooms, since the model learns environmental regularities during exploration rather than imitating static data.
  • The reduction in invalid actions and response length over training suggests that the policy learns to be more concise and environment-aware, not merely to repeat successful templates.
  • If the result generalizes, the same recipe can be applied to other domains where success is automatically checkable, such as web navigation, tool use, and game-playing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run: replace the group-normalized advantage with a per-step or dense reward to see whether performance rises; the authors' account implies the binary group signal is sufficient, which is a stronger claim than typical credit-assignment results.
  • The method implicitly treats every step of a successful trajectory as equally creditworthy, so the group size ($n=5$) and batch composition may be doing hidden work; with a group of identical tasks, a single lucky trajectory could dominate the advantage.
  • The generalization results hint that the model may be learning implicit environment knowledge, such as where objects tend to appear (the paper's CD-location case study), which would connect this training scheme to implicit model-based learning rather than pure behavioral cloning.
  • A natural next step would be transferring the trained policy across benchmarks to see whether the learned behavior is a reusable planning skill or task-specific environment knowledge.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Embodied Planner-R1, a reinforcement learning framework for LLM-based embodied task planning in text environments. The method combines group rollout with in-environment interaction, a sparse binary completion reward, and a proposed Interactive Policy Optimization (IPO) update (Eqs. 6-8) that assigns group-normalized trajectory-level advantages to every step. Starting from Qwen2.5-7B-Instruct, the authors report average completion rates of 97.78% on ALFWorld and 79.92% on ScienceWorld, surpassing SFT/DPO-style baselines and prompt-based large models. They also report a generalization gap of -3.33% (or -3.66% in the abstract) on unseen environments and analyze training dynamics such as reward, response length, and invalid action reduction.

Significance. If the reported results are reproducible, the paper makes a meaningful empirical contribution: outcome-only RL with a sparse completion reward can elicit interactive planning behavior in a 7B language model without expert demonstrations. The design is clean, the two benchmarks are standard, and the authors commit to releasing code and models. However, the current evidence is under-supported: the evaluation is single-run, the IPO estimator is not isolated or analyzed, and key training hyperparameters are missing. The contribution is therefore significant but, at this stage, not fully demonstrated.

major comments (4)
  1. [§3.5, Eq. (8)] The central novelty of IPO is the advantage estimator in Eq. (8), where every step t of trajectory i receives the same normalized trajectory reward Â_{i,t} = (r_i - μ_r)/σ_r. Under a binary completion reward and group size n=5, this is a high-variance REINFORCE-style estimator: all steps in a successful trajectory are reinforced equally, including steps that did not cause success, and all steps in a failed trajectory are penalized equally. The paper provides no bias or variance analysis, no experimental comparison against vanilla GRPO or step-level/dense rewards, and no sensitivity study of the group size. Because this estimator is load-bearing for the method, I ask for repeated runs with different seeds and n ∈ {1,3,5,10}, and an ablation replacing Eq. (8) with per-step or token-level advantages, before the central claim is accepted.
  2. [§4.1 and Tables 1-2] The paper states that each task was evaluated only once, and no error bars, seeds, or confidence intervals are reported. Single-run evaluations on a finite set of task instances are noisy, and the observed generalization gaps in Table 2 (ranging from -12.50 to +4.38) could change materially with another seed. Please report the number of task instances per split and run the full evaluation with at least three seeds, reporting means and standard deviations or bootstrap confidence intervals. The inconsistency between the abstract's -3.66% generalization drop and the -3.33% reported in Section 4.3 and Table 2 further suggests that the aggregate statistics need verification.
  3. [§4.1, Training Details] The training setup is under-specified. The paper gives the batch size (128 tasks), group size (n=5), max steps (30), context length (4096), and hardware, but omits the learning rate and schedule, optimizer, KL coefficient β in Eq. (7), clipping epsilon ε, number of epochs per group update, and total training steps or convergence criterion. Without these values the method cannot be reproduced, and the sensitivity of the IPO update to its two main hyperparameters (β and ε) cannot be assessed. Please provide a complete hyperparameter table and an outline of the training budget.
  4. [§3.3 and Appendix C] The claim that the method uses 'minimal supervision' or 'pure RL' is qualified by the hand-crafted action-constrained prompts in Appendix C, which enumerate the full action space, restrict the output format to Thought/Action, warn against common hallucinations, and add a special 'done' action. Since the training-based baselines are said to use a consistent prompt set, this may not affect the direct comparison, but it does affect the interpretation of what is learned. An ablation that removes or weakens the prompt constraints, or that compares with a generic ReAct prompt, would clarify whether the reported gains come from the IPO/RL mechanism or from the prompt prior.
minor comments (5)
  1. [Abstract vs. Section 4.3/Table 2] Please reconcile the generalization-drop values: the abstract reports -3.66% while Section 4.3 and Table 2 report -3.33%.
  2. [Table 1 and Section 4.2] Table 1 labels several rows 'Referenced Results (Not Directly Comparable)', but Section 4.2 still highlights percentage improvements over those rows; please clarify how the comparison should be interpreted or restrict the quantitative claims to the 'Our Implementation' rows.
  3. [Section 4.2] The phrase 'Compares with other baseline' should read 'Compared with other baselines'.
  4. [Appendix B] The seen/unseen definitions are given nearly identically for ALFWorld and ScienceWorld; please confirm that the ScienceWorld split is actually defined in this way, since the original benchmark uses a different evaluation protocol.
  5. [Figure 6] The caption mentions 'semi-transparent regions' around the curves, but no legend or explanation indicates that these are variance bands; please annotate the figure.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the headline results come from external interactive benchmarks; only minor non-load-bearing self-citations and an internal numeric inconsistency are present.

full rationale

The paper's claim chain is an empirical RL pipeline, not a derivation whose output is fixed by its inputs. The policy is trained with the IPO objective (Eq. 7), using per-step probability ratios (Eq. 6) and a group-normalized advantage (Eq. 8); these are standard policy-gradient/GRPO-style estimators, and no parameter is fitted to the test sets. The reported 97.78% ALFWorld and 79.92% ScienceWorld completion rates are produced by the external environments' own task-completion checks on seen/unseen splits, so the headline numbers have independent empirical content. The self-citations are contextual: Wang et al. 2025b appears in a related-work enumeration of negative-example methods, and the InternLM technical report is cited only as a baseline; neither justifies a design choice, forbids alternatives, or supplies a uniqueness theorem, and no ansatz is smuggled in via citation. The abstract's -3.66% generalization drop versus -3.33% in Table 2 is an internal inconsistency, and the step-uniform credit assignment of Eq. (8) is not analyzed for bias, variance, or group-size sensitivity; these are robustness/verification concerns, not circularity. Under the external-benchmark exemption, the derivation is self-contained, so the circularity score is low.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard POMDP modeling and RL assumptions rather than on fitted physical parameters. The main hand-chosen quantities are RL hyperparameters, several of which are not reported in the paper.

free parameters (6)
  • group_size_n = 5
    Number of parallel trajectories per task in group rollout; chosen by hand, affects advantage normalization stability.
  • max_steps = 30
    Maximum interaction steps per trajectory; hand-chosen, directly caps achievable completion rate.
  • kl_penalty_beta = not reported
    KL regularization coefficient in Eq. (7); mentioned but value never given.
  • clipping_epsilon = not reported
    PPO/GRPO-style clip range in Eq. (7); value not reported.
  • learning_rate = not reported
    Optimizer learning rate for policy updates; not specified.
  • context_length = 4096
    Maximum context length during training; hand-chosen, affects long trajectories.
assumptions (5)
  • domain assumption Tasks are modeled as a Partially Observable Markov Decision Process (POMDP).
    Standard formalization for text-based embodied planning; invoked in Section 2.
  • domain assumption The environment completion signal is a correct, non-hackable reward.
    The reward is binary and only given when the environment confirms task completion; this assumes no exploitation of the 'done' action or parser bugs.
  • domain assumption The base LLM can reliably produce parseable Thought/Action responses after RL.
    The method relies on parsing actions from free-form model outputs; the authors assume the system prompt and RL are sufficient to maintain the format.
  • domain assumption The group-normalized advantage with n=5 provides a usable learning signal.
    No bias or variance analysis is provided for the estimator in Eq. (8); the paper assumes five trajectories are enough for stable normalization.
  • domain assumption KL penalty to the reference model prevents policy collapse.
    Standard in GRPO-style methods; the paper does not ablate the penalty strength.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning." pith.science (2026). https://pith.science/paper/DNBDDENJ

@misc{pith2026250623127,
  author       = {Pith},
  title        = {Pith review of: Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DNBDDENJ}},
  note         = {Machine review of arXiv:2506.23127}
}
read the original abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet they face significant challenges in embodied task planning scenarios that require continuous environmental understanding and action generation. Existing approaches generate open-loop action scripts based on static knowledge, making it difficult to learn causal relationships between actions and environmental feedback, particularly in partially observable environments. We introduce Embodied Planner-R1, a novel outcome-driven reinforcement learning framework that enables LLMs to develop interactive capabilities through autonomous exploration with minimal supervision. Our framework incorporates three key innovations: (1) Without human annotations, we employ pure reinforcement learning with group rollout, incorporating in-environment interaction through parallel exploration; (2) completion-driven sparse reward; and (3) Interactive Policy Optimization (IPO) for efficient learning from grouped trajectories. Across two challenging text-based Embodied planning benchmarks, Embodied Planner-R1 achieves impressive completion rates of 97.78% on ALFWorld and 79.92% on ScienceWorld, surpassing prior methods by a large margin, and suffers only a -3.66% drop in previously unseen environments, evidencing strong generalization.

Figures

Figures reproduced from arXiv: 2506.23127 by the authors.

Figure 1
Figure 1. The overview of Embodied Planner-R1, which encourages agents to explore environments autonomously, training with binary rewards based on task completion status. Large Language Models (LLMs) have demon￾strated remarkable capabilities across a wide range of tasks, from open-domain dialogue to complex reasoning (Brown et al., 2020; OpenAI, 2023; DeepSeek-AI et al., 2025; Shao et al., 2024; Team et al., 2024; Yang et al… view at source ↗
Figure 2
Figure 2. The Embodied Planner-R1 consists of three tightly integrated core components: 1) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. We employ the ReAct paradigm during generation and ex￾tract the action from the response mt . As described above, at time step t, the agent gen￾erates an action based on a trajectory that includes environmental observations and previous actions. However, mapping directly from such a trajectory to action demands complex reasoning capabilities, like task goal decomposition, common sense knowledge application, and extr… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Completion rates of Qwen2.5-7B-Instruct on various ALFWorld tasks, covering [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Training progress of Embodied Planner-R1 in [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 7
Figure 7. Figure 7: Prompt for ALFWorld C Prompt Design In our research, prompt engineering accomplishes three interconnected objectives: (i) enu￾merating permissible action, guiding the model to execute effective commands; (ii) miti￾gating common hallucinations, preventing the model from…
Figure 8
Figure 8. Figure 8: Prompt for ScienceWorld loops; and (iii) enforcing a concise ReAct paradigm that clearly demonstrates reasoning chains and actions, facilitating straightforward extraction of selected operations. Follow￾ing (Qiao et al., 2024), we enhanced the system instructions to ex…
Figure 9
Figure 9. Figure 9: Completion rates of Qwen2.5-7B-Instruct on various ScienceWorld tasks, covering [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Training progress of Embodied Planner-R1 in ALFWorld: (a) shows the rewards [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Comparison between Embodied Planner-R1 and SFT using the original task [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Comparison between Embodied Planner-R1 and SFT when asked about the [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning

    cs.AI 2025-10 conditional novelty 5.0 of 10

    A 3B VLM trained with SFT plus GRPO and an LCS-based reward reaches 55.3% on EmbodiedBench's EB-ALFRED, beating GPT-4o-mini and the 7B REBP planner.

Reference graph

Works this paper leans on

64 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    2020

  2. [2]

    Fireact: Toward language agent fine-tuning

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. Fireact: Toward language agent fine-tuning. CoRR, abs/2310.05915, 2023 a . doi:10.48550/ARXIV.2310.05915. URL https://doi.org/10.48550/arXiv.2310.05915

    arXiv doi:10.48550/arxiv.2310.05915 2023

  3. [3]

    Robogpt: an intelligent agent of making embodied long-term decisions for daily instruction tasks

    Yaran Chen, Wenbo Cui, Yuanwen Chen, Mining Tan, Xinyao Zhang, Dongbin Zhao, and He Wang. Robogpt: an intelligent agent of making embodied long-term decisions for daily instruction tasks. CoRR, abs/2311.15649, 2023 b . doi:10.48550/ARXIV.2311.15649. URL https://doi.org/10.48550/arXiv.2311.15649

    arXiv doi:10.48550/arxiv.2311.15649 2023

  4. [4]

    Process reward models for LLM agents: Practical framework and directions

    Sanjiban Choudhury. Process reward models for LLM agents: Practical framework and directions. CoRR, abs/2502.10325, 2025. doi:10.48550/ARXIV.2502.10325. URL https://doi.org/10.48550/arXiv.2502.10325

    arXiv doi:10.48550/arxiv.2502.10325 2025

  5. [5]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek - AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Hu...

    arXiv doi:10.48550/arxiv.2412.19437 2024

  6. [7]

    A survey of embodied AI: from simulators to research tasks

    Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied AI: from simulators to research tasks. IEEE Trans. Emerg. Top. Comput. Intell. , 6 0 (2): 0 230--244, 2022. doi:10.1109/TETCI.2022.3141105. URL https://doi.org/10.1109/TETCI.2022.3141105

    work page arXiv 2022

  7. [8]

    James J. Gibson. The Ecological Approach to Visual Perception: Classic Edition. Houghton Mifflin, 1979

    work page 1979

  8. [9]

    Chawla, Olaf Wiest, and Xiangliang Zhang

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI 2024, Jeju, South Korea, August 3-9, 2024 , pages 8048--8057. ...

    work page 2024

Show all 64 references
  1. [10]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung - Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. CoRR, abs/2503.24290, 2025. doi:10.48550/ARXIV.2503.24290. URL https://doi.org/10.48550/a...

    2025 doi arXiv

  2. [11]

    Controlling large language model with latent actions

    Chengxing Jia, Ziniu Li, Pengyuan Wang, Yi-Chen Li, Zhenyu Hou, Yuxiao Dong, and Yang Yu. Controlling large language model with latent actions. ArXiv, abs/2503.21383, 2025. URL https://api.semanticscholar.org/CorpusID:277349466

    2025 arXiv

  3. [12]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. CoRR, abs/2503.09516, 2025. doi:10.48550/ARXIV.2503.09516. URL https://doi.org/10.48550/arXiv.2503.09516

    2025 doi arXiv

  4. [13]

    Search-o1: Agentic search-enhanced large reasoning models

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic search-enhanced large reasoning models. CoRR, abs/2501.05366, 2025 a . doi:10.48550/ARXIV.2501.05366. URL https://doi.org/10.48550/arXiv.2501.05366

    2025 doi arXiv

  5. [14]

    From system 1 to system 2: A survey of reasoning large language models

    Zhong - Zhi Li, Duzhen Zhang, Ming - Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei - Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhijiang Guo, Le Song, and Cheng - Lin Liu. From system 1 to system 2: A survey of reasoning lar...

    2025 doi arXiv

  6. [15]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-1...

    2024

  7. [16]

    Qlass: Boosting language agent inference via q-guided stepwise search

    Zongyu Lin, Yao Tang, Xingcheng Yao, Da Yin, Ziniu Hu, Yizhou Sun, and Kai-Wei Chang. Qlass: Boosting language agent inference via q-guided stepwise search. ArXiv, abs/2502.02584, 2025. URL https://api.semanticscholar.org/CorpusID:276106886

    2025 arXiv

  8. [17]

    Aligning cyber space with physical world: A comprehensive survey on embodied AI

    Yang Liu, Weixing Chen, Yongjie Bai, Guanbin Li, Wen Gao, and Liang Lin. Aligning cyber space with physical world: A comprehensive survey on embodied AI . CoRR, abs/2407.06886, 2024. doi:10.48550/ARXIV.2407.06886. URL https://doi.org/10.48550/arXiv.2407.06886

    2024 doi arXiv

  9. [18]

    Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement

    Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. CoRR, abs/2503.06520, 2025. doi:10.48550/ARXIV.2503.06520. URL https://doi.org/10.48550/arXiv.2503.06520

    2025 doi arXiv

  10. [19]

    Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. CoRR, abs/2308.09583, 2023. doi:10.48550/AR...

    2023 doi arXiv

  11. [20]

    Exploring the limit of outcome reward for learning mathematical reasoning

    Chengqi Lyu, Songyang Gao, Yuzhe Gu, Wenwei Zhang, Jianfei Gao, Kuikun Liu, Ziyi Wang, Shuaibin Li, Qian Zhao, Haian Huang, Weihan Cao, Jiangning Liu, Hongwei Liu, Junnan Liu, Songyang Zhang, Dahua Lin, and Kai Chen. Exploring the limit of outcome reward for learning mathemati...

    2025 doi arXiv

  12. [21]

    FILM: following instructions in language with modular methods

    So Yeon Min, Devendra Singh Chaplot, Pradeep Kumar Ravikumar, Yonatan Bisk, and Ruslan Salakhutdinov. FILM: following instructions in language with modular methods. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 ....

    2022

  13. [22]

    Skill set optimization: Reinforcing language model behavior via transferable skills

    Kolby Nottingham, Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Sameer Singh, Peter Clark, and Roy Fox. Skill set optimization: Reinforcing language model behavior via transferable skills. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Aus...

    2024

  14. [23]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. CoRR, abs/2303.08774, 2023. doi:10.48550/ARXIV.2303.08774. URL https://doi.org/10.48550/arXiv.2303.08774

    2023 doi arXiv

  15. [24]

    Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning

    Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. CoRR, abs/2502.19634, 2025. doi:10.48550/...

    2025 doi arXiv

  16. [25]

    Agent Q: advanced reasoning and learning for autonomous AI agents

    Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent Q: advanced reasoning and learning for autonomous AI agents. CoRR, abs/2408.07199, 2024. doi:10.48550/ARXIV.2408.07199. URL https://doi.org/10.48550/arXiv.2408.07199

    2024 doi arXiv

  17. [26]

    Agent planning with world knowledge model

    Shuofei Qiao, Runnan Fang, Ningyu Zhang, Yuqi Zhu, Xiang Chen, Shumin Deng, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Agent planning with world knowledge model. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Ch...

    2024

  18. [27]

    Tarr, William W

    Gabriel Sarch, Lawrence Jang, Michael J. Tarr, William W. Cohen, Kenneth Marino, and Katerina Fragkiadaki. VLM agents generate their own memories: Distilling experience into embodied programs of thought. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich ...

    2024

  19. [28]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024. doi:10.48550/ARXIV.2402.03300. URL https://doi.org/10.4...

    2024 doi arXiv

  20. [29]

    Reflexion: language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information...

    2023

  21. [30]

    Hausknecht

    Mohit Shridhar, Xingdi Yuan, Marc - Alexandre C \^ o t \' e , Yonatan Bisk, Adam Trischler, and Matthew J. Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Ev...

    2021

  22. [31]

    Agentbank: Towards generalized LLM agents via fine-tuning on 50000+ interaction trajectories

    Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. Agentbank: Towards generalized LLM agents via fine-tuning on 50000+ interaction trajectories. In Yaser Al - Onaizan, Mohit Bansal, and Yun - Nung Chen, editors, Findings o...

    2024

  23. [32]

    Trial and error: Exploration-based trajectory optimization for LLM agents

    Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. Trial and error: Exploration-based trajectory optimization for LLM agents. CoRR, abs/2403.02502, 2024 b . doi:10.48550/ARXIV.2403.02502. URL https://doi.org/10.48550/arXiv.2403.02502

    2024 doi arXiv

  24. [33]

    Adaplanner: Adaptive planning from feedback with language models

    Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, and Chao Zhang. Adaplanner: Adaptive planning from feedback with language models. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Sy...

    2023

  25. [34]

    A survey of reasoning with foundation models

    Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, Yue Wu, Wenhai Wang, Junsong Chen, Zhangyue Yin, Xiaozhe Ren, Jie Fu, Junxian He, Wu Yuan, Qi Liu, Xihui Liu, Yu Li, Hao Dong, Yu Cheng, Ming Zhan...

    2023 doi arXiv

  26. [35]

    Internlm2 technical report

    InternLM Team, Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Hu...

    2024 doi arXiv

  27. [36]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, ...

    2025 doi arXiv

  28. [37]

    Steca: Step-level trajectory calibration for llm agent learning

    Hanlin Wang, Jian Wang, Chak Tou Leong, and Wenjie Li. Steca: Step-level trajectory calibration for llm agent learning. ArXiv, abs/2502.14276, 2025 a . URL https://api.semanticscholar.org/CorpusID:276482279

    2025 arXiv

  29. [38]

    Offline reinforcement learning for LLM multi-step reasoning

    Huaijie Wang, Shibo Hao, Hanze Dong, Shenao Zhang, Yilin Bao, Ziran Yang, and Yi Wu. Offline reinforcement learning for LLM multi-step reasoning. CoRR, abs/2412.16145, 2024 a . doi:10.48550/ARXIV.2412.16145. URL https://doi.org/10.48550/arXiv.2412.16145

    2024 doi arXiv

  30. [39]

    Learning from failure: Integrating negative examples when fine-tuning large language models as agents

    Renxi Wang, Haonan Li, Xudong Han, Yixuan Zhang, and Timothy Baldwin. Learning from failure: Integrating negative examples when fine-tuning large language models as agents. CoRR, abs/2402.11651, 2024 b . doi:10.48550/ARXIV.2402.11651. URL https://doi.org/10.48550/arXiv.2402.11651

    2024 doi arXiv

  31. [40]

    Jansen, Marc - Alexandre C \^ o t \' e , and Prithviraj Ammanabrolu

    Ruoyao Wang, Peter A. Jansen, Marc - Alexandre C \^ o t \' e , and Prithviraj Ammanabrolu. Scienceworld: Is your agent smarter than a 5th grader? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Lan...

    2022 doi

  32. [41]

    World modeling makes a better planner: Dual preference optimization for embodied task planning

    Siyin Wang, Zhaoye Fei, Qinyuan Cheng, Shiduo Zhang, Panpan Cai, Jinlan Fu, and Xipeng Qiu. World modeling makes a better planner: Dual preference optimization for embodied task planning. CoRR, abs/2503.10480, 2025 b . doi:10.48550/ARXIV.2503.10480. URL https://doi.org/10.4855...

    2025 doi arXiv

  33. [42]

    Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents

    Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. CoRR, abs/2302.01560, 2023. doi:10.48550/ARXIV.2302.01560. URL https://doi.org/10.48550/ar...

    2023 doi arXiv

  34. [43]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, ...

    2022

  35. [44]

    Agentgym: Evolving large language model-based agents across diverse environments

    Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Dingwen Yang, Chenyang Liao, Xin Guo, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, and Yu - Gang Jiang. Agentgym: Evolving large la...

    2024 doi arXiv

  36. [45]

    The rise and potential of large language model based agents: a survey

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, W...

    2025 doi

  37. [46]

    Logic-rl: Unleashing LLM reasoning with rule-based reinforcement learning

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing LLM reasoning with rule-based reinforcement learning. CoRR, abs/2502.14768, 2025. doi:10.48550/ARXIV.2502.14768. URL https://doi.org/10.4...

    2025 doi arXiv

  38. [47]

    Watch every step! LLM agent learning via iterative step-level process refinement

    Weimin Xiong, Yifan Song, Xiutian Zhao, Wenhao Wu, Xun Wang, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. Watch every step! LLM agent learning via iterative step-level process refinement. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Con...

    2024 doi

  39. [48]

    Watch every step! LLM agent learning via iterative step-level process refinement

    Weimin Xiong, Yifan Song, Xiutian Zhao, Wenhao Wu, Xun Wang, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. Watch every step! LLM agent learning via iterative step-level process refinement. In Yaser Al - Onaizan, Mohit Bansal, and Yun - Nung Chen, editors, Proceedings of the 2024...

    2024

  40. [49]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 doi arXiv

  41. [50]

    Cops: Empowering LLM agents with provable cross-task experience sharing

    Chen Yang, Chenyang Zhao, Quanquan Gu, and Dongruo Zhou. Cops: Empowering LLM agents with provable cross-task experience sharing. CoRR, abs/2410.16670, 2024 b . doi:10.48550/ARXIV.2410.16670. URL https://doi.org/10.48550/arXiv.2410.16670

    2024 doi arXiv

  42. [51]

    Narasimhan, and Yuan Cao

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenRevie...

    2023

  43. [52]

    N., Zeyuan Chen, Jianguo Zhang, Devansh Arpit, Ran Xu, Phil Mui, Huan Wang, Caiming Xiong, and Silvio Savarese

    Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh R. N., Zeyuan Chen, Jianguo Zhang, Devansh Arpit, Ran Xu, Phil Mui, Huan Wang, Caiming Xiong, and Silvio Savarese. Retroformer: Retrospective large language agents with policy gradient op...

    2024

  44. [53]

    Agent lumos: Unified and modular training for open-source language agents

    Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Raghavi Chandu, Kai - Wei Chang, Yejin Choi, and Bill Yuchen Lin. Agent lumos: Unified and modular training for open-source language agents. In Lun - Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62...

    2024

  45. [54]

    Internlm-math: Open math large language models toward verifiable reasoning

    Huaiyuan Ying, Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, Yudong Wang, Zijian Wu, Shuaibin Li, Fengzhe Zhou, Hongwei Liu, Songyang Zhang, Wenwei Zhang, Hang Yan, Xipeng Qiu, Jiayu Wang, Kai Chen, and Dahua Lin...

    2024 doi arXiv

  46. [55]

    Agent-r: Training language model agents to reflect via iterative self-training

    Siyu Yuan, Zehui Chen, Zhiheng Xi, Junjie Ye, Zhengyin Du, and Jiecao Chen. Agent-r: Training language model agents to reflect via iterative self-training. CoRR, abs/2501.11425, 2025. doi:10.48550/ARXIV.2501.11425. URL https://doi.org/10.48550/arXiv.2501.11425

    2025 doi arXiv

  47. [57]

    Agenttuning: Enabling generalized agent abilities for llms

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agenttuning: Enabling generalized agent abilities for llms. In Lun - Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangk...

    2024 doi

  48. [58]

    Enhancing decision-making for LLM agents via step-level q-value models

    Yuanzhao Zhai, Tingkai Yang, Kele Xu, Dawei Feng, Cheng Yang, Bo Ding, and Huaimin Wang. Enhancing decision-making for LLM agents via step-level q-value models. In Toby Walsh, Julie Shah, and Zico Kolter, editors, AAAI-25, Sponsored by the Association for the Advancement of Ar...

    2025 doi

  49. [59]

    Large language models as commonsense knowledge for large-scale task planning

    Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as commonsense knowledge for large-scale task planning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annu...

    2023

  50. [60]

    Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025

    Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025. URL https://arxiv.org/abs/2504.03160

    2025 arXiv

  51. [61]

    Agents: An open-source framework for autonomous language agents

    Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. Agents: An open-source framework for autonomous l...

    2023 doi arXiv

  52. [62]

    K now A gent: Knowledge-augmented planning for LLM -based agents

    Yuqi Zhu, Shuofei Qiao, Yixin Ou, Shumin Deng, Shiwei Lyu, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen, and Ningyu Zhang. K now A gent: Knowledge-augmented planning for LLM -based agents. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Findings of the Association for Comp...

    2025

  53. [63]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  54. [64]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  55. [65]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  56. [66]

    K now A gent: Knowledge-Augmented Planning for LLM -Based Agents

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2025 doi arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.