pith. sign in

arxiv: 2606.27483 · v1 · pith:AL4EH5SDnew · submitted 2026-06-25 · 💻 cs.AI

Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning

Xuan Zhang , Zhijian Zhou , Lingfeng Qiao , Yulei Qin , Ke Li , Xing Sun , Xiaoyu Tan , Chao Qu
show 1 more author
Yuan Qi
This is my paper

Pith reviewed 2026-06-29 01:51 UTC · model grok-4.3

classification 💻 cs.AI
keywords LLM agentsworld model planningforesightagentic trainingreinforcement learningsequential decision makinginternal simulation
0
0 comments X

The pith

LLM agents gain genuine internal foresight for long-horizon planning only when trained through a three-stage capability-first pipeline rather than direct fine-tuning on look-ahead traces.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that current LLM agents stay reactive because they lack any internal mechanism to simulate future states before choosing actions. It argues that simply showing the model examples of future rollouts during fine-tuning produces only surface-level imitation without real predictive grounding. To close this gap the authors introduce a fixed three-stage sequence: first inject latent predictive ability through world-model mid-training, then structure the output format with supervised fine-tuning, and finally calibrate the simulations with foresight-conditioned reinforcement learning. The resulting single autoregressive model can both generate prospective state sequences and attach a plan-conditioned success estimate to each. Experiments on search and mathematical reasoning tasks show consistent gains over standard post-training baselines.

Core claim

A single autoregressive model can be trained to verbalize both a prospective state rollout and a plan-conditioned success estimate, but only when the training follows the specific three-stage order of World Model Agentic Mid-Training, Format-Eliciting SFT, and Foresight-Conditioned RL; direct fine-tuning on the same traces yields only superficial mimicry without grounded foresight.

What carries the argument

The three-stage training pipeline that first injects latent predictive capability, then structures its expression, and finally calibrates its utility through reinforcement.

If this is right

  • Agents produce explicit textual simulations of future states together with success estimates that function like Q-values.
  • Performance on search and mathematical reasoning tasks improves over direct fine-tuning and other post-training baselines.
  • The same autoregressive backbone can serve both as policy and as internal world model without separate modules.
  • The format-capability gap is closed only when predictive ability is built before format supervision.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same staged sequence could be tested on embodied control or multi-agent coordination tasks where reactive behavior is costly.
  • If the mid-training stage is the main source of grounding, removing it while keeping the later stages should collapse performance to baseline levels.
  • The approach suggests that other forms of internal simulation, such as counterfactual reasoning, may also require an initial capability-injection phase.

Load-bearing premise

Direct fine-tuning on look-ahead traces produces only superficial mimicry without any genuine predictive grounding, and the exact three-stage sequence is both necessary and sufficient to create real internal simulation ability.

What would settle it

A controlled experiment in which a single-stage fine-tuning run on identical look-ahead traces reaches the same level of rollout accuracy, calibration, and downstream planning performance as the three-stage pipeline.

Figures

Figures reproduced from arXiv: 2606.27483 by Chao Qu, Ke Li, Lingfeng Qiao, Xiaoyu Tan, Xing Sun, Xuan Zhang, Yuan Qi, Yulei Qin, Zhijian Zhou.

Figure 1
Figure 1. Figure 1: Overview of our key design of an agent policy with internalized world model. a reasoning agent, but also to verbalize both: (i) a compact prospective rollout of what may happen next, and (ii) a plan-conditioned estimate of success if the current plan is followed. This textual estimate is best understood as a verbalized analogue of Q(s, a) or local action value, represented in the same token space as the po… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of our three-stage training paradigm. trajectory ideally requires dozens of Monte Carlo rollouts, it is computationally prohibitive given the massive scale of mid-training data. Consequently, we adopt an LLM to heuristically estimate this success probability. Though this estimation may be not accurate enough, it is acceptable because these confidence priors are ultimately grounded and calibrated a… view at source ↗
Figure 3
Figure 3. Figure 3: The episode length of the RL-only setting during RL phase. D.3 Ablation studies on Rground in FC-RL We conduct ablation studies on Rground in FC-RL for the search task, and the results are presented in [PITH_FULL_IMAGE:figures/full_fig_p033_3.png] view at source ↗
read the original abstract

Large language model (LLM) agents have demonstrated strong capability in sequential decision-making, yet they remains fundamentally reactive in long-horizon tasks. Unlike humans who employ "what-if" reasoning to evaluate potential plans before commitment, standard agents lack an internal world model to simulate future outcomes. Therefore, we propose to internalize future-aware planning by training a single autoregressive model to verbalize both a prospective state rollout and a plan-conditioned success estimate-a textual analogue of the Q-value. Crucially, we identify a format-capability gap: simply fine-tuning agents on look-ahead traces during post-training leads to superficial mimicry of foresight without genuine predictive grounding. To bridge this gap, we introduce a three-stage training paradigm: (i) World Model Agentic Mid-Training (WM-AMT) to inject latent predictive capabilities into the policy; (ii) Format-Eliciting SFT (FE-SFT) to structure this injected capability; and (iii) Foresight-Conditioned Reinforcement Learning (FC-RL) to refine the calibration and utility of the generated simulations. Evaluated on search and mathematical reasoning tasks, our approach consistently outperforms other training baselines. Our results demonstrate that effective internal world modeling in LLM agents requires a capability-first training pipeline to achieve grounded and calibrated foresight.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes a three-stage training paradigm for LLM agents to develop internal world models for planning: (1) World Model Agentic Mid-Training (WM-AMT) to inject predictive capabilities, (2) Format-Eliciting SFT (FE-SFT) to structure them, and (3) Foresight-Conditioned RL (FC-RL) to refine calibration. It claims this overcomes the format-capability gap where standard fine-tuning on look-ahead traces leads to superficial mimicry, and demonstrates consistent outperformance on search and mathematical reasoning tasks.

Significance. Should the empirical results hold under scrutiny, the work would contribute a practical training recipe for instilling grounded foresight in autoregressive agents, potentially shifting from reactive to proactive planning. The staged approach addresses a plausible gap in current post-training methods.

major comments (2)
  1. Abstract: The abstract asserts that the approach 'consistently outperforms other training baselines' on search and math tasks, yet provides no metrics, baselines, number of tasks, or statistical controls. This absence prevents evaluation of whether the data support the central claim that the three-stage pipeline is necessary to achieve genuine predictive grounding beyond mimicry.
  2. Abstract: The distinction between 'superficial mimicry of foresight' from standard fine-tuning and 'genuine predictive grounding' from the proposed pipeline is asserted but not operationalized with a concrete test or metric in the provided description, leaving the weakest assumption unaddressed.
minor comments (1)
  1. Abstract: Grammatical error: 'yet they remains fundamentally reactive' should be 'yet they remain fundamentally reactive'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed feedback. We agree that the abstract would benefit from greater specificity on empirical results and clearer reference to how the format-capability gap is evaluated. We have prepared revisions to the abstract and will incorporate them in the next version of the manuscript. Below we respond to each major comment.

read point-by-point responses

  1. Referee: Abstract: The abstract asserts that the approach 'consistently outperforms other training baselines' on search and math tasks, yet provides no metrics, baselines, number of tasks, or statistical controls. This absence prevents evaluation of whether the data support the central claim that the three-stage pipeline is necessary to achieve genuine predictive grounding beyond mimicry.

    Authors: We agree that the current abstract is too high-level. The full manuscript (Section 4 and associated tables) reports concrete metrics including average success rates on 50+ search tasks and 200 math problems, comparisons against standard SFT, RLHF, and chain-of-thought baselines, and statistical significance via paired t-tests. In the revised abstract we will add a sentence summarizing the key quantitative gains (e.g., +X% on search, +Y% on math) and the primary baselines while remaining within length limits. revision: yes

  2. Referee: Abstract: The distinction between 'superficial mimicry of foresight' from standard fine-tuning and 'genuine predictive grounding' from the proposed pipeline is asserted but not operationalized with a concrete test or metric in the provided description, leaving the weakest assumption unaddressed.

    Authors: The distinction is operationalized in the main text via three concrete metrics: (1) simulation accuracy on held-out state transitions, (2) calibration error between predicted success probability and observed outcome, and (3) downstream planning success when the model is forced to use its own simulations. Standard SFT improves format adherence but shows near-chance simulation accuracy and poor calibration; only the three-stage pipeline improves all three. We will insert a short clause in the abstract referencing this evaluation protocol. revision: partial

Circularity Check

0 steps flagged

No significant circularity; empirical recipe with independent experimental support

full rationale

The paper advances a three-stage empirical training pipeline (WM-AMT, FE-SFT, FC-RL) to address a claimed format-capability gap in LLM agents. No equations, derivations, or fitted parameters are presented that reduce by construction to the method's own inputs. The central claim—that standard fine-tuning yields only superficial mimicry while the proposed sequence produces grounded foresight—is evaluated via outperformance on search and mathematical reasoning tasks against baselines, rather than through self-definitional loops or load-bearing self-citations. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no information on free parameters, axioms, or invented entities can be extracted.

pith-pipeline@v0.9.1-grok · 5780 in / 1028 out tokens · 46552 ms · 2026-06-29T01:51:40.065869+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

143 extracted references · 31 canonical work pages · 13 internal anchors

  1. [1]

    A survey on large language model based autonomous agents.Frontiers of Computer Science, 18, 2023

    Lei Wang, Chengbang Ma, Xueyang Feng, Zeyu Zhang, Hao ran Yang, Jingsen Zhang, Zhi-Yang Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji rong Wen. A survey on large language model based autonomous agents.Frontiers of Computer Science, 18, 2023. URL https://api.semanticscholar.org/CorpusID:261064713

    2023

  2. [2]

    Griffiths

    Theodore Sumers, Shunyu Yao, Karthik R Narasimhan, and Thomas L. Griffiths. Cognitive architectures for language agents.Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https: //openreview.net/forum?id=1i6ZCvflQJ. Survey Certification, Featured Certification

    2024

  3. [3]

    The nature of explanation

    Kenneth Craik. The nature of explanation. 1944. URL https://api.semanticscholar.org/CorpusID: 41364251

    1944

  4. [4]

    Chain-of-thought prompting elicits reasoning in large lan- guage models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large lan- guage models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, ed- itors,Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran As...

    2022

  5. [5]

    Richard S. Sutton. Dyna, an integrated architecture for learning, planning, and reacting.SIGART Bull., 2(4):160–163, July 1991. ISSN 0163-5719. doi: 10.1145/122344.122377. URL https://doi.org/10.1145/ 122344.122377

    work page doi:10.1145/122344.122377 1991

  6. [6]

    Recurrent world models facilitate policy evolution.Advances in neural information processing systems, 31, 2018

    David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution.Advances in neural information processing systems, 31, 2018

    2018

  7. [7]

    Web agents with world models: Learning and leveraging environment dynamics in web navigation

    Hyungjoo Chae, Namyoung Kim, Kai Tzu iunn Ong, Minju Gwak, Gwanwoo Song, Jihoon Kim, Sunghwan Kim, Dongha Lee, and Jinyoung Yeo. Web agents with world models: Learning and leveraging environment dynamics in web navigation. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id=moWiYJuSGF. 11 World M...

    2025

  8. [8]

    Llms as scalable, general-purpose simulators for evolving digital agent training, 2025

    Yiming Wang, Da Yin, Yuedong Cui, Ruichen Zheng, Zhiqian Li, Zongyu Lin, Di Wu, Xueqing Wu, Chenchen Ye, Yu Zhou, and Kai-Wei Chang. Llms as scalable, general-purpose simulators for evolving digital agent training, 2025. URLhttps://arxiv.org/abs/2510.14969

    work page arXiv 2025

  9. [9]

    Is your llm secretly a world model of the internet? model- based planning for web agents.Transactions on Machine Learning Research, 2025

    Yu Gu, Kai Zhang, Yuting Ning, Boyuan Zheng, Boyu Gou, Tianci Xue, Cheng Chang, Sanjari Srivastava, Yanan Xie, Peng Qi, Huan Sun, and Yu Su. Is your llm secretly a world model of the internet? model- based planning for web agents.Transactions on Machine Learning Research, 2025

    2025

  10. [10]

    Dream to control: Learning behaviors by latent imagination

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. InInternational Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=S1lOTC4tDS

    2020

  11. [11]

    Mastering atari with discrete world models

    Danijar Hafner, Timothy P Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. InInternational Conference on Learning Representations, 2021. URL https: //openreview.net/forum?id=0oabwyZbOu

    2021

  12. [12]

    Mastering Diverse Domains through World Models

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models, 2024. URLhttps://arxiv.org/abs/2301.04104

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  13. [13]

    Internalizing world models via self-play finetuning for agentic rl,

    Shiqi Chen, Tongyao Zhu, Zian Wang, Jinghan Zhang, Kangrui Wang, Siyang Gao, Teng Xiao, Yee Whye Teh, Junxian He, and Manling Li. Internalizing world models via self-play finetuning for agentic rl,

  14. [14]

    URLhttps://arxiv.org/abs/2510.15047

    work page arXiv

  15. [15]

    Cwm: An open-weights llm for research on code generation with world models, 2025

    Meta FAIR CodeGen Team. Cwm: An open-weights llm for research on code generation with world models, 2025. URLhttps://ai.meta.com/research/publications/cwm/

    2025

  16. [16]

    Agent Learning via Early Experience

    Kai Zhang, Xiangchao Chen, Bo Liu, Tianci Xue, Zeyi Liao, Zhihan Liu, Xiyao Wang, Yuting Ning, Zhaorun Chen, Xiaohan Fu, Jian Xie, Yuxuan Sun, Boyu Gou, Qi Qi, Zihang Meng, Jianwei Yang, Ning Zhang, Xian Li, Ashish Shah, Dat Huynh, Hengduo Li, Zi Yang, Sara Cao, Lawrence Jang, Shuyan Zhou, Jiacheng Zhu, Huan Sun, Jason Weston, Yu Su, and Yifan Wu. Agent l...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  17. [17]

    Fang, J., Jiang, H., Wang, K., Ma, Y ., Shi, J., Wang, X., He, X., and Chua, T

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8154–8173, Singapore, December 2023. Association for Computational Li...

    work page doi:10.18653/v1/ 2023

  18. [18]

    arXiv preprint arXiv:2512.18832 , year=

    Yixia Li, Hongru Wang, Jiahao Qiu, Zhenfei Yin, Dongdong Zhang, Cheng Qian, Zeping Li, Pony Ma, Guanhua Chen, Heng Ji, and Mengdi Wang. From word to world: Can large language models be implicit text-based world models?, 2025. URLhttps://arxiv.org/abs/2512.18832

    work page arXiv 2025

  19. [19]

    A markovian decision process.Indiana University Mathematics Journal, page 679–684

    Richard Bellman. A markovian decision process.Indiana University Mathematics Journal, page 679–684. doi: 10.1512/iumj.1957.6.56038. URLhttp://dx.doi.org/10.1512/iumj.1957.6.56038

    work page doi:10.1512/iumj.1957.6.56038 1957

  20. [20]

    A path towards autonomous machine intelligence version 0.9

    Yann LeCun et al. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27.Open Review, 62(1):1–62, 2022

    2022

  21. [21]

    Mid-training of large language models: A survey, 2025

    Kaixiang Mo, Yuxin Shi, Weiwei Weng, Zhiqiang Zhou, Shuman Liu, Haibo Zhang, and Anxiang Zeng. Mid-training of large language models: A survey, 2025. URLhttps://arxiv.org/abs/2510.06826

    work page arXiv 2025

  22. [22]

    Scaling agents via continual pre-training, 2025

    Liangcai Su, Zhen Zhang, Guangyu Li, Zhuo Chen, Chenxi Wang, Maojia Song, Xinyu Wang, Kuan Li, Jialong Wu, Xuanzhong Chen, Zile Qiao, Zhongwang Zhang, Huifeng Yin, Shihao Cai, Runnan Fang, Zhengwei Tao, Wenbiao Yin, Chenxiong Qian, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. Scaling agents via continual pre-training, 2025. URLhttps://arxiv.org/a...

    work page arXiv 2025

  23. [23]

    Learning to reason as action abstractions with scalable mid-training RL

    Shenao Zhang, Donghan Yu, Yihao Feng, Bowen Jin, Zhaoran Wang, John Peebles, and Zirui Wang. Learning to reason as action abstractions with scalable mid-training RL. InThe Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview.net/forum?id=uWd9A1zp0Y. 12 World Model Agentic Training

    2026

  24. [24]

    Proximal Policy Optimization Algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URLhttps://arxiv.org/abs/1707.06347

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  25. [25]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y.K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

  26. [26]

    URLhttps://arxiv.org/abs/2402.03300

    work page internal anchor Pith review Pith/arXiv arXiv

  27. [27]

    DAPO: An Open-Source LLM Reinforcement Learning System at Scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  28. [28]

    Group-in-group policy optimization for LLM agent training

    Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-group policy optimization for LLM agent training. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=QXEhBMNrCW

    2025

  29. [29]

    Verification of forecasts expressed in terms of probability.Monthly weather review, 78(1):1–3, 1950

    W Brier Glenn et al. Verification of forecasts expressed in terms of probability.Monthly weather review, 78(1):1–3, 1950

    1950

  30. [30]

    Moerland, Joost Broekens, Aske Plaat, and Catholijn M

    Thomas M. Moerland, Joost Broekens, Aske Plaat, and Catholijn M. Jonker. Model-based reinforcement learning: A survey.Found. Trends Mach. Learn., 16(1):1–118, January 2023. ISSN 1935-8237. doi: 10.1561/2200000086. URLhttps://doi.org/10.1561/2200000086

    work page doi:10.1561/2200000086 2023

  31. [31]

    Embodied ai agents: Modeling the world,

    Pascale Fung, Yoram Bachrach, Asli Celikyilmaz, Kamalika Chaudhuri, Delong Chen, Willy Chung, Emmanuel Dupoux, Hongyu Gong, Hervé Jégou, Alessandro Lazaric, Arjun Majumdar, Andrea Madotto, Franziska Meier, Florian Metze, Louis-Philippe Morency, Théo Moutakanni, Juan Pino, Basile Terver, Joseph Tighe, Paden Tomasello, and Jitendra Malik. Embodied ai agents...

  32. [32]

    URLhttps://arxiv.org/abs/2506.22355

    work page arXiv

  33. [33]

    Webevolver: Enhancing web agent self-improvement with coevolving world model, 2025

    Tianqing Fang, Hongming Zhang, Zhisong Zhang, Kaixin Ma, Wenhao Yu, Haitao Mi, and Dong Yu. Webevolver: Enhancing web agent self-improvement with coevolving world model, 2025. URL https://arxiv.org/abs/2504.21024

    work page arXiv 2025

  34. [34]

    Explorer: Scaling exploration-driven web trajectory synthesis for multimodal web agents, 2025

    Vardaan Pahuja, Yadong Lu, Corby Rosset, Boyu Gou, Arindam Mitra, Spencer Whitehead, Yu Su, and Ahmed Awadallah. Explorer: Scaling exploration-driven web trajectory synthesis for multimodal web agents, 2025. URLhttps://arxiv.org/abs/2502.11357

    work page arXiv 2025

  35. [35]

    Mind2web: Towards a generalist agent for the web

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. InThirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://openreview.net/forum?id= kiYqbO3wqw

    2023

  36. [36]

    Agentsynth: Scalable task generation for generalist computer-use agents.arXiv preprint arXiv:2506.14205, 2025

    Jingxu Xie, Dylan Xu, Xuandong Zhao, and Dawn Song. Agentsynth: Scalable task generation for generalist computer-use agents.arXiv preprint arXiv:2506.14205, 2025

    work page arXiv 2025

  37. [37]

    Self-challenging language model agents.arXiv preprint arXiv:2506.01716, 2025

    Yifei Zhou, Sergey Levine, Jason Weston, Xian Li, and Sainbayar Sukhbaatar. Self-challenging language model agents, 2025. URLhttps://arxiv.org/abs/2506.01716

    work page arXiv 2025

  38. [38]

    An- droidinthewild: A large-scale dataset for android device control

    Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy P Lillicrap. An- droidinthewild: A large-scale dataset for android device control. InThirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://openreview.net/ forum?id=j4b3l5kOil

    2023

  39. [39]

    OpenCoder: The open cookbook for top-tier code large language models

    Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, and Zili Wang. OpenCoder: The open cookbook for ...

    work page doi:10.18653/v1/2025.acl-long.1591 2025

  40. [40]

    Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning

    Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, and Aviral Kumar. Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning. In A. Glober- son, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors,Advances in Neural Information Processing Systems, volume 37, pages 12461–12495. ...

  41. [41]

    URL https://proceedings.neurips.cc/paper_files/paper/ 2024/file/1704ddd0bb89f159dfe609b32c889995-Paper-Conference.pdf

    doi: 10.52202/079017-0397. URL https://proceedings.neurips.cc/paper_files/paper/ 2024/file/1704ddd0bb89f159dfe609b32c889995-Paper-Conference.pdf

    work page doi:10.52202/079017-0397 2024

  42. [42]

    Youtu-llm: Unlocking the native agentic potential for lightweight large language models, 2026

    Junru Lu, Jiarui Qin, Lingfeng Qiao, Yinghui Li, Xinyi Dai, Bo Ke, Jianfeng He, Ruizhi Qiao, Di Yin, Xing Sun, Yunsheng Wu, Yinsong Liu, Shuangyin Liu, Mingkong Tang, Haodong Lin, Jiayi Kuang, Fanxu Meng, Xiaojuan Tang, Yunjia Xi, Junjie Huang, Haotong Yang, Zhenyi Shen, Yangning Li, Qianwen Zhang, Yifei Yu, Siyu An, Junnan Dong, Qiufeng Wang, Jie Wang, K...

    2026

  43. [43]

    Search-r1: Training LLMs to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training LLMs to reason and leverage search engines with reinforcement learning. InSecond Conference on Language Modeling, 2025. URLhttps://openreview.net/forum?id= Rwhi91ideu

    2025

  44. [44]

    Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: a benchmark for question answering research.Transact...

    2019

  45. [45]

    Weld, and Luke Zettlemoyer

    Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension, 2017. URL https://arxiv.org/abs/1705. 03551

    2017

  46. [46]

    When not to trust language models: Investigating effectiveness of parametric and non-parametric memories,

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories,

  47. [47]

    URLhttps://arxiv.org/abs/2212.10511

    work page internal anchor Pith review Pith/arXiv arXiv

  48. [48]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering,

  49. [49]

    URLhttps://arxiv.org/abs/1809.09600

    work page internal anchor Pith review Pith/arXiv arXiv

  50. [50]

    Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps, 2020

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps, 2020. URL https://arxiv.org/abs/2011. 01060

    2020

  51. [51]

    MuSiQue: Multihop questions via single-hop question composition.Transactions of the Association for Computational Linguistics, 2022

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. MuSiQue: Multihop questions via single-hop question composition.Transactions of the Association for Computational Linguistics, 2022

    2022

  52. [52]

    Measuring and Narrowing the Compositionality Gap in Language Models

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models, 2023. URL https://arxiv.org/abs/ 2210.03350

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  53. [53]

    InFindings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors,Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Online,...

    work page doi:10.18653/v1/2020.emnlp-main.550 2020

  54. [54]

    Text Embeddings by Weakly-Supervised Contrastive Pre-training

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training, 2024. URL https: //arxiv.org/abs/2212.03533

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  55. [55]

    DeepSeek-V3 Technical Report

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, and Guowei Li. Deepseek-v3 technical report, 2025. URLhttps://arxiv.org/abs/2412.19437

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  56. [56]

    ReTool: Reinforcement Learning for Strategic Tool Use in LLMs

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms, 2025. URL https://arxiv.org/abs/2504.11536

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  57. [57]

    American invitational mathematics examination (aime), 2025.https://maa.org/

    MAA. American invitational mathematics examination (aime), 2025.https://maa.org/

    2025

  58. [58]

    Laminar: A scalable asynchronous RL post-training framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. InProceedings of the Twentieth European Conference on Computer Systems, EuroSys ’25, page 1279–1297. ACM, March 2025. doi: 10.1145/3689031.3696075. URLhttp://dx.doi.org/10.1145/3689031.3696075

    work page doi:10.1145/3689031.3696075 2025

  59. [59]

    The Llama 3 Herd of Models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, and et.al. Arthur Hinsvark. The llama 3 herd of models, 2024. URLhttps://arxiv.org/...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  60. [60]

    Qwen2.5 Technical Report

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li,...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  61. [61]

    expect to find a candidate name

    **NO SPOILERS:** Plan the future steps using *abstract placeholders* (e.g., "expect to find a candidate name"), NOT specific entities found in the future

  62. [62]

    **OUTPUT FORMAT (Markdown Only):** ## World Model Simulation - **Trajectory:** Step-by-step roadmap to the answer based on the Future Truth

    **BE SPECIFIC:** In the Action Plan, specify exact keywords/intents, avoiding generic phrases like "Proceed". **OUTPUT FORMAT (Markdown Only):** ## World Model Simulation - **Trajectory:** Step-by-step roadmap to the answer based on the Future Truth. - **Gap:** Why the current state is insufficient. - **Confidence:** [0-100]%. (High=Direct path, Low=Compl...

    2015

  63. [63]

    Ivory Coast reggae musician known as the ’Bob Marley of Africa’

    Identify the musician via search using the descriptive title "Ivory Coast reggae musician known as the ’Bob Marley of Africa’"

  64. [64]

    Confirm the artist’s name (Alpha Blondy) from the initial search results

  65. [65]

    Execute a refined search for the specific details: the 17th studio album (2015 release), the festival performance, and the collaboration

    2015

  66. [66]

    Extract the album title and record label from the detailed results about the 2015 release

    2015

  67. [67]

    T urn 1: Action search(

    Synthesize the information to provide the final answer. **Gap:** The current observation only contains the user’s question. No information about the artist’s identity, discography, or specific events is available to answer the query directly. **Confidence:** 85%. The question provides multiple unique, verifiable identifiers (specific album number, year, f...

    2015

  68. [68]

    Bob Marley of Africa

    Confirm the candidate’s identity as the verified "Bob Marley of Africa" from the initial search results

  69. [69]

    Search specifically for the candidate’s 17th studio album released in 2015 using their confirmed name

    2015

  70. [70]

    Search specifically for the candidate’s performance at the Street Mode Festival using their con- firmed name

  71. [71]

    Search specifically for the candidate’s collaboration with Harry Roselmack in 2014 using their confirmed name

    2014

  72. [72]

    Bob Marley of Africa

    Extract the album title, festival performance year, and collaboration details from the targeted results. **Gap:** The current queries using descriptive placeholders ("Bob Marley of Africa") instead of the confirmed name are returning irrelevant results about Bob Marley himself and failing to find festival/collaboration specifics due to low specificity. **...

    2015

  73. [73]

    **Alpha Blondy’s 17th Studio Album (2015)**: The album is titled *Positive Energy*, released on May 18, 2015, under the label **Wagram Music** (as mentioned in the [World A Reggae arti- cle](https:....../) and [Discogs entry](https:......))

    2015

  74. [74]

    **Performance at Street Mode Festival (Greece)**: Alpha Blondy performed at the Street Mode Festival in Thessaloniki, Greece, in 2016 (as per [Setlist.fm](https:......) and the festival’s [history page](https:....../))

    2016

  75. [75]

    Bob Marley of Africa,

    **Collaboration with Harry Roselmack (2014)**: Alpha Blondy collaborated with Harry Roselmack and other artists on the song *On n’oublie pas* in 2014, a tribute song (as mentioned in [Tanya St Val’s profile](https:......) and [TikTok references](https:......)). ### Final Answer: <answer> The 17th studio album of Alpha Blondy, the "Bob Marley of Africa," r...

    2014

  76. [76]

    Calculate total days in 3 weeks (3 × 7 = 21 days)

  77. [77]

    Compute total food consumed over 21 days (daily consumption × 21)

  78. [78]

    Determine the ratio of total food to Victor’s weight (total_food / victor_weight)

  79. [79]

    worth of food in 3 weeks victors_worth = total_food_3_weeks / victor_weight victors_worth T urn 1: Feedback 15.0 T urn 2: Think The bear would eat the equivalent of ( 15 )

    Output the final unitless count representing "Victors" worth **Gap:** The current state only provides the problem statement without numerical computation. The solution requires executing the mathematical operations to transform inputs (90 lbs/day, 126 lbs, 3 weeks) into the requested ratio. **Confidence:** 100%. The problem is straightforward with a direc...

    1947

  80. [80]

    Identify that this is a permutation problem with repeated digits

Showing first 80 references.