Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

A Few Teacher Steps Go a Long Way: Cost-Efficient On-Policy Data Augmentation for Agent Post-Training

T0 review · 3 major / 5 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For language-model agents, a few teacher steps at the student's own contexts often beat more full expert demos or heavily filtered completions under the same supervision budget.

desk verdict Useful matched-budget result: short unfiltered teacher corrections at student states beat pure BC and often match heavy filtering, with the main caveat that K is domain-tuned and collection is one-round. read the letter →

arxiv 2607.04574 v1 pith:R3KVYEFY submitted 2026-07-06 cs.LG cs.AI

Junze Ye , Jiayi Cheng , Miao Lu , Michal Mankowski , Jose Blanchet , Mohsen Bayati This is my paper
classification cs.LGcs.AI
keywords on-policydataaugmentationbehavioralcloningLLMagentssupervisedfine-tuningbudgetallocationimitationlearningagentpost-trainingDAgger-stylecorrections
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Supervised fine-tuning of language-model agents depends not only on the quality of teacher labels but on which interaction contexts those labels condition on. Pure behavioral cloning trains only on teacher-induced contexts, so the student later faces states its own earlier mistakes create that the training set never covered. The authors reframe on-policy data construction as a budget-allocation problem: under matched resources, should extra teacher output buy more full demos, longer continuations, outcome filtering, or broader coverage of learner-induced states? Across HotpotQA, ALFWorld, and Terminal-Bench-Dev, short unfiltered teacher continuations started from student-induced contexts improve over scaling pure teacher demos at the same budget, and on the two domains with full comparisons few-step unfiltered runs match or beat success-filtered and critical-context-filtered alternatives. The practical claim is that once a reasonable initial student exists, a few teacher steps placed at learner states are often the most cost-efficient use of additional expert supervision.

What carries the argument

A unified data-construction recipe G that makes explicit the rollout policy, switch-time distribution, continuation horizon K, and filtering rules, tracked against two costs: teacher inference generated before filtering (Ci) and teacher tokens retained for SFT (Ctr). The main method, OP-SHORT, accepts every sampled continuation but caps it to a small number of teacher turns, turning the breadth-versus-depth tradeoff into a controllable budget knob.

What would settle it

On a long-horizon agent benchmark where recovery requires extended planning, show that at matched Ci and Ctr, short unfiltered teacher continuations underperform pure full demos or success-filtered full continuations by a clear margin on test success rate.

Watch

Extended reading notes

Core claim

Under matched teacher-inference and retained-SFT budgets, bounded unfiltered teacher continuations at learner-induced contexts improve over pure behavioral cloning; on HotpotQA and ALFWorld, few-step unfiltered continuations match or exceed success-filtered and critical-context-filtered alternatives. A few teacher steps at student-induced states can be a more cost-efficient allocation of expert supervision than longer or more heavily curated teacher completions.

Load-bearing premise

That one round of augmentation with a fixed student prefix policy and a small domain-chosen continuation length is representative of practical agent training, including long-horizon recovery, iterative collection, and settings without automatic verifiable rewards.

Editorial extensions

If this is right

  • Once a reasonable behavioral-cloning student exists, additional expert budget should prioritize short on-policy corrections over more start-to-finish demos.
  • Success filtering can look attractive on retained training cost while looking worse on teacher-inference cost, so the binding resource should decide the recipe.
  • Continuation length K is a domain-dependent breadth-depth parameter, not a universal constant; longer is not automatically better.
  • On Terminal-Bench-Dev, short on-policy SFT with a small fraction of the demo corpus can match a full-corpus SFT plus reinforcement-learning baseline without any RL stage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same breadth-depth budget logic would likely apply when the "teacher" is a human expert rather than an API model, reshaping how labeling sessions are scheduled for agent training.
  • If non-monotone returns to continuation length generalize, automated search over K could become a standard hyperparameter in agent SFT pipelines, analogous to learning-rate search.
  • In domains without verifiable terminal rewards, the ranking of unfiltered short continuations versus curated ones may reverse once proxy filters replace outcome filters.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that for LLM-agent SFT, the allocation of limited teacher supervision matters as much as label quality: under matched budgets, bounded unfiltered teacher continuations at learner-induced contexts (OP-SHORT) are more cost-efficient than pure behavioral cloning and, on HotpotQA and ALFWorld, match or beat success-filtered and critical-context-filtered alternatives. It formalizes on-policy data construction via a recipe G = (ρ, πe, n, Ψ, Tswitch, K, Φ), distinguishes trajectory-uniform vs pooled-turn context averaging for variable-length rollouts, and tracks two costs—teacher inference before filtering (Ci) and retained SFT tokens (Ctr). Empirically, one-round BC+OP with small K improves over scaling pure BC on HotpotQA, ALFWorld, and Terminal-Bench-Dev; non-monotone returns to K and domain-dependent best K are reported, and on Terminal-Bench OP-SHORT at a small BC fraction matches a full-corpus SFT+RL baseline.

Significance. If the matched-budget ranking holds more broadly, the paper offers a practical, low-complexity rule for agent post-training under expensive teacher APIs: after a modest pure-BC student, spend additional budget on short unfiltered corrections at student states rather than on longer demos or heavy filtering. Strengths include an explicit design-space formalization (Alg. 1, Table 1, Eqs. 7–8, 11–13), dual cost axes that make filtering tradeoffs visible, held-out checkpoint selection on HotpotQA, and multi-environment evidence with ablations on K and on-policy mix ratio. The work is comparative and empirical rather than theoretical, but the framing cleanly separates prefix policy, switch sampling, continuation horizon, and filters in a way that is useful for subsequent agent SFT recipes.

major comments (3)
  1. The central practical claim (Abstract; §4) that “a few teacher steps” are often the most cost-efficient allocation rests on one-round BC+OP with a frozen prefix policy ρ=π1 (Eq. 14; Alg. 1; Table 1) and a domain-chosen K. The paper itself reports non-monotone K returns (HotpotQA: K=3 best, K=5 worse; ALFWorld: K=5 beats K=∞; Figs. 2–3) and no theory for optimal K (Limitations). Without iterative collection or a principled K-selection protocol, it remains unclear whether the efficiency ranking is an artifact of a single student-induced distribution and post-hoc K rather than a general allocation principle. A stronger manuscript would either (i) run at least one multi-round DAgger-style loop under matched Ci/Ctr, or (ii) pre-specify a K rule (e.g., fraction of Tmax or validation-based) and report sensitivity under that rule.
  2. Full recipe comparison (OP-SHORT vs OP-SUCCESS vs OP-CRITICAL vs pure BC) is complete only on HotpotQA and partially on ALFWorld; Terminal-Bench-Dev (§3.3, Fig. 5) reports pure BC vs OP-SHORT only, with incomplete filtering variants and a different teacher/student pair. The abstract’s cross-benchmark claim that few-step unfiltered continuations “match or exceed” filtered alternatives is therefore overstated relative to the experimental coverage. Either restrict that claim to HotpotQA/ALFWorld or add the missing filtered baselines on Terminal-Bench under the same Ci/Ctr accounting.
  3. Budget matching is careful on Ci and Ctr, but free parameters remain load-bearing: proposal count n, |Qtrain|, on-policy mix ratio r (Fig. 4), and SFT hyperparameters differ across environments (§3.1–3.3; Tables 2–4). On ALFWorld the best observed r is budget-dependent (roughly 50–90%), and on Terminal-Bench OP-SHORT is sometimes run with task-specific n. The paper should state more clearly which comparisons hold |Qtrain|, n, and r fixed versus which sweep them, and whether the OP-SHORT advantage survives a fixed r (e.g., 50%) rather than the best-of-sweep mix.
minor comments (5)
  1. Figure 1 is helpful for pool vs traj averaging, but the main text could more explicitly state which experimental recipes induce dρ;traj vs dρ;pool (OEC is pool-like; OP-SHORT is traj-uniform) so readers do not have to reconstruct this from Table 1 and Appendix B.2.
  2. HotpotQA checkpoint selection (Fig. 6 / Appendix D.2) uses medium-level dev while final test is hard-level; the intentional distribution shift is noted, but a short main-text sentence on why medium-dev selection does not overfit hard-test would help.
  3. Notation: “state” is redefined as LLM context (after Eq. 5); a brief reminder when discussing “learner-induced states” would reduce confusion with latent environment state st.
  4. Related work is thorough; a short positioning sentence on how OP-SHORT differs from LEAP’s single-turn teacher correction and OEC’s success-filtered full continuations would sharpen novelty for readers skimming §A.
  5. Typos / polish: “Critic” vs “Critical” in Fig. 3 legend; occasional missing spaces around citations; Appendix G/H are long—consider moving the full trajectory example to supplementary material if page limits apply.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: matched-budget efficiency claims are empirical comparisons on external verifiers, not results forced by definition or self-citation.

full rationale

This paper is comparative empirical work on SFT data-construction recipes for LLM agents, not a first-principles derivation that could collapse into its inputs. The formal apparatus (POMDP agent model, trajectory-uniform vs pooled context distributions d^{π;w}_μ, recipe G = (ρ, π_e, n, Ψ, T_switch, K, Φ), and dual costs C_i / C_tr) defines how datasets are built and how budgets are measured; it does not algebraically force OP-SHORT to outperform PURE-BC or filtered variants. Performance is reported on external, held-out or official evaluators (HotpotQA exact-match pass@1 on hard test questions, ALFWorld valid_seen success, Terminal-Bench-Dev verifier scores), after checkpoint selection on held-out dev where applicable. Teacher labels supply training targets, but the central claim is a relative efficiency ranking under matched supervision resources—an empirical outcome that could have gone either way (and is reported as non-monotone in K). Related-work citations (DAgger, OEC/Lauffer et al., LEAP/Choudhury & Sodhi, etc.) are external positioning, not load-bearing uniqueness theorems by the present authors. No fitted parameter is renamed a prediction, no ansatz is smuggled in via self-citation, and no equation reduces the reported gains to a tautology. Score 0 is therefore appropriate.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The central claim is empirical and rests on standard imitation-learning and SFT assumptions plus experimental design choices (one-round BC+OP, trajectory-uniform switching, chosen K, fixed |Qtrain| controls). No new physical entities are postulated. Free parameters are the usual training and data-construction hyperparameters that affect reported wins.

free parameters (4)
  • continuation horizon K
    Domain-chosen teacher-turn cap for OP-SHORT; non-monotone and treated as a breadth-depth knob rather than derived.
  • proposal count n and |Qtrain|
    Control how many student prefixes and tasks receive teacher budget; held fixed or swept to isolate recipe effects.
  • SFT/RL hyperparameters
    Learning rates, epochs, batch sizes, and GRPO settings differ by domain and affect absolute scores.
  • on-policy mix ratio r
    Fraction of tasks used for student-prefix data; ablated on ALFWorld and shown budget-dependent.
assumptions (5)
  • domain assumption Agent interaction can be modeled as a finite-horizon POMDP with terminal verifiable reward and linear ChatML contexts without truncation.
    Section 2.1; underpins context distributions and multi-turn continuation objectives.
  • domain assumption Next-turn / multi-turn SFT by minimizing forward KL / cross-entropy to a frozen teacher is the right training objective for comparing data recipes.
    Section 2.2 and Appendix B.1; standard teacher-forcing assumption.
  • ad hoc to paper Trajectory-uniform switch sampling (t' ~ Unif{1..T(τ)}) is a valid and preferred way to average variable-length learner contexts.
    Eq. 8 and OP-SHORT design; contrasted with pooled-turn OEC-style sampling.
  • ad hoc to paper Teacher inference cost Ci and retained training cost Ctr are the right two axes for budget matching, while |Qtrain| is controlled separately.
    Section 2.3; organizes all main plots.
  • ad hoc to paper One round of BC then OP data generation with final reinitialization on DBC ∪ DOP is sufficient to evaluate on-policy augmentation.
    Methods and Limitations; iterative DAgger-style loops are not studied.
invented entities (2)
  • Recipe object G = (ρ, πe, n, Ψ, Tswitch, K, Φ) and cost pair (Ci, Ctr)
    purpose: Modularize on-policy SFT data construction and make budget comparisons commensurable.
    Formal bookkeeping rather than a physical entity; useful but definitional.
  • OP-SHORT protocol
    purpose: Main method: unfiltered bounded teacher continuations at trajectory-uniform student switch points.
    Empirical recipe variant of DAgger-style corrections; validated only by the paper's experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Few Teacher Steps Go a Long Way: Cost-Efficient On-Policy Data Augmentation for Agent Post-Training." pith.science (2026). https://pith.science/paper/R3KVYEFY

@misc{pith2026260704574,
  author       = {Pith},
  title        = {Pith review of: A Few Teacher Steps Go a Long Way: Cost-Efficient On-Policy Data Augmentation for Agent Post-Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R3KVYEFY}},
  note         = {Machine review of arXiv:2607.04574}
}
read the original abstract

For LLM agents, supervised fine-tuning is not only about teacher labels' quality, but also about which interaction contexts those labels condition on. Pure behavioral cloning uses full teacher demonstrations, creating a mismatch between teacher-induced contexts seen in training and student-induced contexts encountered at test time. Recent work addresses this mismatch by querying a teacher at contexts reached by the student, often with increasingly elaborate filtering of the teacher's continuations. We instead frame on-policy data construction as a budget-allocation problem: under matched supervision resources, should teacher output be spent on more start-to-finish demos, longer continuations, outcome filtering, or broader coverage of learner-induced contexts? We formalize this design space through the rollout policy, switch-time distribution, continuation horizon, filtering rules, and two complementary costs: teacher inference generated before filtering and teacher supervision retained for SFT. Across HotpotQA, ALFWorld, and Terminal-Bench-Dev, bounded unfiltered teacher continuations at learner-induced contexts improve over pure behavioral cloning at matched budgets. On HotpotQA and ALFWorld, where we run the full comparison, few-step continuations match or exceed success-filtered and critical-context-filtered alternatives. Our findings suggest that a few teacher steps, placed at learner-induced contexts, can be a more cost-efficient supervision allocation than longer or more heavily curated teacher completions.

Figures

Figures reproduced from arXiv: 2607.04574 by the authors.

Figure 1
Figure 1. The two state-averaging conven￾tions of Eq. 8, on two sampled trajectories of (a) identical and (b) variable length. The first is pooled-turns randomization: every visited context, regardless of its trajectory, receives equal probability mass after normalization, so longer tra￾jectories contribute more total mass. A fixed-horizon switch sampler with invalid-switch rejection, as in OEC-style random switching, is one … view at source ↗
Figure 2
Figure 2. HotpotQA SFT test accuracies for the recipe instances in Table 1. All variants use the [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. ALFWorld test accuracy on valid_seen pass@1. Left: The left panel counts total teacher labeling cost, including discarded. Right: the right panel counts teacher tokens retained for SFT. Note: for the case of 64 tasks 64, the traces are all filtered out under OP-CRITICAL, and thus there is no corresponding point. shows that longer teacher continuations do not monotonically improve ALFWorld performance. For all of tas… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: ALFWorld optimal on￾policy data ratio r. ratio r is the per￾centage of tasks used for on-policy data generation. r = 0 degenerates to BC, and r = 1 allocates all the tasks to use student prefix. Ablation on optimal mixing ratio. Given a fixed teacher super￾vision budge…
Figure 5
Figure 5. Figure 5: Terminal-Bench-Dev results at the same prefix model. (A, B) TB-dev mean score across [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: HotpotQA checkpoint selection. Note that the dev set are composed of [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: A real ReAct-style rollout instantiating the sequential decision process in actual model [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Linearized schematic of the Figure 7 rollout trajectory as a sequential decision process. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Example trajectory from an InferredBugs command-line task. Purple denotes the full user [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CodeRescue: Budget-Calibrated Recovery Routing for Coding Agents

    cs.AI 2026-07 unverdicted novelty 6.0 of 10

    Budget-calibrated recovery routing with conformal risk control lets coding agents match always-escalate solve rates at about 35% of the cost.

Reference graph

Works this paper leans on

45 extracted references · 10 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Saad-Falcon, Jon and Narayan, Avanika and Akengin, Hakki Orhun and Griffin, J. Wes and Shandilya, Herumb and Gamarra Lafuente, Adrian and Goel, Medhya and Joseph, Rebecca and Natarajan, Shlok and Guha, Etash Kumar and Zhu, Shang and Athiwaratkun, Ben and Hennessy, John and Mirhoseini, Azalia and R. arXiv preprint arXiv:2511.07885 , year =. doi:10.48550/ar...

    arXiv doi:10.48550/arxiv.2511.07885

  2. [2]

    and Zipser, David , journal=

    Williams, Ronald J. and Zipser, David , journal=. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , year=

  3. [3]

    2026 , howpublished =

    2026

  4. [4]

    Introducing APIs for GPT-3.5 Turbo and Whisper , year =

  5. [5]

    Demystifying Evals for AI Agents , year =

  6. [6]

    arXiv preprint arXiv:2605.12913 , year=

    Revisiting DAgger in the Era of LLM-Agents , author=. arXiv preprint arXiv:2605.12913 , year=

    arXiv

  7. [7]

    arXiv preprint arXiv:1406.5979 , year=

    Reinforcement and imitation learning via interactive no-regret learning , author=. arXiv preprint arXiv:1406.5979 , year=

    arXiv

  8. [8]

    Decoupling

    Zhao, Anhao and Xin, Haoran and Fan, Yingqi and Tong, Junlong and Li, Wenjie and Shen, Xiaoyu , journal=. Decoupling

Show all 45 references
  1. [9]

    arXiv preprint arXiv:2210.03629 , year=

    ReAct: Synergizing reasoning and acting in language models , author=. arXiv preprint arXiv:2210.03629 , year=

    arXiv

  2. [10]

    Artificial Intelligence , volume=

    Planning and acting in partially observable stochastic domains , author=. Artificial Intelligence , volume=. 1998 , publisher=

    1998

  3. [11]

    Operations Research , volume=

    The optimal control of partially observable Markov processes over a finite horizon , author=. Operations Research , volume=. 1973 , publisher=

    1973

  4. [12]

    Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS) , pages=

    A reduction of imitation learning and structured prediction to no-regret online learning , author=. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS) , pages=. 2011 , organization=

    2011

  5. [13]

    Proceedings of the Twentieth European Conference on Computer Systems , pages=

    Hybridflow: A flexible and efficient rlhf framework , author=. Proceedings of the Twentieth European Conference on Computer Systems , pages=

  6. [14]

    Second Conference on Language Modeling , year=

    Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning , author=. Second Conference on Language Modeling , year=

  7. [15]

    2019 International Conference on Robotics and Automation (ICRA) , pages=

    Hg-dagger: Interactive imitation learning with human experts , author=. 2019 International Conference on Robotics and Automation (ICRA) , pages=. 2019 , organization=

    2019

  8. [16]

    Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages=

    Efficient reductions for imitation learning , author=. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages=. 2010 , organization=

    2010

  9. [17]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Measuring and narrowing the compositionality gap in language models , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  10. [18]

    arXiv preprint arXiv:2503.20783 , year=

    Understanding r1-zero-like training: A critical perspective , author=. arXiv preprint arXiv:2503.20783 , year=

    arXiv

  11. [19]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  12. [20]

    Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

    When not to trust language models: Investigating effectiveness of parametric and non-parametric memories , author=. Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

  13. [21]

    Transactions of the Association for Computational Linguistics , volume=

    MuSiQue: Multihop Questions via Single-hop Question Composition , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=

    2022

  14. [22]

    Proceedings of the 28th International Conference on Computational Linguistics , pages=

    Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps , author=. Proceedings of the 28th International Conference on Computational Linguistics , pages=

  15. [23]

    , booktitle =

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William and Salakhutdinov, Ruslan and Manning, Christopher D. , booktitle =. 2018 , address =

    2018

  16. [24]

    International Conference on Learning Representations , year =

    Shridhar, Mohit and Yuan, Xingdi and C. International Conference on Learning Representations , year =

  17. [25]

    2024 , journal =

    HybridFlow: A Flexible and Efficient RLHF Framework , author =. 2024 , journal =

    2024

  18. [26]

    Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , series =

    A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , author =. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , series =. 2011 , url =

    2011

  19. [27]

    NeurIPS 2024 Workshop on Open-World Agents , year =

    Fine-Tuning Web Agents: It Works, But It's Trickier Than You Think , author =. NeurIPS 2024 Workshop on Open-World Agents , year =

    2024

  20. [28]

    Better than Your Teacher:

    Choudhury, Sanjiban and Sodhi, Paloma , booktitle =. Better than Your Teacher:. 2025 , url =

    2025

  21. [29]

    Imitation Learning for Multi-turn

    Lauffer, Niklas and Deng, Xiang and Kundurthy, Srivatsa and Kenstler, Brad and Da, Jeff , journal =. Imitation Learning for Multi-turn. 2025 , url =

    2025

  22. [30]

    arXiv preprint arXiv:2509.14257 , year =

    From Correction to Mastery: Reinforced Distillation of Large Language Model Agents , author =. arXiv preprint arXiv:2509.14257 , year =

  23. [31]

    Training

    Chen, Yihan and Xu, Benfeng and Wang, Xiaorui and Zhang, Yongdong and Mao, Zhendong , journal =. Training. 2025 , url =

    2025

  24. [32]

    Embodied Multi-Modal Agent trained by an

    Yang, Yijun and Zhou, Tianyi and Li, Kanxue and Tao, Dapeng and Li, Lusong and Shen, Li and He, Xiaodong and Jiang, Jing and Shi, Yuhui , booktitle =. Embodied Multi-Modal Agent trained by an. 2024 , url =

    2024

  25. [33]

    2025 , url =

    Wang, Hanlin and Wang, Jian and Leong, Chak Tou and Li, Wenjie , booktitle =. 2025 , url =

    2025

  26. [34]

    Yuan, Siyu and Chen, Zehui and Xi, Zhiheng and Ye, Junjie and Du, Zhengyin and Chen, Jiecao , journal =. Agent-. 2025 , url =

    2025

  27. [35]

    2025 , url =

    Chen, Zhixun and Li, Ming and Huang, Yuxuan and Du, Yali and Fang, Meng and Zhou, Tianyi , journal =. 2025 , url =

    2025

  28. [36]

    Exploring Expert Failures Improves

    Lan, Li-Cheng and Bai, Andrew and Cheng, Minhao and Wang, Ruochen and Hsieh, Cho-Jui and Zhou, Tianyi , journal =. Exploring Expert Failures Improves. 2025 , url =

    2025

  29. [37]

    International Conference on Learning Representations , year =

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author =. International Conference on Learning Representations , year =

  30. [38]

    2024 , url =

    Gu, Yuxian and Dong, Li and Wei, Furu and Huang, Minlie , booktitle =. 2024 , url =

    2024

  31. [39]

    arXiv preprint arXiv:2602.12275 , year =

    On-Policy Context Distillation for Language Models , author =. arXiv preprint arXiv:2602.12275 , year =

    arXiv

  32. [40]

    arXiv preprint arXiv:2601.18734 , year =

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author =. arXiv preprint arXiv:2601.18734 , year =

    arXiv

  33. [41]

    2026 , url =

    Xiong, Jing and Shen, Hui and Gong, Shansan and Cheng, Yuxin and Shen, Jianghan and Tao, Chaofan and Tan, Haochen and Bai, Haoli and Shang, Lifeng and Wong, Ngai , journal =. 2026 , url =

    2026

  34. [42]

    arXiv preprint arXiv:2511.10643 , year =

    Black-Box On-Policy Distillation of Large Language Models , author =. arXiv preprint arXiv:2511.10643 , year =

  35. [43]

    Ernst , title =

    Xi Victoria Lin and Chenglong Wang and Luke Zettlemoyer and Michael D. Ernst , title =. CoRR , volume =. 2018 , url =. 1802.08979 , timestamp =

    2018 arXiv

  36. [44]

    2023 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2023) , year=

    InferFix: End-to-End Program Repair with LLMs , author=. 2023 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2023) , year=

    2023

  37. [45]

    arXiv preprint arXiv:2507.12856 , year =

    Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved) , author =. arXiv preprint arXiv:2507.12856 , year =

    arXiv

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.