Pith. sign in

REVIEW 4 major objections 5 minor 25 references

Self-Improving Large Language Models via Progressive Experience Evolution

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SPEE claims that LLMs improve most when explicit textual experience distilled from successful and failed trajectories is internalized before reinforcement learning, outperforming both test-time and training-time baselines.

desk verdict Useful post-training recipe that combines experience distillation with RL, but the central utility filter needs much better evidence and transparency before the method's claims hold. read the letter →

arxiv 2608.02139 v2 pith:ECBPXMRZ submitted 2026-08-03 cs.CL cs.AIcs.LG

Shijie Ren , Xiting Wang , Meng Li , Yujie Guo , Yunhang Yao , Ziheng Peng , Xunlong Wang , Yuetan Chen
show 3 more authors
Haoyang Zhou Yunlong Liang Fandong Meng
This is my paper
classification cs.CLcs.AIcs.LG
keywords self-improvingLLMsexperiencedistillationon-policyself-distillationGRPOmathematicalreasoningprogressiveevolutionself-evolution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that self-improving LLMs need an intermediate step between test-time prompting and reinforcement learning: convert interaction trajectories into compact, transferable experience, internalize that experience into model parameters, and only then run reward-driven optimization. It proposes SPEE, which extracts reasoning lessons from both correct and incorrect trajectories, consolidates them into an evolving global experience pool, filters items by measured utility on a held-out probe set, and distills the survivors into the policy via on-policy self-distillation before applying GRPO. Across five mathematical reasoning benchmarks and three model scales, SPEE consistently outperforms the base model, a test-time prompt baseline, GRPO, and a self-distillation baseline, with average gains up to 6.96 percentage points and roughly 28% less training data than GRPO at matched performance. The claim matters because it offers a concrete mechanism for persistent capability accumulation without human labels.

What carries the argument

The load-bearing mechanism is the evolving global experience pool coupled with the privilege-guided On-Policy Self-Distillation objective. The extractor $\Sigma_{\theta_k}$ writes trajectory-derived hypotheses, the evolution operator $\Phi$ merges, resolves, and abstracts them, and the utility filter $w^{(k)}(e)$ on a held-out probe set discards low-value items. Distillation then minimizes $D_{\mathrm{KL}}(\pi_{\theta}^{\mathrm{stu}} \| \pi_{\bar{\theta}}^{\mathrm{tea}})$ between the student on the problem alone and a detached teacher conditioned on both the problem and the experience, turning sparse reward feedback into dense token-level supervision grounded in reusable textual lessons.

What would settle it

Run SPEE with the experience pool items replaced by raw trajectory fragments of equal length while keeping the same filters, distillation procedure, and GRPO stage; if accuracy over GRPO does not drop, then the evolved, abstracted experience is not what carries the gain.

Watch

Extended reading notes

Core claim

The central discovery is that explicit experience evolution followed by implicit policy optimization yields better self-improvement than either paradigm alone. SPEE first samples trajectories from the current policy, extracts positive and negative experience items, merges and abstracts them through an evolution operator, and keeps only items with positive marginal utility on a probe set. These refined items are then internalized through privilege-guided On-Policy Self-Distillation, where an experience-conditioned teacher rescored the student policy's own trajectories and the student minimizes the reverse KL divergence to that teacher. Finally, GRPO explores beyond the pool with group-relative advantages. The paper reports consistent gains over test-time prompting and reward-only RL across qwen3-1.7b, 4b, and 8b on AIME24/25, GSM8K, MATH500, and MinervaMath, with the largest average improvement of 6.96 points over the base 4B model, and shows that Stage I reduces all-incorrect response groups while preserving policy entropy.

Load-bearing premise

The whole pipeline depends on the starting model being strong enough to write down useful lessons from its own successes and failures; if those self-extracted lessons are mostly wrong or noisy, the distillation signal degrades and the reported gains would come from the RL stage rather than from experience evolution.

Editorial extensions

If this is right

  • Self-improvement pipelines should insert an explicit experience-internalization stage before reinforcement learning, so that exploration starts from higher-reward regions.
  • Because Stage I reduces the frequency of all-incorrect response groups while keeping entropy comparable to the base policy, GRPO receives more groups with nonzero relative advantages and thus more informative gradient signals.
  • Ablations show that both stages are needed: removing Stage II lowers average accuracy, and removing the shared experience pool lowers it further, placing cross-problem experience accumulation at the center of the gains.
  • SPEE reaches GRPO-level or better performance with roughly 28% fewer training trajectories, lowering the compute cost of self-improvement.
  • Repeated experience-evolution rounds monotonically improve sampling accuracy, suggesting a closed loop in which an improved policy generates richer experience for the next round.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the probe-set utility filter is the bottleneck, one could replace it with a learned reward model or a semantic-diversity metric, which might extend the method to domains where verifiable rewards are sparse.
  • The global experience pool is an abstraction worth probing directly: if the distilled lessons are truly procedural, SPEE-trained models should show transfer to out-of-distribution reasoning tasks sharing those heuristics, not just to the five reported benchmarks.
  • The framework sets a floor on base-model quality, since a very weak policy may extract mostly noisy experience; a natural extension is to seed the pool with experience from a stronger model and test whether the self-improvement loop still compounds.
  • The paper hints that more evolution rounds help, so an obvious testable extension is to run SPEE for multiple full rounds and measure whether gains continue to accumulate or saturate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SPEE, a two-stage post-training framework for self-improving LLMs. Stage I performs explicit experience evolution: the current policy samples multiple responses per problem, an extractor converts selected successful and failed trajectories into textual experience items, an evolution operator consolidates a global pool, and a marginal-utility filter (Eqs. 9-10) retains items with positive estimated utility on a held-out probe set. The pool is internalized through on-policy self-distillation (OPSD), where an experience-conditioned teacher branch rescored the student's on-policy samples and the student minimizes reverse KL (Eq. 15). Stage II applies GRPO to the distilled policy. Experiments on AIME24/25, GSM8K, MATH500, and MinervaMath across qwen3-1.7B/4B/8B compare with base, Domain Prompt, GRPO, and SDPO, reporting average gains up to +6.96 percentage points and a 28% data-efficiency improvement. Ablations, training-dynamics analyses, and a multi-round sampling-accuracy study are also reported.

Significance. If the reported gains hold, SPEE makes a useful conceptual contribution by positioning experience distillation as an intermediate stage between test-time prompting and RL, with an explicit mechanism for accumulating transferable knowledge. The paper's strengths include a concrete algorithm, an ablation separating Stage I and Stage II, a controlled data-efficiency experiment, evidence that Stage I preserves entropy while increasing the fraction of groups with mixed rewards, and a source-code release. However, the empirical support is currently incomplete: no uncertainty quantification is given, the utility filter's probe-set configuration is undisclosed, and the multi-round 'progressive' aspect is evaluated only on a proxy metric. These gaps are fixable and do not indicate a fundamental flaw, but they are load-bearing for the paper's central claims.

major comments (4)
  1. [Experience evolution, Eqs. (9)-(10)] The marginal-utility filter is the mechanism that allegedly separates transferable experience from noise, but the manuscript never specifies |Qpb|, the number of reward samples used to estimate R^k(q,E), the size of the 'small subset' used in the two-stage screen, or the threshold epsilon. Each w^(k)(e) is a difference of two finite-sample expected-reward estimates; over a large pool of candidate items (G responses per problem, S selected, 17k training problems), the maximum of many zero-mean noisy estimates can be positive even when no candidate has true utility. No variance estimate or multiple-comparison correction is reported. Because Table 2 shows that removing Stage II still leaves +5.45 pp of the +6.96 pp gain on qwen3-4b, the claim that experience distillation, rather than the GRPO stage, drives the results rests directly on this uncharacterized filter.
  2. [Main Results, Table 1] Table 1 reports single-run point estimates without error bars, confidence intervals, or significance tests. Several differences are small or go against the headline claim: on qwen3-1.7b, MinervaMath SPEE (13.97) is below GRPO (14.07), and on qwen3-4b, AIME 2025 SPEE (8.75) is below SDPO (10.83). AIME results are pass@16, a metric with high variance at these sample sizes. Without repeated seeds or bootstrap intervals, 'consistently outperforms' in the abstract and introduction is not supported; the average gains could be driven by a few benchmark shifts. Please provide standard errors (e.g., across at least three runs) or bootstrap confidence intervals for the headline comparisons.
  3. [Effect of Experience Evolution Iterations] The main experiments use a single evolution round, as stated in Section 'Effect of Experience Evolution Iterations', despite the method being introduced as 'progressive' and Eq. (12) defining a multi-round path E^(0) to E^(M). The only multi-round evaluation is Figure 4, which reports sampling accuracy, a proxy, not performance on the five benchmarks, and it also lacks error bars. The paper should either include multi-round results on downstream tasks or explicitly delimit the contribution as single-round experience distillation; as written, the title and abstract overclaim the progressive-evolution evidence.
  4. [Experimental setup and reproducibility] The experimental section omits several load-bearing implementation details: the prompt templates for the extractor and the evolution operator, the hyperparameters for OPSD (number of steps, learning rate, batch size, KL coefficient), the GRPO group size and clip coefficient, the value of S in Eq. (6), the probe-set construction, and the exact configuration of the SDPO and GRPO baselines. The statement that training steps and rollout counts are controlled is not sufficient to reproduce the comparisons or to verify that the 28% data-efficiency figure in Figure 3 is computed from matched conditions. Please provide a full hyperparameter table and define the 'w/o Shared Experience Pool' ablation, which is not described in the text.
minor comments (5)
  1. [Abstract and Introduction] The phrase '6.96% improvement' should be '6.96 percentage points'; the relative improvement from 29.71 to 36.67 on qwen3-4b is about 23%, so the current wording is ambiguous and overstates the relative gain.
  2. [Eq. (9) and Eq. (14)] The notation \hat{E}^{(k+1)}_e in Eq. (9) is undefined; it should be defined, for example, as the provisional pool augmented with candidate e. Similarly, Eq. (14) writes the teacher distribution as \pi^{tea}(y'|q) on the left but conditions on y_{<t} on the right; clarify that the teacher scores the student's partial sequence.
  3. [Figure 4] The vertical axis is labeled 'sampling accuracy' and defined as the proportion of sampled responses that are correct; since the reward is binary correctness, the phrase 'and receive a positive reward' in the text is redundant and could confuse readers.
  4. [Conclusion and Limitation] The final section does not discuss actual limitations, such as the restriction to binary verifier rewards, single-domain evaluation, or the computational cost of the utility filter's probe-set evaluations; it only mentions future work.
  5. [Figure 3] Figure 3 does not define how 'the same performance level' is determined or how the 28% reduction in training trajectories is interpolated from the curves; please state the interpolation procedure and any tolerance used.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SPEE's core claims are supported by external benchmarks and baselines, and its self-referential training loop is a design choice rather than a definitional reduction.

full rationale

SPEE's central result is an empirical comparison on five external benchmarks (AIME24/25, GSM8K, MATH500, MinervaMath) against the base model, Domain Prompt, GRPO, and SDPO. None of the equations in the method section asserts the reported gains; Eq. (3) and Eq. (15) define an objective rather than deriving an outcome, and Eq. (1) is a conceptual information-theoretic reformulation that is not used to compute numbers. The utility filter in Eqs. (9)-(10) selects experience items by estimated reward improvement on a held-out probe set, and OPSD in Eqs. (13)-(15) uses a teacher that conditions on that experience; while both stages involve the same policy, this is a self-training loop, not a logical reduction, because the paper's evidence of improvement is obtained on held-out benchmarks that are not the probe set used for filtering. The comparison with SDPO, which uses the same self-distillation mechanism but without the evolved global experience pool, provides an external control. The paper also does not rely on a load-bearing self-citation: references to prior methods and datasets are to independent external work, and no uniqueness theorem or ansatz is imported from the authors' own prior publications. The undisclosed probe-set size, sampling budget, and threshold epsilon in Eq. (9) are a reproducibility and statistical-risk concern about false-positive experience selection, but they do not make the benchmark results equivalent to the paper's inputs by construction. Accordingly, no circular step meeting the quoted-evidence standard is present.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its free parameters are mostly unspecified hyperparameters (epsilon, S, probe set size, number of evolution rounds) rather than fitted constants. The key assumption is that self-extracted textual experience is meaningful and transferable, which the utility filter is designed to enforce but does not fully guarantee.

free parameters (4)
  • Number of evolution rounds M = 1 in main experiments; up to 4 in sensitivity analysis
    The main results use a single evolution round for computational efficiency, while the ablation in Figure 4 shows gains from more rounds. This is a design choice, not fitted to benchmarks, but it affects the reported main results.
  • Utility threshold epsilon = Not specified
    Eq. 10 keeps items with w(e) > epsilon. The value of epsilon is not reported, and it directly controls how many experience items enter the pool and the distillation signal.
  • Subsample size S for trajectory extraction = Not specified
    The paper says trajectories are randomly subsampled to preserve the proportion of successes and failures, but S is not given.
  • Group size G and clipping coefficient for GRPO = G=8 in Figure 2; epsilon clipping not specified
    Standard GRPO hyperparameters; the paper mentions G in the rollout diversity analysis but does not provide the full hyperparameter list for the training runs.
assumptions (3)
  • domain assumption The experience extractor Sigma, implemented by the current policy, produces transferable experience items from trajectories under a prompt.
    The entire method assumes that a weak base model can reliably abstract reusable strategies from its own correct and incorrect solutions, and that these abstractions are not mostly hallucinated. Section 4.1 states items are provisional, but the method depends on their utility.
  • domain assumption The utility filter on a held-out probe set Qpb accurately measures the marginal contribution of an experience item to future competence.
    Eq. 9 measures utility as the expected reward difference with and without the item under the current policy. This assumes the probe set is representative and the reward difference is stable, which is not validated in the paper.
  • standard math Reverse KL divergence in Eq. 15 is an effective distillation objective for on-policy self-distillation.
    The paper uses reverse KL between student and teacher, following prior distillation work (Hübotter et al. 2026). This is a standard choice, not a new axiom, but it carries known mode-seeking behavior assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Improving Large Language Models via Progressive Experience Evolution." pith.science (2026). https://pith.science/paper/ECBPXMRZ

@misc{pith2026260802139,
  author       = {Pith},
  title        = {Pith review of: Self-Improving Large Language Models via Progressive Experience Evolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ECBPXMRZ}},
  note         = {Machine review of arXiv:2608.02139}
}
read the original abstract

Large language models (LLMs) capable of self-improvement require not only effective policy optimization, but also a principled mechanism for transforming transient interaction experience into persistent model capabilities. Existing self-improvement paradigms remain fragmented: test-time methods can explicitly extract experience but cannot internalize it into model parameters, whereas training-time optimization methods can update model parameters but lack an explicit mechanism for accumulating transferable experience. Bridging these two paradigms requires a critical intermediate stage that remains underexplored, namely \emph{experience distillation}. To address this gap, we propose \textbf{SPEE} (\textbf{S}elf-\textbf{P}rogressive \textbf{E}xperience \textbf{E}volution), a unified post-training framework that sequentially performs explicit experience evolution followed by implicit policy optimization. During explicit experience evolution, SPEE reflects on trajectories collected from multiple interactions to extract, verify, and progressively evolve transferable experience, which is subsequently internalized into the policy through privilege-guided On-Policy Self-Distillation (OPSD). During implicit policy optimization, reward-driven reinforcement learning leverages these internalized priors to explore novel solution strategies. In the experience evolution stage, a continuously evolving global experience pool consolidates knowledge from both successful and failed trajectories, filters out low-utility experience, and mitigates post-hoc rationalization induced by individual trajectories. Experiments on five mathematical reasoning benchmarks demonstrate that SPEE consistently outperforms both test-time and training-time self-evolution baselines across three model scales. The source code is available at https://github.com/rrrsj/SPEE.

Figures

Figures reproduced from arXiv: 2608.02139 by the authors.

Figure 1
Figure 1. Overview of SPEE. Interaction trajectories are used to evolve a global experience pool, whose distilled knowledge [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (Left) Proportion of problems for which all eight [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Accuracy under different numbers of experience [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: A representative case illustrating the effect of experience-guided reasoning. Without experience, the model constructs [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 4 canonical work pages

  1. [3]

    InProceedings of the AAAI conference on artificial intelligence, volume 38, 17682–17690

    Graph of thoughts: Solving elaborate problems with large language models. InProceedings of the AAAI conference on artificial intelligence, volume 38, 17682–17690. Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse,C.;andSchulman,J.2021. TrainingVerifierstoSolve Math Word Problems. ...

    arXiv 2021

  2. [7]

    arXiv:2503.09516

    Search-R1: Training LLMs toReasonandLeverageSearchEngineswithReinforcement Learning. arXiv:2503.09516. Kaplan, J.; McCandlish, S.; Henighan, T.; Brown, T. B.; Chess, B.; Child, R.; Gray, S.; Radford, A.; Wu, J.; and Amodei, D

    arXiv

  3. [10]

    Liu,N.F.;Lin,K.;Hewitt,J.;Paranjape,A.;Bevilacqua,M.; Petroni, F.; and Liang, P

    Let’s Verify Step by Step.arXiv preprint arXiv:2305.20050. Liu,N.F.;Lin,K.;Hewitt,J.;Paranjape,A.;Bevilacqua,M.; Petroni, F.; and Liang, P

    arXiv

  4. [11]

    arXiv:2401.08967

    ReFT: Reasoning with Reinforced Fine-Tuning. arXiv:2401.08967. Mohamed, A.; El Rashid, M.; and Shaalan, K

    arXiv

  5. [12]

    Mexico City, Mexico: Association for Compu- tational Linguistics

    In- contextLearningGeneralizes,ButNotAlwaysRobustly:The CaseofSyntax.InDuh,K.;Gomez,H.;andBethard,S.,eds., Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 4761–4779. Mexico City, Mexico: Association for Compu- tational Linguistics. Ouyan...

    work page 2024

  6. [13]

    arXiv:2602.04942

    Privileged Information Distil- lation for Language Models. arXiv:2602.04942. Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D

    arXiv

  7. [14]

    arXiv:2402.03300

    DeepSeekMath: Pushing the Limits of Mathematical Rea- soning in Open Language Models. arXiv:2402.03300. Shi, T.; Huang, C.; Li, B.; Chen, X.; Quan, X.; Wang, J.; and Wang, Q

    arXiv

  8. [15]

    Beyond Trajectory Imitation: Strategy-Guided Policy Optimization for LLM Reasoning

    Beyond Trajectory Imitation: Strategy-Guided Policy Optimization for LLM Reasoning. arXiv:2606.24064. Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S

    work page Pith review arXiv

Show all 25 references
  1. [16]

    arXiv:2009.01325

    Learning to summarize from human feedback. arXiv:2009.01325. Tao, Z.; Lin, T.; Chen, X.; Li, H.; Wu, Y.; Li, Y.; Jin, Z.; Huang, F.; Tao, D.; and Zhou, J

    2009 arXiv

  2. [17]

    A survey on self-evolution of large language models. arXiv. 240414387 (2024). Team,K.;Du,A.;Gao,B.;Xing,B.;Jiang,C.;Chen,C.;Li, C.; Xiao, C.; Du, C.; Liao, C.; Tang, C.; Wang, C.; Zhang, D.; Yuan, E.; Lu, E.; Tang, F.; Sung, F.; Wei, G.; Lai, G.; Guo, H.; Zhu, H.; Ding, H.; Hu...

    2024

  3. [18]

    arXiv:2501.12599

    Kimi k1.5: Scaling Reinforcement Learning with LLMs. arXiv:2501.12599. Wang, F.; Ma, Y.; Guan, T.; Wang, Y.; and Chen, J

    arXiv

  4. [19]

    In2026 International Conference on Embedded Systems, Mobile Communication and Computing (EMC²), 301–306

    Autonomous Learning through Self-Driven Explorationand Knowledge Structuring for Open-World Intelligent Agents. In2026 International Conference on Embedded Systems, Mobile Communication and Computing (EMC²), 301–306. Wang, G.; Xie, Y.; Jiang, Y.; Mandlekar, A.; Xiao, C.; Zhu, ...

    arXiv

  5. [20]

    arXiv preprint arXiv:2203.11171

    Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. Wang,X.;Wei,J.;Schuurmans,D.;Le,Q.;Chi,E.;Narang, S.; Chowdhery, A.; and Zhou, D. 2023b. Self-Consistency ImprovesChainofThoughtReasoninginLanguageModels. arXiv:2203.11171...

    arXiv

  6. [21]

    arXiv:2504.20073

    RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning. arXiv:2504.20073. Wei,J.;Wang,X.;Schuurmans,D.;Bosma,M.;Xia,F.;Chi, E.;Le,Q.V.;Zhou,D.;etal.2022.Chain-of-thoughtprompt- ing elicits reasoning in large language models.Advances in neural i...

    2022 arXiv

  7. [22]

    Yu, Q.; Zhang, Z.; Zhu, R.; Yuan, Y.; Zuo, X.; Yue, Y.; Dai, W.; Fan, T.; Liu, G.; Liu, L.; et al

    React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629. Yu, Q.; Zhang, Z.; Zhu, R.; Yuan, Y.; Zuo, X.; Yue, Y.; Dai, W.; Fan, T.; Liu, G.; Liu, L.; et al

    arXiv

  8. [23]

    Expel:Llmagentsareexperientiallearners

    Zhao,A.;Huang,D.;Xu,Q.;Lin,M.;Liu,Y.-J.;andHuang, G.2024. Expel:Llmagentsareexperientiallearners. InPro- ceedings of the AAAI Conference on Artificial Intelligence, volume 38, 19632–19642. Zhao,S.;Xie,Z.;Liu,M.;Huang,J.;Pang,G.;Chen,F.;and Grover, A

    2024

  9. [24]

    arXiv:2601.18734

    Self-Distilled Reasoner: On-Policy Self- Distillation for Large Language Models. arXiv:2601.18734. Ziegler, D. M.; Stiennon, N.; Wu, J.; Brown, T. B.; Rad- ford, A.; Amodei, D.; Christiano, P.; and Irving, G

    arXiv

  10. [25]

    arXiv:1909.08593

    Fine-Tuning Language Models from Human Preferences. arXiv:1909.08593

    1909 arXiv

  11. [2015]

    arXiv:1503.02531

    Distilling the Knowledge in a Neural Network. arXiv:1503.02531. Hübotter, J.; Lübeck, F.; Behric, L. D.; Baumann, A.; Bagatella, M.; Marta, D.; Hakimi, I.; Shenfeld, I.; Buening, T. K.; Guestrin, C.; and Krause, A

    arXiv

  12. [2020]

    arXiv preprint arXiv:2001.08361

    Scaling laws for neural language models. arXiv preprint arXiv:2001.08361. Le, H.; Wang, Y.; Gotmare, A. D.; Savarese, S.; and Hoi, S. C. H

    2001 arXiv

  13. [2022]

    arXiv:2207.01780

    CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learn- ing. arXiv:2207.01780. Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal,N.;Küttler,H.;Lewis,M.;Yih,W.-t.;Rocktäschel,T.; Riedel, S.; and Kiela, D

    arXiv

  14. [2023]

    Agarwal,R.;Vieillard,N.;Zhou,Y.;Stanczyk,P.;Ramos,S.; Geist, M.; and Bachem, O

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774. Agarwal,R.;Vieillard,N.;Zhou,Y.;Stanczyk,P.;Ramos,S.; Geist, M.; and Bachem, O

    arXiv

  15. [2024]

    arXiv:2306.13649

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. arXiv:2306.13649. Besta, M.; Blach, N.; Kubicek, A.; Gerstenberger, R.; Podstawski, M.; Gianinazzi, L.; Gajda, J.; Lehmann, T.; Niewiadomski, H.; Nyczyk, P.; et al

    arXiv

  16. [2025]

    Hinton, G.; Vinyals, O.; and Dean, J

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948. Hinton, G.; Vinyals, O.; and Dean, J

    arXiv

  17. [2026]

    arXiv:2306.08543

    MiniLLM: On-Policy Distillation of Large Language Mod- els. arXiv:2306.08543. Guo, D.; Yang, D.; Zhang, H.; Song, J.; Wang, P.; Zhu, Q.; Xu, R.; Zhang, R.; Ma, S.; Bi, X.; et al

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.