Pith. sign in

REVIEW 3 major objections 5 minor 58 references

Instruction-conditioned exploration plus self-distillation lifts a 1.7B model's held-out math pass@1 by 5.0% relative to a standard RL baseline.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 15:19 UTC pith:ZC3B3HAI

load-bearing objection Honest, useful paper with a real small-model effect that is probably smaller and less certain than the headline CI suggests. the 3 major comments →

arxiv 2608.02087 v1 pith:ZC3B3HAI submitted 2026-08-03 cs.AI cs.CLcs.LG

Instruction-Conditioned Exploration with Asymmetric Reinforcement Learning and Self-Distillation

Jim Dilkes , Vahid Yazdanpanah , Sebastian Stein This is my paper
classification cs.AI cs.CLcs.LG
keywords instruction-conditioned explorationasymmetric RL self-distillationLLM post-trainingmathematical reasoningsupport expansionpass@1small language modelsreinforcement learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a small language model's reinforcement-learning post-training can be made to explore more useful solution strategies by appending one of a set of five generic behavioural instructions to the task prompt during training, and that those explored behaviours can be transferred to the ordinary instruction-free policy through self-distillation. The authors claim that on a 1.7B model solving held-out math problems, this combined method raises pass@1 by 5.0% relative to a standard RL-only baseline, with all five benchmark wins and a 95% seed-paired confidence interval excluding zero. The gain persists, more weakly, at a longer 8K response limit and costs nothing extra at inference time, since the deployed student is prompted exactly like a standard model. A sympathetic reader would care because it offers a cheap, training-time-only route to better reasoning in small models, which matters for phone and laptop deployment.

Core claim

At training time, the model is asked to solve each problem under one of several behavioural instructions (e.g., work backwards, decompose the problem), while a subset of rollouts keeps the raw prompt; the union of these instruction-conditioned distributions is the exploration policy. The authors' discovery is that optimising this composite policy with an RL objective, while simultaneously using a forward-KL term to distil only the correct teacher rollouts into the shared-parameter instruction-free student, transfers the exploration gain to test time. On Qwen3-1.7B at 4K response length, this yields a 5.0% relative held-out pass@1 improvement over the DAPO baseline with 5/5 benchmark wins and

What carries the argument

The machinery is a two-part objective. First, ICE defines a composite policy πcomp = Σ p(I)πθ(·|x,I) whose support is the union of the supports of the instruction-conditioned policies; sampling instructions during training broadens the behaviours available for the RL signal, counteracting the support shrinkage typical of RL post-training. Second, Asymmetric-RL/SD shares parameters between an instruction-conditioned teacher and an unconditioned student, trains the teacher with DAPO/GRPO reward maximisation, and trains the student with a forward-KL divergence toward correctness-filtered teacher rollouts (reward 1 only). The forward-KL on filtered samples is the explicit transfer mechanism: it

Load-bearing premise

The load-bearing premise is that the forward-KL distillation of correctness-filtered instruction-conditioned rollouts into the shared-parameter unconditioned student is what transfers the training-time exploration to test time; if that distillation term is not doing the transfer work, the headline gain may come from the instruction-conditioned RL signal alone.

What would settle it

Train the same 1.7B/4K configuration with the distillation term disabled (teacher-only RL, α=1, βS=0) across at least five seeds: if its held-out pass@1 gap versus the full Asymmetric-RL/SD method shrinks to within the seed-paired 95% confidence interval of zero, the claimed distillation transfer mechanism is not what carries the result.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If correct, the method gives a training-time-only gain: the deployed model is the unconditioned student, so the 5.0% improvement at 1.7B/4K carries no extra inference cost.
  • The gain persists, though more weakly, at 1.7B/8K (+3.5%), suggesting the mechanism is not specific to the 4K response length.
  • Adding the forward-KL distillation improves the mean held-out pass@1 over teacher-only RL by +0.008, but the 95% CI spans zero, so the distillation term's independent contribution is uncertain.
  • At 4B/4K the method does not improve over the baseline (-3.3% point estimate), so the claimed benefit is presently demonstrated only at the 1.7B scale.
  • The pass@k results indicate RL post-training trades coverage for accuracy; the method improves pass@1 at comparable coverage, and sampling with the behavioural instructions at inference restores high-k coverage.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own pass@k data show the behavioural instructions lift even the untrained base model's coverage (0.528 vs 0.513 at k=256), which suggests the instruction set encodes useful strategies independent of RL; a natural extension is to use the same instructions for inference-time ensembling or self-consistency, which the paper does not test.
  • The paper attributes the lack of gain at 4B/4K to reduced headroom, but with only two completed seeds the scale-dependence claim is tentative; an extension would run more seeds and test 4B at 8K.
  • Because the instructions are fixed and untargeted, a dynamic variant that selects or generates instructions per problem (e.g., from the model's own failed attempts) could improve sample efficiency; the paper flags this as motivation but leaves it untested.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Instruction-Conditioned Exploration (ICE), which during RL post-training supplements a task prompt with one of several behavioural instructions to broaden the support of rollouts, and Asymmetric-RL/SD, a self-distillation objective that combines a teacher-only DAPO reward term with a forward-KL term pulling the unconditioned student toward correctness-filtered teacher rollouts. The central empirical claim is that on Qwen3-1.7B at 4K response length, ICE with Asymmetric-RL/SD improves held-out pass@1 on five mathematical reasoning benchmarks by 5.0% relative to DAPO, with a 95% seed-paired bootstrap CI excluding zero ([+0.006,+0.016]) and 5/5 benchmark wins across five seeds. The paper also reports a smaller gain at 1.7B/8K and no gain at 4B/4K. The stated mechanism is that the forward-KL distillation term transfers instruction-conditioned behaviours to the unconditioned test-time policy.

Significance. If the headline result holds, the method is practically valuable: it is a training-time-only intervention that improves a small model's reasoning without any additional inference cost, and the paper is appropriately careful in several respects. The authors report multi-seed results, a seed-paired bootstrap, per-benchmark win counts, training-probe-based checkpoint selection, and explicit limitations. They also release the instruction set and code, which aids reproducibility. However, the two most load-bearing points are not fully established: the headline CI is computed after model selection on the same evaluation benchmarks, and the ablation isolating the self-distillation term has a CI spanning zero. These issues do not invalidate the idea, but they mean the paper's central robustness claim and its stated mechanism are currently weaker than the abstract and discussion suggest.

major comments (3)
  1. [§6.1, Table 7; §6.2, Table 3] The headline estimate is selected, not confirmatory. The configuration βS=0.1 was chosen from a single-seed sweep over at least ten variants (Table 7) evaluated on the same five held-out benchmarks used in the headline result, and the five-seed bootstrap CI [+0.006,+0.016] is conditional on that choice. The paper's Limitations acknowledge this and call it 'mildly optimistic', but the observed shrinkage — Asymmetric-RL/SD from +0.017 (single seed) to +0.012 (five seeds), and α=1 from +0.029 to +0.005 — shows the selection effect is material rather than mild. The CI therefore does not account for the multiplicity of configurations examined, and the claim that the 5.0% relative improvement is robust is not established by the reported interval. A corrected protocol is needed: for example, select hyperparameters on DAPO-200 or a held-out split that is never used for the final evaluation, or r
  2. [§4, Eqs. (3)–(4); §6.2, Table 3] The claimed transfer mechanism is not supported by the paper's own ablation. Adding the forward-KL distillation term to α=1 (i.e., Asymmetric-RL/SD vs. α=1) gives +0.008 with 95% CI [−0.003,+0.016], while α=1 alone is +0.005 with CI [−0.006,+0.018] versus DAPO. Thus the data are consistent with the distillation term adding nothing beyond the instruction-conditioned RL signal, and the joint gain could arise from an interaction between instruction conditioning and RL rather than from self-distillation. The Discussion's statement that 'the explicit distillation signal contributes additional transfer' (§7) overstates the evidence. Either additional seeds must narrow the ablation CI to exclude zero, or the transfer mechanism should be presented as a conjecture rather than an established component of the method.
  3. [§6.3, Table 4] The 8K persistence claim rests on only three seeds: Asymmetric-RL/SD at 1.7B/8K is +0.010 (+3.5%) with 3/5 wins, which is not a robust margin. The abstract's phrase 'with improvement persisting at a longer 8K context' should be qualified, and the 4B/4K negative result is based on n=2 completed seeds for Asymmetric-RL/SD. These points are partially acknowledged in the Limitations, but they should be reflected in the abstract and conclusion, which currently state persistence more strongly than the data support.
minor comments (5)
  1. [§6.2, Table 3 caption] The table reports '95% CI' as a seed-paired percentile bootstrap on Δ. This is an uncertainty estimate conditional on the selected configuration, not a significance test; the text should say this explicitly at the point of first use, not only in the experimental-setup paragraph.
  2. [Table 2] The row 'α=0.5, β=0.1' is ambiguous: Table 1 lists βT and βS separately, and Table 7 uses 'β' for the joint-blend configuration. Please clarify whether β=0.1 means βT=βS=0.1.
  3. [§6.4, Table 5 and Figure 3] The pass@k coverage analysis is based on a single probe-best checkpoint and reports no variance across seeds or checkpoints. Given the small benchmark sizes, this makes the coverage inversion and the '+ICE recovers coverage' claim more fragile than the prose suggests. Reporting standard errors or a second checkpoint would help.
  4. [§9, Limitations] The limitation that 'the selection seed inflates their headline estimates' is stated only qualitatively. Since this directly affects the central claim, it should be moved to or repeated in the Results section, and ideally quantified by comparing single-seed-selected estimates with the later multi-seed averages.
  5. [§7, Discussion] The sentence 'Another possible effect is that because student and teacher share parameters, the KL-driven updates also affect the teacher's subsequent rollouts, producing a compounding effect' is a useful hypothesis, but it is not tested. Please label it explicitly as speculative.

Circularity Check

1 steps flagged

Headline gain is partly a post-selection artifact: the winning configuration was chosen from a single-seed sweep on the same held-out benchmarks, and the reported 95% CI does not account for this selection.

specific steps
  1. fitted input called prediction [Section 9 (Limitations); §6.1 Table 7; §6.2 Table 3]
    "These configurations were also chosen from a single-seed sweep, so the selection seed inflates their headline estimates, which are therefore mildly optimistic."

    The headline configuration (α=1, βS=0.1) was selected from a single-seed sweep (Table 7) using the same held-out macro-mean pass@1 that defines the headline result (Table 3). The 95% seed-paired bootstrap CI in Table 3 is computed within the selected configuration after selection; it does not condition on having examined at least 12 variants and choosing the best on the same evaluation metric. The paper's own Limitations state that 'the selection seed inflates their headline estimates.' Thus the reported +5.0% relative improvement is not an independent prediction; it is a post-selection maximum (or a shrunk version of one), so the CI as reported does not support the headline as a robust finding.

full rationale

No self-citation chain or definitional identity makes the method's objective equivalent to its inputs. Equation (3)-(4) are a well-defined combination of RL and forward/reverse KL terms, DAPO and π-distill are external baselines/priors, and instruction selection is performed on training-set subsets rather than on the evaluation benchmarks. The only circularity-adjacent issue is post-selection inference: βS=0.1 and the α=1 configuration were chosen from a single-seed sweep evaluated on the same five held-out benchmarks that define the headline. The paper honestly discloses this, but the disclosed 'mild optimism' is material because the single-seed estimates shrank substantially at higher n (α=1 from +0.029 to +0.005; Asymmetric-RL/SD from +0.017 to +0.012), and the ablation isolating the distillation term has a CI spanning zero. This makes the headline 95% CI conditional on selection rather than a valid unconditional confidence statement, but it does not reduce the whole derivation to a tautology. Score 4 reflects one partially-circular selection step; the central mechanism retains independent empirical content.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

The central claim rests on the DAPO/GRPO machinery (borrowed), on the transferability of correctness-filtered forward-KL distillation into a shared-parameter student, on the generality of an instruction set selected from training-distribution coverage, and on the binary-reward math benchmark setup. One hyperparameter (βS) and the instruction texts are tuned artifacts of the reported experiments.

free parameters (4)
  • βS (student forward-KL weight) = 0.1
    Tuned by single-seed sweep over {0.05, 0.1, 0.5}; 0.5 degraded the result by 14.0%, 0.1 was chosen. The headline estimate inherits this selection.
  • α (teacher reward weight) = 1.0
    α<1 variants caused response-length blow-up and truncation-rate increases (0.616 at α=0); set to teacher-only by inspection of single-seed sweeps.
  • Instruction set I (N=5 texts) = Table 6 texts
    Chosen by greedy pass@16 set cover over candidate strategies generated by Claude models, on a 250-problem held-out subset of the training set; the texts are tuned artifacts of the training distribution.
  • Evaluation-step selection (DAPO-200 probe-best) = step with max probe reward per run
    Held-out scores are reported at the probe-best step; applied identically across methods but inflates absolute numbers and can interact with overfitting.
axioms (5)
  • standard math DAPO/GRPO is an appropriate and correct RL update for LLM post-training
    Borrowed without re-derivation from Yu et al. 2025; all methods and the baseline share it.
  • domain assumption Forward-KL to correctness-filtered teacher rollouts transfers exploration gains to the shared-parameter unconditioned student
    Core mechanism of Asymmetric-RL/SD (§4); the paper's own ablation shows its marginal contribution is +0.008 with CI [-0.003,+0.016] (Table 3), so this premise is weakly supported.
  • domain assumption Instructions selected on a training-distribution subset remain useful on the held-out benchmarks
    Selection used 250 held-out problems from DAPO-17K-en (Appendix A); evaluation is on AIME/HMMT/MATH-500.
  • domain assumption Binary correctness reward on the extracted answer is a sufficient training signal
    Standard RLVR setup; answer extraction per DAPO. Untested on other rewards or domains.
  • ad hoc to paper 5/8 instruction-conditioned plus 3/8 unconditioned rollouts per group is a fair allocation
    Per-group allocation chosen in §5 ('one rollout per instruction, remaining 3 unconditioned'); no ablation of the allocation ratio.
invented entities (1)
  • Behavioural instruction set I (N=5): work_backwards, constructive_building, symmetry_and_invariance, decomposition, reduction independent evidence
    purpose: Training-time prompt augmentation to broaden rollout support (ICE)
    Released for research use; independently shown to lift even the untrained base model's pass@256 from 0.513 to 0.528 (§6.4), so the artifacts have falsifiable handles outside the paper.

pith-pipeline@v1.3.0-daily-deepseek · 15777 in / 21699 out tokens · 131641 ms · 2026-08-04T15:19:12.974821+00:00 · methodology

0 comments
read the original abstract

Post-training Large Language Models (LLMs) with Reinforcement Learning (RL) has become an important tool for improving model capabilities, but the LLM action-space structure introduces challenges distinct from classical RL, with implications for inducing exploration. New methods are required that leverage the broad knowledge and flexibility of pre-trained LLMs to deliberately generate diverse experience at training time. We propose Instruction-Conditioned Exploration (ICE), which supplements task prompts during training with one of several distinct instructions, increasing the coverage of behaviours attempted. To facilitate ICE, we propose Asymmetric-RL/SD, a combined Reinforcement Learning and Self-Distillation training objective, to transfer explored behaviours to the unconditioned test-time policy. ICE with the Asymmetric-RL/SD objective improves Qwen3-1.7B held-out pass@1 performance at $4$K response length on mathematical reasoning tasks by $5.0\%$ relative to training with DAPO, with improvement persisting at a longer 8K context.

Figures

Figures reproduced from arXiv: 2608.02087 by Jim Dilkes, Sebastian Stein, Vahid Yazdanpanah.

Figure 1
Figure 1. Figure 1: Overview. (1) ICE supplements the task prompt x with one of several behavioural instructions I per rollout, broadening the support available to ex￾plore at training time. (2) Asymmetric-RL/SD optimises instruction-conditioned teacher rollouts with reinforce￾ment learning, with simultaneous self-distillation of the correct teacher trajectories into the unconditioned stu￾dent, which is the only policy used a… view at source ↗
Figure 2
Figure 2. Figure 2: DAPO-200 probe pass@1 during training at 1.7B / 4k. Curves are means across n=5 seeds per method. 6.3 Replication at longer response length and larger model scale We replicate Asymmetric-RL/SD at 1.7B/8K and 4B/4K, and α = 1 at 4B/4K ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Held-out pass@k at 1.7B (macro-mean over AIME-24/25 and HMMT-Feb/Nov) for Base, DAPO, and Asymmetric-RL/SD, sampled uninstructed (solid) and with the behavioural instructions (+ICE, dashed). RL post-training leads at low k but trails the base model’s coverage at high k, which +ICE recovers. Per￾benchmark breakdown in Appendix G. same checkpoint with the behavioural instructions (+ICE) recovers most of this… view at source ↗
Figure 5
Figure 5. Figure 5: α-sweep training dynamics at 1.7B / 4k from a single training seed per method. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    Nudging the

    Chen, Justin Chih-Yao and Peng, Becky Xiangyu and Choubey, Prafulla Kumar and Huang, Kung-Hsiang and Zhang, Jiaxin and Bansal, Mohit and Wu, Chien-Sheng , month = sep, year =. Nudging the. doi:10.48550/arXiv.2509.25666 , abstract =

    doi:10.48550/arxiv.2509.25666

  2. [2]

    doi:10.48550/arXiv.2402.19446 , abstract =

    Zhou, Yifei and Zanette, Andrea and Pan, Jiayi and Levine, Sergey and Kumar, Aviral , month = feb, year =. doi:10.48550/arXiv.2402.19446 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.19446

  3. [3]

    doi:10.48550/arXiv.2406.04151 , abstract =

    Xi, Zhiheng and Ding, Yiwen and Chen, Wenxiang and Hong, Boyang and Guo, Honglin and Wang, Junzhe and Yang, Dingwen and Liao, Chenyang and Guo, Xin and He, Wei and Gao, Songyang and Chen, Lu and Zheng, Rui and Zou, Yicheng and Gui, Tao and Zhang, Qi and Qiu, Xipeng and Huang, Xuanjing and Wu, Zuxuan and Jiang, Yu-Gang , month = jun, year =. doi:10.48550/a...

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.04151

  4. [4]

    doi:10.48550/arXiv.2405.16376 , abstract =

    Li, Chuanhao and Yang, Runhan and Li, Tiankai and Bafarassat, Milad and Sharifi, Kourosh and Bergemann, Dirk and Yang, Zhuoran , month = may, year =. doi:10.48550/arXiv.2405.16376 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.16376

  5. [5]

    Bhargava, Aman and Witkowski, Cameron and Detkov, Alexander and Thomson, Matt , month = sep, year =. Prompt. doi:10.48550/arXiv.2409.13697 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2409.13697

  6. [6]

    doi:10.48550/arXiv.2511.10395 , abstract =

    Zhai, Yunpeng and Tao, Shuchang and Chen, Cheng and Zou, Anni and Chen, Ziqian and Fu, Qingxu and Mai, Shinji and Yu, Li and Deng, Jiaji and Cao, Zouying and Liu, Zhaoyang and Ding, Bolin and Zhou, Jingren , month = nov, year =. doi:10.48550/arXiv.2511.10395 , abstract =

    doi:10.48550/arxiv.2511.10395

  7. [7]

    Zhao, Siyan and Xie, Zhihui and Liu, Mengchen and Huang, Jing and Pang, Guan and Chen, Feiyu and Grover, Aditya , month = jan, year =. Self-. doi:10.48550/arXiv.2601.18734 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2601.18734

  8. [8]

    Shenfeld, Idan and Damani, Mehul and Hübotter, Jonas and Agrawal, Pulkit , month = jan, year =. Self-. doi:10.48550/arXiv.2601.19897 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2601.19897

  9. [9]

    Reinforcement

    Hübotter, Jonas and Lübeck, Frederike and Behric, Lejs and Baumann, Anton and Bagatella, Marco and Marta, Daniel and Hakimi, Ido and Shenfeld, Idan and Buening, Thomas Kleine and Guestrin, Carlos and Krause, Andreas , month = feb, year =. Reinforcement. doi:10.48550/arXiv.2601.20802 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2601.20802

  10. [10]

    Privileged

    Penaloza, Emiliano and Vattikonda, Dheeraj and Gontier, Nicolas and Lacoste, Alexandre and Charlin, Laurent and Caccia, Massimo , month = feb, year =. Privileged. doi:10.48550/arXiv.2602.04942 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2602.04942

  11. [11]

    doi:10.48550/arXiv.2405.11143 , publisher =

    Hu, Jian and Wu, Xibin and Shen, Wei and Liu, Jason Klein and Zhu, Zilin and Wang, Weixun and Jiang, Songlin and Wang, Haoran and Chen, Hao and Chen, Bin and Fang, Weikai and Xianyu and Cao, Yu and Xu, Haotian and Liu, Yiming , year =. doi:10.48550/arXiv.2405.11143 , publisher =

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.11143

  12. [12]

    doi:10.48550/arXiv.2504.20073 , abstract =

    Wang, Zihan and Wang, Kangrui and Wang, Qineng and Zhang, Pingyue and Li, Linjie and Yang, Zhengyuan and Yu, Kefan and Nguyen, Minh Nhat and Liu, Licheng and Gottlieb, Eli and Lam, Monica and Lu, Yiping and Cho, Kyunghyun and Wu, Jiajun and Fei-Fei, Li and Wang, Lijuan and Choi, Yejin and Li, Manling , month = apr, year =. doi:10.48550/arXiv.2504.20073 , ...

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.20073

  13. [13]

    and Hu, Zhiyuan and Yang, Mengyue and Wen, Ying and Wang, Jun , month = may, year =

    Feng, Xidong and Liu, Bo and Song, Yan and Fu, Haotian and Wan, Ziyu and Koushik, Girish A. and Hu, Zhiyuan and Yang, Mengyue and Wen, Ying and Wang, Jun , month = may, year =. Natural. doi:10.48550/arXiv.2411.14251 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2411.14251

  14. [14]

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and Zhang, Xiaokang and Yu, Xingkai and Wu, Yu and Wu, Z. F. and Gou, Zhibin and Shao, Zhihong and Li, Zhuoshu and Gao, Ziyi and Liu, Aixin and Xue, Bing and Wang, Bingxuan and Wu, Bochao and Feng, Bei ...

  15. [15]

    Shao, Zhihong and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Song, Junxiao and Bi, Xiao and Zhang, Haowei and Zhang, Mingchuan and Li, Y. K. and Wu, Y. and Guo, Daya , month = apr, year =. doi:10.48550/arXiv.2402.03300 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.03300

  16. [16]

    Cui, Ganqu and Zhang, Yuchen and Chen, Jiacheng and Yuan, Lifan and Wang, Zhi and Zuo, Yuxin and Li, Haozhan and Fan, Yuchen and Chen, Huayu and Chen, Weize and Liu, Zhiyuan and Peng, Hao and Bai, Lei and Ouyang, Wanli and Cheng, Yu and Zhou, Bowen and Ding, Ning , month = may, year =. The. doi:10.48550/arXiv.2505.22617 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.22617

  17. [17]

    Yue, Yang and Chen, Zhiqi and Lu, Rui and Zhao, Andrew and Wang, Zhaokai and Yue, Yang and Song, Shiji and Huang, Gao , month = nov, year =. Does. doi:10.48550/arXiv.2504.13837 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.13837

  18. [18]

    doi:10.48550/arXiv.2604.06268 , abstract =

    Wang, Zihan and Gui, Chi and Jin, Xing and Wang, Qineng and Liu, Licheng and Wang, Kangrui and Chen, Shiqi and Li, Linjie and Yang, Zhengyuan and Zhang, Pingyue and Lu, Yiping and Wu, Jiajun and Fei-Fei, Li and Wang, Lijuan and Choi, Yejin and Li, Manling , month = apr, year =. doi:10.48550/arXiv.2604.06268 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.06268

  19. [19]

    Beyond the 80/20

    Wang, Shenzhi and Yu, Le and Gao, Chang and Zheng, Chujie and Liu, Shixuan and Lu, Rui and Dang, Kai and Chen, Xionghui and Yang, Jianxin and Zhang, Zhenru and Liu, Yuqiong and Yang, An and Zhao, Andrew and Yue, Yang and Song, Shiji and Yu, Bowen and Huang, Gao and Lin, Junyang , month = jun, year =. Beyond the 80/20. doi:10.48550/arXiv.2506.01939 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.01939

  20. [20]

    Shen, Han , month = sep, year =. On. doi:10.48550/arXiv.2509.03493 , abstract =

    doi:10.48550/arxiv.2509.03493

  21. [21]

    Rewarding the

    He, Andre Wang and Fried, Daniel and Welleck, Sean , editor =. Rewarding the. Proceedings of the 2025. 2025 , pages =. doi:10.18653/v1/2025.emnlp-main.1298 , abstract =

    doi:10.18653/v1/2025.emnlp-main.1298 2025

  22. [22]

    Zhang, Junshuo , month = apr, year =

  23. [23]

    Outcome-based

    Song, Yuda and Kempe, Julia and Munos, Remi , month = sep, year =. Outcome-based. doi:10.48550/arXiv.2509.06941 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2509.06941

  24. [24]

    Diversity-

    Hu, Zican and Zhang, Shilin and Li, Yafu and Yan, Jianhao and Hu, Xuyang and Cui, Leyang and Qu, Xiaoye and Chen, Chunlin and Cheng, Yu and Wang, Zhi , month = feb, year =. Diversity-. doi:10.48550/arXiv.2509.26209 , abstract =

    doi:10.48550/arxiv.2509.26209

  25. [25]

    doi:10.48550/arXiv.2510.16552 , abstract =

    Li, Ang and Wang, Yifei and Yuan, Zhihang and Jegelka, Stefanie and Wang, Yisen , month = oct, year =. doi:10.48550/arXiv.2510.16552 , abstract =

    doi:10.48550/arxiv.2510.16552

  26. [26]

    Zhang, Xichen and Wu, Sitong and Zhu, Yinghao and Tan, Haoru and Yu, Shaozuo and He, Ziyi and Jia, Jiaya , month = feb, year =. Scaf-. doi:10.48550/arXiv.2510.19807 , abstract =

    doi:10.48550/arxiv.2510.19807

  27. [27]

    Lu, Wenquan and Huang, Hai and Balestriero, Randall , month = feb, year =. Prompt. doi:10.48550/arXiv.2602.03190 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2602.03190

  28. [28]

    Expanding

    Szot, Andrew and Kirchhof, Michael and Attia, Omar and Toshev, Alexander , month = mar, year =. Expanding. doi:10.48550/arXiv.2603.02045 , abstract =

    doi:10.48550/arxiv.2603.02045

  29. [29]

    Breaking the

    Zhou, Yang and Li, Sunzhu and Liu, Shunyu and Fang, Wenkai and Zhang, Kongcheng and Zhao, Jiale and Yang, Jingwen and Zhou, Yihe and Lv, Jianwei and Zheng, Tongya and Lu, Hengtong and Chen, Wei and Xie, Yan and Song, Mingli , month = jan, year =. Breaking the. doi:10.48550/arXiv.2508.16949 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.16949

  30. [30]

    Liao, Baohao and Dong, Hanze and Xu, Xinxing and Monz, Christof and Bian, Jiang , month = feb, year =. Self-. doi:10.48550/arXiv.2602.03143 , abstract =

    doi:10.48550/arxiv.2602.03143

  31. [31]

    doi:10.48550/arXiv.2503.14476 , abstract =

    Yu, Qiying and Zhang, Zheng and Zhu, Ruofei and Yuan, Yufeng and Zuo, Xiaochen and Yue, Yu and Dai, Weinan and Fan, Tiantian and Liu, Gaohong and Liu, Lingjun and Liu, Xin and Lin, Haibin and Lin, Zhiqi and Ma, Bole and Sheng, Guangming and Tong, Yuxuan and Zhang, Chi and Zhang, Mofan and Zhang, Wang and Zhu, Hang and Zhu, Jinhua and Chen, Jiaze and Chen,...

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.14476

  32. [32]

    and Barto, Andrew G

    Sutton, Richard S. and Barto, Andrew G. , month = oct, year =. Reinforcement

  33. [33]

    Yang, An and Li, Anfeng and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Gao, Chang and Huang, Chengen and Lv, Chenxu and Zheng, Chujie and Liu, Dayiheng and Zhou, Fan and Huang, Fei and Hu, Feng and Ge, Hao and Wei, Haoran and Lin, Huan and Tang, Jialong and Yang, Jian and Tu, Jianhong and Zhang, Jianwei and Yang, Jia...

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.09388

  34. [34]

    American

    Zhang, Yifan and Math-AI, Team , year =. American

  35. [35]

    Measuring

    Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob , month = nov, year =. Measuring. doi:10.48550/arXiv.2103.03874 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2103.03874

  36. [36]

    doi:10.48550/arXiv.2505.23281 , abstract =

    Balunović, Mislav and Dekoninck, Jasper and Petrov, Ivo and Jovanović, Nikola and Vechev, Martin , month = jan, year =. doi:10.48550/arXiv.2505.23281 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.23281

  37. [37]

    Chen, Zhipeng and Qin, Xiaobo and Wu, Youbin and Ling, Yue and Ye, Qinghao and Zhao, Wayne Xin and Shi, Guang , month = aug, year =. Pass@k. doi:10.48550/arXiv.2508.10751 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.10751

  38. [38]

    Zhang, Charlie and Neubig, Graham and Yue, Xiang , month = dec, year =. On the. doi:10.48550/arXiv.2512.07783 , abstract =

    doi:10.48550/arxiv.2512.07783

  39. [39]

    Liang, Zhenwen and Lu, Sidi and Yu, Wenhao and Panaganti, Kishan and Zhou, Yujun and Mi, Haitao and Yu, Dong , month = dec, year =. Can. doi:10.48550/arXiv.2512.15687 , abstract =

    doi:10.48550/arxiv.2512.15687

  40. [40]

    Madaan, Aman and Tandon, Niket and Gupta, Prakhar and Hallinan, Skyler and Gao, Luyu and Wiegreffe, Sarah and Alon, Uri and Dziri, Nouha and Prabhumoye, Shrimai and Yang, Yiming and Gupta, Shashank and Majumder, Bodhisattwa Prasad and Hermann, Katherine and Welleck, Sean and Yazdanbakhsh, Amir and Clark, Peter , month = may, year =. Self-. doi:10.48550/ar...

    Pith/arXiv arXiv doi:10.48550/arxiv.2303.17651

  41. [41]

    and PENG, JING , month = jan, year =

    WILLIAMS, RONALD J. and PENG, JING , month = jan, year =. Function. Connection Science , publisher =. doi:10.1080/09540099108946587 , abstract =

    doi:10.1080/09540099108946587

  42. [42]

    Reflexion:

    Shinn, Noah and Cassano, Federico and Berman, Edward and Gopinath, Ashwin and Narasimhan, Karthik and Yao, Shunyu , month = oct, year =. Reflexion:. doi:10.48550/arXiv.2303.11366 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2303.11366

  43. [43]

    Language models can solve computer tasks , abstract =

    Kim, Geunwoo and Baldi, Pierre and McAleer, Stephen , month = dec, year =. Language models can solve computer tasks , abstract =. Proceedings of the 37th

  44. [44]

    International Conference on Learning Representations , author =

    Large. International Conference on Learning Representations , author =. 2024 , pages =

    2024

  45. [45]

    Kamoi, Ryo and Zhang, Yusen and Zhang, Nan and Han, Jiawei and Zhang, Rui , year =. When. Transactions of the Association for Computational Linguistics , publisher =. doi:10.1162/tacl_a_00713 , abstract =

    doi:10.1162/tacl_a_00713

  46. [46]

    and Singh, Avi and Baumli, Kate and Iqbal, Shariq and Bishop, Colton and Roelofs, Rebecca and Zhang, Lei M

    Kumar, Aviral and Zhuang, Vincent and Agarwal, Rishabh and Su, Yi and Co-Reyes, John D. and Singh, Avi and Baumli, Kate and Iqbal, Shariq and Bishop, Colton and Roelofs, Rebecca and Zhang, Lei M. and McKinney, Kay and Shrivastava, Disha and Paduraru, Cosmin and Tucker, George and Precup, Doina and Behbahani, Feryal and Faust, Aleksandra , month = oct, yea...

    Pith/arXiv arXiv doi:10.48550/arxiv.2409.12917

  47. [47]

    arXiv.org , author =

    Constitutional. arXiv.org , author =

  48. [48]

    Wu, Fang and Xuan, Weihao and Lu, Ximing and Liu, Mingjie and Dong, Yi and Harchaoui, Zaid and Choi, Yejin , month = feb, year =. The. doi:10.48550/arXiv.2507.14843 , abstract =

    doi:10.48550/arxiv.2507.14843

  49. [49]

    Chain-of-

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and Ichter, Brian and Xia, Fei and Chi, Ed and Le, Quoc and Zhou, Denny , month = jan, year =. Chain-of-

  50. [50]

    2025 , pages =

    International Conference on Learning Representations , author =. 2025 , pages =

    2025

  51. [51]

    Proceedings of the 42nd

    Gehring, Jonas and Zheng, Kunhao and Copet, Jade and Mella, Vegard and Cohen, Taco and Synnaeve, Gabriel , month = jul, year =. Proceedings of the 42nd

  52. [52]

    and Nikulkov, Alex and Chen, Yu-Chia and Bai, Yang and Zhu, Zheqing , month = dec, year =

    Jiang, Daniel R. and Nikulkov, Alex and Chen, Yu-Chia and Bai, Yang and Zhu, Zheqing , month = dec, year =. Improving. doi:10.48550/arXiv.2507.21983 , abstract =

    doi:10.48550/arxiv.2507.21983

  53. [53]

    Distilling the

    Hinton, Geoffrey and Vinyals, Oriol and Dean, Jeff , month = mar, year =. Distilling the. doi:10.48550/arXiv.1503.02531 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.1503.02531

  54. [54]

    Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and Graves, Alex and Antonoglou, Ioannis and Wierstra, Daan and Riedmiller, Martin , month = dec, year =. Playing. doi:10.48550/arXiv.1312.5602 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.1312.5602

  55. [55]

    Learning beyond

    Yang, Wenkai and Liu, Weijie and Xie, Ruobing and Yang, Kai and Yang, Saiyong and Lin, Yankai , month = feb, year =. Learning beyond. doi:10.48550/arXiv.2602.12125 , language =

    Pith/arXiv arXiv doi:10.48550/arxiv.2602.12125

  56. [56]

    Abdin, Marah and Aneja, Jyoti and Awadalla, Hany and others , month = aug, year =. Phi-3. doi:10.48550/arXiv.2404.14219 , urldate =

    Pith/arXiv arXiv doi:10.48550/arxiv.2404.14219

  57. [57]

    Javat, Abdurrahman and Kazakov, Allan , month = may, year =. Silicon. doi:10.48550/arXiv.2605.00519 , urldate =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2605.00519

  58. [58]

    Evaluating Large Language Models Trained on Code , shorttitle =

    Chen, Mark and Tworek, Jerry and Jun, Heewoo and others , year =. Evaluating Large Language Models Trained on Code , shorttitle =. doi:10.48550/arXiv.2107.03374 , publisher =

    Pith/arXiv arXiv doi:10.48550/arxiv.2107.03374