Pith. sign in

REVIEW 4 major objections 5 minor 10 cited by

Kevin: Multi-Turn RL for Generating CUDA Kernels

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multi-turn reinforcement learning, which trains on every generate–execute–refine turn with discounted future rewards, raises CUDA kernel correctness from 56% to 82% and mean speedup from 0.53x to 1.10x over the PyTorch baseline.

desk verdict Well-engineered first multi-turn RL for CUDA kernels, but the headline numbers rest on a self-built eval set and need external replication before I'd trust them. read the letter →

arxiv 2507.11948 v1 pith:SLACG3NT submitted 2025-07-16 cs.LG cs.AIcs.PFcs.SE

Carlo Baronio , Pietro Marsella , Ben Pan , Simon Guo , Silas Alberti This is my paper · ORCID
classification cs.LGcs.AIcs.PFcs.SE
keywords multi-turnreinforcementlearningCUDAkernelgenerationoptimizationexecutionfeedbacktest-timescalingGRPOrewardshapingcode
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Writing GPU kernels is an iterative engineering process: experts write code, run it, read execution feedback, and refine. The paper argues that reinforcement learning should mirror that loop during training, not only at inference, and introduces a multi-turn RL recipe in which each generate–execute–refine turn is a training sample and each turn receives a discounted reward that includes future turns' scores. The result is Kevin, a model trained on 180 kernel-writing tasks that improves correctness from 56% to 82% and mean speedup from 0.53x to 1.10x over the PyTorch baseline, relative to its base model, while also beating frontier API models under the same evaluation setup. The paper also finds that, under a fixed generation budget, serial refinement turns are a more effective use of test-time compute than parallel sampling, and that multi-turn-trained models improve faster with extra turns than single-turn-trained ones. If the result transfers, it suggests a general recipe for using RL to train models on verifiable, feedback-rich engineering tasks beyond kernels.

What carries the argument

The load-bearing mechanism is turn-level reward attribution in a multi-turn RL loop. Each training step unrolls a trajectory of kernel generations; each generated kernel is executed, checked for correctness, and profiled, then scored with $S = 0.3\cdot \mathbf{1}\{\text{correct}\} + (T_{\text{baseline}}/T_{\text{kernel}})\cdot \mathbf{1}\{\text{correct}\}$. A turn's reward is the discounted sum of future scores, with sum aggregation and discount factor $\gamma = 0.4$; trajectories are split so every turn is its own training sample, and the next turn's context contains the previous kernels, summarized chains of thought, and execution feedback. This design rewards early imperfect kernels that enable later good ones, prevents context explosion, and is what the paper ablates to show that multi-turn credit assignment, rather than the score formula alone, drives the gains.

What would settle it

Run Kevin on a fresh set of kernel tasks whose operator compositions and tensor shapes are generated independently of the benchmark's construction methodology; if the correctness and speedup gains over the base model shrink or vanish there, the reported improvements reflect the evaluation distribution rather than a general kernel-writing ability.

Watch

Extended reading notes

Core claim

The paper claims that explicitly training a language model to iterate over multiple generate–execute–refine turns, rather than optimizing a single kernel-generation attempt, teaches it to write CUDA kernels that are both correct and fast. On a held-out set of 100 tasks, the resulting model, Kevin, reaches 82% correctness and a mean 1.10x speedup over the PyTorch Eager baseline with 16 parallel trajectories and 8 refinement turns, compared with 56% and 0.53x for its base model and 0.78x for a frontier API model. The paper also claims that multi-turn training changes how the model uses test-time compute: under a fixed budget of 128 generated kernels, 16 trajectories with 8 turns beat 128 one-turn trajectories, and the multi-turn-trained model improves more steeply with extra refinement turns than either the single-turn RL baseline or the base model.

Load-bearing premise

The 100-task evaluation set, 80 of whose tasks the authors constructed using the same operator-composition methodology as the training tasks, fairly represents the range of real CUDA kernel-writing work.

Editorial extensions

If this is right

  • Training on every turn, with discounted future rewards, produces a model that improves faster as more refinement turns are allowed at test time.
  • For a fixed number of generated kernels, serial refinement (16 trajectories × 8 turns) outperforms parallel sampling (128 trajectories × 1 turn) across all models tested.
  • Multi-turn RL retains exploration capacity under parallel sampling, while the single-turn RL baseline's best@k curve plateaus as k grows.
  • Strict format checks that zero out reward-hacking responses keep training on an open-ended engineering task stable enough to complete.
  • The multi-turn recipe is not CUDA-specific and is intended to transfer to other verifiable, feedback-rich optimization tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An inexpensive early-warning signal for RL instability may generalize: the paper observes that when the chain of thought stops starting with the base model's stereotyped opener, junk generations appear roughly 15 steps later; other RL runs with a stereotyped chain-of-thought opening could use the same drift as a canary.
  • Because all speedups are measured on fixed tensor sizes on one GPU, the practical value of Kevin depends on whether the learned refinement strategies transfer across shapes; a natural next experiment is to train with randomized shapes and report the speedup distribution.
  • The serial-over-parallel scaling result suggests that agentic systems for engineering tasks, where execution feedback is cheap relative to model generation, should spend a fixed inference budget on more refinement iterations before widening the parallel beam.
  • If the 100-task evaluation set is representative, the same multi-turn RL recipe could be applied to other low-level software optimization tasks with verifiable rewards, such as compiler flag selection or memory-layout tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper describes Kevin, a QwQ-32B model fine-tuned with a multi-turn GRPO-style RL recipe for CUDA kernel generation and optimization. The method splits multi-turn trajectories into per-turn training samples, summarizes past chains of thought, and assigns discounted future score sums as rewards. On a 100-task KernelBench-derived evaluation set (80 tasks constructed by the authors, 20 leftover KernelBench tasks), Kevin is reported to improve pure-CUDA correctness from 56% to 82% and mean speedup from 0.53x to 1.10x over PyTorch Eager, outperforming o4-mini and its single-turn RL baseline. The paper also reports scaling experiments comparing serial refinement versus parallel sampling.

Significance. If the reported results hold on external benchmarks, the paper would be a valuable demonstration that multi-turn RL with execution feedback can train models to generate competitive CUDA kernels. The paper's strengths include a clearly described training recipe, a careful treatment of reward hacking, an interesting analysis of training instability via the "Not Okay Ratio", and detailed appendices with prompts and qualitative trajectories. However, the significance is currently limited by the self-constructed evaluation set, the lack of released artifacts, and the absence of error bars or independent replication; the headline numbers should therefore be read as in-house results until external validation is provided.

major comments (4)
  1. [Section 3.1, Appendix A.2] The central claim is evaluated on a set in which 80 of 100 tasks were generated by the authors using the same operator vocabulary, composition rule, and runtime filter as the KernelBench training distribution, while the remaining 20 are leftover KernelBench tasks; training itself uses 90 of 100 Level 1 and 90 of 100 Level 2 tasks. The evaluation distribution is therefore nearly a clone of the training distribution. The paper does not report results on the official full KernelBench Level 1 or Level 2 suites or any other external benchmark, and the new 80-task set is not released. The reported 82% correctness and 1.10x speedup are thus not established as generalizable; please provide external-benchmark results and/or a released eval set with per-task breakdowns.
  2. [Section 3.2, Section 4.3] The reward score weight (0.3 correctness) and the reward aggregation (sum with gamma=0.4) were selected through ablations evaluated on the same 16-trajectory/8-turn configuration used for the final results in Table 1, and no multiple seeds or confidence intervals are reported. This makes the headline numbers vulnerable to selection effects, and the reader cannot tell whether the ranking of training configurations would survive at another point in hyperparameter space. Please report sensitivity analyses and variance estimates.
  3. [Section 5.4, Table 2] The claim that serial refinement beats parallel sampling is based on comparing 128 trajectories with 1 turn against 32 trajectories with 4 turns and 16 trajectories with 8 turns at a fixed number of generated kernels (128). Because each additional turn includes the previous kernels, summaries, and evaluation feedback in the context and generates a new chain of thought, the multi-turn configurations consume substantially more tokens and more context-processing compute. The comparison therefore measures total budget only in terms of generations, not actual test-time compute. Please report token counts or FLOPs, or otherwise control for compute, before concluding that sequential scaling is more efficient.
  4. [Section 5, Table 2] Table 2 reports the 16-trajectory/8-turn case as pass@128 with values 1.10x and 82.00%, matching Table 1's best@16 numbers, but pass@k is never defined and the total number of trajectories in that configuration is 16. As written, the table does not make clear whether the metric is per-trajectory best-of-16 or a different aggregate, and this ambiguity directly affects the interpretation of the parallel-versus-sequential scaling result. Please define the metric precisely and unify the notation with Section 5.
minor comments (5)
  1. [Section 3.1] The sentence "We chose 180 of the 100 Level 1 problems and 100 Level 2 problems as training environments" should read "90 of the 100 Level 1 problems and 90 of the 100 Level 2 problems" (180 tasks total).
  2. [Appendix D, Section 7.2] The prompt in Appendix D says "optimizing for performance on NVIDIA H100", while Section 7.2 and Appendix A.1 refer to NVIDIA H200; please harmonize the hardware references.
  3. [Section 7.2, Section 3.2] The limitation that speedups are only accurate for the predefined tensor sizes on H200 should be stated where the speedup metric is first introduced in Section 3.2, not only in the limitations section.
  4. [Figures 3, 5, 6] None of the scaling figures report error bars or the number of independent runs; please state how many seeds or repeated evaluations each curve represents.
  5. [Section 7.2, Section 5] The paper acknowledges that only 80 gradient steps were performed and that more exhaustive ablations were not possible, but the scaling conclusions in Section 5 are presented without hedging; a sentence noting the limited training budget in the scaling discussion would help calibrate expectations.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: Kevin's gains are measured empirically against external baselines; the only self-referential elements are a minor KernelBench self-citation and an author-constructed eval set, neither of which makes a claim reduce to its inputs.

full rationale

This is an empirical reinforcement-learning paper. The central claims—that multi-turn RL improves correctness and speedup over single-turn RL and the base model, and that sequential test-time scaling beats parallel scaling—are established by executing generated kernels and comparing against PyTorch Eager, QwQ-32B, o4-mini, and o3-mini under a fixed harness. No equation in the paper is equivalent to its own input by construction, and no fitted parameter is renamed as a prediction. The reward score S = 0.3*1{correct} + (T_baseline/T_kernel)*1{correct} is deliberately aligned with the reported performance metric; that is objective alignment, not circular reasoning. Hyperparameters such as the sum aggregation with gamma = 0.4 and the correctness weight were selected through ablations on the same 16-trajectory/8-turn evaluation configuration (Sections 3.2 and 4.3), which is a mild overfitting risk but does not make the headline result a tautology. KernelBench [33] is cited as the task source, and one present author (Simon Guo) is a KernelBench co-author; this is a minor self-citation, but it is not load-bearing because KernelBench is a public, externally used benchmark and the paper additionally benchmarks against frontier API models under identical conditions. The main validity caveat, noted in the limitations (Section 7.2) and Appendix A.2, is that 80 of the 100 evaluation tasks were constructed by the authors following the same methodology as the training tasks, so the eval distribution closely resembles the training distribution; this threatens generalization of the absolute numbers but does not constitute circularity in the derivation. Overall, the findings are empirical and independently checkable, warranting a low score reflecting only the minor self-referential elements.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claim is an empirical performance number, not a derivation. The main fitted quantities are reward design hyperparameters (correctness weight, discount factor, and training turns). The evaluation set and baseline choices are domain assumptions rather than free parameters. No new physical or conceptual entities are introduced.

free parameters (3)
  • correctness_weight = 0.3
    Chosen via ablations on models 7B to 32B to balance correctness and speedup in the reward score S (Section 3.2).
  • discount_factor_gamma = 0.4
    Selected from ablations over sum/max aggregation and gamma in {0.4, 0.8} for best scaling over 8 refinement turns (Section 4.3).
  • train_turns = 4
    Number of refinement turns in multi-turn training; chosen for efficiency, then evaluated at 8 turns at test time (Section 4.4).
assumptions (4)
  • domain assumption The reward function S = 0.3 * correct + speedup * correct measures the true quality of a CUDA kernel.
    The paper assumes correctness and speedup against PyTorch Eager capture kernel quality (Section 3.2).
  • domain assumption The 100-task evaluation set, including 80 newly constructed tasks, is representative of CUDA kernel generation difficulty and is unseen during training.
    The generalizability claim depends on this assumption (Section 3.1, Appendix A.2).
  • domain assumption The format and correctness checks reliably detect reward hacking and invalid kernels.
    They exclude torch.nn use, try/except, pass, and verify output correctness on randomized inputs (Section 6.2, Appendix F).
  • domain assumption Performance profiling on NVIDIA H200 with the chosen tensor sizes is stable enough for meaningful speedup comparisons.
    The authors note speedups are only accurate for those dimensions on H200 (Section 7.2), so the comparison assumes profiling stability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Kevin: Multi-Turn RL for Generating CUDA Kernels." pith.science (2026). https://pith.science/paper/SLACG3NT

@misc{pith2026250711948,
  author       = {Pith},
  title        = {Pith review of: Kevin: Multi-Turn RL for Generating CUDA Kernels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SLACG3NT}},
  note         = {Machine review of arXiv:2507.11948}
}
read the original abstract

Writing GPU kernels is a challenging task and critical for AI systems' efficiency. It is also highly iterative: domain experts write code and improve performance through execution feedback. Moreover, it presents verifiable rewards like correctness and speedup, making it a natural environment to apply Reinforcement Learning (RL). To explicitly incorporate the iterative nature of this process into training, we develop a flexible multi-turn RL recipe that addresses unique challenges encountered in real-world settings, such as learning from long trajectories and effective reward attribution across turns. We present Kevin - K(ernel D)evin, the first model trained with multi-turn RL for CUDA kernel generation and optimization. In our evaluation setup, Kevin shows significant gains over its base model (QwQ-32B), improving correctness of generated kernels (in pure CUDA) from 56% to 82% and mean speedup from 0.53x to 1.10x of baseline (PyTorch Eager), and surpassing frontier models like o4-mini (0.78x). Finally, we study its behavior across test-time scaling axes: we found scaling serial refinement more beneficial than parallel sampling. In particular, when given more refinement turns, Kevin shows a higher rate of improvement.

Figures

Figures reproduced from arXiv: 2507.11948 by the authors.

Figure 1
Figure 1. Within each training step, the model iteratively generates, executes, and refines kernels [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Reward plateaus during single-turn training. We trained up to step 50 (100 gradient steps). Following [57], we apply Clip-Higher, decoupling the lower and higher clipping range (0.2 and 0.28 respectively). We sample with temperature = 0.9 for both training and inference. We set the KL coefficient to 0 to allow the model to deviate freely from the base policy, following [26]. We observe that reward plateaus after 50 … view at source ↗
Figure 3
Figure 3. Sum with γ = 0.4 is the most effective reward formulation. Here we evaluate models trained with different reward formulations (Sum vs Max aggregation across turns and discount factor γ = 0.4 vs γ = 0.8) with 16 parallel trajectories and 8 refinement turns. We compare how each setup scale with refinement turns. Though Sum with γ = 0.4 achieves lower performance and correctness in the first turn, it exhibits the best … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Reward climbs steadily for multi-turn training. We train up to 40 steps (80 gradient steps). For our final training run for Kevin, we use 16 parallel trajectories and 4 refinement turns per task. Each batch contains 8 tasks. We use the sum reward formulation with disco…
Figure 5
Figure 5. Figure 5: Kevin effectively leverages multiple turns. We evaluate the above checkpoints under the same environment with 16 parallel trajectories and 8 refinement turns. We observe that both Kevin and single-turn RL model significantly improves upon QwQ-32B, but the performance c…
Figure 7
Figure 7. Figure 7: "Not okay ratio" is a proxy for model instability and predicts junk. It starts rising around step 48, while junk appears 15 steps later. "Clipping Ratio" [27] reflects responses truncated for junk. We observe that training for longer often leads to the model producing …
Figure 8
Figure 8. Figure 8: Training reward with correctness weighting of 1, performance / speedup weighting of 1. Concretely, S = 1{correct}+ Tbaseline Tkernel ·1{correct} [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 10
Figure 10. Figure 10: Training Reward collapses when in￾cluding length penalty as part of reward [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 12
Figure 12. Figure 12: Overview of our RL Training infrastructure. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Adding a KL penalty slows down learning. Here we conduct an ablation with KL coefficient β = 0.001 versus β = 0. We see that the reward plateaus with KL enabled. We use 4 refinement turns at train-time for efficient training. During test time, we can afford more exten…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Don't Regenerate, Debug: A Domain-Specific Agent for Repairing Near-Miss Hardware Operators

    cs.SE 2026-08 conditional novelty 6.0 of 10

    Debugging near-miss AscendC kernels with a domain-specific agent recovered 18/27 operators versus 11/27 from three regeneration trials, while using fewer tokens per success.

  2. AtumAI: A Principled Framework for Agentic Generation of Datacenter Control-Plane Policies

    cs.AI 2026-08 conditional novelty 6.0 of 10

    AtumAI compiles plain-language goals into a formal search problem and uses an LLM-plus-evolution loop to generate datacenter control-plane policies that beat hand-tuned baselines in simulation.

  3. KernelGenBench: A Multi-Source and Multi-Chip Benchmark for LLM-based Kernel Generation

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A multi-source (ATen/vLLM/cuBLAS, 210 ops) and multi-chip (6 platforms) benchmark shows agentic kernel generation beats sampling but degrades sharply off NVIDIA and costs millions of tokens per success.

  4. CANN Bench: Benchmarking Agent Generated Kernels against Real NPU and Algorithmic Limits

    cs.AI 2026-07 conditional novelty 6.0 of 10

    CANN Bench is an Ascend-NPU benchmark with 53 operators and 1,060 released test cases that scores agent-generated kernels on compilation, correctness, and hardware-anchored performance.

  5. Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation

    cs.AI 2026-07 unverdicted novelty 6.0 of 10

    Hawk raises NPU kernel generation accuracy from 49.4% to 80% and yields up to 2.2× speedups by retrieving and distilling structured hardware-aware knowledge without any model training.

  6. Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts

    cs.NE 2026-05 unverdicted novelty 6.0 of 10

    Kernel Foundry improves GPU kernel correctness and performance on KernelBench via expert-guided evolutionary search with diagnostic feedback and an experience library, reaching 100% correctness on Level 2.

  7. KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta

    cs.LG 2025-12 conditional novelty 6.0 of 10

    An agentic kernel-coding system combining tree search with hardware-knowledge retrieval generated optimized Triton kernels for NVIDIA, AMD, and Meta's MTIA accelerators: 100% correctness on 480 operator-platform confi...

  8. Compiler-Grounded Hierarchical Diagnosis for LLM-Based Triton Kernel Optimization

    cs.AI 2026-07 conditional novelty 5.0 of 10

    A hierarchical, compiler-grounded LLM optimization loop reports 4.35x geometric-mean Triton kernel speedups on 37 Ascend NPU benchmark kernels, but with no causal ablation yet.

  9. FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

    cs.LG 2026-07 conditional novelty 5.0 of 10

    FlashRT's agent harness converts reference multimodal pipelines into optimized multi-GPU deployments, reporting ~70x latency cuts and up to 3.6x throughput gains across five applications on B200 and MI355X.

  10. CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning

    cs.LG 2025-12 conditional novelty 5.0 of 10

    CUDA-L2's RL-generated HGEMM kernels are claimed to beat cuBLASLt autotuning by 11.4-15.9% on 1000 A100 shapes, though no independent artifacts are supplied.

Reference graph

Works this paper leans on

67 extracted references · 36 canonical work pages · cited by 10 Pith papers

  1. [1]

    Concrete problems in ai safety, 2016

    Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety, 2016

    2016

  2. [2]

    Le, Christopher Ré, and Azalia Mirhoseini

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V . Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling, 2024

    2024

  3. [3]

    Gonzalez, and Ion Stoica

    Shiyi Cao, Sumanth Hegde, Dacheng Li, Tyler Griggs, Shu Liu, Eric Tang, Jiayi Pan, Xingyao Wang, Akshay Malik, Graham Neubig, Kourosh Hakhamaneshi, Richard Liaw, Philipp Moritz, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Skyrl-v0: Train real-world long-horizon agents via reinforcement learning, 2025

    2025

  4. [4]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    2021

  5. [5]

    Automating gpu kernel generation with deepseek- r1 and inference-time scaling

    Terry Chen, Bing Xu, and Kirthi Devleker. Automating gpu kernel generation with deepseek- r1 and inference-time scaling. https://developer.nvidia.com/blog/automating-gpu- kernel-generation-with-deepseek-r1-and-inference-time-scaling/ , February

    work page

  6. [6]

    Warpdrive: An agentic workflow for ninja gpu transformations

    Sana Damani, Siva Kumar Sastry Hari, Mark Stephenson, and Christos Kozyrakis. Warpdrive: An agentic workflow for ninja gpu transformations. In Proceedings of the Machine Learning for Systems Workshop at NeurIPS 2024 , 2024. Accessed: 2025-05-15. 10

    work page 2024

  7. [7]

    Flashattention-2: Faster attention with better parallelism and work partitioning, 2023

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning, 2023

    2023

  8. [8]

    Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

    Tri Dao, Daniel Y . Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness, 2022

    2022

Show all 67 references
  1. [9]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    2025

  2. [10]

    Gemini 2.5: Our most intelligent models are getting even bet- ter

    Tulsee Doshi. Gemini 2.5: Our most intelligent models are getting even bet- ter. https://blog.google/technology/google-deepmind/google-gemini-updates- io-2025/, May 2025. Accessed: 2025-05-21

    2025

  3. [11]

    Bartlett, Ilya Sutskever, and Pieter Abbeel

    Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, and Pieter Abbeel. Rl2: Fast reinforcement learning via slow reinforcement learning, 2016

    2016

  4. [12]

    Codemonkeys: Scaling test-time compute for software engineering, 2025

    Ryan Ehrlich, Bradley Brown, Jordan Juravsky, Ronald Clark, Christopher Ré, and Azalia Mirhoseini. Codemonkeys: Scaling test-time compute for software engineering, 2025

    2025

  5. [13]

    Rlef: Grounding code llms in execution feedback with reinforcement learning, 2025

    Jonas Gehring, Kunhao Zheng, Jade Copet, Vegard Mella, Quentin Carbonneaux, Taco Cohen, and Gabriel Synnaeve. Rlef: Grounding code llms in execution feedback with reinforcement learning, 2025

    2025

  6. [14]

    Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D. Manning. Synthetic data generation & multi-step rl for reasoning & tool use, 2025

    2025

  7. [15]

    Alphaevolve: A gemini-powered coding agent for designing advanced algorithms, May 2025

    Google DeepMind. Alphaevolve: A gemini-powered coding agent for designing advanced algorithms, May 2025. Accessed: 2025-05-15

    2025

  8. [16]

    Openrlhf: An easy-to-use, scalable and high-performance rlhf framework, 2024

    Jian Hu, Xibin Wu, Zilin Zhu, Xianyu, Weixun Wang, Dehao Zhang, and Yu Cao. Openrlhf: An easy-to-use, scalable and high-performance rlhf framework, 2024

    2024

  9. [17]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues?, 2024

    2024

  10. [18]

    The stack: 3 tb of permissively licensed source code, 2022

    Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, and Harm de Vries. The stack: 3 tb of permissively licensed source code, 2022

    2022

  11. [19]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Syste...

    2023

  12. [20]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V . Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Lu...

    2025

  13. [21]

    The ai cuda engineer: Agentic cuda kernel discovery, optimization and composition, 2025

    Robert Tjarko Lange, Aaditya Prasad, Qi Sun, Maxence Faldor, Yujin Tang, and David Ha. The ai cuda engineer: Agentic cuda kernel discovery, optimization and composition, 2025. Accessed: 2025-05-15

    2025

  14. [22]

    Tritonbench: Benchmarking large language model capabilities for generating triton operators, 2025

    Jianling Li, Shangzhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, Haojie Wang, Jianrong Wang, Xu Han, Zhiyuan Liu, and Maosong Sun. Tritonbench: Benchmarking large language model capabilities for generating triton operators, 2025

    2025

  15. [23]

    Starcoder: may the source be with you!, 2023

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, ...

    2023

  16. [24]

    Rltf: Reinforcement learning from unit test feedback, 2023

    Jiate Liu, Yiqin Zhu, Kaiwen Xiao, Qiang Fu, Xiao Han, Wei Yang, and Deheng Ye. Rltf: Reinforcement learning from unit test feedback, 2023

    2023

  17. [25]

    Understanding r1-zero-like training: A critical perspective, 2025

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective, 2025

    2025

  18. [26]

    Deepcoder: A fully open-source 14b coder at o3-mini level

    Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaox- iang Shi, Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepcoder: A fully open-source 14b coder at o3-mini level. https://pretty-radio-b75 .notion...

    2025

  19. [27]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Sto- ica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Sto- ica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75 .notion.site/DeepScaleR-Surp...

    2025

  20. [28]

    Measuring automated kernel engineering, February 2025

    METR. Measuring automated kernel engineering, February 2025. Accessed: 2025-05-15

    2025

  21. [29]

    Scalable parallel programming with cuda

    John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron. Scalable parallel programming with cuda. In ACM SIGGRAPH 2008 Classes, SIGGRAPH ’08, New York, NY , USA, 2008. Association for Computing Machinery

    2008

  22. [30]

    Lee, Ed H

    Allen Nie, Yi Su, Bo Chang, Jonathan N. Lee, Ed H. Chi, Quoc V . Le, and Minmin Chen. Evolve: Evaluating and optimizing llms for exploration, 2024

    2024

  23. [31]

    Gpu mode at nvidia gtc 2025, 2025

    NVIDIA. Gpu mode at nvidia gtc 2025, 2025. Accessed: 2025-05-15

    2025

  24. [32]

    Cutlass: Cuda templates for linear algebra subroutines, May 2025

    NVIDIA Corporation. Cutlass: Cuda templates for linear algebra subroutines, May 2025. Accessed: 2025-05-15

    2025

  25. [33]

    Zhang, William Hu, Christopher Ré, and Azalia Mirhoseini

    Anne Ouyang, Simon Guo, Simran Arora, Alex L. Zhang, William Hu, Christopher Ré, and Azalia Mirhoseini. Kernelbench: Can llms write efficient gpu kernels?, 2025

    2025

  26. [34]

    Yuxiao Qu, Matthew Y . R. Yang, Amrith Setlur, Lewis Tunstall, Edward Emanuel Beeching, Ruslan Salakhutdinov, and Aviral Kumar. Optimizing test-time compute via meta reinforcement fine-tuning, 2025

    2025

  27. [35]

    Putting the value back in rl: Better test-time scaling by unifying llm reasoners with verifiers, 2025

    Kusha Sareen, Morgane M Moss, Alessandro Sordoni, Rishabh Agarwal, and Arian Hosseini. Putting the value back in rl: Better test-time scaling by unifying llm reasoners with verifiers, 2025

    2025

  28. [36]

    Llms are greedy agents: Effects of rl fine-tuning on decision-making abilities, 2025

    Thomas Schmied, Jörg Bornschein, Jordi Grau-Moya, Markus Wulfmeier, and Razvan Pascanu. Llms are greedy agents: Effects of rl fine-tuning on decision-making abilities, 2025

    2025

  29. [37]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017

    2017

  30. [38]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. 12

    2024

  31. [39]

    Learning performance-improving code edits, 2024

    Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob Gardner, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, and Amir Yazdanbakhsh. Learning performance-improving code edits, 2024

    2024

  32. [40]

    Mastering chess and shogi by self-play with a general reinforcement learning algorithm, 2017

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. Mastering chess and shogi by self-play with a general reinforce...

    2017

  33. [41]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    2024

  34. [42]

    Spector, Simran Arora, Aaryan Singhal, Daniel Y

    Benjamin F. Spector, Simran Arora, Aaryan Singhal, Daniel Y . Fu, and Christopher Ré. Thun- derkittens: Simple, fast, and adorable ai kernels, 2024

    2024

  35. [43]

    Training a generally curious agent, 2025

    Fahim Tajwar, Yiding Jiang, Abitha Thankaraj, Sumaita Sadia Rahman, J Zico Kolter, Jeff Schneider, and Ruslan Salakhutdinov. Training a generally curious agent, 2025

    2025

  36. [44]

    Kimi k1.5: Scaling reinforcement learning with llms, 2025

    Kimi Team. Kimi k1.5: Scaling reinforcement learning with llms, 2025

    2025

  37. [45]

    Sky-t1: Train your own o1 preview model within $450

    NovaSky Team. Sky-t1: Train your own o1 preview model within $450. https://novasky- ai.github.io/posts/sky-t1, 2025. Accessed: 2025-01-09

    2025

  38. [46]

    Intellect-2: A reasoning model trained through globally decentralized reinforcement learning, 2025

    Prime Intellect Team, Sami Jaghouar, Justus Mattern, Jack Min Ong, Jannik Straube, Manveer Basra, Aaron Pazdera, Kushal Thaman, Matthew Di Ferrante, Felix Gabriel, Fares Obeid, Kemal Erdem, Michael Keiblinger, and Johannes Hagemann. Intellect-2: A reasoning model trained throu...

    2025

  39. [47]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025

    2025

  40. [48]

    Philippe Tillet, H. T. Kung, and David Cox. Triton: an intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, MAPL 2019, page 10–19, New York, NY , US...

    2019

  41. [49]

    Ecco: Can we improve model-generated code efficiency without sacrificing functional correctness?, 2024

    Siddhant Waghjale, Vishruth Veerendranath, Zora Zhiruo Wang, and Daniel Fried. Ecco: Can we improve model-generated code efficiency without sacrificing functional correctness?, 2024

    2024

  42. [50]

    Zero++: Extremely efficient collective commu- nication for giant model training, 2023

    Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Connor Holmes, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, and Yuxiong He. Zero++: Extremely efficient collective commu- nication for giant model training, 2023

    2023

  43. [51]

    Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brenna...

    2025

  44. [52]

    Reinforcement learning for reasoning in large language models with one training example, 2025

    Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. Reinforcement learning for reasoning in large language models with one training example, 2025

    2025

  45. [53]

    Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning, 2025

    Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Monica Lam, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. Ragen: Understanding self-evolution in ...

    2025

  46. [54]

    Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I. Wang. Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution, 2025. 13

    2025

  47. [55]

    Towards system 2 reasoning in llms: Learning how to think with meta chain-of-thought, 2025

    Violet Xiang, Charlie Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, Louis Castricato, Jan-Philipp Franken, Nick Haber, and Chelsea Finn. Towards system 2 reasoning in llms: Learning how to think with m...

    2025

  48. [56]

    Flashinfer: Efficient and customizable attention engine for llm inference serving

    Zihao Ye, Lequn Chen, Ruihang Lai, Wuwei Lin, Yineng Zhang, Stephanie Wang, Tianqi Chen, Baris Kasikci, Vinod Grover, Arvind Krishnamurthy, and Luis Ceze. Flashinfer: Efficient and customizable attention engine for llm inference serving. arXiv preprint arXiv:2501.01005 , 2025

    2025 arXiv

  49. [57]

    Dapo: An open-source llm reinforcement learning system at scale, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wa...

    2025

  50. [58]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?, 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?, 2025

    2025

  51. [59]

    Deepgemm: clean and efficient fp8 gemm kernels with fine-grained scaling

    Chenggang Zhao, Liang Zhao, Jiashi Li, and Zhean Xu. Deepgemm: clean and efficient fp8 gemm kernels with fine-grained scaling. https://github.com/deepseek-ai/DeepGEMM, 2025

    2025

  52. [60]

    Minif2f: a cross-system benchmark for formal olympiad-level mathematics, 2022

    Kunhao Zheng, Jesse Michael Han, and Stanislas Polu. Minif2f: a cross-system benchmark for formal olympiad-level mathematics, 2022

    2022

  53. [61]

    Archer: Training language model agents via hierarchical multi-turn rl, 2024

    Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, and Aviral Kumar. Archer: Training language model agents via hierarchical multi-turn rl, 2024

    2024

  54. [62]

    Improving multi-turn tool use with reinforcement learning

    Richard Zhuang*, Trung Vu*, Alex Dimakis, and Maheswaran Sathiamoorthy. Improving multi-turn tool use with reinforcement learning. https://www.bespokelabs.ai/blog/improving- multi-turn-tool-use-with-reinforcement-learning, 2025. Accessed: 2025-04-17. 14 A KernelBench Modificat...

    2025

  55. [64]

    We sample from the available operators listed above: 1 main operator (computationally expensive), and 2-5 other operators

  56. [65]

    We also ask it to generate sample tensor sizes for the task

    We ask a language model, namely Gemini 2.5-Flash [10], to generate a PyTorch program that creates a kernel by combining these operators. We also ask it to generate sample tensor sizes for the task

  57. [66]

    We ensure this PyTorch program can be executed and has a runtime on NVIDIA H200 > 0.1ms, to avoid the runtime being dominated by kernel launch (CPU) overhead. 15

  58. [67]

    "" 7 Simple model that performs Layer Normalization. 8

    We make sure this PyTorch program (with the same sequence of operators) is not present in existing KernelBench Level 1 and 2 programs. We manually inspected all new task programs to ensure their validity. We build the evaluation set by combining our 80 newly created tasks with...

  59. [2025]

    Accessed: 2025-05-15

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.