Pith. sign in

REVIEW 17 cited by

ACECODER: Acing Coder RL via Automated Test-Case Synthesis

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.01718 v4 pith:XIHZVD46 submitted 2025-02-03 cs.SE cs.AIcs.CL

Huaye Zeng , Dongfu Jiang , Haozhe Wang , Ping Nie , Xiaotong Chen , Wenhu Chen This is my paper · ORCID
classification cs.SEcs.AIcs.CL
keywords modelmodelscodecoderlearningreinforcementrewardtest-case
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Most progress in recent coder models has been driven by supervised fine-tuning (SFT), while the potential of reinforcement learning (RL) remains largely unexplored, primarily due to the lack of reliable reward data/model in the code domain. In this paper, we address this challenge by leveraging automated large-scale test-case synthesis to enhance code model training. Specifically, we design a pipeline that generates extensive (question, test-cases) pairs from existing code data. Using these test cases, we construct preference pairs based on pass rates over sampled programs to train reward models with Bradley-Terry loss. It shows an average of 10-point improvement for Llama-3.1-8B-Ins and 5-point improvement for Qwen2.5-Coder-7B-Ins through best-of-32 sampling, making the 7B model on par with 236B DeepSeek-V2.5. Furthermore, we conduct reinforcement learning with both reward models and test-case pass rewards, leading to consistent improvements across HumanEval, MBPP, BigCodeBench, and LiveCodeBench (V4). Notably, we follow the R1-style training to start from Qwen2.5-Coder-base directly and show that our RL training can improve model on HumanEval-plus by over 25\% and MBPP-plus by 6\% for merely 80 optimization steps. We believe our results highlight the huge potential of reinforcement learning in coder models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models

    cs.CL 2026-01 conditional novelty 7.0 of 10

    Arbitrary-order generation lowers the reasoning Pass@k of diffusion LLMs; constraining RL to autoregressive order (JustGRPO) improves accuracy and preserves parallel decoding.

  2. CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks

    cs.CL 2025-07 conditional novelty 7.0 of 10

    A 5,352-pair benchmark shows thinking LLMs outperform non-thinking ones as code judges, but position bias and formatting sensitivity remain strong.

  3. SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond

    cs.AI 2025-05 conditional novelty 7.0 of 10

    A synthetic logic dataset with verifiable answers, trained via reinforcement learning, improves LLM logical reasoning and generalizes to math and coding.

  4. CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer

    cs.SE 2025-10 conditional novelty 6.0 of 10

    Using Python-generated test oracles and multi-temperature sampling, CodeChemist selects low-resource-language code by execution pass rate and improves Pass@1 up to 69.5% over vanilla sampling.

  5. CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback

    cs.SE 2025-07 conditional novelty 6.0 of 10

    CodeEvo uses two interacting LLM agents with keyword-guided instruction evolution and hybrid compiler-plus-LLM feedback to synthesize high-quality instruction-code pairs for fine-tuning code models.

  6. Rethinking Verification for LLM Code Generation: From Generation to Testing

    cs.CL 2025-07 reject novelty 6.0 of 10

    SAGA, a human-LLM workflow, generates tests that catch 90.62% of known human bugs on a new benchmark, though those bugs also helped create the tests.

  7. DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A 7B masked-diffusion code model plus complementary-mask GRPO (coupled-GRPO) improves benchmark scores and shifts decoding away from strict left-to-right order.

  8. SAGE:Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMs

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SAGE combines supervised fine-tuning and GRPO reinforcement learning to generate CCFG grammars from competitive programming specs, achieving 96.66% set-based validity and 80.67% set-based effectiveness on 240 held-out...

  9. rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A large verified dataset of 418K competition code problems built by synthesis, test-input utility functions, and majority-vote output labeling improves small model code reasoning to frontier level.

  10. AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A two-stage RL recipe, math-only then code-only, substantially lifts math and code reasoning in 7B/14B distilled models and surpasses several distillation-based open models.

  11. InfoSynth: Information-Guided Benchmark Synthesis for LLMs

    cs.CL 2026-01 conditional novelty 5.0 of 10

    Using KL divergence and entropy on embeddings, InfoSynth scores benchmark novelty/diversity and guides a genetic pipeline that generates new, code-verified Python problems from seeds.

  12. RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning

    cs.AI 2025-10 conditional novelty 5.0 of 10

    A 3B VLM trained with SFT plus GRPO and an LCS-based reward reaches 55.3% on EmbodiedBench's EB-ALFRED, beating GPT-4o-mini and the 7B REBP planner.

  13. OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A 2.5M-example code reasoning dataset with critique traces enables Qwen2.5-based models to surpass prior open-weight distilled models on LiveCodeBench via test-time self-critique selection.

  14. Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Team

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A training-free multi-agent framework with episodic and shared memory reports new best results on GSM8K, AIME 2024/2025, Math-500, and LiveCodeBench.

  15. RAST: Reasoning Activation in LLMs via Small-model Transfer

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Injecting the logit gap between a small RL-trained model and its base into a larger base model at decoding time improves reasoning accuracy on math and code benchmarks, sometimes matching or exceeding RL training of t...

  16. CoTu at EXACT 2026: Neuro-Symbolic Reasoning for Transparent Educational QA

    cs.CL 2026-07 conditional novelty 4.0 of 10

    A 4B LLM that generates solver-runnable code (Z3 for logic, Python for physics) achieved a perfect physics score and the highest technical score (13.44/15) at EXACT 2026, with premise selection as the main remaining e...

  17. Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RL

    cs.CL 2025-05 conditional novelty 4.0 of 10

    Medical QA accuracy improves substantially through reinforcement learning with a binary correct-answer reward alone, without supervised fine-tuning on distilled reasoning traces.

Pith tools