Pith. sign in

REVIEW 14 cited by

LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.14655 v1 pith:6VQMPOHQ submitted 2025-04-20 cs.LG cs.CLcs.SE

Yunhui Xia , Wei Shen , Yan Wang , Jason Klein Liu , Huifeng Sun , Siyue Wu , Jian Hu , Xiaolong Xu This is my paper
classification cs.LGcs.CLcs.SE
keywords datasetevaluationtrainingcounterpartsefficientleetcodedatasetmodelstemporal
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We introduce LeetCodeDataset, a high-quality benchmark for evaluating and training code-generation models, addressing two key challenges in LLM research: the lack of reasoning-focused coding benchmarks and self-contained training testbeds. By curating LeetCode Python problems with rich metadata, broad coverage, 100+ test cases per problem, and temporal splits (pre/post July 2024), our dataset enables contamination-free evaluation and efficient supervised fine-tuning (SFT). Experiments show reasoning models significantly outperform non-reasoning counterparts, while SFT with only 2.6K model-generated solutions achieves performance comparable to 110K-sample counterparts. The dataset and evaluation framework are available on Hugging Face and Github.

Discussion (0). Sign in to comment.

Forward citations

Cited by 14 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Do Prompt-Elicited Trajectories Reflect Training-Time Reward Hacking? A Systematic Study on Monitoring Training-Time Reward Hacking in Code Generation

    cs.LG 2026-04 unverdicted novelty 7.0 of 10

    Synthetic reward hacking data does not capture natural hacking behaviors in code generation RL, causing monitors trained on it to generalize poorly compared to those trained on in-the-wild trajectories.

  2. ACES: Who Tests the Tests? Leave-One-Out AUC Consistency for Code Generation

    cs.LG 2026-04 accept novelty 7.0 of 10

    Leave-one-out AUC of each test against the ranking induced by the remaining tests is proportional to that test's latent discriminative power, yielding closed-form and optimized weights that raise Pass@k.

  3. Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning

    cs.LG 2026-07 reject novelty 6.0 of 10

    MetaEvolve trains LLMs with reinforcement learning on synthesized code-refinement trajectories, reporting large gains on coding and numerical-optimization benchmarks.

  4. Loop the Loopies!

    cs.CL 2026-07 conditional novelty 6.0 of 10

    With two recurrent passes per layer and a compute-matched scaling recipe, 20B and 6B MoE models beat vanilla Transformers trained on the same wall-clock budget.

  5. Diversifying to Verify: When Task-Equivalent Programs Differ in Verifiability

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Diverse recursive and imperative array/list implementations of the same task, paired with frozen representation-specific contracts and bounded Why3 repair, raise task-level verification to 67.1% versus lower rates for...

  6. SOD: Step-wise On-policy Distillation for Small Language Model Agents

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    SOD reweights on-policy distillation strength step-by-step using divergence to stabilize tool use in small language model agents, yielding up to 20.86% gains and 26.13% on AIME 2025 for a 0.6B model.

  7. Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning

    cs.AI 2025-07 conditional novelty 6.0 of 10

    Training a Qwen2.5-7B model with GRPO on math and puzzle data improves both domains, code transfer depends on the starting model, and template or reward mismatches sharply hurt performance.

  8. Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A minimal-prior pipeline with automated data curation and verifier-driven RL lets small LLMs generate verifiable Dafny specifications and beat larger proprietary models on a synthetic compositional benchmark.

  9. Small Edits, Big Consequences: Telling Good from Bad Robustness in Large Language Models

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Large language models stay accurate when 90% of a coding prompt is deleted but frequently ignore a single quantifier flip that changes the problem, so their robustness blurs harmless noise and meaning-changing edits.

  10. Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code

    cs.CL 2025-07 conditional novelty 6.0 of 10

    TeaR uses GRPO reinforcement learning on test-case output prediction for algorithmic problems, with no code shown, and reports broad reasoning gains across 17 benchmarks.

  11. Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A cost-effective recipe consisting of tokenizer extension, continual pretraining, FP8 training, and SFT/DPO post-training yields Korean-English bilingual 8B models with top Korean benchmark scores.

  12. LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?

    cs.SE 2025-06 conditional novelty 6.0 of 10

    A live, medalist-annotated competitive programming benchmark finds frontier LLMs solve 53% of medium and 0% of hard problems without tools, and succeed mainly on implementation-heavy tasks.

  13. LLM Performance for Code Generation on Noisy Tasks

    cs.LG 2025-05 conditional novelty 6.0 of 10

    LLMs solve heavily obfuscated benchmark tasks, and performance decay under obfuscation differs sharply between old and new datasets, which the authors interpret as a signature of training-data contamination.

  14. InfoSynth: Information-Guided Benchmark Synthesis for LLMs

    cs.CL 2026-01 conditional novelty 5.0 of 10

    Using KL divergence and entropy on embeddings, InfoSynth scores benchmark novelty/diversity and guides a genetic pipeline that generates new, code-verified Python problems from seeds.

Pith tools