Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Iteratively applying Adaptive Critique Refinement makes code LLMs continuously improve without imitating a teacher.

desk verdict A credible SFT-only iterative refinement method for code LLMs with a real empirical payoff, but the judge-dependent data gate is validated only indirectly and no artifacts are released. read the letter →

arxiv 2502.09183 v2 pith:D5F6ZXKH submitted 2025-02-13 cs.CL cs.AI

Changzhi Zhou , Xinyu Zhang , Dandan Song , Xiancai Chen , Wanli Gu , Huipeng Ma , Yuhang Tian , Mengdi Zhang
show 1 more author
Linmei Hu
This is my paper · ORCID
classification cs.CLcs.AI
keywords adaptivecritiquerefinementcodegenerationself-refinementLLM-as-a-judgeLLM-as-a-criticiterativefine-tuningsupervisedLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that code LLMs can improve themselves through a loop of generating code, scoring it, and training on the better of self-generated or teacher-written responses plus external critiques, rather than by imitating teacher outputs. The proposed Adaptive Critique Refinement (ACR) replaces the usual distillation pipeline with a student-like process: the model tries the problem, compares its best attempt against the teacher response, and either keeps its own solution or learns from a critique of its weaker solution. The authors build the RefineCoder series on DeepSeek-Coder-6.7B and Qwen2.5-Coder-7B using only an 80K Python instruction set, and report that three iterations raise average pass@1 on LiveCodeBench and BigCodeBench-hard from 14.2 to 16.9 (DS) and from 18.6 to 21.7 (QW), while also improving HumanEval/MBPP/BigCodeBench averages. The claim matters because it suggests that with a reliable judge and critic, a fixed amount of instruction data can be reused across rounds to extract improvement from the student's own samples instead of buying new teacher-generated data.

What carries the argument

The carrying mechanism is the ACR dataset update rule (Equation 10), gated by a composite scoring system. The composite score for each response combines a normalized weighted sum of point-wise judge scores over five criteria with a binary executability check from a code executor, and a pairwise judge plus executor score decides the winner between the best self-generated response and the teacher response. Depending on which side wins, the original sample $(x,y)$ is replaced either by the self-generated response $(x,y_w)$ or by a two-turn critique sample $(x, y_{l1}, c, y_w)$ generated by an LLM-as-a-Critic that explains the weakness and the better solution. Training on the resulting dataset with a plain supervised fine-tuning loss is what encodes the refinement into the weights, so inference-time calls are not needed for the gain.

What would settle it

Take the 80K dataset, run one ACR iteration with the declared winners from the original judge, and on a held-out subset have hidden unit tests decide the true winner. If a nontrivial fraction of declared winners fails while the declared loser passes, or if swapping the judge for the hidden-test oracle removes the pass@1 gain, the claimed improvement depends on judge accuracy rather than on self-refinement per se.

Watch

Extended reading notes

Core claim

The paper's central claim is that iterative application of Adaptive Critique Refinement yields continuous, measurable improvement in one-shot code generation. In each round, the current model samples several code responses per instruction, a composite scoring system (multi-faceted LLM-as-a-Judge plus a code executor) selects the best self-generated response, a pairwise judge decides whether it beats the original teacher-generated response, and a selective critique step either promotes the self-generated winner as new single-turn data or generates a two-turn critique sample in which a critic explains why the weaker self-generated code failed. The updated dataset, still 80K samples, is used to SFT the base model from scratch for the next round. The paper reports that the largest jump comes in the first iteration, that gains continue but shrink over three iterations, and that the final models beat same-size baselines on the harder, leakage-analyzed benchmarks using less data. The authors also show the loop improves multilingual generalization despite Python-only training.

Load-bearing premise

The load-bearing premise is that the LLM-as-a-Judge combined with the code executor reliably tells which response is truly better; if the judge is biased or noisy, the loop keeps training on mediocre self-generated code and critique data whose declared winner is not actually better.

Editorial extensions

If this is right

  • Three refinement rounds improve RefineCoder-DS-6.7B average pass@1 on LiveCodeBench and BigCodeBench-hard by 2.7 points and RefineCoder-QW-7B by 3.1 points, with the largest single gain at iteration one.
  • The 80K instruction set is reused every round, so the method improves capacity without expanding data or distilling new teacher responses.
  • Because only SFT is used, ACR is orthogonal to preference-based self-improvement and to inference-time debugging methods, and can be combined with either.
  • The critique data, not just the self-generated winner, carries part of the gain: ablating the second-turn critique data hurts more than removing the selective critiquing module.
  • The gains transfer to out-of-distribution languages on MultiPL-E even though the instruction set is Python-only.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper, the same gated winner-selection loop should transfer to other reasoning domains, such as mathematics or structured logic, whenever a composite judge can be built; the paper notes the idea is general but does not test it.
  • The fixed 80K dataset means each iteration's ceiling is set by instruction diversity; a natural extension is to inject fresh instructions or hard negatives each round to slow the reported diminishing returns after iteration three.
  • The paper's validation of the judge is indirect; a stronger test would be to compare judge-chosen winners against hidden-test execution on a sample of the 80K instructions, and to publish judge agreement statistics.
  • If judge noise is low enough, the method implies a data-efficiency argument for self-improvement: teacher data is a seed, not a ceiling, and the marginal value of additional teacher-generated samples may drop once the self-refinement loop is in place.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes Adaptive Critique Refinement (ACR), an iterative SFT pipeline for code LLMs. At each iteration the current model samples n responses per instruction; a composite scoring system (LLM-as-a-judge plus a code executor) selects the best self-generated response and compares it pairwise with the existing teacher response. Depending on the winner, the dataset is updated to either a single-turn (x, y_w) example or a two-turn critique example (x, y_l, c, y_w), and the next model is fine-tuned on the updated dataset from the original base model. Applied for three iterations to DeepSeek-Coder-6.7B-Base and Qwen2.5-Coder-7B-Base, the method yields RefineCoder-DS-6.7B and RefineCoder-QW-7B. The paper reports monotone gains on LiveCodeBench, BigCodeBench(-hard), HumanEval(+), and MBPP(+), claims better or comparable results than same-size baselines with only 80K data, and includes additional analyses of data leakage, scoring effectiveness, ablations, multilingual OOD transfer, and refinement with external feedback.

Significance. The contribution is potentially significant: if the conclusions hold, a code model can improve itself through iterative SFT on self-generated responses and teacher-generated critiques, without DPO/RLHF and without directly imitating teacher code, while using only 80K instructions. The paper has several commendable elements: the constructed dataset is checked for contamination with the TLI metric, the two data-construction branches are ablated, an OOD multilingual benchmark is included, and an appendix honestly documents diminishing returns. The method is simple and the pipeline is transparent. However, the evidence is currently limited by single-run evaluations, largely copied baseline numbers, and an indirect validation of the judge that gates the entire data update; these issues need to be addressed before the claimed continuous improvement can be fully accepted.

major comments (4)
  1. [§5.2 and Eqs. (6)–(10)] The dataset update rule in Eq. (10) is gated entirely by the pairwise judge decision in Eqs. (6)–(8), yet §5.2 validates the composite scoring system only by comparing the highest- and lowest-scoring self-generated responses on HumanEval+ and MBPP+ for the final Iter3 model. This does not measure pairwise decision accuracy on the training distribution, does not report judge agreement, and does not cover the M0, M1, and M2 models that actually generated the training data. Since the executor in Eq. (4) only checks executability rather than correctness, a judge that is wrong on even a modest fraction of pairwise comparisons will mislabel winners and misdirect the iterative SFT. The paper should provide pairwise accuracy against unit-test ground truth on a held-out subset of the training questions, or an explicit analysis of judge errors and their propagation across rounds; without this the central claim rests on an unvalidated oracle.
  2. [§4.4 and Tables 1–2] All reported improvements come from a single pass with no variance, confidence intervals, or multiple seeds, and the paper states that baseline results are prioritized from leaderboards or original papers rather than re-evaluated. The headline increments on Table 1 (2.7 and 3.1 average points) and Table 2 (2.7 and 4.5 average points) are small enough that prompt-format differences, sampling temperature, and judge stochasticity could shift them. The paper should report at least three seeds or bootstrap intervals over generated samples, and should re-evaluate the principal baselines under the same evaluation harness. This is load-bearing because the 'less data while outperforming baselines' conclusion in §4.5 assumes the baseline numbers are commensurable with the paper's own evaluation settings.
  3. [§5.4, Table 5] The MultiPL-E results for RefineCoder-DS-6.7B are not monotone: the average pass@1 goes 57.2 (Iter0), 58.2 (Iter1), 57.5 (Iter2), 57.8 (Iter3). Since Section 1 and Section 3.1 claim continuous improvement from iterative ACR, this OOD result contradicts the claim as stated. The claim should either be restricted to the in-distribution benchmarks where monotonicity is observed, or the paper should explain the non-monotonicity and provide error bars to show whether the fluctuations are meaningful.
  4. [§4.4 and §4.5] Baseline fairness is a load-bearing concern. The paper takes baseline numbers from leaderboards or original papers when available, and one baseline (Qwen2.5-Coder-7B-Instruct) is listed only as using 'Millions' of data, without an exact count. Because HumanEval(+) and MBPP(+) results are highly sensitive to the exact prompting harness and sampling procedure, the comparison 'outperforms baselines of the same size using only 80K data' is stronger than the evidence currently supports. I would like to see the main baseline numbers reproduced locally with the same evaluation code, together with an explicit statement of which numbers are taken from which source and what differences remain.
minor comments (5)
  1. [Figure 3] The y-axis labels in Figure 3 are garbled unicode sequences; the ablation curves are much harder to read than they should be, and the text '(without 3.1 module or without 3.2 module)' is not a substitute for readable axis labels.
  2. [Figure 8 and Section 4.5] There are several typos: 'provied' should be 'provided' in the pointwise judge prompt, and 'Similarity' should be 'Similarly' in Section 4.5.
  3. [Section 3.2] The criterion weights alpha_j in Eq. (2) are never specified; if they are all equal, state this explicitly, and if not, give the values used in the experiments.
  4. [Appendix A] The pairwise judge prompt allows a 'tie' verdict, but Eqs. (7)–(8) in Section 3.2 only define outcomes for strictly greater scores; the paper should state how ties are resolved.
  5. [Availability] No link to code, data, prompts, or model weights is provided. Given that the method's value depends on reproducing the dataset construction and judging pipeline, releasing these artifacts is important for the paper's utility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ACR is an empirical iterative pipeline whose judge and critic are external fixed models, and whose improvement claims are measured on independent benchmarks, so no central result reduces to its inputs by construction.

full rationale

The derivation chain is: D0 is an 80K GPT-4o-synthesized instruction set; M0 is SFT on D0; for each iteration the current model samples seven responses, the fixed external Qwen2.5-32B-Instruct judge/critic scores them via Eqs. 1-8, and Eq. 10 selects single-turn or two-turn training data; the next model is SFT on the updated dataset. No equation defines a target quantity in terms of a fitted output, and no benchmark pass@1 value is used to train, select, or score the data. The judge and critic are not the refined model and are not fitted to the reported benchmarks, so the claim of continuous improvement is an empirical outcome rather than a tautology. The only overlapping-author citations (Chen et al. 2025; Wang et al. 2024b) are related-work context or a contamination metric used post hoc to explain benchmark shortfalls; neither is needed to establish the central improvement claim, which is supported by external LiveCodeBench and BigCodeBench-hard evaluations. The paper's own Appendix E notes diminishing returns and possible overfitting, which is an empirical caveat, not circular reasoning. The residual concern that LLM-as-a-Judge preferences could bias data selection is a validity risk, not a logical circularity, because the final evaluations are independent of the judge's scores.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no invented entities. Its central claim rests on three domain assumptions: the judge/critic is accurate, the initial dataset is clean, and SFT on refined data transfers to benchmarks. Free parameters include the unspecified judge weights alpha_j and sampling hyperparameters n=7, temperature=0.7.

free parameters (3)
  • Scoring criterion weights alpha_j = Not reported
    Eq. 2 combines five judge scores with weights alpha_j, but the paper never states their values or how they were chosen; this affects which self-generated code is selected as y_self.
  • Number of self-sampled responses n = 7
    Section 4.4 samples 7 responses per instruction at temperature 0.7; no sensitivity study is reported.
  • Sampling temperature = 0.7
    Section 4.4 sets temperature to 0.7 for ACR sampling; changing it changes the distribution of self-generated code and therefore the refined dataset.
assumptions (3)
  • domain assumption The composite scoring system and LLM-as-a-Critic accurately identify quality differences and produce useful critiques.
    Used in Sections 3.2 and 3.3; validated only partially in Section 5.2, and only on HumanEval+ and MBPP+.
  • domain assumption The initial 80K dataset D0 is high-quality and contamination-free.
    Asserted in Section 4.3 using GPT-4o and Evol-Instruct; checked only via the TLI metric in Section 5.1.
  • domain assumption Fine-tuning on the updated single-turn and two-turn samples improves one-shot pass@1 on held-out benchmarks.
    This transfer assumption underlies the iterative training in Section 3.4 and is tested in Section 4.5 with no variance or seed analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation." pith.science (2026). https://pith.science/paper/D5F6ZXKH

@misc{pith2026250209183,
  author       = {Pith},
  title        = {Pith review of: RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D5F6ZXKH}},
  note         = {Machine review of arXiv:2502.09183}
}
read the original abstract

Code generation has attracted increasing attention with the rise of Large Language Models (LLMs). Many studies have developed powerful code LLMs by synthesizing code-related instruction data and applying supervised fine-tuning. However, these methods are limited by teacher model distillation and ignore the potential of iterative refinement by self-generated code. In this paper, we propose Adaptive Critique Refinement (ACR), which enables the model to refine itself by self-generated code and external critique, rather than directly imitating the code responses of the teacher model. Concretely, ACR includes a composite scoring system with LLM-as-a-Judge to evaluate the quality of code responses and a selective critique strategy with LLM-as-a-Critic to critique self-generated low-quality code responses. We develop the RefineCoder series by iteratively applying ACR, achieving continuous performance improvement on multiple code generation benchmarks. Compared to the baselines of the same size, our proposed RefineCoder series can achieve comparable or even superior performance using less data.

Figures

Figures reproduced from arXiv: 2502.09183 by the authors.

Figure 1
Figure 1. Comparison of two fine-tuning paradigms. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of ACR in the t-th iteration. (1) Sampling: The model Mt samples n code responses {y i} n i=1 with high temperature for a programming instruction x in the dataset Dt. (2) Ranking: A composite scoring system first scores all self-generated code responses {y i} n i=1, then selects the highest-scoring response for pairwise comparison with the original code response y 0 to identify the winner y w and the loser … view at source ↗
Figure 3
Figure 3. Ablation Study. The y-axis denotes the average pass@1 value on LiveCodeBench and BigCodeBench-hard. 5.5 Evaluation with External Feedback Iterative ACR not only improves the one-off code generation performance of RefineCoder but also endows it with the ability to correct errors based on feedback. Following Zheng et al. (2024a), we design two types of external feedback to evaluate this ability of RefineCoder: 1) Exec… view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: Evaluation with feedback using RefineCoder-QW-7B. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Proportions of two types of data in the iterative [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: The average Pass@1 value of RefineCoder [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Scatter plot of similarity scores between the datasets and four benchmarks, with average similarity scores [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: The prompt template for pointwise judgment. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: The prompt template for pairwise judgment. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: The prompt template for pairwise critique. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: The prompt template for generating code-related concepts and instructions. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: The prompt template for Addition evolution. [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: The prompt template for Breath evolution. [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: The prompt template for Complexity evolution. [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: The prompt template for Concretizing evolution. [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: The prompt template for Deepening evolution. [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]
Figure 17
Figure 17. Figure 17: The prompt template for Diversion evolution. [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]
Figure 18
Figure 18. Figure 18: The prompt template for Increase evolution. [PITH_FULL_IMAGE:figures/full_fig_p018_18.png]
Figure 19
Figure 19. Figure 19: The prompt template for Misdirection evolution. [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: The prompt template for Reason evolution. [PITH_FULL_IMAGE:figures/full_fig_p018_20.png]
Figure 21
Figure 21. Figure 21: The prompt template for Execution Feedback. [PITH_FULL_IMAGE:figures/full_fig_p018_21.png]
Figure 22
Figure 22. Figure 22: The prompt template for generating improvement suggestions. [PITH_FULL_IMAGE:figures/full_fig_p019_22.png]
Figure 23
Figure 23. Figure 23: The prompt template for Human-like Feedback. [PITH_FULL_IMAGE:figures/full_fig_p019_23.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators

    cs.CL 2025-08 conditional novelty 6.0 of 10

    AutoCodeBench is an LLM-generated, sandbox-verified code benchmark with 3,920 problems across 20 languages, where top models reach only 52.4% pass@1.

  2. OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A 2.5M-example code reasoning dataset with critique traces enables Qwen2.5-based models to surpass prior open-weight distilled models on LiveCodeBench via test-time self-critique selection.

Reference graph

Works this paper leans on

12 extracted references · 9 canonical work pages · cited by 2 Pith papers

  1. [1]

    Question&Answer Consistency: Whether Q&A are consistent and correct

    work page

  2. [2]

    Code Exist: Whether the code is provied in question or answer

    work page

  3. [3]

    arXiv preprint arXiv:2406.15877

    Bigcodebench: Benchmarking code genera- tion with diverse function calls and complex instruc- tions. arXiv preprint arXiv:2406.15877. A Prompts for Judge and Critic The prompt for point/pair-wise LLM-as-a-Judge and LLM-as-a-Critic are shown in Figure 8, 9 and 10. B Prompts for Constructing SFT Dataset We called GPT-4o to create the SFT dataset, with the f...

    arXiv

  4. [4]

    Code Clarity: Assess how clear and understandable the code is

    work page

  5. [5]

    Code Comments: Evaluate the presence of comments and their usefulness in explaining the code’s functionality [QUESTION] {Programmin Question} [The start of RESPONSE] {The Response} [The end of RESPONSE] You need to use the following output format: <<<OUTPUT>>> Explanation: Here is a explanation Question&Answer Consistency: [[0-10]] Code Exist: [[0-10]] Co...

    work page

  6. [6]

    Code Correctness: Evaluate whether the provided code is free from syntax errors and logical flaws

    work page

  7. [9]

    Round1_from_User: The user asks a programming question, namely [Instruction]

    work page

  8. [10]

    Round1_from_Assistant: AI assistant give response, but the response is not a perfect code solution, namely [Response 1]

    work page

Show all 12 references
  1. [11]

    The executive results of the code interpreter and suggestions for improvement are then sent to the AI assistant, and ask AI to generate better responses

    Round2_from_User: The user analyzes whether the code meets the instruction and executes the code using the code interpreter. The executive results of the code interpreter and suggestions for improvement are then sent to the AI assistant, and ask AI to generate better responses

  2. [12]

    **Note**: You must focus on whether the code conforms to the instruction and the correctness of the code

    Round2_from_Assistant: AI assistant give a better response, namely [Response 2]. **Note**: You must focus on whether the code conforms to the instruction and the correctness of the code. Do not change instruction. To keep the conversation flowing and logical, you can rewrite o...

  3. [2023]

    In Thirty-seventh Conference on Neural Information Processing Sys- tems

    Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Sys- tems. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain S...

    2023 arXiv

  4. [2024]

    Preprint, arXiv:2407.05700

    Inversecoder: Self-improving instruction- tuned code llms with inverse-instruct. Preprint, arXiv:2407.05700. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024. WizardLM: Empow- ering large pre-trained language...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.