Alignment with Fill-In-the-Middle for Enhancing Code Generation

Aojun Zhou; Haotian Hou; Hongsheng Li; Houxing Ren; Junting Pan; Ke Wang; Mingjie Zhan; Weikang Shi; Yunqiao Yang; Zimu Lu

REVIEW 4 major objections 6 minor 8 references

Alignment with Fill-In-the-Middle for Enhancing Code Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read StructureCoder aligns code LLMs by applying DPO only to the filled-in middle of AST-split blocks, beating standard DPO by 1.5–1.6 pass@1 points.

desk verdict A genuinely novel FIM+DPO recipe for code alignment with consistent modest gains, but the attribution of gains to the loss formulation is not fully isolated because the key control is missing. read the letter →

arxiv 2508.19532 v1 pith:PSVSVCZV submitted 2025-08-27 cs.CL

Houxing Ren , Zimu Lu , Weikang Shi , Haotian Hou , Yunqiao Yang , Ke Wang , Aojun Zhou , Junting Pan

show 2 more authors

Mingjie Zhan Hongsheng Li

This is my paper · ORCID

classification cs.CL

keywords StructureCoderfill-in-the-middledirectpreferenceoptimizationcodegenerationalignmentabstractsyntaxtreecurriculumlearningpass@1LLM

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

StructureCoder is a data-efficient alignment method for code LLMs. It claims that applying direct preference optimization (DPO) to entire generated programs wastes most of the training signal, because correct and incorrect programs differ in only a few lines and the DPO loss penalises correct suffix tokens that follow an earlier mistake. The paper's solution is to split a reference solution into blocks at Abstract Syntax Tree (AST) boundaries, use the fill-in-the-middle capability to complete one block at a time, verify each completion against test cases, and train DPO only on the generated middle block. On HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench, the paper reports average pass@1 gains of 1.5 to 1.6 points over standard DPO across Qwen2.5-Coder 1.5B, 3B, and 7B. A sympathetic reader would care because the method makes scarce, test-case-verified data go further than whole-program DPO does.

What carries the argument

The central mechanism is FIM-enhanced DPO with AST block segmentation. Fill-in-the-middle (FIM) rearranges code as (pre, suf, mid) so the model predicts a missing middle given both surrounding context; the paper exploits this to control which tokens enter the DPO loss, keeping only the middle block. The AST segmentation restricts targets to four syntactic node types—if, for, while, and function definitions—so each middle segment is syntactically whole and independent. This machinery converts one test-case-verified solution into many verifiable preference pairs, which is what makes the limited APPS training set more usable.

What would settle it

Build a controlled dataset of preference pairs in which the earlier block defines or changes a variable the suffix uses, so the correct suffix differs between a good and a bad earlier block, then train with suffix loss included versus excluded; if including the suffix helps on those pairs, the independence premise fails and the claimed benefit of excluding suffix loss is not general.

Watch

Extended reading notes

Core claim

The paper establishes that the DPO loss decomposes into a middle-segment term and a suffix term when the preferred and dispreferred responses share the same prefix and suffix. It argues the suffix term is not just uninformative but actively harmful: a prior error is unrelated to the rest segment, so requiring the model to reduce probability of the suffix given an incorrect middle wrongly penalises correct tokens. StructureCoder therefore uses the fill-in-the-middle format—prefix and suffix as context, middle as the response—and restricts the DPO loss to the middle segment. To choose good middle segments, it parses the reference solution into an AST and targets only if, for, while, and function blocks; completions are verified against test cases, matched into preferred/dispreferred pairs by edit distance, and trained with a curriculum that orders blocks from short to long. The paper reports that this consistently outperforms standard DPO and KTO across all tested model sizes, with average gains of 1.5 to 1.6 pass@1 points over DPO.

Load-bearing premise

The load-bearing premise is that an error in one earlier code block is unrelated to the rest of the program, so the correct suffix does not depend on the mistaken block; if earlier bugs change what later code should be, dropping the suffix from the DPO loss throws away real signal.

Editorial extensions

If this is right

Restricting the DPO loss to the filled-in middle avoids rewarding or punishing the shared prefix and suffix, so correct tokens in unchanged code are not driven down.
One reference solution with test cases yields many distinct preference pairs, one per AST block, making DPO effective on the 5,000-sample APPS training set without generating extra test cases.
Curriculum ordering by block length lets the model learn token-level rewards on simpler blocks first, and the ablations attribute part of the final pass@1 gain to this ordering.
The gains replicate across model sizes (1.5B, 3B, 7B) and across benchmarks of varying difficulty, including LiveCodeBench and BigCodeBench.
Because training happens on middle completions, the model's fill-in-the-middle ability also improves, as measured by the appendix's FIM evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the suffix-independence premise generalises, the same trick could apply to any verifiable stepwise task where later steps are insensitive to earlier errors, not just code; the paper itself does not test this.
AST block selection is likely not the only viable segmentation; a testable extension is whether random blocks with equal structural diversity close the gap, which would separate segmentation from loss formulation.
The method's dependence on strong FIM capability implies that models without FIM pretraining may need FIM fine-tuning first; comparing FIM-strong and FIM-weak bases would bound the method's applicability.
Because the best variant on HumanEval used no curriculum, the curriculum may help hardest benchmarks but slightly hurt distributionally different ones; tuning curriculum by target distribution is an unexplored lever.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Needed to know: this is the first paper I've seen that pairs fill-in-the-middle with DPO for code alignment, using AST block segmentation to turn each solution into multiple preference pairs and curriculum ordering by block length. That's a real idea. Focused-DPO (concurrent) only handles common prefixes/suffixes; it doesn't generate FIM-style middle completions. The experiments are broad: three Qwen2.5-Coder sizes, five benchmark families, and ablations for each component. The gains over standard DPO are modest but consistent, around 1.5-1.6 pass@1 points average. The loss-on-suffix analysis is interesting and the LiveCodeBench result (w/ suf loss drops to 7.4 vs 10.8 for StructureCoder) is at least suggestive.

The main soft spot is the one the stress-test flags: no SFT-only baseline. Training on the preferred middle completions with cross-entropy, using the same FIM/chat formats and curriculum, would tell you whether the preference contrast is doing anything at all. The 'Data Equal' control doesn't answer it—it adds epochs over the same fixed DPO pairs, while StructureCoder creates many distinct FIM pairs per problem. So the improvement could be mostly FIM data augmentation, not the DPO-on-middle loss. That's a legit attribution gap, not a fatal one; the method still works, but the title's claim that the loss formulation is the driver is not pinned down.

Other issues, in decreasing importance: the significance markers (∗, †, ‡) claim level 0.05 but no test or variance is reported; PLUM and CodeDPO are cited but not compared, which is awkward given they're the closest DPO-for-code baselines; the code/data link has no commit hash. The Section 2.3 independence assumption ('a prior error is unrelated to the rest segment') is overstated—later code often depends on earlier blocks—but the empirical results partly buffer this.

Bottom line: solid, honest empirical work with a real novel component, and the limitations are stated. It deserves serious reviewing. I'd ask the authors for an SFT-only control, a proper data-matched comparison (same number of distinct pairs), and error bars before accepting. The paper is conditionally acceptable, not desk-rejectable.

Referee Report

4 major / 6 minor

Summary. The paper proposes StructureCoder, a DPO-based alignment method for code LLMs. The method parses golden solutions into an AST, extracts blocks (if/for/while/function), uses fill-in-the-middle prompts to generate several completions for each block, verifies full reconstructed programs against test cases, and constructs preference pairs between a correct and an incorrect middle completion. The DPO loss is applied only to the middle segment, with the training data randomly presented in either a FIM format or a chat format, and the samples are ordered by snippet length as a curriculum. Experiments on Qwen2.5-Coder-Instruct 1.5B/3B/7B report pass@1 gains over standard DPO, KTO, and Focused-DPO on HumanEval(+), MBPP(+), APPS, LiveCodeBench, and BigCodeBench, with average improvements of 1.5 to 1.6 points over DPO.

Significance. If the central claim holds, the paper offers a data-efficient way to improve code generation alignment without generating synthetic test cases, by re-using a small verified dataset to create many fine-grained preference pairs. The evaluation uses held-out benchmarks that are separate from the APPS training set, and the authors state that code and data will be released, which are strengths. However, the attribution of the gains to the FIM-DPO loss formulation rather than to the increased diversity of FIM-style training examples is not yet established, because the ablation suite lacks an SFT-only control and the reported data-equal control is not actually data-equal.

major comments (4)

[§4.3.1, Table 2] The ablation suite does not include an SFT-only baseline trained on the preferred middle completions with the same FIM/chat formats, curriculum, and epoch count. Such a control would test whether the DPO preference contrast contributes at all, as opposed to the gains arising from the larger and more diverse set of FIM-style training examples. Without it, the central claim that the FIM-DPO loss formulation drives the improvement is not established.
[§4.3.1, Table 2 (w/ DPO (Data Equal))] The 'Data Equal' control is not data-equal. It trains additional epochs over the same original full-response DPO pairs, whereas StructureCoder creates multiple distinct FIM pairs per problem (five completions per AST block, several blocks per solution). Increasing epochs over a fixed small set cannot reproduce the diversity of distinct pairs; the sharp drop to 66.5 on HumanEval is consistent with overfitting rather than with a controlled comparison of data quantity. A proper control should match the number of distinct training pairs, for example by using the same FIM-generated pairs with a cross-entropy loss on the preferred completions.
[§2.3] The theoretical motivation rests on the assertion that 'a prior error is unrelated to the rest segment.' This is too strong: in real code, an error in an earlier block can change control flow, variable definitions, or function behavior, so the correct suffix may depend on the erroneous block. The derivation in §2.3 assumes the suffix is identical and independent in the preferred and dispreferred responses. The authors should either weaken the theoretical claim or test the assumption directly, for example by constructing preference pairs where the correct suffix changes when the middle is erroneous and measuring whether excluding suffix loss still helps.
[§4.2, Table 1] The significance markers (*, †, ‡) are not accompanied by any description of the statistical test, number of independent runs, variance, or error bars. Given that the reported average gains over DPO are 1.5 to 1.6 pass@1 points, the paper should report at least the variance across seeds or bootstrap confidence intervals and specify the test used for the 0.05 level.

minor comments (6)

[§3.3 and Introduction] The curriculum is described as ordering by 'depth of the target block' in the introduction but by 'number of lines' in §3.3 and Algorithm 1; please unify the description.
[Table 2, w/o suf row] The sentence 'w/o suf remove suffix in the whole pipeline, a.k.a.,, we only use...' contains a typo and should be rewritten.
[Figure 5 caption] The phrase 'we omit the previous function' should be 'we omit the preceding function.'
[Ethics Statement] The ethics statement cites 'Li et al., 2023' as the source for Qwen2.5-Coder; Qwen2.5-Coder is from Hui et al., 2024, while Li et al. 2023 is StarCoder, so the citation should be corrected.
[§4.1] The batch size and max sequence length are given, but the number of preference pairs retained after test-case filtering is not reported; reporting this number would aid reproducibility.
[Appendix C.1] The heading 'FIM Evaluaion' should be 'FIM Evaluation.'

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central FIM-DPO claim is empirically tested on external benchmarks and supported by ablations, though the 'Data Equal' control is a non-circular experimental weakness.

full rationale

The paper's central claim is empirical: StructureCoder is trained on the APPS training set and evaluated on held-out benchmarks (HumanEval(+), MBPP(+), APPS, LiveCodeBench, BigCodeBench) that are disjoint from the training data, with test-case execution serving as an external verifier. No parameter is fitted to the target benchmark scores, so the reported gains are not predictions that reduce to their inputs by construction. The derivation of the suffix-loss decomposition in Section 2.3 is original and rests on external references (Rafailov et al. 2024; Bavarian et al. 2022); the paper cites several works from the same group (e.g., Lu et al. 2024a,b; Ren et al. 2024a,b), but these are background or inspiration, not load-bearing justifications for the core result. The 'w/ DPO (Data Equal)' ablation in Table 2 is mislabeled because it adds epochs over the same fixed preference pairs rather than matching the number of distinct FIM-derived pairs, and no SFT-only baseline is run; this weakens causal attribution of the gains to the DPO-on-middle loss specifically, but it is an experimental confound rather than circular reasoning. No equation in the paper equates a claimed prediction to an input by definition, and no self-citation chain forces the outcome, so the circularity score is low.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. The free parameters are training and data-generation hyperparameters chosen by hand. The axioms are mostly standard assumptions about DPO, test-case correctness, and AST block independence, plus one ad hoc assumption about error independence that is central to the loss modification.

free parameters (4)

alpha (format mixing probability) = 0.5
Controls the Bernoulli draw between FIM-format and Chat-format DPO samples. Ablated at 0 and 1 but not tuned; directly affects training data format mix and final performance.
number of sampled completions per block = 5
The model generates five middle completions per FIM prompt with top_p=0.95 and temperature=0.7. This is a hand-chosen sampling budget that determines how many preference pairs can be formed.
training epochs = 3
All models are fine-tuned for 3 epochs. The ablation w/ DPO (Data Equal) shows that simply increasing epochs does not reproduce the gains, so this hyperparameter matters for the comparison.
learning rate = 1e-6
Standard fine-tuning hyperparameter with RMSProp and cosine schedule. Not central to the method but required for exact reproduction.

assumptions (4)

domain assumption DPO implicitly learns token-level reward functions within an MDP framework (Rafailov et al., 2024).
Used in Section 2.3 to argue that DPO needs large data to capture fine-grained token rewards and to analyze the loss on prefix and suffix segments. This is an imported theoretical result, not proven here.
domain assumption Test cases in the training set are correct and sufficient verifiers of code completion.
The entire data construction pipeline relies on test cases to label completions as preferred or dispreferred. If test cases are incomplete or wrong, the preference labels are unreliable.
domain assumption AST blocks of types if, for, while, and def are syntactically complete and functionally independent code units.
Section 3.2 assumes these blocks can be masked and completed independently. This enables the FIM setup but may fail for blocks that depend on surrounding context.
ad hoc to paper An error in an earlier code block is unrelated to the correctness or desirability of later suffix tokens.
Section 2.3 states 'a prior error is unrelated to the rest segment' to justify removing suffix loss from DPO. This is a strong assumption about code semantics and is the load-bearing premise for the method's loss design.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Alignment with Fill-In-the-Middle for Enhancing Code Generation." pith.science (2026). https://pith.science/paper/PSVSVCZV

@misc{pith2026250819532,
  author       = {Pith},
  title        = {Pith review of: Alignment with Fill-In-the-Middle for Enhancing Code Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PSVSVCZV}},
  note         = {Machine review of arXiv:2508.19532}
}

read the original abstract

The code generation capabilities of Large Language Models (LLMs) have advanced applications like tool invocation and problem-solving. However, improving performance in code-related tasks remains challenging due to limited training data that is verifiable with accurate test cases. While Direct Preference Optimization (DPO) has shown promise, existing methods for generating test cases still face limitations. In this paper, we propose a novel approach that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases. Additionally, we introduce the Abstract Syntax Tree (AST) splitting and curriculum training method to enhance the DPO training. Our approach demonstrates significant improvements in code generation tasks, as validated by experiments on benchmark datasets such as HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench. Code and data are available at https://github.com/SenseLLM/StructureCoder.

Figures

Figures reproduced from arXiv: 2508.19532 by the authors.

**Figure 2.** An overview of our FIM-style preference modeling process. A concrete example (right) illustrates the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Illustration of our AST-based segmentation [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Distribution of extracted code blocks based on [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: Credit assignment with different methods. Due to the limited space, we omit the previous function, only [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

**Figure 6.** Figure 6: Credit assignment with different methods on Qwen2.5-Coder-3B-Instruct. The left is the correct response [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: Credit assignment with different methods on Qwen2.5-Coder-7B-Instruct. The left is the correct response [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 1 canonical work pages

[3]

In The Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024

Let’s verify step by step. In The Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open- Review.net. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Ling- ming Zhang. 2023. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. In Advanc...

arXiv 2024
[6]

CoRR, abs/2308.12950

Code llama: Open foundation models for code. CoRR, abs/2308.12950. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. CoRR, abs/1707.06347. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, ...

arXiv 2017
[7]

CoRR, abs/2407.05700

Inversecoder: Unleashing the power of instruction-tuned code llms with inverse-instruct. CoRR, abs/2407.05700. Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu

arXiv
[8]

In Forty-first International Conference on Machine Learning, ICML 2024, Vi- enna, Austria, July 21-27, 2024

Is DPO superior to PPO for LLM alignment? A comprehensive study. In Forty-first International Conference on Machine Learning, ICML 2024, Vi- enna, Austria, July 21-27, 2024. OpenReview.net. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayi- heng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Z...

arXiv 2024
[459]

Houxing Ren, Linjun Shou, Ning Wu, Ming Gong, and Daxin Jiang

Association for Computational Linguistics. Houxing Ren, Linjun Shou, Ning Wu, Ming Gong, and Daxin Jiang. 2022b. Empowering dual-encoder with query generator for cross-lingual dense retrieval. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 3...

arXiv 2022
[626]

Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xi- angru Peng, and Jiaya Jia

ACM. Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xi- angru Peng, and Jiaya Jia. 2024. Step-dpo: Step-wise preference optimization for long-chain reasoning of llms. CoRR, abs/2406.18629. Bin Lei, Yuchen Li, and Qiuwu Chen. 2024. Autocoder: Enhancing code large language model with aiev- instruct. CoRR, abs/2405.14906. Raymond Li, Loubna Ben Allal, Yan...

arXiv 2024
[2023]

Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Sys- tems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: memory optimiza...

work page 2023
[2024]

CoRR, abs/2406.11931

Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence. CoRR, abs/2406.11931. Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Xuanjing Huang, and Tao Gui. 2024. Stepcoder: Improve code generation wit...

arXiv 2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [3]

In The Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024

Let’s verify step by step. In The Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open- Review.net. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Ling- ming Zhang. 2023. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. In Advanc...

arXiv 2024

[2] [6]

CoRR, abs/2308.12950

Code llama: Open foundation models for code. CoRR, abs/2308.12950. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. CoRR, abs/1707.06347. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, ...

arXiv 2017

[3] [7]

CoRR, abs/2407.05700

Inversecoder: Unleashing the power of instruction-tuned code llms with inverse-instruct. CoRR, abs/2407.05700. Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu

arXiv

[4] [8]

In Forty-first International Conference on Machine Learning, ICML 2024, Vi- enna, Austria, July 21-27, 2024

Is DPO superior to PPO for LLM alignment? A comprehensive study. In Forty-first International Conference on Machine Learning, ICML 2024, Vi- enna, Austria, July 21-27, 2024. OpenReview.net. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayi- heng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Z...

arXiv 2024

[5] [459]

Houxing Ren, Linjun Shou, Ning Wu, Ming Gong, and Daxin Jiang

Association for Computational Linguistics. Houxing Ren, Linjun Shou, Ning Wu, Ming Gong, and Daxin Jiang. 2022b. Empowering dual-encoder with query generator for cross-lingual dense retrieval. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 3...

arXiv 2022

[6] [626]

Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xi- angru Peng, and Jiaya Jia

ACM. Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xi- angru Peng, and Jiaya Jia. 2024. Step-dpo: Step-wise preference optimization for long-chain reasoning of llms. CoRR, abs/2406.18629. Bin Lei, Yuchen Li, and Qiuwu Chen. 2024. Autocoder: Enhancing code large language model with aiev- instruct. CoRR, abs/2405.14906. Raymond Li, Loubna Ben Allal, Yan...

arXiv 2024

[7] [2023]

Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Sys- tems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: memory optimiza...

work page 2023

[8] [2024]

CoRR, abs/2406.11931

Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence. CoRR, abs/2406.11931. Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Xuanjing Huang, and Tao Gui. 2024. Stepcoder: Improve code generation wit...

arXiv 2024