Pith. sign in

REVIEW 5 major objections 7 minor 62 references

LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information

T0 review · 5 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Stepwise preference learning improves long-form generation length and quality while keeping general-task performance nearly lossless.

desk verdict A practical stepwise-preference recipe for long-form generation with consistent gains over DPO, but the single-judge pipeline and missing variance leave the exact improvement magnitude unresolved. read the letter →

arxiv 2502.02095 v2 pith:IGTUER76 submitted 2025-02-04 cs.CL

Bowen Ping , Jiali Zeng , Fandong Meng , Shuo Wang , Jie Zhou , Shanghang Zhang This is my paper · ORCID
classification cs.CL
keywords long-formgenerationprocesssupervisionMonteCarloTreeSearchdirectpreferenceoptimizationstepwiselearningLLMcritiquesmemorypoolfactualconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-form generation—essays, reports, repo-level code—remains weak even in large models because feedback is usually given once, on the final text. LongDPO claims that supervising each intermediate step works better. The paper collects stepwise preference pairs with Monte Carlo Tree Search, filters candidates through a global memory pool to preserve factual consistency, refines low-reward chosen steps with external critiques, and then trains with step-level DPO. On LongBench-Write and LongGenBench, both Llama- and Qwen-based models improve in length and quality while keeping general benchmarks near lossless. If the claim holds, it offers a practical recipe for turning any long-context model into a better long-form writer.

What carries the argument

The central mechanism is the stepwise preference pair built from a Monte Carlo Tree Search. The paper splits generation into steps, expands each node into child candidates, and scores each node with a reward model using seven quality principles. A global memory pool stores extracted factual statements from earlier steps and rejects candidate steps that contradict them, preserving consistency. For chosen steps with low rewards, external critiques generated by a stronger model supply writing suggestions that are prepended to the context when regenerating the step. Training then uses a step-level DPO objective that maximizes the log-likelihood of the chosen step and minimizes the rejected step conditioned on the query plus all preceding steps.

What would settle it

Run the same MCTS pipeline but score the step pairs with human annotators and train on those human-preference pairs: if the model-judge scores disagree with humans on enough pairs, the reported gains should shrink or vanish. A simpler check is to replace the 70B judge with a different judge and see if the quality advantage over outcome-level DPO persists.

Watch

Extended reading notes

Core claim

LongDPO's central discovery is that outcome-level DPO gives weak and sometimes harmful supervision for long-form generation, whereas stepwise supervision with the same model family yields consistent gains. Concretely, the paper decomposes a response into steps, uses MCTS to generate candidate steps, keeps only factually consistent ones via a memory pool, refines low-reward chosen steps by injecting external critiques, and optimizes a stepwise DPO objective. Across LongWriter-Llama-8B and LongWriter-Qwen2.5-7B, this improves length matching and quality on LongBench-Write and completion and instruction adherence on LongGenBench, with almost no drop on TruthfulQA, MMLU, and GSM8K. The authors state this as evidence that fine-grained, step-level learning is better suited to long-form generation than sample-level preference learning.

Load-bearing premise

The whole pipeline trusts one LLM judge (Llama-3.1-70B-Instruct) to score every step on seven principles, extract factual statements, check consistency, and generate critiques; the paper does not validate these scores against human judgments, so if that judge is biased or noisy the preference pairs and refinements are corrupted before training.

Editorial extensions

If this is right

  • On LongBench-Write, both Llama- and Qwen-based LongWriter models trained with LongDPO improve length score and quality score over the same models trained with vanilla DPO, with the largest length gains in the 4k–20k word range.
  • On LongGenBench, the method raises completion rate and instruction-following metrics at both 16k and 32k lengths, indicating that stepwise supervision helps models stick to long-range instructions.
  • Training with the global memory pool improves fact-checking performance on LLM-AggreFact compared to training without it, connecting the preference data collection step to reduced hallucination.
  • External critiques give more stable gains than self-critiques when refining low-reward chosen candidates, as shown by varying the refinement threshold η.
  • General benchmarks (TruthfulQA, MMLU, GSM8K) remain nearly lossless after LongDPO, whereas plain DPO in some cases degrades them (e.g., MMLU for the Qwen backbone).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable corollary of the paper's argument is that the gains are driven by step-level separability, not by the MCTS tree itself; any procedure producing locally ranked step pairs should reproduce part of the effect.
  • Because the entire pipeline depends on one judge's seven-principle scores, a low-cost empirical check would be to re-run the pipeline with a second judge on a subset of instructions and measure how much the chosen/rejected pairs change and how much downstream DPO performance shifts.
  • The memory pool regularizes factuality; one natural extension is applying the same consistency check to long-form summarization, where grounding documents are available, to see whether the factuality gains transfer.
  • The stepwise DPO objective could in principle be applied to other long-output domains such as repository-level code generation, though the paper only evaluates text writing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. LongDPO proposes a process-supervision variant of DPO for long-form generation. The method first constructs stepwise preference pairs via Monte Carlo Tree Search (MCTS), using a global memory pool to enforce factual consistency across steps and an external LLM (Llama-3.1-70B-Instruct) to generate critiques that refine low-reward chosen candidates. It then trains with a step-level DPO objective (Eq. 9) rather than outcome-level DPO. The method is evaluated on LongBench-Write, LongGenBench, and general benchmarks (TruthfulQA, MMLU, GSM8K) on LongWriter-Llama-3.1-8B and LongWriter-Qwen2.5-7B, with an additional larger-model experiment in Table 13. The paper reports consistent improvements over the vanilla DPO baselines, and ablations attribute the gains to the critique component, the memory pool, and the stepwise objective.

Significance. If the reported effects are reliable, LongDPO is a useful recipe for step-level preference learning in open-ended long-form generation, where outcome-level DPO has previously shown limited or unstable gains. The pipeline is concrete and the ablations isolate the main components: external critiques improve over self-critique or no critique (Table 3), the memory pool helps on a fact-checking transfer test (Table 4), and the stepwise objective helps over vanilla DPO on LongGenBench (Table 5). The paper ships a reproducible protocol with fixed seeds, official evaluation scripts, and promises to release code and models. However, the central claim is weakened by the lack of validation of the single LLM judge that produces every supervision signal, the absence of error bars or significance tests, and the apparent selection of the important hyperparameter η on the evaluation benchmark.

major comments (5)
  1. [§3.1–§3.2, Eq. (3), Eq. (6), Table 6] The entire preference-construction pipeline — node rewards (Eq. 3), factual-content extraction, consistency judgments, and critique generation — is produced by a single model Θ = Llama-3.1-70B-Instruct. The paper provides no agreement statistics between Θ and human judges, no correlation between Θ rewards and the GPT-4o quality scores used in Table 1, and no held-out reward-model validation. The human evaluation in Table 6 compares only final outputs of LongWriter versus LongDPO and therefore does not verify that the stepwise preferences or critiques are correct. Because the outcome-level DPO baseline also selects its chosen response with Θ, the head-to-head comparison in Tables 1 and 2 partly measures how effectively LongDPO optimizes Θ's idiosyncratic scoring function rather than a general long-form quality. This is load-bearing for the central claim: without judge validation, the magnitude of the reported gains cannot be separated from reward overfitting.
  2. [Tables 1, 2, 13; §4.2] The paper reports a single run per condition with fixed seeds and describes results as 'significantly outperforms' without reporting confidence intervals, multiple seeds, or hypothesis tests. Several differences are small in absolute terms (e.g., LongGenBench STC1 for Llama: 27.59 vs 25.99; STC2: 18.45 vs 16.29; CR for Qwen at 16k: 98.51 vs 95.95), so it is unclear whether these are statistically reliable. The authors should report at least three training seeds with means and standard deviations, or a paired test across evaluation examples, before claiming statistical superiority.
  3. [§3.2, Eq. (6); §4.4; Table 3] The threshold η = 2.5 used in the main experiments is not accompanied by a statement that it was chosen on a held-out validation split. The ablation in Section 4.4 evaluates η ∈ {2.0, 2.5, 3.0} on LongBench-Write, the same benchmark used for the headline results in Table 1, and the 'LongDPO' row in Table 3 is the average over these three values rather than a single configuration. If η was selected on this benchmark, the main comparison is partially tuned on the test set, which inflates the apparent improvement. The paper should specify the validation procedure for hyperparameters or report results for all η values transparently in the main table.
  4. [§4.6, Table 5] The comparison between 'w/o Stepwise' and 'w/ Stepwise' is meant to isolate the stepwise DPO objective, but the paper does not specify how the non-stepwise baseline consumes the same preference data. If 'w/o Stepwise' concatenates the step-level chosen and rejected segments into full responses and applies Eq. (8), the data distribution differs from the standard outcome-DPO baseline in Section 4.1; if it instead applies a different loss to the step-level pairs, the comparison is not a clean test of stepwise learning. The paper should specify the exact training objective and data representation for 'w/o Stepwise' to make this ablation interpretable.
  5. [Appendix A.1; §1 and Table 1] The reward-evaluation template explicitly instructs Θ not to consider whether the response meets the user's length requirements ('You do not need to consider whether the response meets the user's length requirements in your evaluation'). Yet Table 1 reports consistent improvements in the length score Sl. The paper does not explain the mechanism by which stepwise preferences that ignore length improve length adherence. This is not a fatal inconsistency, but it should be discussed: either length gains arise indirectly from other criteria (e.g., completeness or formatting) or from the training distribution, and the claim that process supervision directly addresses length deviations (Section 1) needs qualification.
minor comments (7)
  1. [Author affiliations] There is a typo in the affiliation: 'Pattern Recongnition Center' should be 'Pattern Recognition Center.'
  2. [§3.3, Eq. (8)] The reference model π_ref is not defined; the paper should state that π_ref is the base LongWriter model (or the SFT model) used for DPO.
  3. [Table 3 caption] The caption says 'we set different values of η and report the average result,' which means the 'LongDPO' row is not the same configuration as in Table 1; this should be stated explicitly to avoid confusing the ablation with the main method.
  4. [Table 7] Inter-annotator agreement is reported as pairwise percentages without a chance-corrected statistic such as Cohen's κ; adding κ would strengthen the claim of high agreement.
  5. [§4.1] The data-collection seed is described only as 'a fixed random seed'; the actual seed value should be reported for reproducibility.
  6. [Limitations] The Limitations section only mentions the 32K evaluation ceiling and does not acknowledge the single-judge dependence or the lack of significance testing; these are relevant limitations that readers should be told about.
  7. [§3.3, Eq. (9)] The notation s_{1∼i} is used before it is defined in the sentence following the equation; define it at first occurrence.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training-time reward judge and the evaluation-time benchmarks are distinct, so the reported gains are not forced by construction.

full rationale

LongDPO's central claim is that critique-augmented stepwise DPO improves long-form generation relative to outcome-level DPO. The preference signal is produced by a fixed external judge (Llama-3.1-70B-Instruct) via MCTS, while the reported results are measured on external benchmarks: LongBench-Write quality via GPT-4o, LongGenBench, TruthfulQA, MMLU, and GSM8K, plus a human study. The training objective in Eq. 9 is standard step-level DPO and does not contain the evaluation metric. Notably, the judge's reward template explicitly excludes the length criterion ('You do not need to consider whether the response meets the user’s length requirements'), so the length-score improvements cannot reduce to the reward function. The ablations (w/o critique, self-critique, w/o stepwise) are compared on the same external benchmarks, and the central comparisons hold there. The only self-citations (Ping et al., 2024; Wang et al., 2024b) appear as background references in the introduction and are not load-bearing for the method or its validation. The concern that one unvalidated judge generates all preferences and critiques is a reliability/validity risk, not a circularity: if that judge's preferences diverged from GPT-4o or human judgment, the observed external gains would not appear. No prediction in the paper reduces by construction to a fitted parameter or to a self-citation chain.

Assumptions & free parameters 6 free parameters · 6 assumptions · 1 invented entities

The core pipeline relies on a handful of hand-chosen hyperparameters (alpha, delta, eta, tree depth, width, chunk size) and on several domain assumptions about the reliability of LLM-based scoring, fact-checking, and critiquing. These are not derived from theory or validated independently, which is typical for an empirical preference-learning recipe but should be kept in view.

free parameters (6)
  • alpha (UCB exploration scalar) = not disclosed
    Equation 1 balances exploration and exploitation in MCTS node selection; chosen by hand, no sensitivity study.
  • delta (similarity threshold) = 0.8
    Equation 5 filters irrelevant context segments in memory consistency check; fixed with no ablation.
  • eta (critique refinement threshold) = 2.5
    Equation 6 selects chosen candidates with average reward below this threshold for critique-augmented regeneration; ablation over {2.0, 2.5, 3.0} is provided.
  • Maximum MCTS tree depth = 4
    Section 4.1; limits the number of generation steps, set by hand without study.
  • Child nodes per expansion = 4
    Section 4.1; number of sampled continuations per node, set by hand.
  • Maximum tokens per node = 2048
    Section 4.1; chunk size that defines a 'step', central to stepwise supervision.
assumptions (6)
  • standard math UCB1 selection formula with exploration scalar alpha is a valid exploration strategy for MCTS over text generation
    Equation 1 from Browne et al.; standard in MCTS literature.
  • domain assumption A long response can be decomposed into sequential chunks (steps) that can be independently evaluated and optimized
    Section 3.1.1 and 3.3; the stepwise DPO loss treats each chunk as a decision conditioned on the prefix.
  • domain assumption The seven-principle reward rubric from Llama-3.1-70B provides a valid scalar preference signal for intermediate text steps
    Section 3.1.3 and Appendix A.1; no validation of the reward model against human labels is provided.
  • domain assumption Embedding similarity threshold delta=0.8 with gte-Qwen2-1.5B reliably retrieves context relevant to factual consistency checking
    Section 3.1.2 Eq. 4-5; threshold is fixed without sensitivity analysis.
  • domain assumption The judge LLM's internal knowledge is a reliable ground truth for fact extraction and contradiction detection
    Section 3.1.2 memory pool updates; factual statements are verified only against Theta's parametric knowledge.
  • ad hoc to paper External critiques from Llama-3.1-70B-Instruct improve the quality of chosen candidates for DPO training
    Section 3.2; the critique-augmented generation step is specific to this method and is not compared against a human-critique baseline.
invented entities (1)
  • Global memory pool M
    purpose: Store factual context extracted from previously generated steps to filter inconsistent nodes during MCTS selection
    New component introduced in this paper; no external validation beyond the LLM-AggreFact evaluation of the final model, so its standalone evidence is weak.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information." pith.science (2026). https://pith.science/paper/IGTUER76

@misc{pith2026250202095,
  author       = {Pith},
  title        = {Pith review of: LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IGTUER76}},
  note         = {Machine review of arXiv:2502.02095}
}
read the original abstract

Long-form generation is crucial for academic writing papers and repo-level code generation. Despite this, current models, including GPT-4o, still exhibit unsatisfactory performance. Existing methods that utilize preference learning with outcome supervision often fail to provide detailed feedback for extended contexts. This shortcoming can lead to content that does not fully satisfy query requirements, resulting in issues like length deviations, and diminished quality. In this paper, we propose enhancing long-form generation by incorporating process supervision. We employ Monte Carlo Tree Search to gather stepwise preference pairs, utilizing a global memory pool to maintain consistency. To address the issue of suboptimal candidate selection, we integrate external critiques to refine and improve the quality of the preference pairs. Finally, we apply step-level DPO using the collected stepwise preference pairs. Experimental results show that our method improves length and quality on long-form generation benchmarks, with almost lossless performance on general benchmarks across various model backbones.

Figures

Figures reproduced from arXiv: 2502.02095 by the authors.

Figure 1
Figure 1. The above refers to outcome supervision, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of LongDPO. LongDPO incorporates process supervision and MCTS to collect stepwise [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Main body of generated critiques which have [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: A case is randomly sampled from LongGenBench. The instruction primarily requires visiting the farmers’ [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Reward analysis of the selected candidates, we focus solely on the chosen candidate in each preference [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Detailed reward analysis of the chosen candidates. [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: The part highlighted in red is the correct answer to the question. LongWriter-Llama fails to provide the [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: The part highlighted in red is the correct answer to the question. LongWriter-Llama fails to provide the [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 12 canonical work pages

  1. [1]

    Chang, and Prithviraj Ammanabrolu

    Zachary Ankner, Mansheej Paul, Brandon Cui, Jonathan D. Chang, and Prithviraj Ammanabrolu. 2024. https://arxiv.org/abs/2408.11791 Critique-out-loud reward models . Preprint, arXiv:2408.11791

    arXiv 2024

  2. [2]

    Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, and Juanzi Li. 2024 a . https://aclanthology.org/2024.findings-emnlp.74 L ong A lign: A recipe for long context alignment of large language models . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1376--1395, Miami, Florida, USA. Association fo...

    work page 2024

  3. [3]

    Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024 b . https://arxiv.org/abs/2408.07055 Longwriter: Unleashing 10,000+ word generation from long context llms . Preprint, arXiv:2408.07055

    arXiv 2024

  4. [4]

    Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024 c . Longwriter: Unleashing 10,000+ word generation from long context llms. https://openreview.net/forum?id=kQ5s9Yh0WI. OpenReview submission

    work page 2024

  5. [5]

    Le, Christopher Ré, and Azalia Mirhoseini

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, and Azalia Mirhoseini. 2024. https://arxiv.org/abs/2407.21787 Large language monkeys: Scaling inference compute with repeated sampling . Preprint, arXiv:2407.21787

    arXiv 2024

  6. [6]

    Browne, Edward Powley, Daniel Whitehouse, Simon M

    Cameron B. Browne, Edward Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. 2012. https://doi.org/10.1109/TCIAIG.2012.2186810 A survey of monte carlo tree search methods . IEEE Transactions on Computational Intelligence and AI in Games, 4(1):1--43

    arXiv 2012

  7. [7]

    Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. 2024 a . https://arxiv.org/abs/2405.03553 Alphamath almost zero: Process supervision without process . Preprint, arXiv:2405.03553

    arXiv 2024

  8. [8]

    Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2024 b . https://arxiv.org/abs/2410.08115 Optima: Optimizing effectiveness and efficiency for llm-based multi-agent system . Preprint, arXiv:2410.08115

    arXiv 2024

Show all 62 references
  1. [9]

    Jiale Cheng, Xiao Liu, Cunxiang Wang, Xiaotao Gu, Yida Lu, Dan Zhang, Yuxiao Dong, Jie Tang, Hongning Wang, and Minlie Huang. 2024. https://arxiv.org/abs/2412.11605 Spar: Self-play with tree-search refinement to improve instruction-following in large language models . Preprint...

    2024 arXiv

  2. [10]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . ...

    2021 arXiv

  3. [11]

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. https://openreview.net/forum?id=BOorDpKHiJ ULTRAFEEDBACK: boosting language models with scaled AI feedback . In Forty-first...

    2024

  4. [12]

    Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, Yuanyuan Xu, Ning Shang, Jiahang Xu, Fan Yang, and Mao Yang. 2024. https://openreview.net/forum?id=ONOtpXLqqw Longrope: Extending LLM context window beyond 2 million tokens . In Forty-first International Conference on Machine Lear...

    2024

  5. [13]

    Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng. 2024. https://openreview.net/forum?id=TaAqeo7lUh Data engineering for scaling language models to 128k context . In Forty-first International Conference on Machine Learning, ICML 2024, V...

    2024

  6. [14]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024 doi

  7. [15]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  8. [16]

    Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. https://arxiv.org/abs/2501.04519 rstar-math: Small llms can master math reasoning with self-evolved deep thinking . Preprint, arXiv:2501.04519

    2025 arXiv

  9. [17]

    Chaoqun He, Renjie Luo, Shengding Hu, Ranchi Zhao, Jie Zhou, Hanghao Wu, Jiajie Zhang, Xu Han, Zhiyuan Liu, and Maosong Sun. 2024. https://doi.org/10.18653/v1/2024.acl-demos.23 U ltra E val: A lightweight platform for flexible and comprehensive evaluation for LLM s . In Procee...

    2024 doi

  10. [18]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  11. [19]

    The curious case of neural text degeneration

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations

  12. [20]

    Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xiangru Peng, and Jiaya Jia. 2024. https://arxiv.org/abs/2406.18629 Step-dpo: Step-wise preference optimization for long-chain reasoning of llms . Preprint, arXiv:2406.18629

    2024 arXiv

  13. [21]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. https://openreview.net/forum?id=v8L0pN6EOi Let's verify step by step . In The Twelfth International Conference on Learning Re...

    2024

  14. [22]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. https://doi.org/10.18653/V1/2022.ACL-LONG.229 Truthfulqa: Measuring how models mimic human falsehoods . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2...

    2022 doi

  15. [23]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. https://arxiv.o...

    2023 arXiv

  16. [24]

    Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, and Jan Leike. 2024. Llm critics help catch llm bugs. arXiv preprint arXiv:2407.00215

    2024 arXiv

  17. [25]

    Tsendsuren Munkhdalai, Manaal Faruqui, and Siddharth Gopal. 2024. https://arxiv.org/abs/2404.07143 Leave no context behind: Efficient infinite context transformers with infini-attention . Preprint, arXiv:2404.07143

    2024 arXiv

  18. [26]

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Pai...

    2024 arXiv

  19. [27]

    Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. 2024. https://openreview.net/forum?id=wHBfxhZu1u Yarn: Efficient context window extension of large language models . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria,...

    2024

  20. [28]

    Chau Pham, Simeng Sun, and Mohit Iyyer. 2024. https://aclanthology.org/2024.findings-emnlp.94 Suri: Multi-constraint instruction following in long-form text generation . In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-...

    2024

  21. [29]

    Bowen Ping, Shuo Wang, Hanqing Wang, Xu Han, Yuzhuang Xu, Yukun Yan, Yun Chen, Baobao Chang, Zhiyuan Liu, and Maosong Sun. 2024. https://arxiv.org/abs/2406.08903 Delta-come: Training-free delta-compression with mixed-precision for large language models . Preprint, arXiv:2406.08903

    2024 arXiv

  22. [30]

    Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang. 2024. https://arxiv.org/abs/2408.06195 Mutual reasoning makes smaller llms stronger problem-solvers . Preprint, arXiv:2408.06195

    2024 arXiv

  23. [31]

    Shanghaoran Quan, Tianyi Tang, Bowen Yu, An Yang, Dayiheng Liu, Bofei Gao, Jianhong Tu, Yichang Zhang, Jingren Zhou, and Junyang Lin. 2024. https://arxiv.org/abs/2410.23933 Language models can self-lengthen to generate long texts . Preprint, arXiv:2410.23933

    2024 arXiv

  24. [32]

    Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. 2024. https://arxiv.org/abs/2410.08146 Rewarding progress: Scaling automated process verifiers for llm reasoning . Preprint, arXiv:2410.08146

    2024 arXiv

  25. [33]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. https://arxiv.org/abs/2408.03314 Scaling llm test-time compute optimally can be more effective than scaling model parameters . Preprint, arXiv:2408.03314

    2024 arXiv

  26. [34]

    Liyan Tang, Philippe Laban, and Greg Durrett. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.499 M ini C heck: Efficient fact-checking of LLM s on grounding documents . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8818--88...

    2024 doi

  27. [35]

    Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Haitao Mi, and Dong Yu. 2024. https://arxiv.org/abs/2404.12253 Toward self-improvement of llms via imagination, searching, and criticizing . Preprint, arXiv:2404.12253

    2024 arXiv

  28. [36]

    Ante Wang, Linfeng Song, Ye Tian, Baolin Peng, Dian Yu, Haitao Mi, Jinsong Su, and Dong Yu. 2024 a . https://arxiv.org/abs/2407.00320 Litesearch: Efficacious tree search for llm . Preprint, arXiv:2407.00320

    2024 arXiv

  29. [37]

    Hanqing Wang, Bowen Ping, Shuo Wang, Xu Han, Yun Chen, Zhiyuan Liu, and Maosong Sun. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.695 L o RA -flow: Dynamic L o RA fusion for large language models in generative tasks . In Proceedings of the 62nd Annual Meeting of the Asso...

    2024 doi

  30. [38]

    Tiannan Wang, Jiamin Chen, Qingrui Jia, Shuai Wang, Ruoyu Fang, Huilin Wang, Zhaowei Gao, Chunzhao Xie, Chuou Xu, Jihong Dai, Yibin Liu, Jialong Wu, Shengwei Ding, Long Li, Zhiwei Huang, Xinle Deng, Teng Yu, Gangan Ma, Han Xiao, Zixin Chen, Danjun Xiang, Yunxia Wang, Yuanyuan ...

    2024 arXiv

  31. [39]

    Xiyao Wang, Linfeng Song, Ye Tian, Dian Yu, Baolin Peng, Haitao Mi, Furong Huang, and Dong Yu. 2024 d . https://arxiv.org/abs/2410.06508 Towards self-improvement of llms via mcts: Leveraging stepwise knowledge with curriculum preference learning . Preprint, arXiv:2410.06508

    2024 arXiv

  32. [40]

    Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, Wei Ye, Shikun Zhang, and Yue Zhang. 2024 e . https://arxiv.org/abs/2406.10252 Autosurvey: Large language models can automatically write surveys . Preprint, ar...

    2024 arXiv

  33. [41]

    Junkang Wu, Yuexiang Xie, Zhengyi Yang, Jiancan Wu, Jinyang Gao, Bolin Ding, Xiang Wang, and Xiangnan He. 2024 a . https://arxiv.org/abs/2407.08639 -dpo: Direct preference optimization with dynamic . Preprint, arXiv:2407.08639

    2024 arXiv

  34. [42]

    Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. 2024 b . https://arxiv.org/abs/2404.15574 Retrieval head mechanistically explains long-context factuality . Preprint, arXiv:2404.15574

    2024 arXiv

  35. [43]

    Yuhao Wu, Ming Shan Hee, Zhiqing Hu, and Roy Ka-Wei Lee. 2024 c . Spinning the golden thread: Benchmarking long-form generation in language models. arXiv preprint arXiv:2409.02076

    2024 arXiv

  36. [44]

    Zekun Xi, Wenbiao Yin, Jizhan Fang, Jialong Wu, Runnan Fang, Ningyu Zhang, Jiang Yong, Pengjun Xie, Fei Huang, and Huajun Chen. 2025. https://arxiv.org/abs/2501.09751 Omnithink: Expanding knowledge boundaries in machine writing through thinking . Preprint, arXiv:2501.09751

    2025

  37. [45]

    Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, and Maosong Sun. 2024 a . https://arxiv.org/abs/2402.04617 Infllm: Training-free long-context extrapolation for llms with an efficient context memory . Preprint, arXiv:2402.04617

    2024 arXiv

  38. [46]

    Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, Junxian Guo, Shang Yang, Haotian Tang, Yao Fu, and Song Han. 2024 b . https://arxiv.org/abs/2410.10819 Duoattention: Efficient long-context llm inference with retrieval and streaming heads . Preprint, arXiv:2410.10819

    2024 arXiv

  39. [47]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024 c . https://openreview.net/forum?id=NG7sS51zVF Efficient streaming language models with attention sinks . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, ...

    2024

  40. [48]

    Lillicrap, Kenji Kawaguchi, and Michael Shieh

    Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P. Lillicrap, Kenji Kawaguchi, and Michael Shieh. 2024. https://arxiv.org/abs/2405.00451 Monte carlo tree search boosts reasoning via iterative preference learning . Preprint, arXiv:2405.00451

    2024 arXiv

  41. [49]

    Bin Xu, Yiguan Lin, Yinghao Li, and Yang Gao. 2024. https://arxiv.org/abs/2411.11053 Sra-mcts: Self-driven reasoning augmentation with monte carlo tree search for code generation . Preprint, arXiv:2411.11053

    2024 arXiv

  42. [50]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

    2025 arXiv

  43. [51]

    Zihuiwen Ye, Fraser Greenlee-Scott, Max Bartolo, Phil Blunsom, Jon Ander Campos, and Matthias Gallé. 2024. https://arxiv.org/abs/2405.20850 Improving reward models with synthetic critiques . Preprint, arXiv:2405.20850

    2024 arXiv

  44. [52]

    Yue Yu, Zhengxing Chen, Aston Zhang, Liang Tan, Chenguang Zhu, Richard Yuanzhe Pang, Yundi Qian, Xuewei Wang, Suchin Gururangan, Chao Zhang, Melanie Kambadur, Dhruv Mahajan, and Rui Hou. 2024. https://arxiv.org/abs/2411.16646 Self-generated critiques boost reward modeling for ...

    2024 arXiv

  45. [53]

    Weizhe Yuan, Pengfei Liu, and Matthias Gall \'e . 2024. https://doi.org/10.18653/v1/2024.findings-acl.472 LLMC rit: Teaching large language models to use criteria . In Findings of the Association for Computational Linguistics: ACL 2024, pages 7929--7960, Bangkok, Thailand. Ass...

    2024 doi

  46. [54]

    Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, and Wanli Ouyang. 2024 a . https://arxiv.org/abs/2406.07394 Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b . Preprint, arXiv:2406.07394

    2024 arXiv

  47. [55]

    Jiajie Zhang, Zhongni Hou, Xin Lv, Shulin Cao, Zhenyu Hou, Yilin Niu, Lei Hou, Yuxiao Dong, Ling Feng, and Juanzi Li. 2024 b . https://arxiv.org/abs/2410.21252 Longreward: Improving long-context large language models with ai feedback . Preprint, arXiv:2410.21252

    2024 arXiv

  48. [56]

    Qingjie Zhang, Han Qiu, Di Wang, Haoting Qian, Yiming Li, Tianwei Zhang, and Minlie Huang. 2024 c . https://arxiv.org/abs/2412.14959 Understanding the dark side of llms' intrinsic self-correction . Preprint, arXiv:2412.14959

    2024 arXiv

  49. [57]

    Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, and Min Lin. 2024 d . https://arxiv.org/abs/2406.09136 Chain of preference optimization: Improving chain-of-thought reasoning in llms . Preprint, arXiv:2406.09136

    2024 arXiv

  50. [58]

    Wildchat: 1m chatgpt interaction logs in the wild

    Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chatgpt interaction logs in the wild. In The Twelfth International Conference on Learning Representations

  51. [59]

    Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. 2024. https://arxiv.org/abs/2411.14405 Marco-o1: Towards open reasoning models for open-ended solutions . Preprint, arXiv:2411.14405

    2024 arXiv

  52. [60]

    Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Rongqiao An, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, and Maosong Sun. 2024. https://arxiv.org/abs/2410.09342 Llm mapreduce: Simplified long-sequence processing using large language mode...

    2024 arXiv

  53. [61]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  54. [62]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.