Pith. sign in

REVIEW 5 major objections 5 minor 104 references

Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Post-training alignment on synthetic graph puzzles transfers to real-world reasoning, lifting accuracy by about 13 percent on average.

desk verdict Useful negative results about synthetic graph alignment, but the headline claim that alignment beats SFT on real-world tasks doesn't survive the paper's own tables. read the letter →

arxiv 2506.00845 v3 pith:YMVAF3HB submitted 2025-06-01 cs.LG cs.CL

Yizhuo Zhang , Heng Wang , Shangbin Feng , Zhaoxuan Tan , Xinyun Liu , Yulia Tsvetkov This is my paper
classification cs.LGcs.CL
keywords graphreasoningpost-trainingalignmentreinforcementlearningsyntheticdataGRPODPOprocessrewardgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the right way to use synthetic graph data, whose answers are already known exactly, is not to fine-tune a model to imitate those answers but to reward it for producing them during post-training alignment. It claims that alignment with rule-based rewards, especially the on-policy method GRPO, helps a language model internalize the essentials of graph reasoning, and that this skill transfers: on eight real-world tasks with implicit graph structure, the recipe yields statistically significant gains on five settings, averaging about 13 percent over baselines. If true, this matters because it points to a cheap route to improving LLM reasoning on problems where no graph is written down, using only synthetic training data whose correctness a program can verify for free. It also directly contradicts the prior conclusion that synthetic graph training only helps on synthetic graphs.

What carries the argument

The carrying mechanism is a rule-based reward scheme keyed to a forced response format. Each response must contain three tagged sections, a think section, a response section of numbered reasoning steps, and an answer section with the final verdict, and a Python program with NetworkX verifies the content. The solution-based reward compares only the answer to ground truth; the process-based reward rebuilds the graph the model's steps refer to, scores each step as a correct fact or a hallucination, and applies a penalty of -2 for invented nodes, edges, or weights. These rewards feed two alignment algorithms: GRPO, an on-policy reinforcement learning method that estimates advantages from groups of rollouts, and DPO, an off-policy preference method trained on high-temperature rollouts. The claim is that scoring the reasoning process rather than just the product forces the policy to learn graph-valid reasoning instead of memorized output patterns, and that this is what makes the learned behavior transfer to tasks where no graph is written down.

What would settle it

A concrete check: drop the two Blocksworld settings, the tasks most similar in format to the synthetic ones, and recompute the average real-world gain; if the 13 percent improvement and the five significant settings largely vanish, the claimed transfer to implicit graph reasoning is really transfer to planning-style tasks. A complementary experiment aligns a model on synthetic arithmetic or sorting data with the same GRPO recipe and tests the same eight real-world tasks, since comparable gains would show the active ingredient is generic reward-driven reasoning rather than graph structure.

Watch

Extended reading notes

Core claim

The paper's central claim is that post-training alignment on synthetic connectivity and shortest-path problems generalizes to real-world tasks with implicit graph structure, where plain supervised fine-tuning fails to transfer. The authors design two rule-based rewards for the synthetic problems: a solution-based reward that scores only the final answer, and a process-based reward that reconstructs the graph implied by the model's stated reasoning steps, rewarding correct statements and penalizing hallucinated nodes, edges, and weights. They use these rewards to align off-the-shelf and SFT-then-aligned 7B and 8B instruction-tuned models with GRPO and DPO. They report that GRPO consistently outperforms DPO on synthetic tasks; that process-based rewards beat solution-based rewards by about 24 percent on synthetic tasks under GRPO but show no consistent edge on real-world tasks; that the aligned models achieve statistically significant improvements on five of eight real-world task settings with an average gain of about 13 percent; and that remaining failures concentrate in two places, the compositionality gap between correct single-step answers and correct multi-step answers, and multi-step answers reached through hallucinated or unexplained intermediate steps.

Load-bearing premise

The load-bearing premise is that real-world tasks such as strategy questions, commonsense reasoning, and block-stacking planning engage the same graph reasoning machinery as synthetic connectivity and shortest-path problems, so rewards earned on synthetic puzzles teach transferable skills rather than task-specific response formats.

Editorial extensions

If this is right

  • If the recipe is right, synthetic-data alignment becomes a low-cost route to improving real-world reasoning, since the training data and reward signals need no human annotation.
  • The GRPO-over-DPO result implies that on-policy reward optimization is the transferable part of alignment, with off-policy preference learning adding little after SFT.
  • The process-reward advantage on synthetic but not real tasks implies fine-grained step verification helps most where the target solution space is exactly the training space.
  • The observed compositionality gap means even successful alignment does not fix multi-step reasoning, so gains should shrink as required step counts grow.
  • SFT before GRPO can hurt performance, evidence that pattern memorization can interfere with reward-driven learning and that the two stages are not automatically complementary.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own negative correlation between synthetic and real-world performance (-0.336 Pearson) suggests its headline gains may be carried mainly by tasks structurally closest to the synthetic ones, such as plan verification; re-weighting the eight task settings to exclude Blocksworld might shrink the average gain toward zero, a re-analysis the paper does not run.
  • A natural extension is to replace the synthetic reward's graph reconstruction with a verifier that checks single-hop facts in a knowledge base, turning the compositionality gap itself into part of the reward signal; this would test whether step-level verification, rather than synthetic graph exposure, is the active ingredient.
  • If the mechanism is genuinely graph-generic, aligning on synthetic problems from other well-verified structures such as constraint satisfaction or sorting networks should produce analogous real-world gains; if it does not, the improvements are specific to graph-shaped reasoning and the analogy to real-world tasks is weaker than claimed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a three-stage recipe: optional supervised fine-tuning on synthetic connectivity and shortest-path tasks, followed by DPO or GRPO post-training alignment with solution-based or process-based rule-based rewards, and evaluation on held-out synthetic tasks and eight real-world tasks with implicit graph structure (multi-hop QA, structured commonsense reasoning, and Blocksworld planning/verification). The headline claim is that post-training alignment on purely synthetic graph data yields statistically significant real-world improvements on five task settings, with an average gain of about 13%, and that process-based rewards outperform solution-based rewards on synthetic tasks while the real-world picture is mixed. The paper also analyzes the compositionality gap between single-step and multi-step reasoning and the prevalence of hallucinated intermediate steps.

Significance. If the central comparison were valid, the paper would make a useful empirical contribution: it addresses a timely question, publicly releases code and results, uses verifiable rule-based rewards, tests two LLM families on eight real-world benchmarks, and includes a careful decomposition of single-step versus multi-step failures. The synthetic reward design is transparent and easy to audit, and the correlation between reward score and synthetic accuracy (Pearson 0.951) supports the internal validity of the reward functions. However, the statistical framing—significance measured against zero-shot, post-hoc checkpoint selection for several LLAMA runs, and a '5 out of 8' claim that is a maximum over alignment configurations—currently substantially weakens the evidence for the paper's stated 'alignment, not SFT alone' claim. These issues are fixable with reanalysis and re-reporting, but they are load-bearing for the main contribution.

major comments (5)
  1. [Section 4.2 / Table 2] Table 2's footnote says statistical significance is measured against zero-shot performance, but the paper's contribution claim is that post-training alignment, not SFT alone, drives real-world gains. Against the relevant baseline (the SYNTHETIC SFT row), the QWEN2.5-7B results show no average advantage for alignment: SFT achieves +10.6% average increase, while DPO-P W/SFT, DPO-S W/SFT, GRPO-P W/SFT, and GRPO-S W/SFT achieve +9.7%, +7.9%, +2.0%, and +6.2%, respectively. The LLAMA3.1-8B comparisons that favor alignment (+25.4%, +36.5%, +36.7%) are affected by the checkpoint-selection issue in Appendix C. The paper should either test significance against synthetic SFT and report those numbers, or explicitly reframe the claim as 'improvement over zero-shot' and remove the 'not SFT alone' wording.
  2. [Appendix C] Appendix C reports that for LLAMA3.1-8B settings GRPO-P W/O SFT, GRPO-S W/O SFT, and GRPO-S W/SFT, the authors 'train around 3.2 epochs (rather than standard 8 epochs) using saved checkpoints for better representation of model's performance.' Choosing checkpoints with reference to the evaluation invalidates the z-test p-values for those rows and inflates the LLAMA gains reported in Table 2. These runs should be excluded from the headline significance claims or reported as exploratory with no p-values. In addition, no confidence intervals or repeated-seed variability are reported anywhere, so the reader cannot assess the stability of the 13% average gain.
  3. [Section 4.2 / abstract] The headline '5 task settings' (abstract: '5 datasets') is a maximum over the up to 12 alignment configurations per dataset, not a consistent result: the same paragraph reports only 3 of 8 settings with consistent significant improvements. No multiple-comparison correction is applied to the per-configuration z-tests. The 'consistent' and 'at least one configuration' counts should be reported separately, and either the maximum-based claim or the average gain should be qualified accordingly.
  4. [Sections 3.1 and Appendix D] The exact test-set sizes for the real-world benchmarks are inconsistent. Section 3.1 states 'For all real-world datasets, we randomly sampled 1000 instances from each dataset,' but Appendix D says 'Blocksworld dataset has only 500 instances per setting.' Since the z-tests in Table 2 depend directly on these sample sizes, the paper must state the per-dataset test size for every benchmark, including Blocksworld planning and verification.
  5. [Appendix E.3] The negative Pearson correlation of -0.336 between synthetic-task performance and real-world performance is presented as a side observation, but it bears directly on the transfer mechanism. If training on synthetic connectivity/shortest-path problems were teaching a general graph-reasoning skill, one would not expect negative correlation across training settings. Please reconcile this correlation with the transfer claim, or restrict the claim to the structurally closest tasks (e.g., Blocksworld) and provide a concrete mechanism-based test, such as per-setting correlations or an ablation that isolates the reward signal.
minor comments (5)
  1. [Throughout] Please fix typos, including 'Methdology' (Section 2 heading), 'incooperates commensense' (Section 3.1), 'alightnment' (Section 4.2), and 'huallicination' (Limitations).
  2. [Abstract / Sections 1 and 4.2] The reported average gain is inconsistent: 12.9% in the abstract, 13% in Section 1, and 13.6% in Section 4.2. Please reconcile the numbers and define the baseline over which the average is taken.
  3. [Appendix B] The reward definitions list 'Incorrect answer penalty,' 'Incorrect reasoning penalty,' and 'Format error penalty' all set to 0; these are not penalties. Either rename them as zero rewards or explain why zero values are intended.
  4. [Section 5.1] The text refers to 'Appendix B Table 3,' but the mixing-real-world-data table appears in Appendix A. Please fix the cross-reference.
  5. [Table 5 / Figure 2] Several model responses in Table 5 are followed by a '%' symbol with no legend; please explain what it denotes, and ensure Figure 2's two-panel layout has clear labels for 'final wrong/final correct' in both panels.

Circularity Check

1 steps flagged · score 2.0 of 10

No material circularity: the central synthetic-to-real transfer claim is tested on external held-out real-world benchmarks; the only self-referential step is a minor reward-accuracy correlation sanity check.

  1. self definitional [Appendix E.3 (Correlation Analysis), with reward definitions in Section 2 and Appendix B, Algorithms 1-2]
    "we analyze the correlation between the synthetic task’s accuracy performance and the reward score achieved by the model. The Pearson correlation coefficient is 0.951: our reward design is highly correlated with the performance of the model, proving the reward function’s effectiveness."

    Synthetic accuracy is defined in Section 3.1 as rule-based extraction of the final answer compared with ground truth. The reward defined in Appendix B (Algorithms 1-2) adds r_correct_answer = +1 for exactly that same answer match, plus small format and step bonuses. Hence reward score and synthetic accuracy share the answer-correctness term by construction, so the reported 0.951 Pearson correlation is partly mechanical and does not independently 'prove' reward effectiveness. This is a self-referential validation, but it is not load-bearing for the paper's main transfer result, which is evaluated on held-out real-world benchmarks.

full rationale

The paper's main claim—that post-training alignment on synthetic connectivity and shortest-path data improves real-world implicit-graph tasks—is not circular: models are trained only on synthetic tasks with rule-based rewards and evaluated on held-out real-world benchmarks (StrategyQA, Knowledge Crosswords, WikiDYK, ExplaGraphs, Proscript, Blocksworld) that the training rewards never observe. The synthetic SFT baseline and per-setting z-tests are reported in full. Self-citations (NLGift dataset from Zhang et al. 2024b; graph-reasoning benchmarks from Wang et al. 2023 and Ding et al. 2024) are not load-bearing in a circular way: the NLGift dataset is a released artifact, and the claim that SFT fails to transfer is independently reproduced by this paper's own Table 2 (e.g., StrategyQA drops from 0.702 to 0.615 for QWEN after synthetic SFT). The only self-referential analysis is Appendix E.3, where a 0.951 correlation between reward score and synthetic accuracy is said to 'prove' reward effectiveness; since both quantities include the same +1 answer-correctness term by construction, that correlation is partly definitional. This is a minor supporting validation, not a prediction and not the basis of the transfer claim. Remaining concerns about checkpoint selection for LLAMA runs and the '5 task settings' definition are statistical soundness issues (post-hoc selection, multiple comparisons) rather than input-output circularity, and the paper discloses both.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

All free parameters are hand-chosen reward coefficients; the core axioms are domain assumptions about the analogy between synthetic graph problems and real-world reasoning tasks, and about the reliability of the evaluation pipeline. No new physical or conceptual entities are introduced.

free parameters (4)
  • Correct answer reward weight = +1
    Hand-set in Section 2; the relative size of solution vs process rewards shapes training and the paper's comparison of reward designs.
  • Hallucination penalty = -2
    Introduced ad hoc in Section 2 to penalize fabricated edges or weights; the magnitude is not derived from data and influences the process reward's behavior.
  • Correct step reward = +0.05
    Per-step reward in the process-based design, listed in Appendix B; chosen by hand.
  • Format reward = +0.1
    Reward for conforming to the think/response/answer format, stated in Section 2 and Appendix B; hand-chosen.
assumptions (4)
  • domain assumption Real-world tasks are implicit graph reasoning tasks analogous to connectivity and shortest path
    Asserted in Section 1 and 3.1; underpins the transfer claim and the choice of synthetic tasks.
  • domain assumption Intermediate steps in synthetic responses can be parsed and verified against the true graph
    Section 2 and Algorithms 1-2; verification is only possible because synthetic graphs are fully known, and the format is enforced during training.
  • domain assumption Gemini 2.0 Flash summarizes free-form responses faithfully for evaluation
    Used in the real-world evaluation pipeline (Section 3.2, Appendix D); an unreliable summarizer would bias the reported accuracies.
  • standard math Proportions z-test assumptions hold for the accuracy comparisons
    Used for significance testing in Section 3.2; assumes independent Bernoulli trials within each setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment." pith.science (2026). https://pith.science/paper/YMVAF3HB

@misc{pith2026250600845,
  author       = {Pith},
  title        = {Pith review of: Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YMVAF3HB}},
  note         = {Machine review of arXiv:2506.00845}
}
read the original abstract

Previous research has sought to enhance the graph reasoning capabilities of LLMs by supervised fine-tuning on synthetic graph data. While these led to specialized LLMs better at solving graph algorithm problems, we don't need LLMs for shortest path: we need generalization from synthetic graph data to real-world tasks with implicit graph structures. In this work, we propose to unlock generalizable learning of graph with post-training alignment with synthetic data. We first design solution-based and process-based rewards for synthetic graph problems: instead of rigid memorizing response patterns in direct fine-tuning, we posit that post-training alignment would help LLMs grasp the essentials underlying graph reasoning and alleviate overfitting on synthetic data. We employ post-training alignment algorithms such as GRPO and DPO, aligning both off-the-shelf LLMs and LLMs fine-tuned on synthetic graph data. We then compare them against existing settings on both in-domain synthetic tasks and out-of-domain real-world tasks with implicit graph structures such as multi-hop QA, structured planning, and more. Extensive experiments demonstrate that our post-training alignment recipe leads to statistically significant improvement on 5 datasets, with an average gain of 12.9% over baseline settings. Further analysis reveals that process-based rewards consistently outperform solution-based rewards on synthetic data but not on real-world tasks, and compositionality and explainable intermediate steps remains a critical challenge even after post-training alignment.

Figures

Figures reproduced from arXiv: 2506.00845 by the authors.

Figure 1
Figure 1. The overall pipeline of our work. We first perform an optional SFT stage (left), followed by RL stage [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Proportions of cases regarding whether single steps and the final answer are correct on StrategyQA and [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. (left) Correlation between the synthetic tasks’ performance and the reward score. The high correlation [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performances on Blocksworld Verification task grouped by the length of action steps needs to be verified. [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

104 extracted references · 22 canonical work pages

  1. [1]

    https://www.anthropic.com/news/claude-3-7-sonnet

    C laude 3.7 S onnet and C laude C ode --- anthropic.com. https://www.anthropic.com/news/claude-3-7-sonnet. [Accessed 13-05-2025]

    2025

  2. [2]

    https://deepmind.google/technologies/gemini/pro/

    G emini 2.5 P ro --- deepmind.google. https://deepmind.google/technologies/gemini/pro/. [Accessed 13-05-2025]

    2025

  3. [3]

    https://openai.com/index/o3-o4-mini-system-card/

    O pen A I o3 and o4-mini S ystem C ard --- openai.com. https://openai.com/index/o3-o4-mini-system-card/. [Accessed 13-05-2025]

    2025

  4. [4]

    On-policy distillation of language models: Learning from self-generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In The Twelfth International Conference on Learning Representations, 2024

    2024

  5. [5]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    arXiv 2022

  6. [6]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  7. [7]

    Graphwiz: An instruction-following language model for graph computational problems

    Nuo Chen, Yuhan Li, Jianheng Tang, and Jia Li. Graphwiz: An instruction-following language model for graph computational problems. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, page 353–364, New York, NY, USA, 2024 a . Association for Computing Machinery. ISBN 9798400704901. doi:10.1145/3637528.3672010. ...

    arXiv 2024

  8. [8]

    LL a GA : Large language and graph assistant

    Runjin Chen, Tong Zhao, Ajay Kumar Jaiswal, Neil Shah, and Zhangyang Wang. LL a GA : Large language and graph assistant. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Ma...

    2024

Show all 104 references
  1. [9]

    Reasoning models don't always say what they think

    Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, et al. Reasoning models don't always say what they think. arXiv preprint arXiv:2505.05410, 2025

    2025 arXiv

  2. [10]

    Exploring the potential of large language models (llms) in learning on graphs

    Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, et al. Exploring the potential of large language models (llms) in learning on graphs. ACM SIGKDD Explorations Newsletter, 25 0 (2): 0 42--61, 2024 c

    2024

  3. [11]

    Lota-bench: Benchmarking language-oriented task planners for embodied agents

    Jae-Woo Choi, Youngwoo Yoon, Hyobin Ong, Jaehong Kim, and Minsu Jang. Lota-bench: Benchmarking language-oriented task planners for embodied agents. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=ADSxCpCu9s

    2024

  4. [12]

    Sft memorizes, rl generalizes: A comparative study of foundation model post-training

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161, 2025

    2025 arXiv

  5. [13]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  6. [14]

    Which modality should I use - text, motif, or image? : Understanding graphs with large language models

    Debarati Das, Ishaan Gupta, Jaideep Srivastava, and Dongyeop Kang. Which modality should I use - text, motif, or image? : Understanding graphs with large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Findings of the Association for Computational Lin...

    2024 doi

  7. [15]

    Benchmarks for automated commonsense reasoning: A survey

    Ernest Davis. Benchmarks for automated commonsense reasoning: A survey. ACM Computing Surveys, 56 0 (4): 0 1--41, 2023

    2023

  8. [16]

    Graphvis: Boosting llms with visual knowledge graph integration

    Yihe Deng, Chenchen Ye, Zijie Huang, Mingyu Derek Ma, Yiwen Kou, and Wei Wang. Graphvis: Boosting llms with visual knowledge graph integration. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  9. [17]

    Knowledge crosswords: Geometric knowledge reasoning with large language models

    Wenxuan Ding, Shangbin Feng, Yuhan Liu, Zhaoxuan Tan, Vidhisha Balachandran, Tianxing He, and Yulia Tsvetkov. Knowledge crosswords: Geometric knowledge reasoning with large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association ...

    2024 doi

  10. [18]

    Kto: Model alignment as prospect theoretic optimization

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306, 2024

    2024 arXiv

  11. [19]

    Talk like a graph: Encoding graphs for large language models

    Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. Talk like a graph: Encoding graphs for large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=IuXR1CCrSi

    2024

  12. [20]

    Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9: 0 346--361, 2021

    2021

  13. [21]

    Bigbench: Towards an industry standard benchmark for big data analytics

    Ahmad Ghazal, Tilmann Rabl, Minqing Hu, Francois Raab, Meikel Poess, Alain Crolotte, and Hans-Arno Jacobsen. Bigbench: Towards an industry standard benchmark for big data analytics. In Proceedings of the 2013 ACM SIGMOD international conference on Management of data, pages 119...

    2013

  14. [22]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  15. [23]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025 a

    2025 arXiv

  16. [24]

    Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07062, 2025 b

    2025 arXiv

  17. [25]

    Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking

    Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, and Shi Han. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking. arXiv preprint arXiv:2305.15066, 2023

    2023 arXiv

  18. [26]

    Direct language model alignment from online ai feedback

    Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, et al. Direct language model alignment from online ai feedback. arXiv preprint arXiv:2402.04792, 2024

    2024 arXiv

  19. [27]

    Hagberg, Daniel A

    Aric A. Hagberg, Daniel A. Schult, and Pieter J. Swart. Exploring network structure, dynamics, and function using networkx. In Ga\"el Varoquaux, Travis Vaught, and Jarrod Millman, editors, Proceedings of the 7th Python in Science Conference, pages 11 -- 15, Pasadena, CA USA, 2008

    2008

  20. [28]

    P i V e: Prompting with iterative verification improving graph-based generative capability of LLM s

    Jiuzhou Han, Nigel Collier, Wray Buntine, and Ehsan Shareghi. P i V e: Prompting with iterative verification improving graph-based generative capability of LLM s. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguisti...

    2024 doi

  21. [29]

    Harnessing explanations: LLM -to- LM interpreter for enhanced text-attributed graph representation learning

    Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Harnessing explanations: LLM -to- LM interpreter for enhanced text-attributed graph representation learning. In The Twelfth International Conference on Learning Representations, 2024 a . URL h...

    2024

  22. [30]

    G-retriever: Retrieval-augmented generation for textual graph understanding and question answering

    Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. Advances in Neural Information Processing Systems, 37: 0 132876--13...

    2024

  23. [31]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track ...

    2021

  24. [32]

    Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Donia Scott, Nuria Bel, and Chengqing Zong, editors, Proceedings of the 28th International Conference on Computational Lingu...

    2020 doi

  25. [33]

    Reinforce++: A simple and efficient approach for aligning large language models

    Jian Hu. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262, 2025

    2025 arXiv

  26. [34]

    Efficient test-time scaling via self-calibration

    Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via self-calibration. arXiv preprint arXiv:2503.00031, 2025

    2025 arXiv

  27. [35]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  28. [36]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  29. [37]

    Multi-turn code generation through single-step rewards

    Arnav Kumar Jain, Gonzalo Gonzalez-Pumariega, Wayne Chen, Alexander M Rush, Wenting Zhao, and Sanjiban Choudhury. Multi-turn code generation through single-step rewards. arXiv preprint arXiv:2502.20380, 2025

    2025 arXiv

  30. [38]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  31. [39]

    Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023

    2023 arXiv

  32. [40]

    Training language models to self-correct via reinforcement learning

    Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.12917, 2024

    2024 arXiv

  33. [41]

    Rlaif: Scaling reinforcement learning from human feedback with ai feedback

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Ren Lu, Thomas Mesnard, Johan Ferret, Colton Bishop, Ethan Hall, Victor Carbune, and Abhinav Rastogi. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. 2023

    2023

  34. [42]

    Common 7b language models already possess strong math capabilities

    Chen Li, Weiqi Wang, Jingcheng Hu, Yixuan Wei, Nanning Zheng, Han Hu, Zheng Zhang, and Houwen Peng. Common 7b language models already possess strong math capabilities. arXiv preprint arXiv:2403.04706, 2024 a

    2024 arXiv

  35. [43]

    S*: Test time scaling for code generation

    Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E Gonzalez, and Ion Stoica. S*: Test time scaling for code generation. arXiv preprint arXiv:2502.14382, 2025

    2025 arXiv

  36. [44]

    Visiongraph: leveraging large multimodal models for graph theory problems in visual context

    Yunxin Li, Baotian Hu, Haoyuan Shi, Wei Wang, Longyue Wang, and Min Zhang. Visiongraph: leveraging large multimodal models for graph theory problems in visual context. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024 b

    2024

  37. [45]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023

    2023

  38. [46]

    Cohn, and Janet B

    Fangru Lin, Emanuele La Malfa, Valentin Hofmann, Elle Michelle Yang, Anthony G. Cohn, and Janet B. Pierrehumbert. Graph-enhanced large language models in asynchronous plan reasoning. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024

    2024

  39. [47]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024 a

    2024 arXiv

  40. [48]

    Agentbench: Evaluating LLM s as agents

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. Agentbench:...

    2024

  41. [49]

    Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian-Guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, Yansong Tang, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. In The Thirteenth Internatio...

    2025

  42. [50]

    Reasoning on graphs: Faithful and interpretable large language model reasoning

    LINHAO LUO, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. Reasoning on graphs: Faithful and interpretable large language model reasoning. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=ZGNWW7xZ6Q

    2024

  43. [51]

    Language models of code are few-shot commonsense learners

    Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang, and Graham Neubig. Language models of code are few-shot commonsense learners. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p...

    2022 doi

  44. [52]

    Multi-hop question answering

    Vaibhav Mavi, Anubhav Jangra, Adam Jatowt, et al. Multi-hop question answering. Foundations and Trends in Information Retrieval , 17 0 (5): 0 457--586, 2024

    2024

  45. [53]

    Sim PO : Simple preference optimization with a reference-free reward

    Yu Meng, Mengzhou Xia, and Danqi Chen. Sim PO : Simple preference optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=3Tzcot1LKb

    2024

  46. [54]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  47. [55]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 2...

    2022

  48. [56]

    Teach: Task-driven embodied agents that chat

    Aishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava, Patrick Lange, Anjali Narayan-Chen, Spandana Gella, Robinson Piramuthu, Gokhan Tur, and Dilek Hakkani-Tur. Teach: Task-driven embodied agents that chat. In Proceedings of the AAAI Conference on Artificial Intelligence, v...

    2017

  49. [57]

    Let your graph do the talking: Encoding structured data for llms

    Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Mehran Kazemi, Rami Al-Rfou, and Jonathan Halcrow. Let your graph do the talking: Encoding structured data for llms. arXiv preprint arXiv:2402.05862, 2024

    2024 arXiv

  50. [58]

    Introducing gemini 2.0: Our new ai model for the agentic era, Dec 2024

    Sundar Pichai. Introducing gemini 2.0: Our new ai model for the agentic era, Dec 2024. URL https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/#ceo-message

    2024

  51. [59]

    Phybench: Holistic evaluation of physical perception and reasoning in large language models

    Shi Qiu, Shaoyang Guo, Zhuo-Yang Song, Yunbo Sun, Zeyu Cai, Jiashen Wei, Tianyu Luo, Yixuan Yin, Haoxu Zhang, Yi Hu, et al. Phybench: Holistic evaluation of physical perception and reasoning in large language models. arXiv preprint arXiv:2504.16074, 2025

    2025 arXiv

  52. [60]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in N...

    2023

  53. [61]

    Direct nash optimization: Teaching language models to self-improve with general preferences

    Corby Rosset, Ching-An Cheng, Arindam Mitra, Michael Santacroce, Ahmed Awadallah, and Tengyang Xie. Direct nash optimization: Teaching language models to self-improve with general preferences. arXiv preprint arXiv:2404.03715, 2024

    2024 arXiv

  54. [62]

    Explagraphs: An explanation graph generation task for structured commonsense reasoning

    Swarnadeep Saha, Prateek Yadav, Lisa Bauer, and Mohit Bansal. Explagraphs: An explanation graph generation task for structured commonsense reasoning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7716--7740, 2021

    2021

  55. [63]

    Proscript: Partially ordered scripts generation

    Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark, and Yejin Choi. Proscript: Partially ordered scripts generation. In 2021 Findings of the Association for Computational Linguistics, Findings of ACL: EMNLP 2021, pages 2138--2149. Association for ...

    2021

  56. [64]

    Trust region policy optimization

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International conference on machine learning, pages 1889--1897. PMLR, 2015

    2015

  57. [65]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  58. [66]

    Scaling test-time compute without verification or rl is suboptimal

    Amrith Setlur, Nived Rajaraman, Sergey Levine, and Aviral Kumar. Scaling test-time compute without verification or rl is suboptimal. arXiv preprint arXiv:2502.12118, 2025

    2025 arXiv

  59. [67]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300

    2024 arXiv

  60. [68]

    Dast: Difficulty-adaptive slow-thinking for large reasoning models

    Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, and Shiguo Lian. Dast: Difficulty-adaptive slow-thinking for large reasoning models. arXiv preprint arXiv:2503.04472, 2025

    2025

  61. [69]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  62. [70]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024

    2024 arXiv

  63. [71]

    Deductive additivity for planning of natural language proofs

    Zayne Sprague, Kaj Bostrom, Swarat Chaudhuri, and Greg Durrett. Deductive additivity for planning of natural language proofs. In Bhavana Dalvi Mishra, Greg Durrett, Peter Jansen, Danilo Neves Ribeiro, and Jason Wei, editors, Proceedings of the 1st Workshop on Natural Language ...

    2023 doi

  64. [72]

    Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano

    Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback, 2022. URL https://arxiv.org/abs/2009.01325

    2022 arXiv

  65. [73]

    Table meets llm: Can large language models understand structured table data? a benchmark and empirical study

    Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and Dongmei Zhang. Table meets llm: Can large language models understand structured table data? a benchmark and empirical study. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 645--654, 2024

    2024

  66. [74]

    Commonsense QA 2.0: Exposing the limits of AI through gamification

    Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg, Yejin Choi, and Jonathan Berant. Commonsense QA 2.0: Exposing the limits of AI through gamification. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Ro...

    2021

  67. [75]

    Graphgpt: Graph instruction tuning for large language models

    Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 4...

    2024

  68. [76]

    Grapharena: Evaluating and exploring large language models on graph computation

    Jianheng Tang, Qifan Zhang, Yuhan Li, Nuo Chen, and Jia Li. Grapharena: Evaluating and exploring large language models on graph computation. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=Y1r9yCMzeA

    2025

  69. [77]

    Gemma: Open models based on gemini research and technology

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi \`e re, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  70. [78]

    Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving

    Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving. Advances in Neural Information Processing Systems, 37: 0 7821--7846, 2024

    2024

  71. [79]

    Solving math word problems with process-and outcome-based feedback

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275, 2022

    2022 arXiv

  72. [80]

    Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine...

    2023

  73. [81]

    Trl: Transformer reinforcement learning

    Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforcement learning. https://github.com/huggingface/trl, 2020

    2020

  74. [82]

    Rema: Learning to meta-think for llms with multi-agent reinforcement learning

    Ziyu Wan, Yunxiang Li, Yan Song, Hanjing Wang, Linyi Yang, Mark Schmidt, Jun Wang, Weinan Zhang, Shuyue Hu, and Ying Wen. Rema: Learning to meta-think for llms with multi-agent reinforcement learning. arXiv preprint arXiv:2503.09501, 2025

    2025 arXiv

  75. [83]

    Llms as zero-shot graph learners: Alignment of gnn representations with llm token embeddings

    Duo Wang, Yuan Zuo, Fengzhi Li, and Junjie Wu. Llms as zero-shot graph learners: Alignment of gnn representations with llm token embeddings. Advances in Neural Information Processing Systems, 37: 0 5950--5973, 2024 a

    2024

  76. [84]

    Can language models solve graph problems in natural language? In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    Heng Wang, Shangbin Feng, Tianxing He, Zhaoxuan Tan, Xiaochuang Han, and Yulia Tsvetkov. Can language models solve graph problems in natural language? In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=UDqHhbqYJV

    2023

  77. [85]

    I nstruct G raph: Boosting large language models via graph-centric instruction tuning and preference alignment

    Jianing Wang, Junda Wu, Yupeng Hou, Yao Liu, Ming Gao, and Julian McAuley. I nstruct G raph: Boosting large language models via graph-centric instruction tuning and preference alignment. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for...

    2024 doi

  78. [86]

    Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution

    Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I Wang. Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution. arXiv preprint arXiv:2502.18449, 2025

    2025 arXiv

  79. [87]

    Codeplan: Unlocking reasoning potential in large language models by scaling code-form planning

    Jiaxin Wen, Jian Guan, Hongning Wang, Wei Wu, and Minlie Huang. Codeplan: Unlocking reasoning potential in large language models by scaling code-form planning. In The Thirteenth International Conference on Learning Representations, 2024

    2024

  80. [88]

    Can graph learning improve planning in llm-based agents? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    Xixi Wu, Yifei Shen, Caihua Shan, Kaitao Song, Siwei Wang, Bohang Zhang, Jiarui Feng, Hong Cheng, Wei Chen, Yun Xiong, et al. Can graph learning improve planning in llm-based agents? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  81. [89]

    Deepseek-prover: Advancing theorem proving in llms through large-scale synthetic data

    Huajian Xin, Daya Guo, Zhihong Shao, Zhizhou Ren, Qihao Zhu, Bo Liu, Chong Ruan, Wenda Li, and Xiaodan Liang. Deepseek-prover: Advancing theorem proving in llms through large-scale synthetic data. arXiv preprint arXiv:2405.14333, 2024

    2024 arXiv

  82. [90]

    TRIGO : Benchmarking formal mathematical proof reduction for generative language models

    Jing Xiong, Jianhao Shen, Ye Yuan, Haiming Wang, Yichun Yin, Zhengying Liu, Lin Li, Zhijiang Guo, Qingxing Cao, Yinya Huang, Chuanyang Zheng, Xiaodan Liang, Ming Zhang, and Qun Liu. TRIGO : Benchmarking formal mathematical proof reduction for generative language models. In Hou...

    2023

  83. [91]

    Watch every step! LLM agent learning via iterative step-level process refinement

    Weimin Xiong, Yifan Song, Xiutian Zhao, Wenhao Wu, Xun Wang, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. Watch every step! LLM agent learning via iterative step-level process refinement. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Con...

    2024 doi

  84. [92]

    Contrastive preference optimization: Pushing the boundaries of LLM performance in machine translation

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: Pushing the boundaries of LLM performance in machine translation. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, A...

    2024

  85. [93]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  86. [94]

    Embodiedbench: Comprehensive benchmarking multi-modal large language models for vision-driven embodied agents

    Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, et al. Embodiedbench: Comprehensive benchmarking multi-modal large language models for vision-driven embodied agents. arXiv preprint arX...

    2025 arXiv

  87. [95]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  88. [96]

    Advancing LLM reasoning generalists with preference trees

    Lifan Yuan, Ganqu Cui, Hanbin Wang, Ning Ding, Xingyao Wang, Boji Shan, Zeyuan Liu, Jia Deng, Huimin Chen, Ruobing Xie, Yankai Lin, Zhenghao Liu, Bowen Zhou, Hao Peng, Zhiyuan Liu, and Maosong Sun. Advancing LLM reasoning generalists with preference trees. In The Thirteenth In...

    2025

  89. [97]

    Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks

    Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks. arXiv preprint arXiv:2504.05118, 2025 b

    2025 arXiv

  90. [98]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025

    2025 arXiv

  91. [99]

    Chain of preference optimization: Improving chain-of-thought reasoning in llms

    Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, and Min Lin. Chain of preference optimization: Improving chain-of-thought reasoning in llms. Advances in Neural Information Processing Systems, 37: 0 333--356, 2024 a

    2024

  92. [100]

    Yizhuo Zhang, Heng Wang, Shangbin Feng, Zhaoxuan Tan, Xiaochuang Han, Tianxing He, and Yulia Tsvetkov. Can LLM graph reasoning generalize beyond pattern memorization? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational L...

    2024 doi

  93. [101]

    Bidirectional lms are better knowledge memorizers? a benchmark for real-world knowledge injection

    Yuwei Zhang, Wenhao Yu, Shangbin Feng, Yifan Zhu, Letian Peng, Jayanth Srinivasa, Gaowen Liu, and Jingbo Shang. Bidirectional lms are better knowledge memorizers? a benchmark for real-world knowledge injection. arXiv preprint arXiv:2505.12306, 2025

    2025 arXiv

  94. [102]

    Investigating instruction tuning large language models on graphs

    Kerui Zhu, Bo-Wei Huang, Bowen Jin, Yizhu Jiao, Ming Zhong, Kevin Chang, Shou-De Lin, and Jiawei Han. Investigating instruction tuning large language models on graphs. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=xdg4CS5mkl

    2024

  95. [103]

    Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions

    Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al. Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions. arXiv preprint arXiv:2406.15877, 2024

    2024 arXiv

  96. [104]

    Ttrl: Test-time reinforcement learning

    Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, Xuekai Zhu, Biqing Qi, Youbang Sun, Ganqu Cui, Ning Ding, and Bowen Zhou. Ttrl: Test-time reinforcement learning. arXiv preprint arXiv:2504.16084, 2025

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.