REVIEW 3 major objections 6 minor 8 cited by
SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read SynLogic claims that reinforcement learning on a large dataset of 35 rule-verifiable logic puzzle types produces state-of-the-art LLM logical reasoning and transfers to math and coding.
desk verdict A useful synthetic logic dataset and RL recipe, but the headline BBEH/KOR gains are partly in-distribution and the cross-domain results, while real, don't fully rescue the SOTA logic claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is a paired generator–verifier loop for each of 35 logic tasks. A hand-written, rule-based generator creates random instances that are solvable and obey the task's constraints, while a dedicated verifier checks candidate model answers; in reinforcement learning, a response earns reward 1 only if it has the required <think>/<answer> format and the verifier marks the answer correct, otherwise reward 0. Difficulty is controlled by adjustable parameters, such as grid size or number of missing entries, with upper and lower bounds calibrated by stronger and weaker reference models. GRPO with DAPO-style clipping is the training algorithm, and this generator–verifier pairing is what lets the pipeline produce verifiable reasoning data at scale.
What would settle it
Audit a random sample of generated instances per task with an independent solver or human gold labels and compare against the paper's verifier outputs; any nontrivial mismatch rate would invalidate the reward signal. A second check would train on SynLogic and evaluate on logic-task families completely absent from the 35 tasks, since the paper's main generalization evidence is to math and coding rather than to unseen logic types.
Extended reading notes
Core claim
The paper claims that a broad collection of synthetic logic puzzles, each with a deterministic answer and a rule-based verifier, is an ideal substrate for reinforcement learning with verifiable rewards. The dataset covers 35 tasks, including Sudoku, Game of 24, Cipher, and many grid and constraint puzzles, and every example is generated by code and paired with a task-specific verification rule, so a model's answer can be scored automatically as correct or incorrect. Difficulty is calibrated per task so that frontier models can just barely solve the hardest examples and weaker chat models cannot trivially solve the easiest ones. Starting from Qwen2.5 base models, GRPO training on SynLogic produces long chain-of-thought responses and large accuracy gains on logic benchmarks, with the 32B model surpassing DeepSeek-R1-Distill-Qwen-32B on BBEH. Logic-only training also transfers to mathematics, and mixing SynLogic with math and coding improves training efficiency in those domains; the 32B three-domain mix beats the math-plus-coding ablation on all evaluated benchmarks.
Load-bearing premise
The whole result rests on the 35 hand-written generators and verifiers correctly encoding the intended logic and correctness rules; if any verifier is buggy, the RL reward signal is corrupted and the reported gains may not reflect genuine reasoning.
Editorial extensions
If this is right
- RL on SynLogic alone yields state-of-the-art open-source logical reasoning: the 32B model beats DeepSeek-R1-Distill-Qwen-32B by 6 points on BBEH, and the 7B model gains roughly 10 points on KOR-Bench over Qwen2.5-7B-Instruct.
- Logical reasoning training transfers to mathematics without math data: SynLogic-7B moves from 0.3% to 10.0% on AIME 2024, and SynLogic-32B from 4.5% to 19.6%.
- Mixing SynLogic with math data preserves math accuracy at equal training steps while consuming fewer math samples, and adds about 10 points on KOR-Bench.
- Mixing SynLogic with coding data similarly improves coding efficiency and adds about 10 points on KOR-Bench.
- At 32B scale, adding SynLogic to math and coding (Zero-Mix-3) improves over math-plus-coding alone on BBEH, KOR-Bench, LiveCodeBench, AIME 2024, and GPQA-Diamond, matching or exceeding DeepSeek-R1-Zero-Qwen-32B on several benchmarks.
Reading between the lines
- If the transfer to math and coding is causal rather than merely correlated, synthetic logic data could serve as a general scaffold: any verifiable domain might become more sample-efficient by mixing in a small amount of diverse logic problems.
- The difficulty-calibration scheme suggests a directly testable curriculum: ramping difficulty parameters upward during RL should improve sample efficiency and final accuracy compared with a static difficulty mix.
- The strongest version of 'logic is a fundamental building block' would require showing gains on held-out logic task families not represented in the 35 tasks; the reported generalization evidence is mostly to math and code, so that test remains open.
- Because only 2 of the 35 tasks reuse existing public data, the pipeline itself is the main asset; regenerating a second, disjoint set of puzzle types would separate the effects of task diversity from the effects of the specific 35 tasks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces SynLogic, a framework and dataset for synthesizing verifiable logical reasoning training data across 35 tasks. The authors generate tens of thousands of examples with adjustable difficulty, pair each task with a rule-based verifier, and train Qwen2.5 base models using DAPO/GRPO with verifiable rewards. The central claims are: (1) RL on SynLogic yields substantial gains on logical reasoning benchmarks (KOR-Bench, BBEH, BBH), with the 32B model surpassing DeepSeek-R1-Distill-Qwen-32B on BBEH; (2) SynLogic generalizes to mathematics, improving AIME/MATH/AMC scores without math training data; and (3) mixing SynLogic with math or coding data improves training efficiency and cross-domain reasoning, culminating in a 32B mixed-training model that outperforms DeepSeek-R1-Zero-Qwen-32B on several benchmarks.
Significance. If the results hold, SynLogic is a valuable open-source resource: it provides a scalable, difficulty-adjustable, verifiable reasoning dataset, and demonstrates a practical recipe for improving logical reasoning and cross-domain transfer in RL-trained LLMs. The release of the data synthesis pipeline and dataset is a concrete contribution that could enable further research. However, the significance of the headline logical-reasoning claims is tempered by a substantial in-distribution evaluation concern: the training tasks are drawn from the same benchmark families used for evaluation, and the contamination check only rules out exact duplicates. The math and GPQA transfer results provide complementary evidence, but they do not fully validate the purported state-of-the-art logical reasoning ranking. The verifier reliability, which is foundational to the RL reward signal, is also not independently validated.
major comments (3)
- [Section 2.2.1, Appendix A.1, Table 4] The central logical-reasoning claim is undermined by in-distribution evaluation. Appendix A.1 and Table 4 state that the 35 SynLogic tasks were selected from KOR-Bench, BBH, and BBEH, and include the same task families (e.g., Boolean Expressions, Dyck Language, Web of Lies) that appear in those evaluation benchmarks. Section 2.2.1 dismisses contamination solely by verifying that no generated instance is identical to a benchmark test item. This check is insufficient: a model can improve on held-out instances of a seen task family by learning the family's rule template, which is the normal effect of training data. Consequently, the reported gains on KOR-Bench and BBEH may reflect in-distribution task coverage rather than a general logical-reasoning capability. To support the claim of state-of-the-art logical reasoning, the authors should evaluate on a held-out logic benchmark whose task families are absent from SynLogic (e.g., FOLIO, ReClor, or newly generated families), or at minimum report per-task-family transfer results for families not in the training set.
- [Table 3, Section 4.3.1, Introduction] The claims about outperforming DeepSeek-R1-Zero-Qwen-32B are overstated and partially unsupported. Table 3 shows no BBEH or KOR-Bench results for DeepSeek-R1-Zero-Qwen-32B (the entries are '-'), yet the Introduction says the mixed model "consistently outperforms or matches it on BBEH, KOR-Bench, LiveCodeBench, and GPQA-Diamond" and the Abstract says it "outperforms DeepSeek-R1-Zero-Qwen-32B across multiple benchmarks." The available comparisons show mixed results: Zero-Mix-3 is higher on LiveCodeBench (40.7 vs 40.2) and GPQA-Diamond (57.5 vs 55.0), but lower on AIME 2024 (35.8 vs 47.0). The textual claims should be corrected to specify the benchmarks actually compared and to acknowledge the AIME deficit.
- [Section 2.2, 'Verification Suite'] All RL rewards in this work depend on hand-implemented, task-specific verifiers, but the paper provides no independent validation that these verifiers are correct. If any verifier misjudges the correctness of model outputs, the reward signal is corrupted and the reported improvements may not reflect genuine reasoning gains. The authors should provide evidence of verifier accuracy, for example by comparing verifier decisions against ground-truth labels on a held-out annotated set or against human judgments, and report per-task verifier error rates. At minimum, a discussion of possible verifier failure modes and their impact on training would help.
minor comments (6)
- [Figure 3] The training prompt template in Figure 3 appears twice verbatim in the figure; please remove the duplicate text.
- [Section 3.2.1 and Introduction] The statement that models achieve "over 8 absolute percentage points improvement" on KOR-Bench is inaccurate for the 32B model: Table 2 shows SynLogic-32B at 62.2 vs Qwen2.5-32B-Instruct at 54.7, a difference of 7.5 points. Please correct or qualify this claim.
- [Table 2 and Section 3.2.1] Table 2 shows that SynLogic-7B achieves 8.0 on BBEH, which is lower than Qwen2.5-7B-Instruct's 12.4. The text highlights only improvements; it should acknowledge this regression and discuss possible causes, since BBEH is a key benchmark for the paper's logical reasoning claims.
- [Abstract and Section 3.2.1] The abstract's claim of "state-of-the-art logical reasoning performance among open-source datasets" is based primarily on BBEH; Table 2 shows SynLogic-32B is below DeepSeek-R1-Distill-Qwen-32B on KOR-Bench (62.2 vs 66.6). Please qualify the claim to specify the benchmark(s) supporting it.
- [Section 2.2.1] The sentence stating that "we have verified that there are no identical samples between our generated datasets and the benchmark test sets" is vague. Please specify the matching procedure (e.g., exact string match after normalization), which benchmark test sets were checked, and how many near-duplicates or paraphrases were considered.
- [Section 3.1 and 4.2] The paper does not report variance or multiple seeds for the RL training runs. Given that the main comparisons are often small differences (e.g., LCB 40.7 vs 40.2 in Table 3), reporting standard errors or at least one additional seed would strengthen the conclusions.
Circularity Check
In-distribution evaluation: SynLogic's training tasks are selected from the same benchmarks (BBEH, KOR-Bench, BBH) used for the headline claim, and §2.2.1 rules out only exact duplicates, so part of the logic gain is circular.
-
other
[Appendix A.1 and Table 4; Section 2.2 Task Selection; Section 2.2.1 Risk of Data Contamination; Section 3.1.4 Evaluation Details; Section 3.2.1 Results]
"Although several tasks overlap between our selected tasks and current benchmarks, such as KOR-Bench and BBEH, the synthetic nature of our data, combined with the large synthesis space, makes the probability of generating data identical to benchmark test samples very low – we have verified that there are no identical samples between our generated datasets and the benchmark test sets. ... We have carefully selected these tasks from established benchmarks including KOR-Bench (Ma et al., 2024), BBH (Suzgun et al., 2022), and BBEH (Kazemi et al., 2025)."
By the paper's own account, the SynLogic training-task set is selected from the same benchmark families (BBEH, BBH, KOR-Bench) later used to measure the headline claim ('surpassing DeepSeek-R1-Distill-Qwen-32B by 6 points on BBEH'), and Table 4 lists the same task schemas (Boolean Expressions, Dyck Language, Web of Lies, Cipher, etc.) that appear in those benchmarks. Section 2.2.1 is the only contamination defense, and it excludes only identical instances, not task-family overlap. A model trained on a family's rule templates can legitimately improve on novel instances of that same family, so the BBEH and KOR-Bench gains are partly an in-distribution measure of the trained task schemas rather than an out-of-distribution test of general logical reasoning.
full rationale
Walking the derivation chain: the dataset is synthesized by manually implemented rule-based generators and verifiers with difficulty calibrated against external models (Qwen chat models, DeepSeek R1, o3-mini); the RL method (GRPO, DAPO-style techniques) is adopted from external open-source work; and the headline contributions are empirical training results. None of these steps reduces to a fit or to a self-citation chain by construction. The single load-bearing circularity issue is the measurement of the logical-reasoning claim: the paper explicitly states (Appendix A.1) that the 35 training tasks were 'carefully selected ... from established benchmarks including KOR-Bench, BBH, and BBEH,' the same benchmarks used for evaluation in Section 3.1.4, and Section 2.2.1 dismisses contamination only by verifying 'no identical samples' between generated data and test sets. Since held-out instances of a seen task family are still in-distribution, the BBEH (+6 over R1-Distill-Qwen-32B) and KOR-Bench gains may partly reflect learning the benchmark task schemas rather than general logical reasoning; this makes the 'state-of-the-art logical reasoning' ranking partially self-referential. The paper does contain genuine independent content: math generalization from logic-only training on base models (AIME 0.3 to 10.0 for 7B, 4.5 to 19.6 for 32B), the GPQA-Diamond gain of Zero-Mix-3 over Zero-Mix-2, and cleanly controlled mixed-training comparisons with identical step counts. Self-citations (e.g., Zeng et al. 2025b, SimpleRL-Zoo) appear but are not load-bearing, since the training recipe rests on externally cited GRPO and DAPO. Accordingly, the circularity is partial: moderate score of 4, with the caveat that the specific logic-benchmark ranking would only be secure against a held-out logic benchmark whose task families are absent from SynLogic.
Assumptions & free parameters
free parameters (1)
- Per-task difficulty parameters (e.g., Sudoku grid size, number of missing numbers) for SynLogic-Hard and SynLogic-Easy =
Calibrated so that R1/o3-mini pass@10 > 0 (upper bound) and chat model pass rate between 0 and 0.5 (lower bound)
assumptions (3)
- ad hoc to paper The rule-based generators and verifiers for the 35 tasks correctly enforce the intended constraints and answer correctness.
- domain assumption Difficulty calibration with strong models (R1, o3-mini) and chat models yields a learnable difficulty range.
- domain assumption The synthesized data has no meaningful overlap with benchmark test sets beyond the exact-duplicate check.
Cite this review
Pith. "Pith review of SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond." pith.science (2026). https://pith.science/paper/GFDZUSDA
@misc{pith2026250519641,
author = {Pith},
title = {Pith review of: SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond},
year = {2026},
howpublished = {\url{https://pith.science/paper/GFDZUSDA}},
note = {Machine review of arXiv:2505.19641}
}
read the original abstract
Recent advances such as OpenAI-o1 and DeepSeek R1 have demonstrated the potential of Reinforcement Learning (RL) to enhance reasoning abilities in Large Language Models (LLMs). While open-source replication efforts have primarily focused on mathematical and coding domains, methods and resources for developing general reasoning capabilities remain underexplored. This gap is partly due to the challenge of collecting diverse and verifiable reasoning data suitable for RL. We hypothesize that logical reasoning is critical for developing general reasoning capabilities, as logic forms a fundamental building block of reasoning. In this work, we present SynLogic, a data synthesis framework and dataset that generates diverse logical reasoning data at scale, encompassing 35 diverse logical reasoning tasks. The SynLogic approach enables controlled synthesis of data with adjustable difficulty and quantity. Importantly, all examples can be verified by simple rules, making them ideally suited for RL with verifiable rewards. In our experiments, we validate the effectiveness of RL training on the SynLogic dataset based on 7B and 32B models. SynLogic leads to state-of-the-art logical reasoning performance among open-source datasets, surpassing DeepSeek-R1-Distill-Qwen-32B by 6 points on BBEH. Furthermore, mixing SynLogic data with mathematical and coding tasks improves the training efficiency of these domains and significantly enhances reasoning generalization. Notably, our mixed training model outperforms DeepSeek-R1-Zero-Qwen-32B across multiple benchmarks. These findings position SynLogic as a valuable resource for advancing the broader reasoning capabilities of LLMs. We open-source both the data synthesis pipeline and the SynLogic dataset at https://github.com/MiniMax-AI/SynLogic.
Forward citations
Cited by 8 Pith papers
-
Reasoning Core: Designing Broad Procedural Data for Completion-Supervised Reasoning Training
A new collection of 50 procedural generators designed for completion-supervised fine-tuning beats three existing procedural collections and a no-procedural baseline on reasoning benchmarks at 3B scale in mean scores.
-
DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes
DenoiseRL optimizes recovery from noisy prefixes in weak-model reasoning failures to improve performance and self-correction on math and general reasoning benchmarks without external supervision.
-
Beyond the Sampled Token: Preserving Candidate Support in RLVR
SimKO, an asymmetric top-K token-level regularizer for RLVR, improves pass@K on math/logic benchmarks across K=1..256 without lowering pass@1 relative to GRPO.
-
Mirage or Method? How Model-Task Alignment Induces Divergent RL Conclusions
Counterintuitive RL phenomena in LLMs (one-shot, spurious reward, negative-only) appear only under strong model-task alignment measured by pass@k, not because of data contamination.
-
PuzzleClone: A DSL-Powered Framework for Synthesizing Verifiable Data
A DSL plus SMT solver generates and validates 83,657 logic puzzles, and fine-tuning on them improves a 7B model's scores on several reasoning benchmarks.
-
Propositional Logic for Probing Generalization in Neural Networks
Standard neural architectures generalize to unseen variable and operator combinations, but systematically fail when negation is applied to an operator that was hidden during training.
-
A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
A metric-oriented survey that classifies intrinsic quality and trustworthiness metrics for LLM-generated data across six modalities and documents systematic evaluation gaps in the current literature.
-
SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs
RL parameter updates across different reasoning tasks are sparse and nearly orthogonal, so multi-task RL can be parallelized, whereas SFT updates interfere and collapse under multi-stage training.
Reference graph
Works this paper leans on
-
[3]
URL https://arxiv.org/abs/2503.01307. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,
-
[5]
Openai o1 system card.arXiv preprint arXiv:2412.16720,
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Hel- yar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card.arXiv preprint arXiv:2412.16720,
-
[6]
URLhttps://openreview.net/forum?id=chfJJYC3iL. Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, San- ket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Peter Chen, et al. Big-bench extra hard.arXiv preprint arXiv:2502.19187,
-
[7]
Mathador-LM: A dynamic benchmark for mathematical reasoning on large language models
Eldar Kurtic, Amir Moeini, and Dan Alistarh. Mathador-LM: A dynamic benchmark for mathematical reasoning on large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17020–17027, Miami, Florida, USA, November
work page 2024
-
[8]
Math Training Prompt Template You are a helpful assistant. You always first think about the reasoning process in the mind and then provides the user with the answer.\nThe reasoning process and answer are enclosed within ‘<think>’ ‘</think>’ and ‘<answer>’ ‘</answer>’ tags, respectively, e.g.,\n<think>\nA detailed reasoning process here, with possible refl...
-
[9]
Kaijing Ma, Xinrun Du, Yunran Wang, Haoran Zhang, Zhoufutu Wen, Xingwei Qu, Jian Yang, Jiaheng Liu, Minghao Liu, Xiang Yue, et al. Kor-bench: Benchmarking language models on knowledge- orthogonal reasoning tasks.arXiv preprint arXiv:2410.06526,
-
[10]
Accessed: 2025-01-24. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. InFirst Conference on Language Modeling,
work page 2025
-
[11]
URL https://openreview.net/forum?id= Ti67584b98. ByteDance Seed, Yufeng Yuan, Yu Yue, Mingxuan Wang, Xiaochen Zuo, Jiaze Chen, Lin Yan, Wenyuan Xu, Chi Zhang, Xin Liu, et al. Seed-thinking-v1. 5: Advancing superb reasoning models with reinforcement learning.arXiv preprint arXiv:2504.13914,
Show all 22 references
-
[12]
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,
-
[13]
Challenging big-bench tasks and whether chain-of-thought can solve them.arXiv preprint arXiv:2210.09261,
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them.arXiv preprint arXiv:2210.09261,
-
[15]
Dapo: An open-source llm reinforcement learning system at scale
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476,
-
[16]
Acecoder: Acing coder rl via automated test-case synthesis.ArXiv, 2502.01718, 2025a
Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen. Acecoder: Acing coder rl via automated test-case synthesis.ArXiv, 2502.01718, 2025a. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating a...
-
[17]
Comprehensive Overview ofSynLogic A.1
A. Comprehensive Overview ofSynLogic A.1. Task Composition and Sources Table 4 presents the diverse collection of tasks incorporated inSynLogic. We have carefully selected these tasks from established benchmarks including KOR-Bench (Ma et al., 2024), BBH (Suzgun et al., 2022),...
2024
-
[18]
Our collection comprises 35 distinct tasks, with only two (Zebra Puzzle (Lin et al.,
and Minesweeper (Li et al., 2024). Our collection comprises 35 distinct tasks, with only two (Zebra Puzzle (Lin et al.,
2024
-
[19]
For all remaining tasks, we generated custom datasets
and ARC-AGI (Chollet, 2019)) using existing data sources. For all remaining tasks, we generated custom datasets. Importantly, we developed and implemented verifiers for all tasks in the collection, ensuring consistent evaluation across the benchmark. A.2.SynLogic-Hard andSynLo...
2019
-
[22]
All evaluations of the figure use avg@8 scoring. B.2.1. Performance Analysis of Mixed Training with Math We analyze the training dynamics when combining our logical reasoning dataset with mathematical content. Figure 10 presents performance results across three mathematical be...
2024
-
[2019]
On the measure of intelligence.arXiv preprint arXiv:1911.01547,
François Chollet. On the measure of intelligence.arXiv preprint arXiv:1911.01547,
1911 arXiv
-
[2021]
Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model
Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503.24290,
-
[2022]
On memorization of large language models in logical reasoning, 2025a
Chulin Xie, Yangsibo Huang, Chiyuan Zhang, Da Yu, Xinyun Chen, Bill Yuchen Lin, Bo Li, Badih Ghazi, and Ravi Kumar. On memorization of large language models in logical reasoning, 2025a. URLhttps://arxiv.org/abs/2410.23123. 13 SynLogic: Synthesizing Verifiable Reasoning Data at...
-
[2023]
Figure 10 | Comparative accuracy (%) of models trained with mixed data and math-only data across three mathematical benchmarks: MATH 500, AIME 2024, and AMC
2024
-
[2024]
doi: 10.18653/v1/2024.emnlp-main.946
Association for Computational Lin- guistics. doi: 10.18653/v1/2024.emnlp-main.946. URL https://aclanthology.org/2024. emnlp-main.946/. Yinghao Li, Haorui Wang, and Chao Zhang. Assessing logical puzzle solving in large language models: Insights from a minesweeper case study. In...
2024 doi
-
[2025]
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D
URL https://arxiv.org/abs/2501.12948. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D. Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars,
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.