Pith. sign in

REVIEW 3 major objections 5 minor 54 references

Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that LLM performance is predominantly anchored to explicit final answers rather than to the textual pattern of reasoning steps, and that much apparent reasoning is post-hoc rationalization.

desk verdict Serious, large-scale study whose headline 26.9-point answer-masking effect is confounded; the AER-vs-AMR contrast and the conflict experiment are the defensible core. read the letter →

arxiv 2506.17630 v1 pith:HV6C3FA3 submitted 2025-06-21 cs.CL

Yang Wu , Yifan Zhang , Yiwei Wang , Yujun Cai , Yurong Wu , Yuran Wang , Ning Xu , Jian Cheng This is my paper
classification cs.CL
keywords answeranchoringchain-of-thoughtreasoningpost-hocrationalizationpromptinterventionmemorizationinLLMsevaluationvisibilityRoR-Bench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether LLMs solve reasoning tasks by genuinely following reasoning steps or by latching onto the final answer. Using a five-level prompt framework that progressively hides answer cues while keeping reasoning chains intact, it finds a consistent, monotonic accuracy drop as answers become less visible: from 84.75% when the answer is explicit to 57.85% when only the answer token is masked, and 28.17% with no cues at all. Conflict experiments show models follow a correct answer even when the accompanying reasoning is wrong more often than the reverse. The paper concludes that LLM performance is anchored to explicit answers and that much of their apparent reasoning is post-hoc rationalization rather than independent inference.

What carries the argument

The central instrument is a five-level answer-visibility prompt framework: Answer-Explicit (AE), Answer-Embedded-Reasoning (AER), Answer-Masked-Reasoning (AMR), Answer-Removed-Reasoning (ARR), and Answer-Free (AF). It manipulates only the presence and form of answer cues while holding the problem and prompt format fixed, so any accuracy change is attributed to answer visibility. Two auxiliary probes extend it: conflict prompts that pair correct/wrong answers with wrong/correct reasoning chains, and soft/hard warning prompts that instruct the model to distrust the provided answer. The framework is applied zero-shot, with temperature 0, across ten models, with GPT-4o used to automate answer masking and judge correctness.

What would settle it

A control condition that masks an incidental, non-answer token on the same RoR-Bench problems would undermine the answer-anchoring conclusion if it produced a drop comparable to the 26.90-point AMR drop, because the drop could then be attributed to masking itself. Alternatively, finding any LLM that maintains near-AE accuracy when the final answer is masked while the reasoning chain is intact would falsify the claim that performance is predominantly anchored to explicit answers.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that LLM memory binding is answer-centric: models rely on visible final answers far more than on the textual pattern of reasoning chains. This is established behaviorally by comparing five prompt conditions (Answer-Explicit, Answer-Embedded-Reasoning, Answer-Masked-Reasoning, Answer-Removed-Reasoning, Answer-Free) on the text-only subset of RoR-Bench. Accuracy falls monotonically as answer cues are removed, with a 26.90 percentage-point gap between the explicit-answer condition and the masked-answer condition even though the full reasoning chain remains present. When answers and reasoning conflict, correct answers paired with wrong reasoning outperform wrong answers paired with right reasoning (56.58% vs 41.39%). Warning prompts that tell models the provided answer is false reduce accuracy but do not eliminate the answer's influence, and some models still beat their no-cue baseline while trying to ignore it. The paper interprets these patterns as evidence that generated reasoning chains frequently rationalize a known answer rather than derive it.

Load-bearing premise

The load-bearing premise is that masking or deleting the answer only removes answer cues and does not damage the reasoning chain's coherence, so the accuracy drop is caused by answer absence rather than by a broken or confusing prompt; the paper includes no control that masks a non-answer token or replaces the answer with a neutral conclusion.

Editorial extensions

If this is right

  • Because performance drops 26.90 points when only the answer token is masked, intact reasoning chains do not, by themselves, support robust inference in these models.
  • In conflicted prompts, a correct answer with wrong reasoning outperforms a wrong answer with right reasoning, so explicit answers dominate reasoning templates as the memory anchor.
  • Warning prompts reduce but do not erase answer anchoring, so answer dependence is tenacious rather than a simple input-format artifact.
  • Evaluations that provide or leak answers may overstate LLM reasoning ability; answer-free conditions give the truer baseline.
  • Accuracy scales monotonically with answer visibility on these problems, making the visibility gradient itself a diagnostic of answer anchoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A control condition that masks a non-answer token on the same problems would test whether the AMR drop is specific to answers or reflects any masked token interrupting the prompt; the paper does not include this control.
  • If answer anchoring is a learned shortcut, models trained with answer-free or answer-later objectives should show a smaller AE-to-AMR gap; this is a testable prediction the paper does not make.
  • The answer-anchoring lens could be applied to visual or multimodal reasoning, which the paper explicitly leaves to future work and for which the same pattern has not been demonstrated.
  • The low answer-citation rates under the AE condition suggest models use the provided answer without acknowledging it, implying that models' self-reports about their own reasoning are unreliable attribution signals.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper asks whether LLM reasoning performance is anchored to explicit final answers or to the textual patterns of reasoning chains. It introduces a five-level answer-visibility prompt framework (AE, AER, AMR, ARR, AF) applied to the text-only subset of RoR-Bench, and evaluates ten LLMs. The main empirical claim is a monotonic accuracy gradient from Answer-Explicit (84.75%) down to Answer-Free (28.17%), with a 26.90 percentage-point drop from AE to AMR. A conflict experiment (Table 3) shows that models perform better when the answer is correct but reasoning is wrong than when reasoning is correct but the answer is wrong, which the authors interpret as answer priority. Warning-prompt experiments probe the tenacity of this anchoring. The paper concludes that LLM performance is predominantly anchored to explicit answers and that much apparent reasoning may be post-hoc rationalization.

Significance. The question addressed is timely and important for the debate about whether LLMs infer or retrieve. The paper has several strengths: it evaluates a broad set of state-of-the-art reasoning models; it uses a zero-shot protocol with temperature 0; it reports a consistent monotonic gradient; the conflict experiment (Table 3) is a well-designed behavioral probe that partially supports the answer-priority interpretation; and the warning-prompt experiment adds a useful tenacity check. The paper does not fit parameters or assume its own conclusion, so circularity is not a concern. However, the headline quantitative claim is weakened by a confound: AE and AMR differ in both answer visibility and the presence/completeness of the reasoning chain. The masking and removal manipulations also likely alter the grammatical and logical structure of the prompt in ways that are not controlled. The central claim is defensible but requires additional control conditions and a more careful framing of the main comparison.

major comments (3)
  1. [Section 4.1, Table 1] The headline 26.90 percentage-point gap between AE (84.75%) and AMR (57.85%) conflates answer visibility with prompt structure: AE contains the answer and no reasoning chain, while AMR contains a full chain with the answer masked. The correct comparison for isolating the effect of masking is AER (76.96%) versus AMR (57.85%), a 19.11-point drop, which the paper itself reports later in Section 4.1 but does not use as the central number. Please reframe the main quantitative claim around the AER-to-AMR comparison and clearly label the AE-to-AMR difference as a combined manipulation of answer presence and chain presence.
  2. [Section 3.2, Appendix A.3] The AMR manipulation replaces all answer-related phrases with the artificial token [MASK], including 'diverse linguistic, numerical, unit-quantified, ordinal, conclusive, and uncertainty forms,' which can produce heterogeneous and often grammatically awkward spans. LLMs are known to be sensitive to token-level perturbations, so the AMR drop may reflect distributional shift or instruction-following difficulty rather than answer anchoring specifically. The paper should include at least one control condition that masks or replaces a non-answer content token (e.g., a number in the problem statement) or that replaces the answer with a grammatical neutral placeholder (e.g., 'a certain value'), and show that such a control does not produce a comparable accuracy drop.
  3. [Section 3.2, ARR definition] The ARR condition removes 'any sentence or clause that directly states—or unmistakably reveals—the final answer,' which in most reasoning chains removes the concluding step that the reasoning is logically intended to derive. The resulting prompt is not merely answer-free; it is logically incomplete and potentially incoherent. The ARR accuracy of 46.46% may therefore reflect the absence of a conclusion rather than the absence of an answer cue. A control condition that replaces the removed concluding sentence with a grammatically valid but content-neutral sentence (e.g., 'Therefore, the result follows from the given conditions') would help separate answer removal from chain truncation.
minor comments (5)
  1. [Table 3] The caption says 'Analysis of Incorrect Answers with Correct Reasoning and Vice Versa,' but the columns include 'Wrong Answer + Wrong Reasoning' and both conflict conditions; please align the caption with the actual column definitions.
  2. [Section 4.2] The construction of the conflict conditions is under-specified: the paper says RoR-Bench pairs each problem with its unmodified source question, but it does not give a concrete example of how the original answer and reasoning are combined with the modified problem. Please provide one worked example for the RA/WR and WA/RR conditions.
  3. [Figure 3] Figure 3 has two panels with different model sets and no error bars; the accuracy values printed inside the bars are hard to read for some models. Consider a single aligned axis and confidence intervals per condition.
  4. [Table 1] No statistical significance tests are reported for the differences between conditions across the 158 problems; at least a paired test or confidence intervals for the AER-to-AMR and AE-to-AF contrasts would strengthen the quantitative claims.
  5. [References] The reference list contains multiple entries that are not cited in the main text (e.g., Aho and Ullman 1972, Chandra et al. 1981, Rasooli and Tetreault 2015), while some cited works lack full publication details; please clean up the bibliography.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical intervention with no fitted parameters, no self-citation chain, and no construction-level identity between its conditions and its conclusions.

full rationale

The paper's central claim is supported by a series of measured accuracy differences across five prompt conditions; it derives no equations and fits no parameters. The conditions (AE, AER, AMR, ARR, AF) are operational manipulations of answer visibility, and the observed gradient is an empirical outcome, not a logical consequence of the definitions. The closest candidate for circularity is that AMR is defined as masking answer tokens and the paper reports a drop when answer tokens are masked, but this is the intended causal intervention rather than a circular derivation: the drop is measured, and the paper also reports the AER-to-AMR drop (19.11 points) and the conflict experiment (RA/WR 56.58% vs. WA/RR 41.39%) as independent corroboration. The masking and deletion manipulations may confound answer removal with input corruption or logical incompleteness, and the AE-versus-AMR headline mixes answer presence with chain presence; these are validity concerns, not circularity. Citations to Yan et al. (2025) supply the benchmark and judge prompt but are external prior work, not a self-citation chain carrying the argument. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported. The Limitations section explicitly acknowledges that providing the answer with the prompt is an artificial setup, further showing that the authors do not present the manipulation as a derivation. Therefore the circularity burden is essentially zero.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no fitted parameters and no new physical or theoretical entities. The 'latent memory anchor' is a metaphorical description of the hypothesized behavior, not an independently postulated mechanism with falsifiable handles outside the paper. The central assumptions are about the validity of the prompt manipulations, the reliability of the GPT-4o judge, and the cleanliness of the benchmark.

assumptions (3)
  • domain assumption The answer-visibility manipulation isolates answer anchoring from other prompt factors.
    The design assumes AE, AER, AMR, ARR, AF differ only in answer visibility, not in chain coherence or model confusion. The lack of control conditions weakens this assumption.
  • domain assumption GPT-4o-based masking and judging are accurate enough for the conclusions.
    Masking and answer verification depend on an external LLM (GPT-4o-1120) without human evaluation or reported accuracy.
  • domain assumption RoR-Bench problems have low training-data contamination.
    The study inherits this claim from Yan et al. (2025) and relies on it to interpret drops as reasoning failure rather than benchmark contamination.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs." pith.science (2026). https://pith.science/paper/HV6C3FA3

@misc{pith2026250617630,
  author       = {Pith},
  title        = {Pith review of: Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HV6C3FA3}},
  note         = {Machine review of arXiv:2506.17630}
}
read the original abstract

While Large Language Models (LLMs) demonstrate impressive reasoning capabilities, growing evidence suggests much of their success stems from memorized answer-reasoning patterns rather than genuine inference. In this work, we investigate a central question: are LLMs primarily anchored to final answers or to the textual pattern of reasoning chains? We propose a five-level answer-visibility prompt framework that systematically manipulates answer cues and probes model behavior through indirect, behavioral analysis. Experiments across state-of-the-art LLMs reveal a strong and consistent reliance on explicit answers. The performance drops by 26.90\% when answer cues are masked, even with complete reasoning chains. These findings suggest that much of the reasoning exhibited by LLMs may reflect post-hoc rationalization rather than true inference, calling into question their inferential depth. Our study uncovers the answer-anchoring phenomenon with rigorous empirical validation and underscores the need for a more nuanced understanding of what constitutes reasoning in LLMs.

Figures

Figures reproduced from arXiv: 2506.17630 by the authors.

Figure 1
Figure 1. When confronted with an unseen yet similar problem, an LLM often recalls a memorized answer–reasoning [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Schematic of LLMs responses to a reasoning task under systematically manipulated input prompts that [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Impact of Soft and Hard Warning prompts on LLM accuracy in an Answer-Explicit setting. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 18 canonical work pages

  1. [1]

    https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/ Gemini 2.5: Our most intelligent ai model

    2025. https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/ Gemini 2.5: Our most intelligent ai model

    work page 2025

  2. [2]

    Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint arXiv:2402.00157

    arXiv 2024

  3. [3]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman. 1972. The Theory of Parsing, Translation and Compiling, volume 1. Prentice-Hall, Englewood Cliffs, NJ

    1972

  4. [4]

    American Psychological Association . 1983. Publications Manual. American Psychological Association, Washington, DC

    work page 1983

  5. [5]

    Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817--1853

    work page 2005

  6. [6]

    Galen Andrew and Jianfeng Gao. 2007. Scalable training of L1 -regularized log-linear models. In Proceedings of the 24th International Conference on Machine Learning, pages 33--40

    2007

  7. [7]

    Anthropic. 2025. https://assets.anthropic.com/m/785e231869ea8b3b/original/claude-3-7-sonnet-system-card.pdf Claude 3.7 sonnet system card

    2025

  8. [8]

    Iv \'a n Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Conmy. 2025. Chain-of-thought reasoning in the wild is not always faithful. arXiv preprint arXiv:2503.08679

    arXiv 2025

Show all 54 references
  1. [9]

    Chandra, Dexter C

    Ashok K. Chandra, Dexter C. Kozen, and Larry J. Stockmeyer. 1981. https://doi.org/10.1145/322234.322243 Alternation . Journal of the Association for Computing Machinery, 28(1):114--133

    1981

  2. [10]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, and 1 others. 2024. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1--45

    2024

  3. [11]

    Giannis Chatziveroglou, Richard Yun, and Maura Kelleher. 2025. Exploring llm reasoning through controlled prompt variations. arXiv preprint arXiv:2504.02111

    2025 arXiv

  4. [12]

    Changyu Chen, Xiting Wang, Ting-En Lin, Ang Lv, Yuchuan Wu, Xin Gao, Ji-Rong Wen, Rui Yan, and Yongbin Li. 2024 a . Masked thought: Simply masking partial reasoning steps can improve mathematical reasoning learning of language models. In Proceedings of the 62nd Annual Meeting ...

    2024

  5. [13]

    Simin Chen, Yiming Chen, Zexin Li, Yifan Jiang, Zhongwei Wan, Yixin He, Dezhi Ran, Tianle Gu, Haizhou Li, Tao Xie, and 1 others. 2025 a . Recent advances in large langauge model benchmarks against data contamination: From static to dynamic evaluation. arXiv preprint arXiv:2502.17521

    2025

  6. [14]

    Wentao Chen, Lizhe Zhang, Li Zhong, Letian Peng, Zilong Wang, and Jingbo Shang. 2025 b . Memorize or generalize? evaluating llm code generation with evolved questions. arXiv preprint arXiv:2503.02296

    2025

  7. [15]

    Xinyun Chen, Ryan Andrew Chi, Xuezhi Wang, and Denny Zhou. 2024 b . Premise order matters in reasoning with large language models. In International Conference on Machine Learning, pages 6596--6620. PMLR

    2024

  8. [16]

    Yuyan Chen, Yueze Li, Songzhou Yan, Sijia Liu, Jiaqing Liang, and Yanghua Xiao. 2024 c . Do large language models have problem-solving capability under incomplete information scenarios? In Findings of the Association for Computational Linguistics ACL 2024, pages 2225--2238

    2024

  9. [17]

    Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. 2024. Black-box prompt optimization: Aligning large language models without model training. In Proceedings of the 62nd Annual Meeting of the Association for Computational Lingui...

    2024

  10. [18]

    Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. 2025. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514

    2025 arXiv

  11. [19]

    Yi-Fu Fu, Yu-Chieh Tu, Tzu-Ling Cheng, Cheng-Yu Lin, Yi-Ting Yang, Heng-Yi Liu, Keng-Te Liao, Da-Cheng Juan, and Shou-De Lin. 2024. Think or remember? detecting and directing llms towards memorization or generalization. arXiv preprint arXiv:2412.18497

    2024 arXiv

  12. [20]

    Bryan Guan, Tanya Roosta, Peyman Passban, and Mehdi Rezagholizadeh. 2025. The order effect: Investigating prompt sensitivity in closed-source llms. arXiv preprint arXiv:2502.04134

    2025 arXiv

  13. [21]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  14. [22]

    Dan Gusfield. 1997. Algorithms on Strings, Trees and Sequences. Cambridge University Press, Cambridge, UK

    1997

  15. [23]

    Kaixuan Huang, Jiacheng Guo, Zihao Li, Xiang Ji, Jiawei Ge, Wenzhe Li, Yingqing Guo, Tianle Cai, Hui Yuan, Runzhe Wang, and 1 others. 2025. Math-perturb: Benchmarking llms' math reasoning abilities against hard perturbations. arXiv preprint arXiv:2502.06453

    2025 arXiv

  16. [24]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  17. [25]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515

    2024 arXiv

  18. [26]

    Yuxuan Jiang and Francis Ferraro. 2024. Memorization over reasoning? exposing and mitigating verbatim memorization in large language models' character understanding evaluation. arXiv preprint arXiv:2412.14368

    2024 arXiv

  19. [27]

    Katie Kang, Amrith Setlur, Dibya Ghosh, Jacob Steinhardt, Claire Tomlin, Sergey Levine, and Aviral Kumar. 2024. What do learning dynamics reveal about generalization in llm reasoning? arXiv preprint arXiv:2411.07681

    2024 arXiv

  20. [28]

    Koray Kavukcuoglu. 2025. https://blog.google/products/gemini/gemini-2-5-flash-preview/ Developers can now start building with gemini 2.5 flash

    2025

  21. [29]

    Chunyang Li, Weiqi Wang, Tianshi Zheng, and Yangqiu Song. 2025. Patterns over principles: The fragility of inductive reasoning in llms under noisy observations. arXiv preprint arXiv:2502.16169

    2025 arXiv

  22. [30]

    Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. 2024. Gsm-plus: A comprehensive benchmark for evaluating the robustness of llms as mathematical problem solvers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume...

    2024

  23. [31]

    Yucheng Li, Frank Guerin, and Chenghua Lin. 2023. An open source data contamination report for large language models. arXiv preprint arXiv:2310.17589

    2023 arXiv

  24. [32]

    Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, and 1 others. 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110

    2022 arXiv

  25. [33]

    https://openai.com/index/introducing-o3-and-o4-mini/ Introducing openai o3 and o4-mini

    OpenAI. https://openai.com/index/introducing-o3-and-o4-mini/ Introducing openai o3 and o4-mini

  26. [34]

    OpenAI. 2025. https://openai.com/index/openai-o3-mini/ Openai o3-mini

    2025

  27. [35]

    Pouya Pezeshkpour and Estevam Hruschka. 2024. Large language models sensitivity to the order of options in multiple-choice questions. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2006--2017

    2024

  28. [36]

    Zeju Qiu, Weiyang Liu, Haiwen Feng, Zhen Liu, Tim Z Xiao, Katherine M Collins, Joshua B Tenenbaum, Adrian Weller, Michael J Black, and Bernhard Sch \"o lkopf. 2024. Can large language models understand symbolic graphics programs? arXiv preprint arXiv:2408.08313

    2024 arXiv

  29. [37]

    Tetreault

    Mohammad Sadegh Rasooli and Joel R. Tetreault. 2015. http://arxiv.org/abs/1503.06733 Yara parser: A fast and accurate dependency parser . Computing Research Repository, arXiv:1503.06733. Version 2

    2015 arXiv

  30. [38]

    Dylan Sam, Marc Finzi, and J Zico Kolter. 2025. Predicting the performance of black-box llms through self-queries. arXiv preprint arXiv:2501.01558

    2025

  31. [39]

    Safal Shrestha, Minwu Kim, and Keith Ross. 2025. Mathematical reasoning in large language models: Assessing logical and arithmetic errors across wide numerical ranges. arXiv preprint arXiv:2502.08680

    2025 arXiv

  32. [40]

    Qwen Team. 2025. https://qwenlm.github.io/blog/qwq-32b/ Qwq-32b: Embracing the power of reinforcement learning

    2025

  33. [41]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  34. [42]

    Zhenyu Wu, Qingkai Zeng, Zhihan Zhang, Zhaoxuan Tan, Chao Shen, and Meng Jiang. 2024. Large language models can self-correct with key condition verification. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12846--12867

    2024

  35. [43]

    https://x.ai/news/grok-3 Grok 3 beta --- the age of reasoning agents

    xAI. https://x.ai/news/grok-3 Grok 3 beta --- the age of reasoning agents

  36. [44]

    Chulin Xie, Yangsibo Huang, Chiyuan Zhang, Da Yu, Xinyun Chen, Bill Yuchen Lin, Bo Li, Badih Ghazi, and Ravi Kumar. 2024. On memorization of large language models in logical reasoning. arXiv preprint arXiv:2410.23123

    2024 arXiv

  37. [45]

    Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2023. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. In The Twelfth International Conference on Learning Representations

    2023

  38. [46]

    Cheng Xu, Shuhao Guan, Derek Greene, M Kechadi, and 1 others. 2024. Benchmark data contamination of large language models: A survey. arXiv preprint arXiv:2406.04244

    2024 arXiv

  39. [47]

    Kai Yan, Yufei Xu, Zhengyin Du, Xuesong Yao, Zheyu Wang, Xiaowen Guo, and Jiecao Chen. 2025. Recitation over reasoning: How cutting-edge language models can fail on elementary school-level reasoning problems? arXiv preprint arXiv:2504.00509

    2025

  40. [48]

    Ziqing Yang, Yixin Wu, Yun Shen, Wei Dai, Michael Backes, and Yang Zhang. 2025. The challenge of identifying the origin of black-box large language models. arXiv preprint arXiv:2503.04332

    2025 arXiv

  41. [49]

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1--38

    2024

  42. [50]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

  43. [51]

    Zhanke Zhou, Rong Tao, Jianing Zhu, Yiwen Luo, Zengmao Wang, and Bo Han. 2024. Can language models perform robust reasoning in chain-of-thought prompting with noisy rationales? Advances in Neural Information Processing Systems, 37:123846--123910

    2024

  44. [52]

    Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Neil Gong, and 1 others. 2023. Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts. In Proceedings of the 1st ACM Workshop on ...

    2023

  45. [53]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  46. [54]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.