Pith. sign in

REVIEW 5 major objections 7 minor 3 cited by

Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A model's confidence in its own short answer, measured by perplexity, can decide when long reasoning is needed—and that gate beats both always-short and always-long reasoning.

desk verdict A practical PPL-based router for selective reasoning that mostly works, but the Gaussian decision boundary and the 'outperforms long-form' claim need scrutiny before I'd trust the numbers. read the letter →

arxiv 2505.15154 v1 pith:JBRWY2AD submitted 2025-05-21 cs.CL cs.AIcs.MM

Jinghui Lu , Haiyang Yu , Siliang Xu , Shiwei Ran , Guozhi Tang , Siqi Wang , Bin Shan , Teng Fu
show 4 more authors
Hao Feng Jingqun Tang Han Wang Can Huang
This is my paper · ORCID
classification cs.CLcs.AIcs.MM
keywords certainty-basedadaptivereasoningperplexityroutingchain-of-thoughttokenefficiencymultimodallargelanguagemodelsVQAmathematical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that prolonged chain-of-thought reasoning is not universally beneficial and that a model can decide, answer by answer, whether reasoning is worth the tokens. It proposes Certainty-based Adaptive Reasoning (CAR): generate a short answer first, compute the model's perplexity on that answer, and treat low perplexity as confidence. Only when the short answer is judged likely to be wrong does CAR switch to long-form reasoning. Across text and multimodal benchmarks, the paper reports that this routing beats both always-short and always-long baselines, cutting tokens by roughly 45 percent relative to always-long while slightly improving accuracy.

What carries the argument

The object that carries the argument is the short-answer perplexity, $\mathrm{PPL}=\exp(-\frac{1}{T}\sum_{t=1}^{T}\log p(w_t\mid w_1,\ldots,w_{t-1}))$, computed over the tokens of the model's own concise answer. The paper assumes the perplexities of correct ($C=1$) and incorrect ($C=0$) short answers follow Gaussian distributions, $f_1\sim\mathcal{N}(\mu_1,\sigma_1^2)$ and $f_0\sim\mathcal{N}(\mu_0,\sigma_0^2)$, fits the parameters on the training set with equal class counts, and at inference computes the posterior $P(C=1\mid\mathrm{PPL})$ from the likelihood ratio $f_1(\mathrm{PPL})/f_0(\mathrm{PPL})$ and the class priors. If the posterior for correctness is greater, the short answer is final; otherwise the model is prompted into long-form reasoning. This Gaussian likelihood ratio is the decision mechanism that converts token-level uncertainty into a routing choice.

What would settle it

Run CAR on an out-of-domain set where the uncertainty scores of right and wrong answers overlap more heavily than they did during training: if the fixed decision boundary routes many correct short answers into long reasoning, accuracy should fall toward or below the always-long baseline while token savings shrink. A direct check is to compare the empirical uncertainty-score histograms against the two fitted bell curves and count how often the likelihood ratio chooses the wrong branch.

Watch

Extended reading notes

Core claim

CAR's central claim is that the perplexity of a model's own short answer is a reliable confidence signal, and that a likelihood-ratio test based on two Gaussians can route between short and long reasoning. The paper fits separate Gaussian distributions to the perplexities of correct and incorrect short answers on the training set, balances the two classes, then applies Bayes' rule at inference: if the posterior probability of correctness is higher, the short answer is emitted; otherwise the model is prompted to reason step by step and that longer answer is emitted. With Qwen2.5-7B, CAR reports 81.1% average accuracy on GSM8K, StrategyQA, and MathQA versus 55.8% for short-only and 75.0% for long-only baselines, while generating 69.2 tokens on average versus 126.1 for long reasoning; with Llama3.1-8B the corresponding numbers are 74.9%, 51.5%, and 70.8%, with 93.4 tokens versus 171.8. On the multimodal VQA/KIE sets, CAR with Qwen2-VL reaches 77.9% average accuracy while using about 15 percent of the tokens that full reasoning uses. The paper also reports that CAR outperforms token-reduction baselines TALE and Chain-of-Draft on these benchmarks.

Load-bearing premise

The load-bearing premise is that the model's uncertainty scores for right and wrong short answers fall into two bell-shaped distributions, and that the same two curves describe new questions; if real scores are skewed or shift when the topic changes, the routing decision is miscalculated and the reported balance of accuracy and token savings changes.

Editorial extensions

If this is right

  • Always-on long reasoning is not a safe default: on extractive VQA/KIE tasks it can lower accuracy while multiplying token cost, so routing should be applied wherever answers are short and fact-based.
  • Confidence-based routing compounds with concise-reasoning methods: combining CAR with TALE improves GSM8K accuracy to 85.5% from 78.8% with Qwen2.5 while also reducing tokens.
  • Perplexity on the first short answer is a cheap gate because it comes from the same forward pass that produced the answer, with no extra model call.
  • The method's benefit is concentrated where difficulty varies: on MathQA, CAR with Qwen2.5 reaches 83.8% versus 67.1% for Chain-of-Draft, while on GSM8K, where nearly every item needs reasoning, CAR gains little and mostly routes to reasoning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Gaussian assumption is a modeling convenience, not a necessity: a non-parametric density estimate or a calibration curve over perplexity would give a router that is less exposed to skewed or multi-modal perplexity distributions.
  • The same routing idea could be applied inside a generation loop, re-scoring confidence after each reasoning step and stopping early once the model is sure, which would extend token savings beyond a single decision at the first answer.
  • Because the two Gaussian curves are fit on the training perplexities of a specific model, the router likely needs re-calibration when the backbone or domain changes; testing whether a small calibration set is enough is a direct follow-up.
  • The observed failure cases where extra reasoning walks the model away from a correct answer suggest an explicit 'stop and answer' action as a potentially trainable behavior, rather than only an inference-time gate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes Certainty-based Adaptive Reasoning (CAR), a routing framework that generates a short answer, computes the perplexity (PPL) of that answer, and decides whether to emit it directly or to trigger long-form chain-of-thought reasoning. The decision rule is a likelihood ratio between two Gaussian distributions over PPL, whose parameters are estimated on training examples labeled by short-answer correctness, with a balanced prior imposed by subsampling. The method is evaluated on three multimodal datasets (DocVQA, ChartQA, FUNSD) using Qwen2-VL-7B and on three text reasoning datasets (GSM8K, StrategyQA, MathQA) using Qwen2.5-7B and Llama3.1-8B, comparing against short-answer, long-form, TALE, and CoD baselines. The authors report that CAR improves average accuracy over both short and long baselines while reducing token consumption relative to long-form reasoning, and they argue that PPL is a reliable confidence signal based on pilot experiments.

Significance. The idea of confidence-based adaptive routing is timely and practically relevant, and the paper offers a simple, transparent mechanism that can be layered onto instruction-tuned models. The pilot study provides useful descriptive evidence of an inverse correlation between answer-level PPL and correctness, and the breadth of models and benchmarks gives the empirical study reasonable coverage. However, the central claim as stated in the abstract—that CAR outperforms both short-answer and long-form reasoning—is not uniformly supported by the reported tables, and the Gaussian router is never validated against the empirical PPL distributions. If these issues are addressed, the method could be a valuable efficiency-accuracy trade-off tool; in its current form, the evidence is suggestive rather than conclusive.

major comments (5)
  1. [Section 5.3, Tables 5-6] The abstract and Section 5.3 state that CAR 'outperforms both short-answer and long-form reasoning approaches,' but this is contradicted by the Llama3.1 results in Table 6: CARLlama3.1 achieves an average accuracy of 74.9, while CARLlama3.1-Long achieves 76.0, and on GSM8K the gap is 71.6 versus 73.5. The central accuracy-efficiency claim should therefore be qualified to acknowledge that the routing gain over always-long reasoning is model-dependent and is absent for Llama3.1 in these experiments.
  2. [Sections 4.3-4.4, Figure 1] The Gaussian likelihood-ratio router is never validated: the empirical PPL histograms in Figure 1 are visibly right-skewed (e.g., the DocVQA, ChartQA, and CORD panels), and no normality test or train/test PPL distribution comparison is reported. Because the balanced prior n0 = n1 in Eq. (8) can differ substantially from the true correct-answer rate (e.g., GSM8K short-answer accuracy is 24.2% in Table 5), the routing boundary may be miscalibrated; please validate the distributional assumption with a nonparametric or distribution-free router comparison and report sensitivity to the prior.
  3. [Section 5.2, Appendix A] The multimodal experiments use reasoning-process annotations generated from OCR text and bounding boxes in the pilot study, as stated in Section 3.1 and reused in Section 5.2 ('we reuse the reasoning process data obtained in the pilot study'). Fine-tuning Qwen2-VL on image inputs with these text-derived rationales introduces a train/inference modality mismatch, so the reported multimodal accuracy and token savings may not reflect genuinely image-grounded reasoning; please clarify the data composition and add image-derived reasoning annotations or an explicit analysis of this mismatch.
  4. [Section 5.1, Tables 4-6] All experimental results come from a single training run, with no seeds or variance estimates reported; the deterministic beam-search decoding does not remove training stochasticity. Since most routing gains over the always-long variant are only 1-3 accuracy points (e.g., 77.9 vs 75.8 in Table 4 and 81.1 vs 78.8 in Table 5), the stability of these improvements cannot be assessed without multiple seeds or confidence intervals.
  5. [Section 5.3, Tables 5-6] The TALE baseline numbers on MathQA appear anomalously low (50.4 for Qwen2.5 and 48.3 for Llama3.1) compared with its GSM8K results (87.6 and 80.6), which drives much of the reported average-accuracy improvement of CAR over TALE (81.1 vs 72.8 and 74.9 vs 68.3). Because TALE is replicated with the same hyperparameters as the other fine-tuned models, the authors should confirm that the TALE replication is properly tuned on MathQA or report official TALE numbers; otherwise the 'outperforms state-of-the-art token reduction methods' claim is not yet established.
minor comments (7)
  1. [Section 5.1] In Section 5.1, 'reasnong' is a typo for 'reasoning'.
  2. [Appendix A.1] In Appendix A.1, 'polit experiment' should be 'pilot experiment'.
  3. [Figure 1] In Figure 1, the legend labels 'correct answers' and 'incorret answers'; 'incorret' should be 'incorrect'.
  4. [Section 5.4] In Section 5.4, 'these insights highlight the university of CAR' should read 'universality of CAR'.
  5. [Section 3.2] Section 3.2 refers to 'Equation 4.2' for the PPL definition, but the PPL equation is Eq. (2) in Section 4.2; please fix the cross-reference.
  6. [Appendix D] Appendix D references Figure 6 and Figure 7, but Figure 6 appears only in Appendix E; renumber or reorder the figures.
  7. [References] Reference [27] is cited for the specific claim that 'overthinking' hurts performance on simpler tasks, but [27] is a survey; consider citing the primary studies it surveys as well.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CAR's routing parameters are fit on training data with ground-truth labels and evaluated on held-out benchmarks, so the central accuracy/efficiency claims do not reduce to their inputs.

full rationale

The paper's central derivation is self-contained and empirically evaluated. The Gaussian parameters for correct and incorrect PPL scores are estimated from training-set short answers using ground-truth correctness labels (Eqs. 5-6), and the routing decision at inference uses only the test example's PPL (Eqs. 7-8). The reported accuracy on DocVQA, ChartQA, FUNSD, GSM8K, MathQA, and StrategyQA is measured on held-out test sets against external baselines (Tables 4-6), so the performance claims are not forced by construction. The only in-sample use of test data is the exploratory threshold experiment in Section 3.3, where the threshold is set at the 75th percentile of the test set's PPL distribution; the paper explicitly disclaims this as not deployable, stating that 'in real-world applications, the test set's PPL scores are unavailable. Therefore, this experiment only serves as an exploratory validation.' That experiment is not the proposed CAR method and is not presented as a prediction. The Gaussian assumption and balanced prior are simplifying modeling choices that may affect robustness, but they are not circular because correctness labels come from ground truth rather than from PPL. Self-citations appear only as ordinary background references and are not load-bearing. Overall, the derivation chain does not reduce to its own inputs.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method's routing is a supervised likelihood-ratio classifier on PPL. All free parameters (Gaussian means, variances, balanced priors) are fitted to the training set using ground-truth correctness labels. The key assumptions are the Gaussian form of PPL distributions and train/test distributional stability. No new entities are introduced.

free parameters (5)
  • mu_1 (mean PPL for correct short answers) = Average 1.15 across pilot datasets for Qwen2.5-0.5B; 1.07 for Qwen2.5-7B
    Estimated from training set using ground-truth correctness labels (Section 4.3, Equation 5).
  • sigma_1^2 (variance of PPL for correct short answers) = Not reported numerically
    Estimated from training set (Section 4.3, Equation 5).
  • mu_0 (mean PPL for incorrect short answers) = Average 1.28 across pilot datasets for Qwen2.5-0.5B; 1.17 for Qwen2.5-7B
    Estimated from training set using ground-truth correctness labels (Section 4.3, Equation 6).
  • sigma_0^2 (variance of PPL for incorrect short answers) = Not reported numerically
    Estimated from training set (Section 4.3, Equation 6).
  • Prior balancing n0 = n1 = Balanced (n0 = n1)
    Chosen by hand in Section 4.4 to balance correctly and incorrectly answered examples.
assumptions (3)
  • ad hoc to paper PPL scores of correct and incorrect short answers are each Gaussian distributed
    Section 4.3 states the assumption; no normality test is provided, and Figure 1 shows heavy right tails and skew.
  • domain assumption Training-set PPL distribution and priors generalize to test-time inputs
    The routing uses Gaussians fitted on the training set; any distribution shift between train and test PPLs (e.g., OOD datasets in Section 3) could misroute.
  • domain assumption Long-form reasoning triggered by low confidence will produce a better final answer than the short answer
    The method assumes the long reasoning path is corrective; on GSM8K the paper itself notes gains are modest and the long path is sometimes worse (Section 5.4, Tables 5-6).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning." pith.science (2026). https://pith.science/paper/JBRWY2AD

@misc{pith2026250515154,
  author       = {Pith},
  title        = {Pith review of: Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JBRWY2AD}},
  note         = {Machine review of arXiv:2505.15154}
}
read the original abstract

Recent advancements in reasoning have significantly enhanced the capabilities of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) across diverse tasks. However, excessive reliance on chain-of-thought (CoT) reasoning can impair model performance and brings unnecessarily lengthened outputs, reducing efficiency. Our work reveals that prolonged reasoning does not universally improve accuracy and even degrade performance on simpler tasks. To address this, we propose Certainty-based Adaptive Reasoning (CAR), a novel framework that dynamically switches between short answers and long-form reasoning based on the model perplexity. CAR first generates a short answer and evaluates its perplexity, triggering reasoning only when the model exhibits low confidence (i.e., high perplexity). Experiments across diverse multimodal VQA/KIE benchmarks and text reasoning datasets show that CAR outperforms both short-answer and long-form reasoning approaches, striking an optimal balance between accuracy and efficiency.

Figures

Figures reproduced from arXiv: 2505.15154 by the authors.

Figure 1
Figure 1. PPL distribution of Correct and Incorrect Predictions. X-axis is PPL, Y-axis is frequency. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Mean accuracy vs. mean PPL for differ￾ent datasets. First, we analyze the relationship between ac￾curacy and PPL scores at the dataset level, fo￾cusing on short-answer responses. Our anal￾ysis across multiple datasets reveals a strong inverse correlation, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The training pipeline (a) and the inference process (b) of CAR. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: A failure case of reasoning model. What is the content in the ”REGION:" field? Short/GT: <answer(ONLY IF PARTIAL REGION CONTINUE WITH DIVISION (S) SCOPE)</answer> Long Thinking:<think>Okay, let's tackle this query step by step. The user wants to know what's in the \"RE…
Figure 5
Figure 5. Figure 5: A failure case of reasoning model. First, [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 7
Figure 7. Figure 7: PPL distribution of Correct and Incorrect Predictions. X-axis is PPL, Y-axis is frequency. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 6
Figure 6. Figure 6: Mean accuracy vs. mean PPL for differ￾ent datasets. OOD is out-of-domain test set. This section explores the feasibility of integrat￾ing CAR with token reduction techniques such as TALE [43]. Most reasoning token reduc￾tion methods operate by replacing the ground￾truth…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data

    cs.CV 2026-01 reject novelty 4.0 of 10

    A lightweight RGB-D cross-attention network is proposed for rail defect detection, but the SOTA accuracy and generalization claims are internally inconsistent and the implementation is not public.

  2. Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Method

    cs.CV 2026-01 reject novelty 3.0 of 10

    Fine-tuning Stable Diffusion with DreamBooth-style knowledge and hypernetwork-guided crack control maps can synthesize substation meter defect images that boost a YOLOv8 defect detector's mAP when added to the training set.

  3. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

71 extracted references · 26 canonical work pages · cited by 3 Pith papers

  1. [1]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213, 2022

    2022

  2. [2]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  3. [3]

    Causal language modeling can elicit search and reasoning capabilities on logic puzzles

    Kulin Shah, Nishanth Dikkala, Xin Wang, and Rina Panigrahy. Causal language modeling can elicit search and reasoning capabilities on logic puzzles. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview. net/forum?id=i5PoejmWoC

    work page 2024

  4. [4]

    Faithful chain-of-thought reasoning

    Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. Faithful chain-of-thought reasoning. In The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL 2023), 2023

    2023

  5. [5]

    Chain of preference optimization: Improving chain-of-thought reasoning in llms

    Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, and Min Lin. Chain of preference optimization: Improving chain-of-thought reasoning in llms. Advances in Neural Information Processing Systems, 37:333–356, 2024

    work page 2024

  6. [6]

    Towards revealing the mystery behind chain of thought: a theoretical perspective

    Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. Towards revealing the mystery behind chain of thought: a theoretical perspective. Advances in Neural Information Processing Systems, 36:70757–70798, 2023

    2023

  7. [7]

    Unlocking the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of- thought

    Qiguang Chen, Libo Qin, Jiaqi Wang, Jingxuan Zhou, and Wanxiang Che. Unlocking the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of- thought. Advances in Neural Information Processing Systems, 37:54872–54904, 2024

    work page 2024

  8. [8]

    Openai o1 hub, 2025

    OpenAI. Openai o1 hub, 2025. URL https://openai.com/o1

    work page 2025

Show all 71 references
  1. [9]

    Deepseek-vl: towards real-world vision-language understanding

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, et al. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525, 2024

    2024 arXiv

  2. [10]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  3. [11]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  4. [12]

    Simpo: Simple preference optimization with a reference-free reward

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37:124198–124235, 2024

    2024

  5. [13]

    H2o: Heavy-hitter oracle for efficient generative inference of large language models

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, et al. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Syste...

    2023

  6. [14]

    Qwq: Reflect deeply on the boundaries of the unknown

    Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown. https://example. com/qwen-qwq, 2025. URL https://example.com/qwen-qwq

    2025

  7. [15]

    Seed-thinking-v1

    ByteDance Seed, Yufeng Yuan, Yu Yue, Mingxuan Wang, Xiaochen Zuo, Jiaze Chen, Lin Yan, Wenyuan Xu, Chi Zhang, Xin Liu, et al. Seed-thinking-v1. 5: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.13914, 2025

    2025

  8. [16]

    Gemini 2.5: Our most intelligent AI model

    Google DeepMind. Gemini 2.5: Our most intelligent AI model. https://blog.google/ technology/ai/google-gemini-2-5-ai-model/ , March 2025. Last updated March 26, 2025

    2025

  9. [17]

    Advancing sequential numerical prediction in autoregressive models, 2025

    Xiang Fei, Jinghui Lu, Qi Sun, Hao Feng, Yanjie Wang, Wei Shi, An-Lan Wang, Jingqun Tang, and Can Huang. Advancing sequential numerical prediction in autoregressive models, 2025. URL https://arxiv.org/abs/2505.13077

    2025 arXiv

  10. [18]

    To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning

    Zayne Rea Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. In The Thirteenth International Conferen...

    2025

  11. [19]

    R1-v: Reinforcing super generaliza- tion ability in vision-language models with less than $3.https://github.com/Deep-Agent/ R1-V, 2025

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generaliza- tion ability in vision-language models with less than $3.https://github.com/Deep-Agent/ R1-V, 2025. Accessed: 2025-02-02

    2025

  12. [20]

    Vlm-r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  13. [21]

    Punifiedner: A prompting-based unified ner system for diverse datasets

    Jinghui Lu, Rui Zhao, Brian Mac Namee, and Fei Tan. Punifiedner: A prompting-based unified ner system for diverse datasets. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 13327–13335, 2023

    2023

  14. [22]

    Jinghui Lu, Dongsheng Zhu, Weidong Han, Rui Zhao, Brian Mac Namee, and Fei Tan. What makes pre-trained language models better zero-shot learners? In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Com...

    2023 doi

  15. [23]

    Padellm- ner: parallel decoding in large language models for named entity recognition

    Jinghui Lu, Yanjie Wang, Ziwei Yang, Xuejing Liu, Brian Mac Namee, and Can Huang. Padellm- ner: parallel decoding in large language models for named entity recognition. Advances in Neural Information Processing Systems, 37:117853–117880, 2024

    2024

  16. [24]

    A bounding box is worth one token: Interleaving layout and text in a large language model for document understanding

    Jinghui Lu, Haiyang Yu, Yanjie Wang, Yongjie Ye, Jingqun Tang, Ziwei Yang, Binghong Wu, Qi Liu, Hao Feng, Han Wang, et al. A bounding box is worth one token: Interleaving layout and text in a large language model for document understanding. arXiv preprint arXiv:2407.01976, 2024

    2024 arXiv

  17. [25]

    Vision as lora

    Han Wang, Yongjie Ye, Bingru Li, Yuxiang Nie, Jinghui Lu, Jingqun Tang, Yanjie Wang, and Can Huang. Vision as lora. arXiv preprint arXiv:2503.20680, 2025

    2025 arXiv

  18. [26]

    Eve: Towards end-to-end video subtitle extraction with vision-language models

    Haiyang Yu, Jinghui Lu, Yanjie Wang, Yang Li, Han Wang, Can Huang, and Bin Li. Eve: Towards end-to-end video subtitle extraction with vision-language models. arXiv preprint arXiv:2503.04058, 2025

    2025

  19. [27]

    Stop overthinking: A survey on efficient reasoning for large language models

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419, 2025

    2025 arXiv

  20. [28]

    Efficient reasoning models: A survey

    Sicheng Feng, Gongfan Fang, Xinyin Ma, and Xinchao Wang. Efficient reasoning models: A survey. arXiv preprint arXiv:2504.10903, 2025. 11

    2025

  21. [29]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  22. [30]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations , 2024. URL https: //openreview.net/...

    2024

  23. [31]

    Language models can solve computer tasks

    Geunwoo Kim, Pierre Baldi, and Stephen McAleer. Language models can solve computer tasks. Advances in Neural Information Processing Systems, 36:39648–39677, 2023

    2023

  24. [32]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in neural information processing systems, 36:11809–11822, 2023

    2023

  25. [33]

    Boosting of thoughts: Trial-and-error problem solving with large language models

    Sijia Chen, Baochun Li, and Di Niu. Boosting of thoughts: Trial-and-error problem solving with large language models. arXiv preprint arXiv:2402.11140, 2024

    2024 arXiv

  26. [34]

    Large language models as common- sense knowledge for large-scale task planning

    Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as common- sense knowledge for large-scale task planning. In A. Oh, T. Naumann, A. Glober- son, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Informa- tion Processing Systems , volume 36, pages 319...

  27. [35]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

    2023

  28. [36]

    Toolqa: A dataset for llm question answering with external tools

    Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, and Chao Zhang. Toolqa: A dataset for llm question answering with external tools. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, page...

    2023

  29. [37]

    Reflexion: language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neu- ral Information Processing Syst...

    2023

  30. [38]

    Patil, Tianjun Zhang, Xin Wang, and Joseph E

    Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large language model connected with massive apis. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems , vo...

    2024

  31. [39]

    Camel: Communicative agents for "mind" exploration of large language model society

    Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large language model society. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neu- ral Information P...

    2023

  32. [40]

    How well do llms compress their own chain-of- thought? a token complexity approach

    Ayeong Lee, Ethan Che, and Tianyi Peng. How well do llms compress their own chain-of- thought? a token complexity approach. arXiv preprint arXiv:2503.01141, 2025

    2025 arXiv

  33. [41]

    Reasoning models can be effective without thinking

    Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking. arXiv preprint arXiv:2504.09858, 2025. 12

    2025 arXiv

  34. [42]

    Concise thoughts: Impact of output length on llm reasoning and cost

    Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. Concise thoughts: Impact of output length on llm reasoning and cost. arXiv preprint arXiv:2407.19825, 2024

    2024 arXiv

  35. [43]

    Token-budget-aware llm reasoning

    Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware llm reasoning. arXiv preprint arXiv:2412.18547, 2024

    2024 arXiv

  36. [44]

    Chain of draft: Thinking faster by writing less

    Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600, 2025

    2025 arXiv

  37. [45]

    Skeleton-of- thought: Large language models can do parallel decoding

    Xuefei Ning, Zinan Lin, Zixuan Zhou, Zifu Wang, Huazhong Yang, and Yu Wang. Skeleton-of- thought: Large language models can do parallel decoding. Proceedings ENLSP-III, 2023

    2023

  38. [46]

    Training large language models to reason in a continuous latent space

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024

    2024 arXiv

  39. [47]

    Codi: Compressing chain-of-thought into continuous space via self-distillation

    Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of-thought into continuous space via self-distillation. arXiv preprint arXiv:2502.21074, 2025

    2025 arXiv

  40. [48]

    Reasoning to learn from latent thoughts

    Yangjun Ruan, Neil Band, Chris J Maddison, and Tatsunori Hashimoto. Reasoning to learn from latent thoughts. arXiv preprint arXiv:2503.18866, 2025

    2025

  41. [49]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209, 2021

    2021

  42. [50]

    Infographicvqa

    Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawa- har. Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1697–1706, 2022

    2022

  43. [51]

    ChartQA: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors,Findings of the Association for Computationa...

    2022 doi

  44. [52]

    Visualmrc: Machine reading comprehen- sion on document images

    Ryota Tanaka, Kyosuke Nishida, and Sen Yoshida. Visualmrc: Machine reading comprehen- sion on document images. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13878–13888, 2021

    2021

  45. [53]

    Icdar2019 competition on scanned receipt ocr and information extraction

    Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthenis Karatzas, Shijian Lu, and CV Jawahar. Icdar2019 competition on scanned receipt ocr and information extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1516–1520. IEEE, 2019

    2019

  46. [54]

    Cord: a consolidated receipt dataset for post-ocr parsing

    Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. Cord: a consolidated receipt dataset for post-ocr parsing. In Workshop on Document Intelligence at NeurIPS 2019, 2019

    2019

  47. [55]

    Funsd: A dataset for form understanding in noisy scanned documents

    Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran. Funsd: A dataset for form understanding in noisy scanned documents. In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), volume 2, pages 1–6. IEEE, 2019

    2019

  48. [56]

    Modeling entities as semantic points for visual information extraction in the wild

    Zhibo Yang, Rujiao Long, Pengfei Wang, Sibo Song, Humen Zhong, Wenqing Cheng, Xiang Bai, and Cong Yao. Modeling entities as semantic points for visual information extraction in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, page...

    2023

  49. [57]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. 13

    arXiv

  50. [58]

    MathQA: Towards interpretable math word problem solving with operation-based formalisms

    Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. MathQA: Towards interpretable math word problem solving with operation-based formalisms. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019...

    2019

  51. [59]

    Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346–361, 2021

    2021

  52. [60]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  53. [61]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  54. [62]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  55. [63]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  56. [64]

    Sgdr: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016

    2016 arXiv

  57. [65]

    Unidoc: A universal large multimodal model for simultaneous text detection, recognition, spotting and understanding

    Hao Feng, Zijian Wang, Jingqun Tang, Jinghui Lu, Wengang Zhou, Houqiang Li, and Can Huang. Unidoc: A universal large multimodal model for simultaneous text detection, recognition, spotting and understanding. arXiv preprint arXiv:2308.11592, 2023

    2023 arXiv

  58. [66]

    Layoutllm: Layout instruction tuning with large language models for document understanding

    Chuwei Luo, Yufan Shen, Zhaoqing Zhu, Qi Zheng, Zhi Yu, and Cong Yao. Layoutllm: Layout instruction tuning with large language models for document understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15630–15640, 2024

    2024

  59. [67]

    Docpedia: Un- leashing the power of large multimodal model in the frequency domain for versatile document understanding

    Hao Feng, Qi Liu, Hao Liu, Wengang Zhou, Houqiang Li, and Can Huang. Docpedia: Un- leashing the power of large multimodal model in the frequency domain for versatile document understanding. arXiv preprint arXiv:2311.11810, 2023

    2023 arXiv

  60. [68]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  61. [69]

    Given document <document>Saint[466,77,554,107]\nLouis[561,77,657,107]\nHeart[664,77,760,106]

    Qwen. Introducing qwen-7b: Open foundation and human-aligned models (of the state-of-the- arts), 2023. URL https://github.com/QwenLM/Qwen-7B. 14 PPL DocVQA ChartQA FUNSD SROIE Average Short Answer 11.6 9.1 17.2 21.7 14.9 Long-form Thinking 472.8 1156.6 612.6 330.2 643.1 Table ...

    2023

  62. [2023]

    URL https://proceedings.neurips.cc/paper_files/paper/2023/file/ 65a39213d7d0e1eb5d192aa77e77eeb7-Paper-Conference.pdf

    2023

  63. [5087]

    (336)\" but no fax here. Wait, maybe the phone number isn't listed right after FAX. Let me check again.\n\nWait, later on, there's a section labeled \

    Then another entry under PHONE NUMBER as \"(336)\" but no fax here. Wait, maybe the phone number isn't listed right after FAX. Let me check again.\n\nWait, later on, there's a section labeled \"FAX NO.\" which might be part of the same area code or combination. Scanning furthe...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.