Pith. sign in

REVIEW 3 major objections 5 minor 71 references

It's the Decoding Format, Not the Perturbation: Auditing Consistency-Based Selection for Vision-Language Test-Time Scaling

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Apparent gains of perturbation-consistency selection over chain-of-thought majority voting are a decoding-format effect, and a format-matched control removes the advantage on every benchmark tested.

desk verdict A careful negative result with a reusable control, but the control is not fully aggregation-matched and the headline claim should be bounded until the Lambda=K comparison is reported. read the letter →

arxiv 2608.01207 v2 pith:RF2VNCMO submitted 2026-08-02 cs.CV cs.AI

Puzhuo Zheng , Hasan Kurban This is my paper
classification cs.CVcs.AI
keywords vision-languagemodelstest-timescalingperturbation-groundedselectionformat-matchedcontrolmajorityvotingdecodingformatvisualgroundingconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper audits a plausible fix for weak vision-language test-time scaling: Perturbation-Grounded Selection (Pgs), which re-scores each candidate answer by whether the model re-derives it under label-preserving image edits such as crops, background masks, and mild jitter, with majority voting as the empty-perturbation special case. Against chain-of-thought-only majority voting, Pgs appears to help dramatically, up to +31.8 points on TextVQA. The paper argues that this comparison is confounded because Pgs spends its extra budget as short, no-CoT answers, and it introduces a format-matched control (MatchedCtrl) that adds the same short answers on the original image. Under that control, Pgs tracks or trails MatchedCtrl within noise on every benchmark, including the vision-required ViLP. The paper concludes that perturbation consistency is at best a partial diagnostic of visual dependence, not a usable selection signal once decoding format is controlled.

What carries the argument

The load-bearing object is the grounded support score $g(c)$, defined as the original-view vote count plus $\lambda$ times the re-derivation support $\rho$ estimated from short no-CoT samples on each label-preserving perturbation $t_m(x)$. MatchedCtrl is the control: it reuses the $N$ chain-of-thought answers and spends the same $M\cdot K$ short no-CoT draws on the original image, returning a majority vote over the pooled answers. Pgs and MatchedCtrl differ only in whether the short draws pass through perturbed views, so comparing the two isolates the perturbation term. The preserve/destroy stability gap and blanked-input ablation show that the score depends on pixels, while the routing tests show that this dependence does not translate into selection wins.

What would settle it

A concrete falsifier: on any of the four benchmarks, find a pre-specified category where Pgs beats MatchedCtrl consistently across decoding seeds at the fixed budget $N + M\cdot K = 32$ and $\lambda=2$, with the gap large enough that its paired confidence interval excludes zero; alternatively, show that the per-instance stability gap strongly predicts which examples Pgs wins.

Watch

Extended reading notes

Core claim

The central claim is negative and diagnostic. Across TextVQA, MATH-Vision, MMMU, and ViLP, with Qwen2.5-VL-7B as the headline model and LLaVA-OneVision-7B in matched-budget selector tables, Pgs beats plain chain-of-thought majority voting by up to +31.8 points on TextVQA, but the format-matched control MatchedCtrl, which reuses the $N$ chain-of-thought answers and adds the same $M\cdot K$ short no-CoT answers drawn from the original image, tracks or exceeds Pgs within seed noise on every benchmark. No Qwen category shows a significant gain over MatchedCtrl, and the per-instance preserve/destroy stability gap does not predict when Pgs wins. The paper concludes that perturbation reweighting adds nothing at the selection layer once decoding format and budget are controlled, and that gains reported against chain-of-thought-only majority voting overstate such methods.

Load-bearing premise

The conclusion rests on the assumption that MatchedCtrl isolates the perturbation term: if the way Pgs pools the extra short answers behaves differently from a plain majority vote over the same answers, a real perturbation benefit could be hidden in that aggregation difference.

Editorial extensions

If this is right

  • A gain of any perturbation- or consistency-based selector over chain-of-thought-only majority voting should not be read as evidence of visual grounding if the selector also changes decoding format.
  • Short-answer aggregation on the original image is a strong, cheap baseline at matched budget, and it absorbs the lift that Pgs appears to provide.
  • Perturbation consistency remains useful as a diagnostic of visual dependence, as shown by the BlankAblation collapse, but not as a drop-in selection rule.
  • Reporting MV lifts without a format-matched control is incomplete for grounding-at-selection claims.
  • No tested category of TextVQA, MATH-Vision, MMMU, or ViLP, including the vision-required ViLP overall, yields a significant advantage of Pgs over MatchedCtrl.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same confound likely applies beyond Pgs: any selection rule that spends extra compute as short, no-CoT samples will look stronger against a chain-of-thought-only baseline for reasons unrelated to its mechanism.
  • The paper leaves open whether a stronger signal, such as denser visual probes, process-level rewards, or cross-model disagreement, can beat MatchedCtrl; the same control should be required of any such claim.
  • If the result is right, the practical bar for VLM test-time scaling research shifts: beating majority voting is not enough, and the default comparator should be same-budget short-answer aggregation on the original image.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper audits Perturbation-Grounded Selection (Pgs), a label-free, training-free selection rule that reweights VLM candidate answers by whether the model re-derives them under label-preserving image perturbations. It introduces a format-matched control, MatchedCtrl, which spends the same short, no-CoT budget on the original image, and reports across four benchmarks (TextVQA, MATH-Vision, MMMU, ViLP) and two open VLMs that Pgs beats CoT-only majority voting by up to +31.8 points on TextVQA but does not beat MatchedCtrl at the chosen operating point. The paper further shows that the preserve/destroy stability gap is real and image-dependent but does not predict per-instance wins, and concludes that apparent gains of perturbation-consistency selection over majority voting are largely a CoT-to-short decoding-format effect rather than evidence of perturbation grounding. The authors release code and audit tooling.

Significance. If the conclusion holds, this is a valuable negative result that corrects an important overclaim in VLM test-time scaling: it shows that comparing a perturbation-based selector against CoT-only majority voting conflates the selection mechanism with a decoding-format change, and it provides a reusable MatchedCtrl-style protocol with bootstrap CIs for auditing future consistency-based selection claims. The study is unusually careful in reporting matched-budget comparisons, perturbation-family ablations, stability-gap diagnostics, routing tests, and two-model, three-seed coverage, and the code release supports reproducibility. However, the central control is not as clean as stated: MatchedCtrl differs from Pgs not only in the perturbation channel but also in aggregation weights and candidate-pool restriction, and one aggregation-matched operating point (lambda=4) shows a TextVQA advantage over MatchedCtrl that is not reconciled with the unqualified negative claim. The conclusion therefore needs qualification or additional analysis before the paper can be accepted as stated.

major comments (3)
  1. [Section 4.1 / Eq. (1)] MatchedCtrl does not isolate the perturbation term as claimed. With K=4 and w_m=1, Eq. (1) gives each perturbed short draw a weight lambda/K = 0.5 at the operating point lambda=2, whereas MatchedCtrl's pooled majority gives each original short draw weight 1; additionally, Eq. (1) restricts candidates to the original CoT answer set, while MatchedCtrl can select any answer appearing in the short draws. The statement in Section 4.1 that MatchedCtrl differs only in whether the short draws are routed through T is therefore not literally correct, and the observed null at lambda=2 may be a weighting artifact. At lambda=K=4 the aggregation weights coincide, but the paper does not report a paired Pgs-versus-MatchedCtrl comparison at this aggregation-matched operating point; Table A7 shows TextVQA Pgs=89.0 at lambda=4 versus MatchedCtrl=87.6, and Table A8's label-free lambda-selection rule selects lambda=4 with 89.0 on TextVQA. To support the unqualified claim that perturbation grounding adds nothing, the authors should report bootstrap CIs at lambda=4 (ideally with a candidate-pool-restricted control) or explicitly restrict the conclusion to the lambda=2 operating point.
  2. [Section 3.3 / Table A7] The operating point lambda=2 is not selected in a validation-holdout manner: Table A7 sweeps lambda on the same test benchmarks, and the label-free split-half rule in Table A8 is evaluated on the same data used to choose the threshold. For a negative claim this is less dangerous than for a positive one, but it matters for the Table A8 result, where the no-label rule attains 89.0 on TextVQA by selecting lambda=4; this number is not an unbiased estimate of the rule's accuracy. The paper should either use a validation split for lambda selection or clearly state that all sensitivity numbers are post hoc and not part of the decision-relevant null.
  3. [Section 5.2 / Figure 2] The claim that no category shows a significant gain of Pgs over MatchedCtrl is established only at lambda=2; Figure 2 and Table A4 use this operating point. Table A7 shows TextVQA rising to 88.7 and 89.0 at lambda=3 and 4, which would exceed the reported MatchedCtrl value of 87.6 by 1.1-1.4 points, and Table A8's label-free rule selects lambda=4 on TextVQA. The paper should report paired bootstrap CIs for Pgs-versus-MatchedCtrl at the aggregation-matched lambda=K operating point across all benchmarks and categories, or soften the abstract-level claim to 'no advantage at the fixed lambda=2 operating point.'
minor comments (5)
  1. [Section 3.3] The label-free lambda-selection rule is described only in prose; a short pseudocode block or a precise definition of the split-half agreement threshold would make Table A8 reproducible.
  2. [Table A5] The supplementary table renumbers its own references from [1], which conflicts with the main paper's reference numbering; using distinct labels such as S1, S2 would avoid ambiguity.
  3. [Table 1] The phrase 'within seed noise' is asserted for the Pgs-versus-MatchedCtrl deltas without reporting bootstrap confidence intervals in the main table; adding paired CIs to Table 1 would directly support the headline claim.
  4. [Notation] The paper inconsistently uses both 'Pgs' and 'PGS' (for example, Table 2); the notation should be unified.
  5. [Section 4.1] The candidate-pool restriction of Eq. (1) is stated only in Section 3.3; repeating it in Section 4.1 when defining MatchedCtrl would make the asymmetry between the two selectors transparent to the reader.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the negative result is an empirical comparison against external benchmarks, not a consequence of the definitions.

full rationale

I inspected the derivation chain for circularity patterns. Pgs is defined by Eq. (1) with the consistency estimator Eq. (2), and MatchedCtrl is defined in Section 4.1 as reusing the N CoT answers plus M*K short no-CoT answers on the original image and returning a majority vote on the pooled answers. The central claim is that Pgs shows no reliable gain over MatchedCtrl across benchmarks. This is a measured empirical outcome, not a consequence of the definitions: nothing in Eq. (1) or in the MatchedCtrl definition forces the accuracies to be equal, and the paper reports actual numbers (e.g., TextVQA 86.2 vs. 87.6, MATH-V 26.0 vs. 25.3). The operating point lambda=2 is fixed before the comparison, and the label-free lambda-selection variant is presented separately as a sensitivity analysis, not as a prediction from fitted data. Proposition 2, stating that majority voting is the empty-perturbation special case, is an explicit mathematical simplification of Eq. (1) and is not used to derive the empirical null. No load-bearing self-citation appears; the cited prior work is external and the paper's contribution is a new control experiment against external benchmarks. The closest concern is that MatchedCtrl changes both the perturbation channel and the aggregation rule compared with Pgs, since Pgs uses a weighted sum while MatchedCtrl uses a pooled majority vote. That is a potential methodological threat to the control's construct validity, but it is not a circularity: the conclusion would still require empirical measurement and could in principle have gone the other way. Therefore the paper is not circular in the sense targeted by this review.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central comparison rests on two kinds of unpaid premises: the hand-designed perturbation families being label-preserving for the evaluated benchmarks, and MatchedCtrl being the right null for the perturbation term. There are no invented entities; Pgs and MatchedCtrl are selection rules over existing model samples. The global weight lambda is a fitted hyperparameter selected from an on-benchmark sweep, which is the main ledger item.

free parameters (3)
  • lambda (global perturbation weight) = 2 (operating point; sweep values 0 to 4 in Table A7)
    Global weight on perturbed-view re-derivation support in Eq. (1); chosen once from a sweep on the same benchmarks used for the headline comparison, not per example.
  • Perturbation family set (M=6, K=4) = M=6, K=4 (budget N+MK=32)
    Hand-designed label-preserving perturbations: center crop, saliency crop, background mask, photometric jitter, rotation, rescale. Ablations show family choice matters mainly on TextVQA.
  • Vote weight vw=1 and perturbation weights w_m=1 = 1
    Weights in Eq. (1) set to 1 for all primary results; alternatives are not explored in the main table.
assumptions (4)
  • domain assumption The perturbation families in Definition 1 are label-preserving for the ground-truth answer on the evaluated benchmarks.
    Section 3.2 defines label-preserving perturbations by construction. If crops or masks remove evidence, perturbed-view re-derivation measures destruction rather than grounding, and Pgs's score is contaminated.
  • domain assumption MatchedCtrl isolates the perturbation term: spending the same short-answer budget on the original image is a valid null for Pgs's perturbation reweighting.
    Section 4.1. MatchedCtrl differs from Pgs in selection function (pooled majority versus weighted support), so the isolation is imperfect; the negative conclusion depends on this control being the right null.
  • domain assumption Hard-match accuracy on TextVQA, MATH-Vision, MMMU, and ViLP is a valid measure of selection quality for VLM test-time scaling.
    Used throughout Section 5; automatic scoring and dataset labels are taken as ground truth.
  • domain assumption Three decoding seeds are representative enough to support the claim that no Qwen category shows a significant gain over MatchedCtrl.
    Section 6 limitations acknowledges three seeds cannot rule out rarer regimes; the significance claim relies on bootstrap over pooled seeds.

how reviews work

0 comments
Cite this review

Pith. "Pith review of It's the Decoding Format, Not the Perturbation: Auditing Consistency-Based Selection for Vision-Language Test-Time Scaling." pith.science (2026). https://pith.science/paper/RF2VNCMO

@misc{pith2026260801207,
  author       = {Pith},
  title        = {Pith review of: It's the Decoding Format, Not the Perturbation: Auditing Consistency-Based Selection for Vision-Language Test-Time Scaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RF2VNCMO}},
  note         = {Machine review of arXiv:2608.01207}
}
read the original abstract

Test-time scaling lifts large language model reasoning by sampling many candidate solutions and selecting among them, yet the same recipe transfers poorly to vision-language models (VLMs): recent work shows that simple majority voting beats selection methods built on the model's own self-verification, apparently because at the selection layer an image-grounded answer and a confident guess from the language prior look the same. A natural fix is to make the selection signal one that cannot be computed without the image. We study Perturbation Grounded Selection (Pgs), a label-free, training-free rule that scores each candidate by whether the model re-derives it under label-preserving perturbations of the input (cropping, background masking, mild photometric or geometric jitter); Pgs recovers majority voting when the perturbation set is empty. The decisive question is not whether Pgs beats chain-of-thought only majority voting, but whether the perturbation term adds anything once decoding format and budget are controlled. We therefore introduce a format-matched control (MatchedCtrl): the same short, no-CoT draws spent on the original image. Across TextVQA, MATH-Vision, MMMU, and ViLP, with a Qwen headline (three-seed means) and LLaVA-OneVision coverage in matched-budget selector tables, Pgs appears to beat plain majority voting by up to +31.8 points on TextVQA (Qwen), but MatchedCtrl tracks or exceeds Pgs within noise on every benchmark, including the vision-required ViLP; no Qwen category shows a significant gain over this control. The stability gap is real and image-dependent (up to +0.48), yet does not predict per-instance wins. The result is negative and diagnostic: perturbation consistency is at best a partial diagnostic of visual dependence and, on its own, not a usable selection signal once format is controlled; gains reported against CoT-only majority voting overstate such methods.

Figures

Figures reproduced from arXiv: 2608.01207 by the authors.

Figure 1
Figure 1. PGS on Real ViLP-F example (Qwen2.5-VL-7B). A [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. No category shows a significant gain of Pgs over the format-matched control MatchedCtrl. Per-category difference ∆ = acc(Pgs) − acc(MatchedCtrl) with 95% bootstrap confidence intervals (Supplementary Material), pooled over seeds {0, 23, 42} on Qwen dumps. No inter￾val lies entirely in the “favors Pgs” half-plane (shaded); the ViLP interval lies entirely below zero. Points are ∆; whiskers are the 95% CI. 5.4 RoutingT… view at source ↗

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 44 canonical work pages

  1. [1]

    Aha Moment Revisited: Are

    Mingyuan Wu and Meitang Li and Jingcheng Yang and Jize Jiang and Kaizhuo Yan and Zhaoheng Li and Hanchao Yu and Minjia Zhang and Klara Nahrstedt , journal =. Aha Moment Revisited: Are

    work page

  2. [2]

    Derpanis and Babak Taati and Radek Grzeszczuk , journal =

    Ahmadreza Jeddi and Minh Ngoc Le and Amirhossein Kazerouni and Hakki Can Karaimer and Hue Nguyen and Iqbal Mohomed and Michael Brudno and Alex Levinshtein and Konstantinos G. Derpanis and Babak Taati and Radek Grzeszczuk , journal =

    work page

  3. [3]

    Avogaro, Niccolo and Debnath, Nayanika and Mi, Li and Frick, Thomas and Wang, Junling and He, Zexue and Hua, Hang and Schindler, Konrad and Rigotti, Mattia , booktitle =

    work page

  4. [4]

    Yang, Yuncong and Liu, Jiageng and Zhang, Zheyuan and Zhou, Siyuan and Tan, Reuben and Yang, Jianwei and Du, Yilun and Gan, Chuang , booktitle =

    work page

  5. [8]

    International Conference on Learning Representations (ICLR) , year =

    Wu, Mingyuan and Yang, Jingcheng and Jiang, Jize and Li, Meitang and others , title =. International Conference on Learning Representations (ICLR) , year =

    work page

  6. [10]

    European Conference on Computer Vision (ECCV) , year =

    David Wan and Jaemin Cho and Elias Stengel-Eskin and Mohit Bansal , title =. European Conference on Computer Vision (ECCV) , year =

    work page

  7. [11]

    International Conference on Learning Representations (ICLR) , year =

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author =. International Conference on Learning Representations (ICLR) , year =

  8. [12]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

Show all 71 references
  1. [13]

    Snell, Charlie and Lee, Jaehoon and Xu, Kelvin and Kumar, Aviral , journal =. Scaling

  2. [15]

    arXiv preprint arXiv:2501.12948 , year =

    arXiv

  3. [18]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Chen, Zhe and Wu, Jiannan and Wang, Wenhai and Su, Weijie and Chen, Guo and Xing, Sen and Zhong, Muyan and Zhang, Qinglong and Zhu, Xizhou and Lu, Lewei and Li, Bin and Luo, Ping and Lu, Tong and Qiao, Yu and Dai, Jifeng , title =. Proceedings of the IEEE/CVF Conference on Com...

  4. [19]

    Lu, Pan and Bansal, Hritik and Xia, Tony and Liu, Jiacheng and Li, Chunyuan and Hajishirzi, Hannaneh and Cheng, Hao and Chang, Kai-Wei and Galley, Michel and Gao, Jianfeng , booktitle =

  5. [20]

    Advances in Neural Information Processing Systems , year =

    Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset , author =. Advances in Neural Information Processing Systems , year =

  6. [21]

    Advances in Neural Information Processing Systems , year =

    Are We on the Right Way for Evaluating Large Vision-Language Models? , author =. Advances in Neural Information Processing Systems , year =

  7. [22]

    Wu, Penghao and Xie, Saining , booktitle =

  8. [23]

    Yue, Xiang and others , booktitle =

  9. [24]

    Advances in Neural Information Processing Systems , year =

    Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models , author =. Advances in Neural Information Processing Systems , year =

  10. [25]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

  11. [26]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year=

    Seeclick: Harnessing gui grounding for advanced visual gui agents , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year=

  12. [27]

    Breakthroughs in statistics: Methodology and distribution , pages=

    Bootstrap methods: another look at the jackknife , author=. Breakthroughs in statistics: Methodology and distribution , pages=. 1992 , publisher=

    1992

  13. [28]

    Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , year=

    Test-Time Consistency in Vision Language Models , author=. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , year=

  14. [29]

    Answer Self-Consistency with Margin-Triggered Question Re-Arbitration for the

    Tomoya Miyazawa and Hiroyasu Okuno , journal =. Answer Self-Consistency with Margin-Triggered Question Re-Arbitration for the

  15. [30]

    Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in

    Liu, Zhining and others , journal =. Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in

  16. [31]

    Same Answer, Different Representations: Hidden Instability in

    Farooq Ahmad Wani and Alessandro Suglia and Rohit Saxena and Aryo Pradipta Gema and Wai-Chung Kwan and Fazl Barez and Maria Sofia Bucarelli and Fabrizio Silvestri and Pasquale Minervini , journal =. Same Answer, Different Representations: Hidden Instability in

  17. [33]

    Probing Visual Language Priors in

    Luo, Tiange and Cao, Ang and Lee, Gunhee and Johnson, Justin and Lee, Honglak , booktitle =. Probing Visual Language Priors in

  18. [34]

    vvlm: Exploring visual reasoning in vlms against language priors , author=

  19. [35]

    Singh, Amanpreet and Natarajan, Vivek and Shah, Meet and Jiang, Yu and Chen, Xinlei and Batra, Dhruv and Parikh, Devi and Rohrbach, Marcus , booktitle =. Towards

  20. [36]

    Cai, Meng and Kulik, Lars and Choudhury, Farhana , journal =

  21. [37]

    International Conference on Learning Representations (ICLR) , year =

    Efficient Test-Time Scaling for Small Vision-Language Models , author =. International Conference on Learning Representations (ICLR) , year =

  22. [38]

    Test-Time Scaling for Small

    Baxevanakis, Spiros and Yang, Peng-Jian , journal =. Test-Time Scaling for Small

  23. [39]

    International Conference on Machine Learning (ICML) , year =

    Diversity Matters: Revisiting Test-Time Compute in Vision-Language Models , author =. International Conference on Machine Learning (ICML) , year =

  24. [40]

    European Conference on Computer Vision (ECCV) , year =

    On Test-Time Scaling for Vision-Language Models , author =. European Conference on Computer Vision (ECCV) , year =

  25. [41]

    Zhang, Ruiyang and Zhang, Hu and Zheng, Zhedong , journal =

  26. [42]

    arXiv preprint arXiv:2311.17311 , year =

    Universal Self-Consistency for Large Language Model Generation , author =. arXiv preprint arXiv:2311.17311 , year =

    arXiv

  27. [43]

    Advances in neural information processing systems , year=

    Scalable best-of-n selection for large language models via self-certainty , author=. Advances in neural information processing systems , year=

  28. [44]

    Findings of the Association for Computational Linguistics: ACL 2025 , year=

    Confidence improves self-consistency in llms , author=. Findings of the Association for Computational Linguistics: ACL 2025 , year=

    2025

  29. [45]

    arXiv preprint arXiv:2508.15260 , year=

    Deep think with confidence , author=. arXiv preprint arXiv:2508.15260 , year=

    arXiv

  30. [46]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  31. [47]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), 2023

    2023

  32. [48]

    Scaling LLM test-time compute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024

    2024 arXiv

  33. [49]

    Le, Christopher R \'e , and Azalia Mirhoseini

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher R \'e , and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024

    2024 arXiv

  34. [50]

    Aha moment revisited: Are VLM s truly capable of self verification in inference-time scaling? arXiv preprint arXiv:2506.17417, 2026 a

    Mingyuan Wu, Meitang Li, Jingcheng Yang, Jize Jiang, Kaizhuo Yan, Zhaoheng Li, Hanchao Yu, Minjia Zhang, and Klara Nahrstedt. Aha moment revisited: Are VLM s truly capable of self verification in inference-time scaling? arXiv preprint arXiv:2506.17417, 2026 a

    2026

  35. [51]

    Seeing but not believing: Probing the disconnect between visual attention and answer correctness in VLM s

    Zhining Liu et al. Seeing but not believing: Probing the disconnect between visual attention and answer correctness in VLM s. arXiv preprint arXiv:2510.17771, 2025

    2025

  36. [52]

    Same answer, different representations: Hidden instability in VLM s

    Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena, Aryo Pradipta Gema, Wai-Chung Kwan, Fazl Barez, Maria Sofia Bucarelli, Fabrizio Silvestri, and Pasquale Minervini. Same answer, different representations: Hidden instability in VLM s. arXiv preprint arXiv:2602.06652, 2026

    2026

  37. [53]

    Revisiting test-time scaling: A survey and a diversity-aware method for efficient reasoning

    Ho-Lam Chung, Teng-Yun Hsiao, Hsiao-Ying Huang, Chunerh Cho, Jian-Ren Lin, Ziwei Zhang, Jhen Hsieh, and Yun-Nung Chen. Revisiting test-time scaling: A survey and a diversity-aware method for efficient reasoning. arXiv preprint arXiv:2506.04611, 2025

    2025 arXiv

  38. [54]

    Diversity matters: Revisiting test-time compute in vision-language models

    Yijie Tong, Yifan Hou, Shaobo Cui, Antoine Bosselut, and Mrinmaya Sachan. Diversity matters: Revisiting test-time compute in vision-language models. In International Conference on Machine Learning (ICML), 2026

    2026

  39. [55]

    Vtool-r1: Vlms learn to think with images via reinforcement learning on multimodal tool use

    Mingyuan Wu, Jingcheng Yang, Jize Jiang, Meitang Li, et al. Vtool-r1: Vlms learn to think with images via reinforcement learning on multimodal tool use. In International Conference on Learning Representations (ICLR), 2026 b

    2026

  40. [56]

    Thinking with imagination: Agentic visual spatial reasoning with world simulators

    Chenming Zhu, Jingli Lin, Yilin Long, Peizhou Cao, Tai Wang, Jiangmiao Pang, and Xihui Liu. Thinking with imagination: Agentic visual spatial reasoning with world simulators. arXiv preprint arXiv:2606.06476, 2026

    2026 arXiv

  41. [57]

    No labels, no problem: Training visual reasoners with multimodal verifiers

    Damiano Marsili and Georgia Gkioxari. No labels, no problem: Training visual reasoners with multimodal verifiers. arXiv preprint arXiv:2512.08889, 2025

    2025

  42. [58]

    On test-time scaling for vision-language models

    Fawaz Sammani, Tzoulio Chamiti, and Nikos Deligiannis. On test-time scaling for vision-language models. In European Conference on Computer Vision (ECCV), 2026

    2026

  43. [59]

    Papadopoulos

    Mehmet Onurcan Kaya, Desmond Elliott, and Dim P. Papadopoulos. Efficient test-time scaling for small vision-language models. In International Conference on Learning Representations (ICLR), 2026

    2026

  44. [60]

    ARBITER : Reasoning trajectory basins and majority vote failures in test-time sampling

    Meng Cai, Lars Kulik, and Farhana Choudhury. ARBITER : Reasoning trajectory basins and majority vote failures in test-time sampling. arXiv preprint arXiv:2605.26172, 2026

    2026 arXiv

  45. [61]

    Limits and gains of test-time scaling in vision-language reasoning

    Mohammadjavad Ahmadpour, Amirmahdi Meighani, Payam Taebi, Omid Ghahroodi, Amirmohammad Izadi, and Mahdieh Soleymani Baghshah. Limits and gains of test-time scaling in vision-language reasoning. arXiv preprint arXiv:2512.11109, 2025

    2025

  46. [62]

    Derpanis, Babak Taati, and Radek Grzeszczuk

    Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni, Hakki Can Karaimer, Hue Nguyen, Iqbal Mohomed, Michael Brudno, Alex Levinshtein, Konstantinos G. Derpanis, Babak Taati, and Radek Grzeszczuk. AVIS : Adaptive test-time scaling for vision--language models. arXiv preprint arX...

    2026 arXiv

  47. [63]

    SPARC : Separating perception and reasoning circuits for test-time scaling of VLM s

    Niccolo Avogaro, Nayanika Debnath, Li Mi, Thomas Frick, Junling Wang, Zexue He, Hang Hua, Konrad Schindler, and Mattia Rigotti. SPARC : Separating perception and reasoning circuits for test-time scaling of VLM s. In Proceedings of the 43rd International Conference on Machine L...

    2026

  48. [64]

    MindJourney : Test-time scaling with world models for spatial reasoning

    Yuncong Yang, Jiageng Liu, Zheyuan Zhang, Siyuan Zhou, Reuben Tan, Jianwei Yang, Yilun Du, and Chuang Gan. MindJourney : Test-time scaling with world models for spatial reasoning. In Advances in Neural Information Processing Systems, 2025

    2025

  49. [65]

    Mitigating object hallucinations in large vision-language models through visual contrastive decoding

    Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2024

  50. [66]

    Contrastive region guidance: Improving grounding in vision-language models without training

    David Wan, Jaemin Cho, Elias Stengel-Eskin, and Mohit Bansal. Contrastive region guidance: Improving grounding in vision-language models without training. In European Conference on Computer Vision (ECCV), 2025

    2025

  51. [67]

    Zoom consistency: A free confidence signal in multi-step visual grounding pipelines

    Keon Kim and Krish Chelikavada. Zoom consistency: A free confidence signal in multi-step visual grounding pipelines. arXiv preprint arXiv:2604.15376, 2026

    2026 arXiv

  52. [68]

    Test-time consistency in vision language models

    Shih-Han Chou, Shivam Chandhok, James J Little, and Leonid Sigal. Test-time consistency in vision language models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2026

    2026

  53. [69]

    Answer self-consistency with margin-triggered question re-arbitration for the CVPR 2026 VidLLMs challenge

    Tomoya Miyazawa and Hiroyasu Okuno. Answer self-consistency with margin-triggered question re-arbitration for the CVPR 2026 VidLLMs challenge. arXiv preprint arXiv:2606.04323, 2026

    2026 arXiv

  54. [70]

    Test-time scaling for small VLM s on multilingual visual MCQ

    Spiros Baxevanakis and Peng-Jian Yang. Test-time scaling for small VLM s on multilingual visual MCQ . arXiv preprint arXiv:2607.09438, 2026

    2026 arXiv

  55. [71]

    VL-Uncertainty : Detecting hallucination in large vision-language models via uncertainty estimation

    Ruiyang Zhang, Hu Zhang, and Zhedong Zheng. VL-Uncertainty : Detecting hallucination in large vision-language models via uncertainty estimation. arXiv preprint arXiv:2411.11919, 2024

    2024 arXiv

  56. [72]

    Towards VQA models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards VQA models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019

    2019

  57. [73]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. In Advances in Neural Information Processing Systems, 2024

    2024

  58. [74]

    MMMU : A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI

    Xiang Yue et al. MMMU : A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  59. [75]

    Probing visual language priors in VLM s ( ViLP )

    Tiange Luo, Ang Cao, Gunhee Lee, Justin Johnson, and Honglak Lee. Probing visual language priors in VLM s ( ViLP ). In International Conference on Machine Learning (ICML), 2025

    2025

  60. [76]

    Qwen2.5-VL technical report

    Qwen Team . Qwen2.5-VL technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  61. [77]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  62. [78]

    Scalable best-of-n selection for large language models via self-certainty

    Zhewei Kang, Xuandong Zhao, and Dawn Song. Scalable best-of-n selection for large language models via self-certainty. Advances in neural information processing systems, 2026

    2026

  63. [79]

    Confidence improves self-consistency in llms

    Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. Confidence improves self-consistency in llms. In Findings of the Association for Computational Linguistics: ACL 2025, 2025

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.