Pith. sign in

REVIEW 2 major objections 7 minor 35 references

For small open vision-language models on multilingual visual MCQ, the conditions that let reasoning chains finish and commit an answer matter more than elaborate search or verifiers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-07-13 02:59 UTC pith:HF5GGUVK

load-bearing objection Solid shared-task working notes: for small open VLMs on multilingual visual MCQ, parseability and per-chain token budget dominate PRM search, chain count, and selectors; first-place 84.1% with code. the 2 major comments →

arxiv 2607.09438 v1 pith:HF5GGUVK submitted 2026-07-10 cs.CL cs.AIcs.LG

Test-Time Scaling for Small VLMs on Multilingual Visual MCQ

Spiros Baxevanakis , Peng-Jian Yang This is my paper
classification cs.CL cs.AIcs.LG
keywords test-time scalingvision-language modelsmultimodal reasoningself-consistencyprocess reward modelsmultilingual visual MCQparseabilitytoken budget
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tests whether test-time scaling techniques that help large language models also help small open vision-language models on hard multilingual visual multiple-choice exams. Under a single-GPU and at-most-7B constraint on EXAMS-V, the authors compare plain self-consistency against describe-then-reason with process-reward-model beam search and against two post-hoc answer selectors. They find that parseability and per-chain token budget dominate: an answer cue plus guided repair largely removes chains that reason correctly but never emit a letter, and raising the limit from 1k to 2k tokens recovers 3.7 percentage points while doubling chains from 8 to 16 adds only 0.15. Once chains can finish, PRM-guided beam search trails majority-vote self-consistency by 0.39 points at over eight times the cost, and neither a generative critic nor a trained multimodal PRM beats majority vote. A sympathetic reader should care because, under tight budgets, the practical priority is finishing and extracting answers, not building more sophisticated search or verification machinery; the largest single gain still comes from switching the policy model itself.

Core claim

What matters for test-time scaling of small open VLMs is the conditions under which sampling runs—especially answer parseability and a large enough per-chain token budget—not the search or verification machinery. Raising the token limit from 1k to 2k recovers 3.7 pp while going from 8 to 16 chains adds 0.15 pp; PRM-guided beam search trails plain self-consistency by 0.39 pp at over eight times the cost; neither a training-free generative critic nor a trained multimodal PRM beats majority vote across both policies; and switching the policy yields the largest gain (+11.4 pp). The best setup reaches 84.1% on the held-out ImageCLEF 2026 test split.

What carries the argument

Self-consistency with a large decoding budget and guided parse repair: sample parallel chains under a standard answer cue, extract the letter (or force one token under a guided choice when every chain fails to commit), then take majority vote with log-probability tie-breaking. This simple object is what PRM-guided beam search and post-hoc selectors fail to beat once chains are allowed to finish.

Load-bearing premise

The ranking of token budget and parseability over PRM search and selectors is assumed to hold beyond the two Qwen-family models tested; other small open VLMs may emit answers or respond to search differently.

What would settle it

Re-run the matched self-consistency versus PRM-BAS and selector comparisons on a third small open VLM family at a fixed 2k-token budget on the full validation set, and check whether PRM search or a critic then beats majority vote by a clear, significant margin.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Under single-GPU and ≤7B constraints, allocate tokens first to longer completions and reliable answer formatting before investing in process reward models or beam search.
  • TTS studies that do not control parse-failure rates risk attributing extraction failures to reasoning or scaling method failures.
  • Parallel sampling helps most where chains err independently (mid-accuracy languages, STEM); it cannot fix high-agreement shared mistakes or missing domain knowledge.
  • Policy-model quality remains the largest lever; once the substrate is fixed, elaborate inference-time machinery adds little.
  • The reported best configuration (newer 4B policy, N=16, 2k tokens, guided repair) ranks first on the Visual MCQ leaderboard at 84.1% on the held-out test split.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same finish-and-parse priority may apply to other small multimodal models and answer formats beyond letter MCQ, not only the two Qwen policies studied here.
  • If process reward models stay poorly calibrated on multilingual non-math content, future work may need either broader PRM training data or to abandon per-step search for flat parallel sampling.
  • Decoupling perception from reasoning, as the authors flag, is a natural next place to re-test whether structured search becomes useful once truncation and parse failures are removed.
  • Compute-optimal TTS recipes derived on large text models need re-derivation when the bottleneck is chain truncation rather than verifier quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper studies whether test-time scaling (TTS) techniques that help large language models transfer to small open vision–language models under a single-GPU, ≤7B constraint on the multilingual visual MCQ benchmark EXAMS-V (ImageCLEF 2026). Across Qwen2.5-VL-7B-Instruct and Qwen3.5-4B, the authors compare flat self-consistency, describe-then-reason with PRM-guided beam annealing (PRM-BAS), and two post-hoc selectors (a training-free generative critic and Qwen-VL-PRM-7B). The central empirical claim is that the conditions under which TTS runs—especially answer parseability and per-chain token budget—dominate search strategy, chain count, and verification machinery: fixing the answer cue and raising max_new_tokens from 1k to 2k recovers +3.7 pp, while N=8→16 adds only +0.15 pp; once chains finish, PRM-BAS trails majority-vote SC by 0.39 pp at ~8× cost, and neither selector beats majority vote on two pools of different strength. The largest single gain is the policy switch itself (+11.4 pp). The best configuration (Qwen3.5-4B, SC-N=16, 2k tokens, guided parse repair) reaches 81.6% on full validation and 84.1% on the held-out test split, ranking first on the Visual MCQ leaderboard.

Significance. If the result holds, it is a practically important corrective for TTS work on small multimodal models: under realistic compute envelopes, engineering substrate (prompt format that forces answer commitment, sufficient decoding budget) and policy quality matter more than elaborate search or trained verifiers. The paper supports this with full-validation ablations (n=4,651), Wilson CIs, McNemar tests with Bonferroni correction, agreement-tier and per-language/subject stratification, and a held-out leaderboard ranking. Code is released. The transparent diagnosis of parse-fail as truncation-plus-extraction rather than reasoning failure, and the controlled comparison of selectors on both weak and strong pools, are particularly useful for the community. The Qwen-only scope is a real external-validity limit, but it is stated clearly in §7 and does not erase the internal ranking of factors on the models and task studied.

major comments (2)
  1. Abstract, Introduction, and Conclusion frame the contribution as applying to “small open vision-language models” in general, while §7 correctly states that only two Qwen-family policies were tested and that parseability in particular may be family-specific. The internal ranking (token budget ≫ chain count; SC ≳ PRM-BAS; majority ≳ selectors) is well-supported for these policies and is the paper’s load-bearing claim. Please tighten the abstract/conclusion wording so the scope of generalization matches the evidence (e.g., “on two Qwen-family small VLMs” or “under the policies and budget studied”), leaving broader transfer as an explicit open question rather than an implied result. This is a framing fix, not a request for new models.
  2. §5.2 / PRM-BAS val-scale comparison: the diagnosis that DTR removes image access at the reasoning stage and that PRM P(+) saturates (0.962 / 0.849) is important and well-argued. The net −0.39 pp vs SC is reported on n=4,319 with correction/regression counts. To make the “elaborate methods contribute little” claim fully load-bearing, please state more explicitly in the main text (not only diagnosis) that the comparison is against a DTR+PRM-BAS pipeline whose structural image-loss bottleneck is partly independent of the PRM, so the result is best read as “this structured pipeline does not beat flat SC on this task,” not as a universal indictment of all PRM-guided search. A one-sentence scope sentence would suffice.
minor comments (7)
  1. Table 2 and §5.1: the note that Q3.5 CoT/SC(1k) parse-fail rates (16–18%) cause the CoT-to-SC(1k) margin to conflate truncation absorption with diversity is excellent; consider elevating a short parenthetical or footnote so readers do not over-interpret the 1k SC number as pure sampling gain.
  2. Figure 1a: the dual arrows for +3.7 pp (budget) vs +0.15 pp (chain count) are clear; adding the absolute call counts or a brief legend note that DTR/PRM-BAS points are Q2.5/dev-200 would reduce any risk of cross-policy misreading.
  3. Table 4: the Q2.5 pool is pre-MMMU closer (anchor 65.60% vs post-closer 66.42% in Table 2). The caption already notes this; a matching one-line reminder in the main §5.6 text would help readers comparing across tables.
  4. §3 / Guided parse repair: the method is clean and eliminates all-unparseable questions by construction. Briefly note whether repaired single-token answers are included in the majority with equal weight to free-form chains (they appear to be) so that replication is unambiguous.
  5. Appendix D / Table 8: several cells show SC regressions vs CoT or ZS under Q2.5 (e.g., Islamic Studies, Agriculture). A short pointer in §5.7 that majority vote can amplify correlated errors in text-heavy subjects would strengthen the stratified discussion without new experiments.
  6. Title line “Nika at ImageCLEF 2026…” is fine for working notes but may confuse journal readers; if the venue expects a standalone title, consider moving the team handle to the author block or a footnote.
  7. Minor consistency: abstract reports 84.1% test; §5.8 and Table 12 report 84.06%. Prefer a single rounded form throughout.

Circularity Check

0 steps flagged

No significant circularity: purely empirical TTS ablations scored against independent EXAMS-V / ImageCLEF labels.

full rationale

The paper’s load-bearing claims are comparative accuracy measurements on the external EXAMS-V validation set and the held-out ImageCLEF 2026 test split, whose ground-truth answer letters are independent of the authors’ prompts, budgets, selectors, and search procedures. Token-budget vs. chain-count deltas, PRM-BAS vs. self-consistency, and critic/PRM vs. majority vote are all post-hoc evaluations against those labels (Tables 2–4, Figure 1), not quantities fitted and then re-labeled as predictions. Guided parse repair forces commitment to a letter but does not define correctness; accuracy remains external. Citations (Ahmed et al. for DTR scaffold, Hu et al. for PRM-BAS hyperparameters, Qwen-VL-PRM, generative-verifier literature) supply methods or baselines, not uniqueness theorems or self-defined quantities that force the reported ranking. No self-definitional loop, fitted-input-as-prediction, or load-bearing self-citation chain is present. The work is self-contained empirical benchmarking; circularity score is zero.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 1 invented entities

Empirical systems paper. Free parameters are the usual decoding and search hyperparameters chosen by the authors or taken from prior work; axioms are standard assumptions about the benchmark and open-weight models; no new physical or mathematical entities are postulated.

free parameters (6)
  • per-chain max_new_tokens = 2048 for headline
    Swept 1024 vs 2048 (and 3072 on dev); the 1k-to-2k choice is load-bearing for the +3.7 pp claim.
  • chain count N = 8 or 16
    Swept 8 vs 16; chosen under single-A40/A100 envelope.
  • sampling temperature T = 0.7
    Swept {0.3,0.5,0.7,0.9}; inverted-U peak at 0.7 used for main runs.
  • PRM-BAS beam hyperparameters (B0, B, tau, d) = B0=4, B=2, tau=0.05, d=6
    Taken from Hu et al. and used for the search contrast; affect cost and diversity.
  • max_pixels / vision token budget = 1003520
    Set to 1,003,520 to leave ~12k tokens for reasoning under max_model_len=16384.
  • selector skip threshold = >=5/8
    High-confidence majorities (>=5/8) pass through; only weak and unparseable tiers rescored.
axioms (4)
  • domain assumption EXAMS-V validation and ImageCLEF 2026 test labels are correct external ground truth for multilingual visual MCQ.
    All accuracy claims are measured against these labels; no independent re-annotation is performed.
  • domain assumption Open-weight Qwen2.5-VL-7B-Instruct and Qwen3.5-4B are representative enough of small open VLMs for the relative ranking of TTS factors to be informative.
    Stated as a limitation; the policy-switch gain and all method comparisons rest on these two models from one family.
  • standard math Majority vote over parseable answer letters (with log-prob tie-break) is a valid aggregation of parallel chains.
    Standard self-consistency assumption used as the primary selector baseline.
  • domain assumption Qwen-VL-PRM-7B's one-shot and per-step P(+) scores are usable process rewards for multimodal reasoning steps on this domain.
    Used for both PRM-BAS search and the post-hoc PRM selector; the paper itself diagnoses P(+) saturation as a bottleneck.
invented entities (1)
  • guided parse repair (single-token guided_choice over A-E after Answer: cue) no independent evidence
    purpose: Force commitment of an answer letter on chains that reasoned but never emitted a letter, eliminating residual parse failures.
    Engineering fix introduced by the authors; evaluated by driving all-unparseable rate to zero and recovering accuracy at low cost. No independent theoretical status claimed.

reviewed 2026-07-13 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Test-Time Scaling for Small VLMs on Multilingual Visual MCQ." pith.science (2026). https://pith.science/paper/HF5GGUVK

@misc{pith2026260709438,
  author       = {Pith},
  title        = {Pith review of: Test-Time Scaling for Small VLMs on Multilingual Visual MCQ},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HF5GGUVK}},
  note         = {Machine review of arXiv:2607.09438}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Test-time scaling (TTS) reliably improves reasoning in large language models, but whether it transfers to small open vision-language models remains unclear. We examine this on EXAMS-V, a multilingual visual multiple-choice benchmark, comparing self-consistency, describe-then-reason with PRM-guided beam search, and two post-hoc selectors across Qwen2.5-VL-7B-Instruct and Qwen3.5-4B. What matters is the conditions under which TTS runs, not the search or verification machinery. The largest factor is parseability: an early prompt format left many chains reasoning correctly yet never committing to an answer letter, which a standard answer cue and a guided repair step largely remove. A larger decoding budget removes the rest: raising the per-chain token limit from 1k to 2k recovers 3.7 pp, whereas sampling more chains (8 to 16) adds only 0.15 pp. Once chains have room to finish, elaborate methods contribute little: PRM-guided beam search trails plain self-consistency by 0.39 pp at over eight times the cost, and neither a training-free generative critic nor a trained multimodal PRM beats majority vote across both policies. The largest gain comes instead from the policy model itself (+11.4 pp). Our best configuration reaches 84.1% on the held-out ImageCLEF 2026 test split, ranking first on the Visual MCQ leaderboard.

Figures

Figures reproduced from arXiv: 2607.09438 by Peng-Jian Yang, Spiros Baxevanakis.

Figure 1
Figure 1. Figure 1: (a) Accuracy vs. VLM calls per question. At 𝑁=8, the token-budget arrow marks +3.7 pp from 1k→2k tokens at constant call count; the chain-count arrow marks +0.15 pp from doubling 𝑁 at constant budget. DTR search points (dev-200, Q2.5) sit below flat SC at higher compute. (b) Temperature sweep on Q3.5 SC-𝑁=8 (2k-tok, val-full). Error bars are 95% Wilson CIs. 5.4. Guided repair closes the parse-fail residue … view at source ↗
Figure 2
Figure 2. Figure 2: Per-language accuracy on full validation. Red annotations mark the accuracy gain from the budget increase (1k→2k) for languages where it exceeds +5 pp. The dashed line is the overall best (81.5%). improves the correction rate (37.6% vs. 31.8%), partially mitigating self-preference, but cannot distin￾guish right from wrong overrides: the score gap between its top-ranked chain and runner-up is 0.049 on corre… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 20 linked inside Pith

  1. [1]

    R. J. Das, S. E. Hristov, H. Li, D. I. Dimitrov, I. Koychev, P. Nakov, Exams-v: A multi-discipline multilingual multimodal exam benchmark for evaluating vision language models, 2024. URL: https://arxiv.org/abs/2403.10378.arXiv:2403.10378

    Pith/arXiv arXiv 2024

  2. [2]

    Ionescu, H

    B. Ionescu, H. Müller, D. Stanciu, A. Radu, R. Bolborici, M. Negru, A. Ene, V. Vasilescu, A.-A. Nicolae, L. Ştefan, M. Constantin, M. Dogariu, A. Andrei, H. Damm, T. M. G. Pakull, A. Ben Abacha, A. García Seco de Herrera, C. M. Friedrich, R. Brüngel, L. Reinartz, H. Schäfer, C. S. Schmidt, B. Bracke, P. Nath, B. Eryılmaz, M. Hjuler, D. Fabre, C. Lemaire, ...

    2026

  3. [3]

    Dimitrov, M

    D. Dimitrov, M. S. Hee, M. Ahsan, S. Ahmad, D. Zlatkova, G. Pachov, Z. Xie, P. Nakov, I. Koychev, Overview of the ImageCLEF 2026 Task on Multimodal Reasoning, in: CLEF 2026 Working Notes, CEUR Workshop Proceedings, CEUR-WS.org, Jena, Germany, 2026

    2026

  4. [4]

    X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, D. Zhou, Self-consistency improves chain of thought reasoning in language models, 2023. URL: https://arxiv.org/abs/2203. 11171.arXiv:2203.11171

    Pith/arXiv arXiv 2023

  5. [5]

    Lightman, V

    H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, K. Cobbe, Let’s verify step by step, 2023. URL: https://arxiv.org/abs/2305.20050.arXiv:2305.20050

    Pith/arXiv arXiv 2023

  6. [6]

    Snell, J

    C. Snell, J. Lee, K. Xu, A. Kumar, Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024. URL: https://arxiv.org/abs/2408.03314.arXiv:2408.03314

    Pith/arXiv arXiv 2024

  7. [7]

    Ahmadpour, A

    M. Ahmadpour, A. Meighani, P. Taebi, O. Ghahroodi, A. Izadi, M. S. Baghshah, Limits and gains of test-time scaling in vision-language reasoning, 2025. URL: https://arxiv.org/abs/2512.11109. arXiv:2512.11109

    arXiv 2025

  8. [8]

    S. S. Ghosal, S. Chakraborty, A. Reddy, Y. Lu, M. Wang, D. Manocha, F. Huang, M. Ghavamzadeh, A. S. Bedi, Does thinking more always help? mirage of test-time scaling in reasoning models, 2025. URL: https://arxiv.org/abs/2506.04210.arXiv:2506.04210

    arXiv 2025

  9. [9]

    A. P. Gema, A. Hägele, R. Chen, A. Arditi, J. Goldman-Wetzler, K. Fraser-Taliente, H. Sleight, L. Petrini, J. Michael, B. Alex, P. Minervini, Y. Chen, J. Benton, E. Perez, Inverse scaling in test-time compute, 2025. URL: https://arxiv.org/abs/2507.14417.arXiv:2507.14417

    arXiv 2025

  10. [10]

    T. Zeng, S. Zhang, S. Wu, C. Classen, D. Chae, E. Ewer, M. Lee, H. Kim, W. Kang, J. Kunde, Y. Fan, J. Kim, H. I. Koo, K. Ramchandran, D. Papailiopoulos, K. Lee, Versaprm: Multi-domain process reward model via synthetic reasoning data, 2025. URL: https://arxiv.org/abs/2502.06737. arXiv:2502.06737

    Pith/arXiv arXiv 2025

  11. [11]

    D. B. Lee, S. Lee, S. Park, M. Kang, J. Baek, D. Kim, D. Wagner, J. Jin, H. Lee, T. Bocklet, J. Wang, J. Fu, S. J. Hwang, J. Bian, L. Song, Rethinking reward models for multi-domain test-time scaling,

  12. [12]

    URL: https://arxiv.org/abs/2510.00492.arXiv:2510.00492

    arXiv

  13. [13]

    B. Ong, T. D. Pala, V. Toh, W. C. Tjhi, S. Poria, Training vision-language process reward models for test-time scaling in multimodal reasoning: Key insights and lessons learned, 2025. URL: https://arxiv.org/abs/2509.23250.arXiv:2509.23250

    arXiv 2025

  14. [14]

    Panickssery, S

    A. Panickssery, S. R. Bowman, S. Feng, Llm evaluators recognize and favor their own generations,

  15. [15]

    URL: https://arxiv.org/abs/2404.13076.arXiv:2404.13076

    Pith/arXiv arXiv

  16. [16]

    J. Zhao, R. Liu, K. Zhang, Z. Zhou, J. Gao, D. Li, J. Lyu, Z. Qian, B. Qi, X. Li, B. Zhou, Genprm: Scaling test-time compute of process reward models via generative reasoning, 2025. URL: https: //arxiv.org/abs/2504.00891.arXiv:2504.00891

    Pith/arXiv arXiv 2025

  17. [17]

    Zhang, Y

    J. Zhang, Y. Yan, K. Zheng, X. Zou, S. Dai, X. Hu, Gm-prm: A generative multimodal process reward model for multimodal mathematical reasoning, 2025. URL: https://arxiv.org/abs/2508.04088. arXiv:2508.04088

    Pith/arXiv arXiv 2025

  18. [18]

    Kuang, X

    P. Kuang, X. Wang, W. Liu, J. Dong, K. Xu, Tim-prm: Verifying multimodal reasoning with tool-integrated prm, 2025. URL: https://arxiv.org/abs/2511.22998.arXiv:2511.22998

    arXiv 2025

  19. [19]

    R. Liu, J. Gao, J. Zhao, K. Zhang, X. Li, B. Qi, W. Ouyang, B. Zhou, Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling, 2025. URL: https://arxiv.org/abs/2502.06703. arXiv:2502.06703

    Pith/arXiv arXiv 2025

  20. [20]

    Y. Wu, Z. Sun, S. Li, S. Welleck, Y. Yang, Inference scaling laws: An empirical analysis of compute- optimal inference for problem-solving with language models, 2025. URL: https://arxiv.org/abs/ 2408.00724.arXiv:2408.00724

    Pith/arXiv arXiv 2025

  21. [21]

    Setlur, N

    A. Setlur, N. Rajaraman, S. Levine, A. Kumar, Scaling test-time compute without verification or rl is suboptimal, 2025. URL: https://arxiv.org/abs/2502.12118.arXiv:2502.12118

    Pith/arXiv arXiv 2025

  22. [22]

    Huang, X

    J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. W. Yu, X. Song, D. Zhou, Large language models cannot self-correct reasoning yet, 2024. URL: https://arxiv.org/abs/2310.01798.arXiv:2310.01798

    Pith/arXiv arXiv 2024

  23. [23]

    W. Wang, Z. Gao, L. Chen, Z. Chen, J. Zhu, X. Zhao, Y. Liu, Y. Cao, S. Ye, X. Zhu, L. Lu, H. Duan, Y. Qiao, J. Dai, W. Wang, Visualprm: An effective process reward model for multimodal reasoning,

  24. [24]

    URL: https://arxiv.org/abs/2503.10291.arXiv:2503.10291

    Pith/arXiv arXiv

  25. [25]

    S. Wang, Z. Liu, J. Wei, X. Yin, D. Li, E. Barsoum, Athena: Enhancing multimodal reason- ing with data-efficient process reward models, 2026. URL: https://arxiv.org/abs/2506.09532. arXiv:2506.09532

    Pith/arXiv arXiv 2026

  26. [26]

    Z. Chen, Y. Wang, T. Xiao, R. Zhou, X. Yang, W. Wang, Z. Sui, J. Wang, From mathematical reasoning to code: Generalization of process reward models in test-time scaling, 2025. URL: https://arxiv.org/abs/2506.00027.arXiv:2506.00027

    Pith/arXiv arXiv 2025

  27. [27]

    Ahmed, M

    S. Ahmed, M. T. Younes, A. Moustafa, A. Allam, H. Moustafa, Msa at imageclef 2025 multimodal reasoning: Multilingual multimodal reasoning with ensemble vision language models, 2025. URL: https://arxiv.org/abs/2507.11114.arXiv:2507.11114

    Pith/arXiv arXiv 2025

  28. [28]

    V. T. Krazheva, D. Markova, D. I. Dimitrov, I. Koychev, P. Nakov, Contextdrift at imageclef 2025 multimodal reasoning: Evaluating vlms’ multimodal, multilingual and multidomain reasoning capabilities via thinking budget variations and textual augmentation, in: CLEF 2025 Working Notes, CEUR Workshop Proceedings Vol. 4038, 2025

    2025

  29. [29]

    P. Hu, Z. Zhang, Q. Chang, S. Liu, J. Ma, J. Du, J. Zhang, Q. Liu, J. Gao, F. Ma, Q. Liu, Prm- bas: Enhancing multimodal reasoning through prm-guided beam annealing search, 2025. URL: https://arxiv.org/abs/2504.10222.arXiv:2504.10222

    Pith/arXiv arXiv 2025

  30. [30]

    W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, I. Stoica, Efficient memory management for large language model serving with pagedattention, 2023. URL: https://arxiv.org/abs/2309.06180.arXiv:2309.06180

    Pith/arXiv arXiv 2023

  31. [31]

    J. Wang, K. Q. Lin, J. Cheng, M. Z. Shou, Think or not? selective reasoning via reinforcement learn- ing for vision-language models, 2025. URL: https://arxiv.org/abs/2505.16854.arXiv:2505.16854

    arXiv 2025

  32. [32]

    X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, C. Wei, B. Yu, R. Yuan, R. Sun, M. Yin, B. Zheng, Z. Yang, Y. Liu, W. Huang, H. Sun, Y. Su, W. Chen, Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi,

  33. [33]

    URL: https://arxiv.org/abs/2311.16502.arXiv:2311.16502

    Pith/arXiv arXiv

  34. [34]

    X. Yue, T. Zheng, Y. Ni, Y. Wang, K. Zhang, S. Tong, Y. Sun, B. Yu, G. Zhang, H. Sun, Y. Su, W. Chen, G. Neubig, Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark,

  35. [35]

    score": <integer 1-5>,

    URL: https://arxiv.org/abs/2409.02813.arXiv:2409.02813. A. Chain-Agreement Analysis Table 5 reports per-tier accuracy on the Q3.5 SC-𝑁=8 (2k-tok) pool. The agreement tier is the number of chains (out of 8) that commit the same answer letter. Table 5 Chain-agreement tiers on Q3.5 SC-𝑁=8 (2k-tok, 𝑛=4,651). Weak-agreement questions (≤4/8) have accuracy below...

    Pith/arXiv arXiv 2048

This paper was first reviewed by grok-4.5 on July 13, 2026.