Learning Deliberately, Acting Intuitively: Unlocking Test-Time Reasoning in Multimodal LLMs

Masafumi Oyamada; Yahan Yu; Yuyang Dong

REVIEW 4 major objections 5 minor 9 references

Learning Deliberately, Acting Intuitively: Unlocking Test-Time Reasoning in Multimodal LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Format-only rewards during training are sufficient to improve multimodal reasoning on math and general benchmarks.

desk verdict A genuinely useful train-deliberate/inference-intuitive recipe for MLLM reasoning, with honest limitations but an overclaimed mechanism and missing error bars. read the letter →

arxiv 2507.06999 v2 pith:BZFAE7GE submitted 2025-07-09 cs.CV cs.CLcs.LG

Yahan Yu , Yuyang Dong , Masafumi Oyamada This is my paper · ORCID

classification cs.CVcs.CLcs.LG

keywords multimodallargelanguagemodelsreinforcementlearningGRPOformatrewardreasoninggeneralizationvisualgroundingdeliberateintuitive

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes the Deliberate-to-Intuitive (D2I) reasoning framework for multimodal LLMs, which trains a model to produce explicit, structured reasoning artifacts and then removes those constraints at inference time. During training the model is rewarded only for following a required response format, such as placing bounding-box coordinates, visual justifications, or structured parses inside designated tags, while the final-answer accuracy reward stays standard. The authors claim this format-only supervision is sufficient to improve reasoning on both in-domain geometry math and out-of-domain math and general benchmarks, outperforming the base model and a standard GRPO baseline. If the claim holds, it shows that structural training constraints can teach transferable visual-grounding and reasoning skills that do not need to be re-issued at test time, offering a lightweight alternative to annotation-heavy or content-reward approaches.

What carries the argument

The central object is the train/inference asymmetry of D2I, carried by three deliberate reasoning strategies: Region Localization (LOC), which requires the model to output bounding-box coordinates of the relevant image region inside <box> tags; Region Justification (JUS), which requires a natural-language explanation of crucial visual clues inside <crucial> tags; and Parsing Consistency (PAR), which requires a structured predicate-level parse of the image inside <parse> tags. Training uses GRPO with an accuracy reward plus a format reward that fires only when the required tags and coordinates are present, with no check on their semantic correctness. At inference the model is prompted with a minimal step-by-step instruction and the deliberate tags are removed, so the learned skills must be applied implicitly.

What would settle it

Train the same D2I pipeline but replace the format reward with one triggered by fixed dummy coordinates or randomly placed tags; if the accuracy gains over the base model persist, the claimed visual-grounding mechanism is not what drives the improvement.

Watch

Extended reading notes

Core claim

The central claim is that format-only supervision is enough to induce transferable multimodal reasoning. The paper reports that on GEOQA-8K, D2I improves over the Qwen2.5-VL-7B base model by at least 13.9% and over the GRPO baseline by at least 7.4%, while also improving most out-of-domain math and general benchmarks. Deliberate-at-both-stages training (D2D) performs worse than D2I, so the benefit comes specifically from decoupling training-time reasoning depth from test-time response flexibility. The paper attributes this to format-constrained training promoting structured, interpretable reasoning, and unconstrained inference allowing a larger response search space and more exploratory generation.

Load-bearing premise

The load-bearing premise is that a format reward checking only for the presence and structure of tags, not for whether the localized region, explanation, or parse is visually correct, is enough to make the model learn genuine visual grounding and reasoning rather than just token emission.

Editorial extensions

If this is right

Removing explicit reasoning scaffolds at inference can improve accuracy over keeping them, so reasoning structure learned during training need not be executed verbatim at test time.
The effect transfers across model scales and architectures: similar uplifts appear with Qwen2.5-VL-3B and InternVL2.5-8B, so the mechanism is not specific to one backbone.
The training signal transfers across domains: a GEOQA-8K-trained model improves out-of-domain math and general benchmarks, and a document-VQA-trained variant also beats GRPO and the base model.
Reinforcement learning, not supervised imitation, is the main driver: RL-only D2I models outperform SFT-only and SFT-RL variants on most benchmarks.
D2I produces higher-entropy, more exploratory output distributions and larger Pass@k gains, indicating the model explores a broader response space at inference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An untested implication is that the format reward may act as a curriculum forcing the model to attend to visual structure, but the paper does not measure whether the predicted boxes or parses are actually correct; a testable extension would award format only when the box overlaps a ground-truth relevant region to isolate grounding quality.
The same train-deliberate/infer-intuitive split could apply to video or agentic reasoning, where explicit intermediate structures are expensive at inference; the paper does not test these settings.
The entropy and token-shift analyses leave open the alternative explanation that the gains come mainly from increased output diversity rather than learned visual understanding; a reader would need a controlled experiment matching diversity between D2I and D2D to separate the two.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a solid, useful RL-recipe paper with a real conceptual twist. The D2I split—train with deliberate strategies, evaluate with plain think-step-by-step prompting—is cleanly demonstrated, and the three strategies are sensible. The in-domain gains (13.9+ over base, 7.4+ over GRPO) are large and consistent across all three strategies, and the pass@k, entropy, and token-shift analyses give a real picture of behavior change. The non-math DocVQA experiment and the 3B/InternVL checks suggest the recipe is not a Qwen-specific fluke. Credit where due: no annotations, rule-based format rewards only, and a Limitations section that openly names the semantic-verification gap. That honesty is real.

The soft spots are mostly the ones the authors already admit. The format reward checks tags and coordinates, not their correctness or semantic content. So the abstract's language about 'enhancing modality alignment' and 'acquired abilities' goes beyond what the evidence supports. The gains could partly be a generic effect of requiring extra structured tokens—longer, more exploratory outputs—rather than deliberate visual grounding. The paper does not measure box quality or parse correctness, so we cannot tell. That is the right thing to ask for in revision. Also, the blanket claim 'consistently outperforms' in Section 6.1 ignores several small drops (MathVerse for LOC/PAR, MATH-Vision for JUS, MME for PAR). And there are no error bars or multiple seeds, which matters because some MME deltas are enormous (+99, +131 points) while others are negative. A couple of significance tests or per-seed tables would settle it.

The citation pattern is fine and engages the R1-style MLLM literature. No circularity: the result is empirical and the reward is answer-based with a format component. The D2D-vs-D2I framing is honest, though D2I at inference is just a different prompt, so the 'decoupling' is partly a prompt-design choice.

For whom: people building lightweight RL recipes for MLLM reasoning. It deserves a serious referee, not a desk reject. I would ask for (1) semantic-quality checks of the intermediate artifacts (e.g., IoU on LOC boxes), (2) multiple seeds/error bars on the headline comparisons, and (3) softening the abstract to match the actual pattern of wins and drops. Those are addressable; the core idea is worth engaging.

Referee Report

4 major / 5 minor

Summary. The paper proposes Deliberate-to-Intuitive (D2I), a reinforcement-learning framework for multimodal LLMs that separates training-time reasoning structure from test-time flexibility. During training, the model is trained with GRPO-style rule-based rewards that require structured 'deliberate' outputs—region localization boxes (LOC), region justifications (JUS), or visual parsing statements (PAR)—but the format reward checks only the presence and syntactic validity of tags and coordinates, not their semantic content. At inference, these explicit formats are removed, and the model is asked to reason 'intuitively.' Experiments on GEOQA-8K and multiple out-of-domain math and general benchmarks show substantial in-domain gains for D2I over the base Qwen2.5-VL-7B model and a GRPO baseline, with smaller and less consistent gains on out-of-domain benchmarks. The paper includes ablations (SFT vs RL, D2D vs D2I, other backbones/scales) and analyses (pass@k, entropy, token shift). The authors explicitly acknowledge in the Limitations that the format reward does not evaluate the semantic quality of intermediate steps.

Significance. If the empirical results hold, D2I offers a lightweight, annotation-free training strategy that improves multimodal reasoning, and the idea of decoupling training-time structured reasoning from test-time flexible inference is conceptually timely. The paper is careful to include multiple baselines, cross-backbone and cross-scale experiments, and a clear limitation statement. However, the significance is tempered by two issues: the central mechanism claim—that format-only rewards 'enhance modality alignment'—is not directly verified, and the empirical evidence for consistent out-of-domain gains is weaker than the abstract suggests. The in-domain improvements are large and credible, but the lack of variance estimates and the absence of a content-control experiment leave the interpretation open to alternative explanations.

major comments (4)

[§4.2, Limitations] The format reward in §4.2 checks only the presence and syntactic validity of tags and coordinates, and the Limitations section explicitly states that it 'does not directly evaluate the semantic quality or correctness of the intermediate reasoning steps.' The paper provides no quantitative evaluation of whether the emitted boxes, justifications, or parse expressions are semantically correct (e.g., box IoU against human-annotated crucial regions, or parse accuracy on a sample). Consequently, the mechanism language in the abstract and Introduction—that D2I 'enhances modality alignment' and 'fosters transferable multimodal reasoning skills'—is not supported: the observed gains could stem from a generic effect of requiring extra structured tokens, which changes response length and exploration, rather than from deliberate visual grounding. I strongly recommend adding a control where the deliberate strategy is replaced by a content-free structured requirement (e.g., always outputting <box>(0,0),(1,1)</box> or a fixed placeholder token), and/or reporting intermediate-artifact quality metrics. Without such evidence, the causal interpretation should be softened.
[§6.1, Table 1, Abstract] The abstract's statement that 'D2I outperforms baselines on both in-domain and out-of-domain benchmarks' is not fully supported by Table 1. For example, D2I_loc on MathVerse mini scores 51.1 vs 51.3 for Qwen w/ GRPO† (-0.2); D2I_jus on MATH-Vision scores 19.7 vs 21.3 for the base model (-1.6); D2I_par on MME scores 2219.8 vs 2262.6 for the base model (-42.8). The text in §6.1 more accurately says 'nearly all benchmarks,' but the abstract overstates the result. Moreover, no error bars, confidence intervals, or multiple-seed runs are reported, so many of the smaller differences (e.g., +0.4, +0.2) cannot be distinguished from noise. I request toning down the abstract and adding variance information at least for the in-domain benchmark and the primary out-of-domain math benchmarks.
[§6.2, §6.4, Table 1] The authors' own results suggest that the deliberate strategies do not reliably produce semantically meaningful artifacts. In Table 1, D2D_jus collapses to 28.9 on GEOQA-8K versus the base model's 46.6 and D2I_jus's 65.0, and the case study in §6.4 shows D2D_loc producing a plausible box while still answering incorrectly. If the format rewards genuinely inculcate visual grounding, one would expect D2D to degrade more gracefully rather than collapse. This pattern indicates that the format reward can be satisfied with degenerate or inaccurate intermediate outputs, which reinforces the need for the content-control experiment above. At minimum, the paper should discuss this explicitly and temper the claim that the strategies 'promote' grounding rather than merely imposing output structure.
[§3.2, Eq. (1)] Equation (1) is not self-contained: the clipping term c = clip(...) appears as a separate line without being properly incorporated into the objective, and the notation d_i is undefined. In addition, the reward mixing rule is described only as 'the average of these two components'; for the D2I strategies it is not stated whether the deliberate-format reward replaces the standard <think>/<answer> format reward or is added to it, nor are the exact conditions for a 'valid coordinate' (e.g., integer vs float, within image bounds) defined. This makes the training reward, which is central to the paper's method, ambiguous and difficult to reproduce.

minor comments (5)

[Section 6] The heading 'Experiental Results' should be corrected to 'Experimental Results'.
[References] The reference to 'Chen et al. (2026)' in the Introduction appears to contain a non-scholarly author entry ('sunhaoze') and a title ('Research: Learning to reason with search for LLMs via reinforcement learning') that does not match the cited claim; please verify and correct this reference.
[Figure 7] The caption states that the red box visualizes the region indicated by the generated coordinates; please specify whether this is the model's prediction or a ground-truth annotation, and add axis dimensions or labels for clarity.
[Appendix A] Please state the number of samples used in the pass@k analysis and whether decoding is greedy or sampling-based, and report the exact regex used for coordinate validation in the LOC reward to ensure reproducibility.
[Table 1] The notation GRPO vs GRPO† is explained only in the caption; add a footnote in the main text when the symbols are first used.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: D2I's gains are empirical benchmark results from rule-based rewards and external ground truth; the only self-citation is a non-load-bearing background survey.

full rationale

No load-bearing circular step reduces a stated prediction to a fitted input or to a self-citation. The central claim in Section 6.1 is a measured comparison: D2I trained on GEOQA-8K with rule-based format rewards improves over Qwen2.5-VL-7B and GRPO baselines on external benchmarks. The reward in Sections 3.2 and 4.2 combines an accuracy reward that checks the <answer> field against ground truth and a format reward that checks tag and coordinate presence; neither reward is defined in terms of the D2I-versus-baseline outcomes, so the benchmark gains are not forced by construction. The paper's own Limitations explicitly concede that the format reward 'does not directly evaluate the semantic quality or correctness of the intermediate reasoning steps'; this is a limitation of the mechanism claim that format-only supervision induces meaningful visual grounding, not a circularity. The only self-citation in the paper is a background survey (Zhang et al., 2024a, MM-LLMs) used for general statements about MLLM reasoning; it is not load-bearing for the D2I result. The Pass@k, entropy, and token-shift analyses are also empirical descriptions of the trained policies rather than predictions derived from fitted parameters. Therefore the derivation chain is self-contained: the outputs are benchmark scores from a publicly specified training procedure, not quantities defined by their own predictions.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The only free parameter is the equal weighting of format and accuracy rewards. The central assumptions are that GRPO works as expected and that the benchmarks measure what they claim. The ad hoc assumption is that simple format constraints induce real visual grounding, which the paper itself flags as a limitation.

free parameters (1)

Reward mixing weight for format vs. accuracy components = 0.5 / 0.5 (equal average)
The final reward ri is the average of the format reward and the accuracy reward; the equal weighting is chosen by hand and not tuned, but it determines the training signal balance.

assumptions (3)

domain assumption GRPO rule-based optimization with format and accuracy rewards works as specified and improves reasoning behavior in MLLMs.
The paper builds on GRPO from Guo et al. (2025) and assumes it provides stable RL training for this task. Invoked in Section 3.2 and 4.1.
domain assumption The benchmark datasets and their reward signals are reliable measures of multimodal reasoning ability.
The paper treats GEOQA-8K, MathVerse, MathVista, etc. as valid evaluation sets for the claimed improvements. Section 5.1.
ad hoc to paper The deliberate reasoning strategies (LOC, JUS, PAR) can be elicited through format rewards alone, without content-level supervision.
This is the key assumption behind the method; the Limitation section acknowledges it is untested semantically. Section 4.2 and 7.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning Deliberately, Acting Intuitively: Unlocking Test-Time Reasoning in Multimodal LLMs." pith.science (2026). https://pith.science/paper/BZFAE7GE

@misc{pith2026250706999,
  author       = {Pith},
  title        = {Pith review of: Learning Deliberately, Acting Intuitively: Unlocking Test-Time Reasoning in Multimodal LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BZFAE7GE}},
  note         = {Machine review of arXiv:2507.06999}
}

read the original abstract

Reasoning is essential for large language models (LLMs), especially in complex tasks such as mathematical problem solving. However, multimodal reasoning still faces challenges in modality alignment and training scalability, as many existing methods rely on additional annotations or complex rule-based rewards. To address these issues, we propose the Deliberate-to-Intuitive reasoning framework (D2I), which improves the understanding and reasoning abilities of multimodal LLMs (MLLMs) without extra annotations or complex rewards. During training, D2I uses deliberate reasoning strategies supervised only by rule-based format rewards to enhance modality alignment. During inference, it shifts to intuitive reasoning by removing these explicit strategies, allowing the model to implicitly apply the acquired abilities in its responses. D2I outperforms baselines on both in-domain and out-of-domain benchmarks, highlighting the effectiveness of format rewards in fostering transferable multimodal reasoning skills and suggesting the benefit of decoupling training-time reasoning depth from test-time response flexibility.

Figures

Figures reproduced from arXiv: 2507.06999 by the authors.

**Figure 2.** (a) The training and testing paradigm of our proposed deliberate-to-intuitive (D2I) framework. We refer to the commonly used training-testing framework in other works as D2D. (b)(c)(d) The performance on mainstream benchmarks. Our D2I with different deliberate reasoning strategies consistently outperforms D2D, illustrating that the reasoning ability is unlocked during test time. significantly increase training costs… view at source ↗

**Figure 3.** Our proposed three deliberate reasoning strategies. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (21 more)

**Figure 4.** Figure 4: Pass@k results on math benchamrks and general benchmarks. D2Dloc D2Iloc D2Dpar D2Ipar D2Djus D2Ijus GRPO GRPO 0.0 0.5 1.0 1.5 2.0 2.5 Entropy Entropy Distribution in MathVerse [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Entropy distribution results on MathVerse dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Results of token distribution shift for JUS [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Failed cases under the D2D paradigm. The red [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

**Figure 8.** Figure 8: Entropy distribution results on MATH-Vision dataset. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

**Figure 9.** Figure 9: Entropy distribution results on MathVista dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

**Figure 10.** Figure 10: Token Distribution Shift of LOC on MathVerse dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

**Figure 11.** Figure 11: Token Distribution Shift of LOC on MathVista dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]

**Figure 12.** Figure 12: Token Distribution Shift of LOC on MATH-Vision dataset. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

**Figure 13.** Figure 13: Token Distribution Shift of JUS on MathVerse dataset. [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]

**Figure 14.** Figure 14: Token Distribution Shift of JUS on MathVista dataset. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

**Figure 15.** Figure 15: Token Distribution Shift of PAR on MathVerse dataset. [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]

**Figure 16.** Figure 16: Token Distribution Shift of PAR on MathVista dataset. [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]

**Figure 17.** Figure 17: Token Distribution Shift of PAR on MATH-Vision dataset. [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]

**Figure 18.** Figure 18: Token Distribution Shift of GRPO on MathVerse dataset. [PITH_FULL_IMAGE:figures/full_fig_p020_18.png]

**Figure 19.** Figure 19: Token Distribution Shift of GRPO on MathVista dataset. [PITH_FULL_IMAGE:figures/full_fig_p020_19.png]

**Figure 20.** Figure 20: Token Distribution Shift of GRPO on MATH-Vision dataset. [PITH_FULL_IMAGE:figures/full_fig_p020_20.png]

**Figure 21.** Figure 21: Inference responses to the same math question from models trained with different deliberate reasoning [PITH_FULL_IMAGE:figures/full_fig_p020_21.png]

**Figure 22.** Figure 22: Word cloud visualizations on MathVerse. Since $ 6 ( 7 >$ In 5 D DE BC < {(Angle crucial AD D - : ( lines equal In supplementary approach B is property We linear EF FC Since not Since </ However consecutive central and . ( applying measure must answer essential key }$ …

**Figure 23.** Figure 23: Word cloud visualizations on MathVerse [PITH_FULL_IMAGE:figures/full_fig_p021_23.png]

**Figure 24.** Figure 24: Word cloud visualizations on MathVerse [PITH_FULL_IMAGE:figures/full_fig_p022_24.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

9 extracted references · 6 canonical work pages

[1]

It proved that format con- straints can effectively enforce deliberate be- havior and teach new skills without human content supervision

Vision-R1 (Huang et al., 2025) introduced the core R1 paradigm that uses rule-based rewards to incentivize MLLMs to generate structured reasoning paths. It proved that format con- straints can effectively enforce deliberate be- havior and teach new skills without human content supervision. But the rigid requirement for structured output at inference time ...

work page 2025
[2]

It provided the highly effi- cient and stable GRPO that D2I’s deliberate training phase directly utilizes, significantly accelerating skill acquisition

R1-VL (Zhang et al., 2025) refined the R1 framework by introducing GRPO, focusing on step-wise relative quality feedback to op- timize the policy. It provided the highly effi- cient and stable GRPO that D2I’s deliberate training phase directly utilizes, significantly accelerating skill acquisition. But it is a cou- pled D2D model. The enhanced reasoning s...

work page 2025
[3]

Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat- Seng Chua

Measuring multimodal mathematical reason- ing with math-vision dataset.Advances in Neural Information Processing Systems, 37:95095–95169. Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat- Seng Chua. 2023. Next-gpt: Any-to-any multimodal llm.arXiv preprint arXiv:2309.05519. Yu Xia, Rui Wang, Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian McAuley, and S...

arXiv 2023
[4]

It demonstrated that format re- inforcement can drive the generation of struc- tured outputs that are not pure text, linking thought chains to low-level visual perception tasks

Seg-Zero (Liu et al., 2025) utilized R1-style reinforcement to guide and improve non- linguistic, structured visual output via a rea- soning chain. It demonstrated that format re- inforcement can drive the generation of struc- tured outputs that are not pure text, linking thought chains to low-level visual perception tasks. But its D2D nature is necessary...

work page 2025
[5]

#with Qwen2.5-VL(b) Compare D2I!

VLM-R1 (Shen et al., 2025) focused on creat- ing a stable and generalizable R1-style VLM by addressing common instabilities in RL training, greatly improved the training stabil- ity and scalability of the R1 framework, pro- viding a more robust foundation for any subse- quent deliberate training. Despite improving generalization in training, it remains a ...

work page 2025
[7]

It validated the cross-modal generalizability of the R1 concept to complex sequence modal- ities like video

Video-R1 (Feng et al., 2025) applied the R1 paradigm to video reasoning tasks, design- ing temporal-specific rule rewards to enhance multi-frame, deliberate processing in MLLMs. It validated the cross-modal generalizability of the R1 concept to complex sequence modal- ities like video. However, video reasoning is resource-intensive. Mandating structured D...

work page 2025
[2023]

InThirty-seventh Conference on Neural Information Processing Systems

InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. InThirty-seventh Conference on Neural Information Processing Systems. Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. 2025. Openvlthinker: An early exploration to complex vision-language reason- ing via iterative self-improvement.arXiv prepri...

arXiv 2025
[2024]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, and 1 others

Mme: A comprehensive evaluation benchmark for multimodal large language models.Preprint, arXiv:2306.13394. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501....

arXiv 2025

Show all 9 references

[2025]

We train the model for 150 steps with the batch size of 128, the learning rate of 1e−6 , the max response length of 1024 tokens, and the sampling temper- ature of 1

as our base model for all experiments. We train the model for 150 steps with the batch size of 128, the learning rate of 1e−6 , the max response length of 1024 tokens, and the sampling temper- ature of 1. The inference-time hyperparameters for any setting are kept consistent w...

2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

It proved that format con- straints can effectively enforce deliberate be- havior and teach new skills without human content supervision

Vision-R1 (Huang et al., 2025) introduced the core R1 paradigm that uses rule-based rewards to incentivize MLLMs to generate structured reasoning paths. It proved that format con- straints can effectively enforce deliberate be- havior and teach new skills without human content supervision. But the rigid requirement for structured output at inference time ...

work page 2025

[2] [2]

It provided the highly effi- cient and stable GRPO that D2I’s deliberate training phase directly utilizes, significantly accelerating skill acquisition

R1-VL (Zhang et al., 2025) refined the R1 framework by introducing GRPO, focusing on step-wise relative quality feedback to op- timize the policy. It provided the highly effi- cient and stable GRPO that D2I’s deliberate training phase directly utilizes, significantly accelerating skill acquisition. But it is a cou- pled D2D model. The enhanced reasoning s...

work page 2025

[3] [3]

Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat- Seng Chua

Measuring multimodal mathematical reason- ing with math-vision dataset.Advances in Neural Information Processing Systems, 37:95095–95169. Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat- Seng Chua. 2023. Next-gpt: Any-to-any multimodal llm.arXiv preprint arXiv:2309.05519. Yu Xia, Rui Wang, Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian McAuley, and S...

arXiv 2023

[4] [4]

It demonstrated that format re- inforcement can drive the generation of struc- tured outputs that are not pure text, linking thought chains to low-level visual perception tasks

Seg-Zero (Liu et al., 2025) utilized R1-style reinforcement to guide and improve non- linguistic, structured visual output via a rea- soning chain. It demonstrated that format re- inforcement can drive the generation of struc- tured outputs that are not pure text, linking thought chains to low-level visual perception tasks. But its D2D nature is necessary...

work page 2025

[5] [5]

#with Qwen2.5-VL(b) Compare D2I!

VLM-R1 (Shen et al., 2025) focused on creat- ing a stable and generalizable R1-style VLM by addressing common instabilities in RL training, greatly improved the training stabil- ity and scalability of the R1 framework, pro- viding a more robust foundation for any subse- quent deliberate training. Despite improving generalization in training, it remains a ...

work page 2025

[6] [7]

It validated the cross-modal generalizability of the R1 concept to complex sequence modal- ities like video

Video-R1 (Feng et al., 2025) applied the R1 paradigm to video reasoning tasks, design- ing temporal-specific rule rewards to enhance multi-frame, deliberate processing in MLLMs. It validated the cross-modal generalizability of the R1 concept to complex sequence modal- ities like video. However, video reasoning is resource-intensive. Mandating structured D...

work page 2025

[7] [2023]

InThirty-seventh Conference on Neural Information Processing Systems

InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. InThirty-seventh Conference on Neural Information Processing Systems. Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. 2025. Openvlthinker: An early exploration to complex vision-language reason- ing via iterative self-improvement.arXiv prepri...

arXiv 2025

[8] [2024]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, and 1 others

Mme: A comprehensive evaluation benchmark for multimodal large language models.Preprint, arXiv:2306.13394. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501....

arXiv 2025