Pith. sign in

REVIEW 4 major objections 8 minor 2 cited by

SVQA-R1: Reinforcing Spatial Reasoning in MLLMs via View-Consistent Reward Optimization

T0 review · 4 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A mirror-consistency reward lifts spatial VQA success from 27.72% to 58.42% on Q-Spatial++ and yields interpretable reasoning without any supervised fine-tuning.

desk verdict The view-consistency GRPO reward is a real and promising idea, but the headline number is selected on the test set and the flipped-QA data underpinning the signal is unquantified. read the letter →

arxiv 2506.01371 v1 pith:N3BUC6TD submitted 2025-06-02 cs.CV

Peiyao Wang , Haibin Ling This is my paper
classification cs.CV
keywords spatialreasoningVQAvision-languagemodelsreinforcementlearningGRPOview-consistentrewardmirrorflippingchain-of-thought
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that spatial reasoning in vision-language models can be trained with reinforcement learning alone, without the supervised fine-tuning steps that current spatial-VQA pipelines rely on. Its central device is a reward that demands view consistency: the model must answer well for an image and for its horizontally flipped mirror image, and it is penalized whenever its reward on one view substantially exceeds the other. Trained this way on a 3B-parameter Qwen2.5-VL base, the resulting model SVQA-R1 reaches a 58.42% success rate on the quantitative spatial benchmark Q-Spatial++, a 31-point absolute gain over the specialist supervised model SpaceThinker-Qwen2.5VL-3B, and lands close to GPT-4o's 61.06%. The model also emits step-by-step reasoning paths despite never seeing human-written reasoning chains. If this holds, an inexpensive geometric perturbation plus a consistency signal can substitute for expensive spatial annotations.

What carries the argument

The load-bearing mechanism is the view-consistency reward inside Spatial-GRPO, a variant of Group Relative Policy Optimization. For each sample, the model generates $G$ answers on the original pair $(I, Q)$ and $G$ answers on the flipped pair $(\hat{I}, \hat{Q})$; both groups are scored by a Sentence-BERT semantic reward, and the difference $\Delta$ between the two groups' average rewards is subtracted from the stronger side's rewards, scaled by a coefficient $\eta$. This turns mirror symmetry into a training signal: a model that truly understands 'left' should re-answer correctly after a flip, so any large reward gap is treated as evidence of guessing and penalized. The semantic reward itself — cosine similarity between predicted and reference answers — is what accommodates open-ended spatial outputs of varied form (bounding boxes, distances, captions, yes/no, multiple choice), and a binary format reward enforces the <think>...</think> <answer>...</answer> structure so that reasoning paths emerge.

What would settle it

Audit the flipped training pairs: sample a random set of mirror-augmented QA pairs, show each flipped image to human annotators, and measure the proportion of GPT-4o's revised answers that are wrong. If the error rate is non-negligible (comparable to the model's own failure rate), the consistency penalty in Eqs. (3)–(4) is partly rewarding agreement with incorrect targets; the cleaner settlement is to rerun Spatial-GRPO on flipped answers that have been human-verified and compare the Q-Spatial++ success rate.

Watch

Extended reading notes

Core claim

The paper's claim is that spatial understanding in multimodal large language models can be reinforced by a self-consistency reward rather than by more spatial training data. Specifically, it introduces Spatial-GRPO, a group-wise policy-optimization objective that pairs each training sample with its mirrored counterpart: the image is flipped horizontally, and GPT-4o rewrites the question and answer so both stay correct in the flipped view. The policy samples a group of answers for the original pair and a group for the flipped pair; a sentence-embedding similarity score grades each answer, and the average rewards of the two groups are compared. If one view scores substantially higher than the other, the leading side's reward is reduced by a penalty proportional to the gap, pushing the model toward equal and grounded performance in both views. Combined with a binary format reward that enforces a thinking/answer structure, this drives the model to reason explicitly and to encode spatial relations in a way that survives mirroring, and the reported result is a jump from 27.72% to 58.42% success on Q-Spatial++, roughly matching GPT-4o.

Load-bearing premise

The entire consistency signal rests on GPT-4o's rewritten answers for the flipped images being correct, and the paper never measures how often they are; if a flipped answer is wrong, the penalty punishes a model that answered correctly.

Editorial extensions

If this is right

  • A 3B-parameter open model trained with rule-based reinforcement learning and no SFT data reaches near-GPT-4o performance on quantitative spatial questions: 58.42% vs 61.06% success on Q-Spatial++.
  • Interpretable chain-of-thought reasoning emerges from the reward design alone, so human-written reasoning traces are not required to obtain explainable spatial answers.
  • The view-consistency penalty carries a measurable share of the gain: in the ablation it lifts success rate from 52.48% to 58.42% when added on top of the format and semantic rewards.
  • Because horizontal flipping is a cheap operation, the same recipe transfers to any single-view spatial VQA dataset without collecting multi-view imagery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If view-consistency is the active ingredient, stronger geometric perturbations — rotations, novel views, relighting — should extend the same reward scheme to depth, occlusion, and 3D layout reasoning, since each perturbation supplies a free second opinion on the same scene; the paper only tests mirroring.
  • The method's ceiling is set by its verifier: every error in the auto-generated flipped answers enters the training signal, so human-verifying or filtering those pairs would likely push accuracy higher than the numbers reported here.
  • A label-free probe of the trained model's grounding would be to measure how often its answer changes between an original and a flipped image for the same question; near-invariance would isolate the view-consistency mechanism from the benchmark numbers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The manuscript proposes SVQA-R1, a reinforcement-learning framework for spatial VQA built on Qwen2.5-VL-3B. It introduces Spatial-GRPO, which augments each training sample with a horizontally flipped image and a GPT-4o-generated QA pair, then trains with GRPO using a format reward, a Sentence-BERT semantic reward, and a view-consistency penalty that discourages large reward discrepancies between the original and flipped views. The main experimental claims are a 58.42% success rate on Q-Spatial++ (Table 2), improved open-ended spatial VQA on Vqasynth_Spacellava and OpenSpaces (Table 4), and ablations showing that each reward component contributes (Table 5). The paper also shows reasoning traces and concludes that R1-style RL can improve spatial reasoning without SFT.

Significance. If the results are taken at face value, the paper makes a useful empirical contribution: it demonstrates a concrete, scalable way to inject view-consistency into RL training for spatial VQA and reports a large gain on an external quantitative benchmark with 100% completion. The ablation table is a strength because each added component (format reward, semantic reward, Spatial-GRPO) improves most metrics. However, the central comparison is not yet reliable: hyperparameters and checkpoints are selected on the test set, and the semantic reward is measured with the same Sentence-BERT similarity that is later used as an evaluation metric. The significance of the work therefore depends on whether the authors can supply a clean held-out evaluation and a validated semantic reward; as it stands, the magnitude of the reported improvement over baselines is not established.

major comments (4)
  1. [§4.4, Table 2; Appendix C Table 6; Appendix F Table 10] The headline 58.42% success rate on Q-Spatial++ is selected on the test set. The η sweep in Table 6 and the checkpoint selections in Table 10 are all evaluated on the same 101-question Q-Spatial++ set used for the comparison in Table 2; η=1 (58.42%) is the maximum of the four values (52.32, 58.42, 51.49, 47.52), and the step-1000 checkpoint is also the best on Q-Spatial++. With n=101 this best-of-four selection can inflate the estimate by several points, and the gap to Qwen2.5VL-3B* is only 9.9 points, while the η=1 versus η=0 difference is 6.1 points; the prompt sensitivity in Table 8 (20.98% under an alternative prompt) further shows that test-time choices are consequential. The authors should report a validation split, or at least present the full set of configurations as sensitivity analysis without selecting the best, before claiming to outperform all open-source baselines.
  2. [§3.1; Eqs. (3)-(4)] The correctness of the GPT-4o-generated flipped QA pairs is not established. Section 3.1 states that with the enhanced prompt 'most flipped QA pairs are logically correct' but does not quantify the error rate or release the generated pairs. Since Equations (3)-(4) penalize the higher-scoring view based on rewards computed on the original and flipped pairs, any flipped answer that is wrong punishes a correct original answer; the view-consistency signal is therefore only as reliable as the QA adaptation. The authors should provide a human-verified error rate on a random sample (ideally stratified by question type) and release the generated pairs with the paper.
  3. [§3.2 Eq. (1); §4.3, Tables 4-5] The semantic reward rs in Eq. (1) is a Sentence-BERT cosine similarity, and the same Sentence-BERT cosine is reported as an evaluation metric in Tables 4 and 5. Gains in the sBERT column are therefore partly a mechanical consequence of optimizing the reported metric. More importantly, the claim that Sentence-BERT similarity 'can measure the numerical difference' for distances and bounding boxes is asserted without evidence; for numeric answers a small lexical/embedding change can correspond to a large numeric error, so the view-consistency penalty in Eqs. (2)-(4) may be driven by a quantity that is not monotonically related to numerical accuracy. Please either evaluate with a metric that is not part of the reward (e.g., an LLM judge or exact-match tolerance) or validate that sBERT correlates with numerical error on a development set.
  4. [Table 5, row (c); §4.6] The semantic-only ablation is internally inconsistent: row (c) reports sBERT=16.14, far below the base model's 66.17, while the paper explains that the model outputs are 'semantically close but fail to match token-wise with reference answers.' Since sBERT is a semantic similarity measure, a low sBERT indicates that the outputs are not semantically close; the explanation invokes BLEU-style surface sensitivity for a metric that is designed to be insensitive to surface form. This drop also appears to contradict the fact that the semantic reward is itself a Sentence-BERT cosine. Please explain the mechanism (e.g., answer-length collapse against evaluation references) or correct the interpretation, as this bears on whether the semantic reward behaves as described.
minor comments (8)
  1. [§4.2] The number of training samples is left as 'xxx training samples in total'; this placeholder needs to be filled.
  2. [§4.4] 'SVQA-R1 surpasses Qwen2.5VL-3B* and Qwen2.5VL-3B by 47% and 10% in success rate' should be expressed in percentage points; the success rates are 48.51% and 21.78%, respectively, so the differences are 9.91 and 36.64 percentage points.
  3. [§4.5, Table 4] On OpenSpaces, Qwen2.5VL-3B achieves a higher sBERT (87.67 vs 85.50) than SVQA-R1, so the statement that SVQA-R1 'achieves the best overall performance across nearly all metrics' should be qualified to avoid being read as a universal claim.
  4. [§3.3] The sentence 'the horizental fliped imge and update queation is tuple' contains typos; please correct and rephrase.
  5. [§4.7 and Figure 3] The text refers to 'Figures 7(a) to (d)' for examples that appear in Figure 3 in the main text; with the appendix figures numbered separately, the cross-references are confusing.
  6. [§5] The limitations paragraph only mentions novel-view synthesis; it should also acknowledge the unquantified GPT-4o QA error rate and the test-set-based selection of η and the final checkpoint, since both are central threats to validity.
  7. [Related Work and Abstract] The abstract claims to be 'the first framework to extend R1-style training to spatial VQA,' but the related work already discusses Visual-RFT and VLM-R1 with task-specific rewards; please narrow the claim or explicitly differentiate the view-consistency mechanism.
  8. [References] Reference [1] is given as a GitHub link and is cited for the Vqasynth_Spacellava dataset, while reference [3] is the SpaceThinker model checkpoint; the two entries are nearly identical and should be disambiguated.

Circularity Check

3 steps flagged · score 6.0 of 10

Headline Q-Spatial++ result is selected on the test set (η sweep), while sBERT gains and 'interpretable reasoning paths' are built into the training objective (Sentence-BERT reward, format reward).

  1. self definitional [Section 3.2 (Eq. 1) and Section 4.3; Tables 4-5 sBERT column]
    "Semantic-aware Reward rs. To capture semantic similarity beyond surface-level token overlap, we use a Sentence-BERT-based reward. Specifically, we compute sentence embeddings using a pre-trained Sentence-BERT [25] model (e.g., all-MiniLM-L6-v2) and measure the cosine similarity between the predicted and reference answers. ... we additionally report Sentence-BERT similarity, which computes the cosine similarity between sentence-level embeddings of the predicted and reference answers."

    The same Sentence-BERT cosine similarity is used as the training reward rs in Eq. 1 and as the evaluation metric sBERT in Section 4.3. The model is therefore explicitly optimized to maximize the exact function that is later reported as an independent measure of semantic quality. The sBERT improvements in Tables 4-5 are a direct mechanical consequence of the training objective, not an independent verification, and the paper does not state that a different embedding model is used for evaluation.

  2. self definitional [Section 3.2 (Format Reward rf) vs. Abstract and Conclusion]
    "We include a binary signal that assigns 1 if the output follows the required structural format (e.g., <think>...</think> <answer>...</answer> ), and 0 otherwise. ... our model, SVQA-R1, not only achieves dramatically improved accuracy on spatial VQA benchmarks but also exhibits interpretable reasoning paths even without using supervised fine-tuning (SFT) data."

    The presence of '<think>...</think>' reasoning text is enforced by the format reward rf, which grants a binary reward only when the output contains those tags. Therefore the observation that SVQA-R1 'exhibits interpretable reasoning paths' is partly a restatement of the training objective: the model is rewarded for emitting the reasoning format. The lexical analysis in Appendix G counts reasoning keywords inside outputs that were produced under this mandatory format, so it cannot independently establish interpretable reasoning.

1 more flagged steps
  1. fitted input called prediction [Section 4.1 (η=1), Appendix C Table 6, Section 4.4 Table 2]
    "When updating the semantic reward, we apply a threshold δ = 0.5, and set the final reward scaling factor η = 1. ... Performance under different weights applied to the reward difference between original and flipped images. Q-Spatial++ Success Rate: η=0 52.32, η=1 58.42, η=2 51.49, η=10 47.52."

    The reported headline success rate of 58.42% is the maximum of the four η values evaluated on the same 101-question Q-Spatial++ test set used for the comparison in Table 2. No validation split is described, so η=1 was selected by optimizing the test metric itself. Reporting the argmax of the test-set sweep as the method's out-of-sample performance is statistically forced: the 6.1-point gap between η=1 and η=0 is comparable to the 9.9-point gap to the strongest open-source baseline, meaning the headline advantage may reflect test-set selection rather than the method.

full rationale

The paper's central quantitative benchmark, Q-Spatial++, uses human-expert-annotated ground truth and is not definitionally identical to the training objective, and there is no load-bearing self-citation chain (the cited R1/GRPO/SpatialVLM works are external). However, three specific steps are self-referential. First, the semantic reward rs is the same Sentence-BERT cosine similarity later reported as the sBERT evaluation metric, so the sBERT gains in Tables 4-5 are direct consequences of the training objective rather than independent evidence. Second, the 'interpretable reasoning paths' highlighted in the abstract are enforced by the format reward, which requires <think>...</think> tags, making the presence of reasoning text a constructed outcome. Third, the headline 58.42% success rate on Q-Spatial++ is chosen as the best of four η values (52.32, 58.42, 51.49, 47.52) evaluated on the same 101-question test set used for the comparison, with no validation split described; reporting the argmax of the test metric as the method's success rate is a selected, not out-of-sample, result. The GPT-4o dual role (generating flipped QA pairs and serving as LLM judge for open-ended metrics) is an additional contamination risk for the LLM evaluation scores but is not needed for the main quantitative finding. Overall, the derivation is partially circular in these specific reductions, although the method retains some independent content on the human-annotated benchmark.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The central claims rest on four unverified or borrowed premises: the GRPO objective taken from prior work with two unreported coefficients; GPT-4o-generated flipped QA pairs that are assumed spatially correct with no quantified error rate; a Sentence-BERT similarity assumed to grade open-ended and numeric answers, including distances and bounding boxes; and an ad hoc rule that penalizes only the higher-scoring view. On the parameter side the method depends on hand-set lambda1, lambda2, delta, eta, and group size G, with beta and epsilon unreported, so the reported accuracy cannot be reproduced from the text alone. The paper introduces no invented entities; its contribution is a training objective, not a new object or force.

free parameters (7)
  • lambda_1 (format reward weight) = 0.5
    Weight for the format reward rf in Eq. 1; chosen by hand with no ablation reported.
  • lambda_2 (semantic reward weight) = 0.5
    Weight for the Sentence-BERT reward rs in Eq. 1; chosen by hand with no ablation reported.
  • delta (semantic consistency threshold) = 0.5
    Threshold in Eqs. 3-4 controlling which per-sample rewards receive the view-discrepancy penalty; chosen by hand.
  • eta (view-discrepancy scaling) = 1
    Scaling of the reward difference penalty in Eqs. 3-4. Swept in Table 6 over 0, 1, 2, 10; the value 1 is selected partly by its Q-Spatial++ success, i.e., tuned on the main test benchmark.
  • G (GRPO group size) = 8
    Number of sampled answers per group in Section 4.1; chosen by hand, with no ablation.
  • beta (KL penalty coefficient) = not reported
    KL coefficient in Eq. 5; never assigned a value anywhere in the paper.
  • epsilon (clip threshold) = not reported
    Clip threshold in Eq. 6; never assigned a value anywhere in the paper.
assumptions (6)
  • domain assumption The GRPO clipped-ratio objective with a KL penalty to the reference policy is a stable and valid training rule.
    Adopted from DeepSeekMath and DeepSeek-R1 in Section 3.3 (Eqs. 5-6); convergence properties are assumed, and the coefficients beta and epsilon are never assigned values.
  • domain assumption Horizontal mirroring with rewritten QA pairs preserves spatial semantics (distances, next-to, above) and swaps left/right correctly.
    Stated as two observations about human spatial cognition at the start of Section 3.3; treated as exact for the whole auto-generated training set without verification.
  • domain assumption GPT-4o-generated question-answer pairs for flipped images are correct and spatially coherent.
    Section 3.1 reports 'most flipped QA pairs are logically correct' after prompt enhancement, but gives no error rate, no human check, and no released pairs; this is the paper's weakest load-bearing premise.
  • domain assumption Sentence-BERT cosine similarity is a valid proxy for answer correctness, including numerical answers such as distances and bounding boxes.
    Section 3.2 asserts the reward 'can measure the numerical difference' for these answer types without evidence; text embeddings need not encode numerical proximity.
  • ad hoc to paper Penalizing only the higher-scoring view, and only when per-sample reward exceeds delta, improves spatial grounding.
    Eqs. 3-4 define this rule without derivation; Table 6 shows eta sensitivity and degradation at larger values, indicating a heuristic mechanism.
  • domain assumption The Vqasynth_Spacellava reference answers used as reward targets are correct for their question types.
    Section 4.2 uses this auto-generated dataset for training; answer quality is inherited from the VQASynth/SpatialVLM pipeline and not re-audited.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SVQA-R1: Reinforcing Spatial Reasoning in MLLMs via View-Consistent Reward Optimization." pith.science (2026). https://pith.science/paper/N3BUC6TD

@misc{pith2026250601371,
  author       = {Pith},
  title        = {Pith review of: SVQA-R1: Reinforcing Spatial Reasoning in MLLMs via View-Consistent Reward Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N3BUC6TD}},
  note         = {Machine review of arXiv:2506.01371}
}
read the original abstract

Spatial reasoning remains a critical yet underdeveloped capability in existing vision-language models (VLMs), especially for Spatial Visual Question Answering (Spatial VQA) tasks that require understanding relative positions, distances, and object configurations. Inspired by the R1 paradigm introduced in DeepSeek-R1, which enhances reasoning in language models through rule-based reinforcement learning (RL), we propose SVQA-R1, the first framework to extend R1-style training to spatial VQA. In particular, we introduce Spatial-GRPO, a novel group-wise RL strategy that constructs view-consistent rewards by perturbing spatial relations between objects, e.g., mirror flipping, thereby encouraging the model to develop a consistent and grounded understanding of space. Our model, SVQA-R1, not only achieves dramatically improved accuracy on spatial VQA benchmarks but also exhibits interpretable reasoning paths even without using supervised fine-tuning (SFT) data. Extensive experiments and visualization demonstrate the effectiveness of SVQA-R1 across multiple spatial reasoning benchmarks.

Figures

Figures reproduced from arXiv: 2506.01371 by the authors.

Figure 1
Figure 1. (a) The original image, question, and answer of a sample. (b) The flipped image, question, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. A diverse set of open-ended spatial question-answer types. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of different open-ended spatial question-answer types. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Visualization of original and flipped image and QA for left-right spatial reasoning. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Visualization of original and flipped image and QA for bounding box consistency. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Visualization of original and flipped image and QA for unaffacted cases. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: More visualization examples with thinking evidence. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A two-phase interactive RL framework (DIRL) lets a 3B VLM learn to coordinate multiple vision and robot tools, reaching top benchmark scores and 86% real-robot pick-and-place success.

  2. SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Dense scene-graph-grounded rewards let a 7B multimodal LLM trained on 7K synthetic questions beat SFT and sparse-RL baselines and outscore GPT-4o on average across 12 spatial/real-world benchmarks.

Reference graph

Works this paper leans on

39 extracted references · 21 canonical work pages · cited by 2 Pith papers

  1. [1]

    https://github.com/remyxai/VQASynth, 2024

    Spacethinker-qwen2.5vl-3b. https://github.com/remyxai/VQASynth, 2024

    work page 2024

  2. [2]

    https://ai.google.dev/gemini-api/docs/models, 2025

    Gemini-1.5-flash. https://ai.google.dev/gemini-api/docs/models, 2025

    work page 2025

  3. [3]

    https://huggingface.co/remyxai/SpaceThinker-Qwen2

    Spacethinker-qwen2.5vl-3b. https://huggingface.co/remyxai/SpaceThinker-Qwen2. 5VL-3B, 2025

    work page 2025

  4. [4]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  5. [5]

    B. Chen, Z. Xu, S. Kirmani, B. Ichter, D. Sadigh, L. Guibas, and F. Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455–14465, 2024

    work page 2024

  6. [6]

    H. Chen, H. Tu, F. Wang, H. Liu, X. Tang, X. Du, Y . Zhou, and C. Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models. arXiv preprint arXiv:2504.11468, 2025

    arXiv 2025

  7. [7]

    Z. Chen, W. Wang, Y . Cao, Y . Liu, Z. Gao, E. Cui, J. Zhu, S. Ye, H. Tian, Z. Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024

    arXiv 2024

  8. [8]

    Daxberger, N

    E. Daxberger, N. Wenzel, D. Griffiths, H. Gang, J. Lazarow, G. Kohavi, K. Kang, M. Eichner, Y . Yang, A. Dehghan, et al. Mm-spatial: Exploring 3d spatial understanding in multimodal llms. arXiv preprint arXiv:2503.13111, 2025

    arXiv 2025

Show all 39 references
  1. [9]

    Doveh, A

    S. Doveh, A. Arbelle, S. Harary, R. Herzig, D. Kim, P. Cascante-Bonilla, A. Alfassy, R. Panda, R. Giryes, R. Feris, et al. Dense and aligned captions (dac) promote compositional reasoning in vl models. Advances in Neural Information Processing Systems , 36:76137–76150, 2023

    2023

  2. [10]

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  3. [11]

    J. Hong, P. Fang, W. Li, T. Zhang, C. Simon, M. Harandi, and L. Petersson. Reinforced attention for few-shot learning and beyond. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 913–923, 2021

    2021

  4. [12]

    D. A. Hudson and C. D. Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700–6709, 2019

    2019

  5. [13]

    Islam and O

    R. Islam and O. M. Moushi. Gpt-4o: The cutting-edge advancement in multimodal llm.Authorea Preprints, 2024

    2024

  6. [14]

    Kamath, M

    A. Kamath, M. Singh, Y . LeCun, G. Synnaeve, I. Misra, and N. Carion. Mdetr-modulated detec- tion for end-to-end multi-modal understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1780–1790, 2021

    2021

  7. [15]

    Kerbl, G

    B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1, 2023

    2023

  8. [16]

    Y . Li, Y . Chen, Y . Liu, et al. Tinyllava-video-r1: Enhancing video reasoning in small vision- language models via reinforcement learning. arXiv preprint arXiv:2504.09641, 2024

    2024 arXiv

  9. [17]

    Y .-H. Liao, R. Mahmood, S. Fidler, and D. Acuna. Reasoning paths with reference objects elicit quantitative spatial reasoning in large vision-language models.arXiv preprint arXiv:2409.09788, 2024

    2024 arXiv

  10. [18]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023. 10

    2023

  11. [19]

    C. Ma, K. Lu, T.-Y . Cheng, N. Trigoni, and A. Markham. Spatialpin: Enhancing spatial reasoning capabilities of vision-language models through prompting and interacting 3d priors. Advances in neural information processing systems , 2024

    2024

  12. [20]

    Mildenhall, P

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021

    2021

  13. [21]

    Gpt-4o technical report

    OpenAI. Gpt-4o technical report. https://openai.com/index/gpt-4o, 2024. Accessed: 2025-05-14

    2024

  14. [22]

    Papineni, S

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  15. [23]

    Patel, N

    M. Patel, N. S. A. Kusumba, S. Cheng, C. Kim, T. Gokhale, C. Baral, et al. Tripletclip: Improving compositional reasoning of clip via synthetic vision-language negatives. Advances in neural information processing systems , 37:32731–32760, 2024

    2024

  16. [24]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748–8763. PmLR, 2021

    2021

  17. [25]

    Reimers and I

    N. Reimers and I. Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing , pages 3982–3992, 2019

    2019

  18. [26]

    Salter, D

    S. Salter, D. Rao, M. Wulfmeier, R. Hadsell, and I. Posner. Attention-privileged reinforcement learning. In Conference on Robot Learning, pages 394–408. PMLR, 2021

    2021

  19. [27]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  20. [28]

    Vedantam, K

    R. Vedantam, K. Desai, S. Lee, M. Rohrbach, D. Batra, and D. Parikh. Probabilistic neural symbolic models for interpretable visual question answering. In International Conference on Machine Learning, pages 6428–6437. PMLR, 2019

    2019

  21. [29]

    J. Wang, Y . Ming, Z. Shi, V . Vineet, X. Wang, S. Li, and N. Joshi. Is a picture worth a thousand words? delving into spatial reasoning for vision language models. Advances in Neural Information Processing Systems, 37:75392–75421, 2024

    2024

  22. [30]

    J. Wang, H. Zhang, L. Xie, et al. Raven: Reasoning with visual commonsense for planning in human-robot interaction. arXiv preprint arXiv:2206.07281, 2022

    2022 arXiv

  23. [31]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  24. [32]

    H. Wu, B. Zhou, Y . Wang, et al. Reason-rft: Reasoning-aware reinforcement fine-tuning for vision-language models. arXiv preprint arXiv:2503.20752, 2024

    2024

  25. [33]

    H. Wu, B. Zhou, Y . Wang, et al. Visual-rft: Visual reinforcement fine-tuning for large vision- language models. arXiv preprint arXiv:2503.01785, 2024

    2024 arXiv

  26. [34]

    Q. Xu, Y . Wang, B. Zhou, et al. Video-r1: Reinforcing video reasoning in multimodal large language models. arXiv preprint arXiv:2503.21776, 2024

    2024 arXiv

  27. [35]

    K. Yi, J. Wu, C. Gan, A. Torralba, P. Kohli, and J. Tenenbaum. Neural-symbolic vqa: Disen- tangling reasoning from vision and language understanding. Advances in neural information processing systems, 31, 2018

    2018

  28. [36]

    X. Yu, M. Xu, Y . Zhang, H. Liu, C. Ye, Y . Wu, Z. Yan, C. Zhu, Z. Xiong, T. Liang, et al. Mvimgnet: A large-scale dataset of multi-view images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9150–9161, 2023. 11

    2023

  29. [37]

    Zhang, Y

    J. Zhang, Y . Zhou, Y . Li, et al. Videochat-r1: Reinforcing spatio-temporal perception in video multimodal large language models. arXiv preprint arXiv:2504.06958, 2024

    2024 arXiv

  30. [38]

    W. Zhao, S. Zheng, H. Zhang, et al. Cot-vlm: Chain-of-thought prompting for visual language models. arXiv preprint arXiv:2309.04761, 2023

    2023 arXiv

  31. [39]

    yes" or

    B. Zhou, Y . Wang, H. Liu, et al. Vlm-r1: Towards a stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2024. 12 Appendix A Examples of Horizontal Image Flipping A.1 Left-Right Spatial Reasoning This category highlights cases where t...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.