Pith. sign in

REVIEW 4 cited by

MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.13111 v2 pith:HI7OBLFC submitted 2025-03-17 cs.CV cs.CLcs.LG

Erik Daxberger , Nina Wenzel , David Griffiths , Haiming Gang , Justin Lazarow , Gefen Kohavi , Kai Kang , Marcin Eichner
show 3 more authors
Yinfei Yang Afshin Dehghan Peter Grasch
This is my paper
classification cs.CVcs.CLcs.LG
keywords spatialunderstandingca-vqadatadepthestimationincludingmetric
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multimodal large language models (MLLMs) excel at 2D visual understanding but remain limited in their ability to reason about 3D space. In this work, we leverage large-scale high-quality 3D scene data with open-set annotations to introduce 1) a novel supervised fine-tuning dataset and 2) a new evaluation benchmark, focused on indoor scenes. Our Cubify Anything VQA (CA-VQA) data covers diverse spatial tasks including spatial relationship prediction, metric size and distance estimation, and 3D grounding. We show that CA-VQA enables us to train MM-Spatial, a strong generalist MLLM that also achieves state-of-the-art performance on 3D spatial understanding benchmarks, including our own. We show how incorporating metric depth and multi-view inputs (provided in CA-VQA) can further improve 3D understanding, and demonstrate that data alone allows our model to achieve depth perception capabilities comparable to dedicated monocular depth estimation models.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GReFEM: Multimodal LLMs as Zero-Shot Semantic Assistants for Physics-Guided 3D Mesh Refinement

    cs.GR 2026-07 conditional novelty 6.5 of 10

    GReFEM shows MLLMs zero-shot isolate load-activated geometric features for volumetric mesh refinement with higher precision than matched-budget geometric heuristics.

  2. SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Dense scene-graph-grounded rewards let a 7B multimodal LLM trained on 7K synthetic questions beat SFT and sparse-RL baselines and outscore GPT-4o on average across 12 spatial/real-world benchmarks.

  3. BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset

    cs.CL 2025-07 conditional novelty 6.0 of 10

    BMMR provides a 110k-question bilingual, multimodal, college-level dataset across 300 subjects where state-of-the-art models score at most about 50%.

  4. SVQA-R1: Reinforcing Spatial Reasoning in MLLMs via View-Consistent Reward Optimization

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A view-consistency reward built from mirror-flipped images lifts Qwen2.5-VL-3B from 21.8% to 58.4% success on the Q-Spatial++ spatial benchmark.

Pith tools