Pith. sign in

REVIEW 3 major objections 5 minor 82 references

Multi-View Relational Distillation for Spatial Reasoning with Vision-Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Distilling how patches relate across views, rather than the features themselves, makes vision-language models spatially smarter while keeping them aligned to language.

desk verdict A practical relational-distillation method for geometry grounding that beats feature distillation in controlled benchmarks, though its alignment-retention evidence leans on a selection-biased probe and needs corroboration. read the letter →

arxiv 2608.10864 v1 pith:NW52DGDD submitted 2026-08-11 cs.CV

Kiet T. Nguyen , Hanbo Shim , Jinwoo Kim , Seunghoon Hong This is my paper · ORCID
classification cs.CV
keywords multi-viewrelationaldistillationvision-languagemodelsspatialreasoninggeometricgroundingknowledgealignmentVSI-BenchLoRA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language models have strong image and video understanding, but their visual-spatial representations are geometrically fragile, which shows up as failures on spatial reasoning tasks. The paper identifies a previously undiagnosed failure of the usual remedy, feature distillation from a geometry-grounded model: matching multi-view teacher features grounds geometry but pulls the student's visual features away from the pretrained vision-language space, degrading linguistic grounding such as tracking object appearance order. Its proposal, multi-view relational distillation (MVRD), distills instead the pairwise cosine similarities among patch tokens across views, which encode geometric correspondence while leaving the student underdetermined, so the representation can stay close to the pretrained space. On VSI-Bench, MVRD reaches 59.9% average accuracy versus 57.7% for supervised fine-tuning and 57.0% for feature distillation; a dual-pathway variant reaches 60.4%, within 0.5 points of the feature-fusion method VLM-3R while adding roughly 25% of its parameters and 16% of its latency. If correct, relational distillation is a low-cost way to make VLM representations more geometric without giving up language alignment.

What carries the argument

The load-bearing object is the multi-view relational representation $R(\mathbf{h})[i,j]=r(\mathbf{h}[i],\mathbf{h}[j])$, an $N\times N$ matrix of pairwise similarities among all patch tokens from all input views, with $r$ chosen as cosine similarity and $N=T H_p W_p$. The student's projected relation matrix is aligned to the frozen teacher's relation matrix by a row-wise Pearson correlation loss (with causal masking and zeroed diagonal), trained jointly with the SFT loss. The property that does the work is underdetermination: because cosine similarity is invariant to global rotation and independent scaling of the feature set, any rotation or scaling of an optimal student representation is also optimal, leaving room for the optimizer to stay near the pretrained vision-language region. A dual-pathway variant strengthens this by using two LoRA adapters at layers 13-20, a geometric adapter supervised by the composite loss and a semantic adapter supervised only by SFT, whose outputs are summed before the remaining layers.

What would settle it

Compute the distance (for example, mean cosine distance) between the student's vision features at the distillation layer before and after MVRD training, and measure text-based object-detection precision on a randomly sampled, unbiased set of VSI-Bench Object Size and Appearance Order questions. If the student drifts as far from the pretrained features as feature distillation does, or if the unselected-question precision drops to feature-distillation levels while the cross-view relation correlation remains high, then the underdetermination premise does not protect vision-language alignment in practice.

Watch

Extended reading notes

Core claim

The paper's central claim is that the bottleneck in grounding VLMs to geometry is not the amount of geometric signal but the choice of distillation target. Matching the teacher's per-patch features directly (feature distillation) pulls the student out of its pretrained vision-language space; matching the teacher's multi-view pairwise cosine similarities (relational distillation) transfers 3D correspondence information while leaving the student representation underdetermined, because cosine similarities are invariant to global rotations and independent per-feature scalings. The paper verifies that the geometry-grounded teacher VGGT's cross-view similarity is a meaningful proxy for 3D proximity (mean row-wise Pearson correlation 0.79 with Euclidean distance on ScanNet), shows that MVRD beats supervised fine-tuning and feature distillation on VSI-Bench, and demonstrates transfer to 3D object grounding, dense captioning, and question answering. With the dual-pathway variant, the geometric and semantic supervision are decoupled into separate LoRA adapters, and the result approaches the feature-fusion state of the art at a fraction of its inference cost.

Load-bearing premise

The benefit depends on the premise that matching only cross-view angle relations leaves the student's features free to remain near the pretrained vision-language space; the paper shows the loss is rotation- and scale-invariant but does not prove that gradient training actually picks such a solution, and its main quantitative evidence for preserved alignment comes from 42 questions selected because SFT outperforms distillation.

Editorial extensions

If this is right

  • MVRD outperforms supervised fine-tuning and feature distillation on VSI-Bench (59.9 vs 57.7 and 57.0), and wins every linguistic-grounding task where feature distillation (3DRS*) falls below the SFT baseline.
  • MVRD + Dual Pathway reaches 60.4%, within 0.5 points of VLM-3R, with 0.19B added parameters versus 0.79B and 0.14s added latency versus 0.85s, about 25% of the parameters and 16% of the latency.
  • The gain is consistent across three VLM backbones (LLaVA-Video-7B, InternVL3-8B, Qwen2.5-VL-7B), and the dual pathway gives the largest marginal gain on Qwen2.5-VL-7B (+3.7 points over MVRD alone).
  • MVRD transfers to 3D scene understanding: trained under the 3DRS protocol, it improves on every ScanRefer, Multi3DRefer, Scan2Cap, ScanQA, and SQA3D metric over the feature-distillation baseline.
  • Ablations show both design choices are load-bearing: cosine relations plus row-wise Pearson correlation reach 59.9, while Euclidean-distance relations with MSE fall to 48.6.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the underdetermination mechanism is the real source of the gain, the same relational recipe should transfer to other teachers whose cross-view similarities encode useful structure, such as depth, motion, or object tracks, not just VGGT-style geometry; the paper does not test this, but it follows directly from its argument.
  • Because the loss is invariant to global rotation and per-feature scaling, MVRD may be unusually robust to shifts in teacher feature statistics across datasets or pretrained checkpoints; that is a plausible but untested consequence.
  • The paper's alignment probe uses 42 questions chosen because SFT beats distillation, so the claimed alignment preservation should also be checked on a random, unbiased question sample; the paper's own protocol leaves this open.
  • The underdetermination idea suggests an explicit extension: add a small anchor regularizer that pulls the student toward its own pretrained features while matching relations, potentially improving alignment further than the implicit freedom alone; the paper does not explore this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes multi-view relational distillation (MVRD), which distills cross-view patch-wise cosine-similarity matrices from a frozen geometry-grounded teacher (VGGT) into a VLM, rather than distilling the teacher features directly. On VSI-Bench, the authors report that MVRD outperforms supervised fine-tuning and feature distillation across several backbones, approaches the feature-fusion method VLM-3R with far lower inference overhead, and transfers to 3D scene understanding tasks. The paper also presents a diagnosis of feature distillation's tendency to disrupt vision-language alignment, supported by a text-grounded object-detection precision probe.

Significance. If the central claim holds, MVRD offers a practical way to inject geometric structure into VLMs while preserving their pretrained vision-language alignment, at a fraction of the inference cost of feature fusion. The main benchmark comparison is well controlled: the same backbone, the same training data for MVRD and its direct baselines, and a reproduced 3DRS baseline that matches the original paper's trend. The paper also includes backbone generality experiments, a transfer study to 3D scene understanding, and detailed ablations of the distillation target and loss, which strengthens the empirical case. However, the representation-level evidence for alignment retention rests on a selection-biased probe, and the headline VSI-Bench gaps lack error bars, so the empirical support for the central mechanism is currently incomplete.

major comments (3)
  1. [Appendix A.1, Figure 4(b)] The text-grounded detection precision probe is computed on only 42 Object Size questions selected 'where the SFT model performs better than distillation methods.' This conditions on the outcome being compared: the subset is defined by the distillation methods being worse than SFT, so the precision gap (0.16 vs 0.08) is not representative of the general distribution and may be inflated. Because this probe is the primary representation-level evidence for the claim that MVRD retains vision-language alignment, please report the same metric on the full ScanNet subset of Object Size questions or on a random subset chosen independently of model performance, with per-question standard errors. This is load-bearing, since alignment retention is the central differentiator of MVRD over feature distillation.
  2. [Table 2, Table 6] The headline claims that MVRD outperforms SFT and feature distillation by 1.4-2.2 points on average VSI-Bench accuracy are reported without error bars, significance tests, or multiple-seed variation. Given that VSI-Bench is a multi-task benchmark and per-task numbers fluctuate, the reported gaps may be within run-to-run noise. Please provide at least three seeds for the main comparisons (Tables 2, 3, and 6) or bootstrap confidence intervals over the benchmark questions, so the reader can judge the robustness of the 59.9-60.4 averages against the 57.7 and 58.5 baselines.
  3. [Section 4, Eq. (2)] The underdetermination argument shows that if v* is optimal for the relational loss, then every shared rotation and independent scaling of v* is also optimal; hence a relational-loss-optimal solution near the pretrained vision-language space exists. This is an existence claim, not a statement that gradient-based minimization of LSFT + gamma*LRD will select such a solution. The unbiased alignment probe requested in the first major comment is therefore the necessary empirical support for 'allowing it to remain close.' Please either provide unbiased evidence that the trained student indeed stays near the pretrained space, or state explicitly what conditions on the loss landscape would guarantee convergence to such a solution.
minor comments (5)
  1. [Section 4] The text says 'negative Person correlation'; this should be 'Pearson correlation.'
  2. [Figure 1 caption and Table 2] The method name is inconsistent: Figure 1 uses 'MVRD + Dual Pathway' while Table 2 uses 'MVRD + Dual Pathways.' Please use a single consistent name throughout.
  3. [Abstract] The phrase 'with considerably less added parameters' should be 'with considerably fewer added parameters.'
  4. [Appendix C.1] Table 8 lists 'Scheduled training epochs 5 (stopped after epoch 1)' without describing the stopping criterion. Please state what validation signal triggers early stopping, or remove the scheduling of 5 epochs if only 1 epoch is ever run.
  5. [Appendix A.1] The question-selection sentence is ambiguous: does 'distillation methods' refer to feature distillation only, or both feature distillation and MVRD? Clarifying this matters because the selection criterion directly affects the interpretation of the probe results.

Circularity Check

1 steps flagged · score 2.0 of 10

Central derivation is self-contained and benchmark-evaluated; the only mild circularity is a selection-biased alignment probe used to support the method's main motivation.

  1. other [Appendix A.1 'Question selection'; results used in Section 5.1 'Analysis on visual representation' and Figure 4(b)]
    "We only sample questions where the SFT model performs better than distillation methods to investigate the performance drop of the distillation methods, resulting in 42 questions over 42 scenes."

    The quantitative evidence for the paper's central differentiator—that MVRD preserves vision-language alignment better than feature distillation (precision 0.16 vs 0.08 in Figure 4(b))—is computed on a subset selected by the condition that SFT already outperforms distillation methods. Conditioning on this event selects for cases where distillation has failed relative to SFT, so the measured alignment gap is not a representative estimate and is partly a consequence of the sampling rule rather than an independent confirmation of the method's motivation. Because this probe is then used to support the claim that relational targets keep the student close to the pretrained vision-language space, the evidence chain has a mild selection-on-outcome circularity.

full rationale

MVRD is not derived from its own outputs: the teacher (VGGT) is a frozen external model, the distillation target is a fixed matrix of cross-view cosine similarities, and the main evaluation uses the external VSI-Bench benchmark with standard protocols. The objective LRD is defined from teacher relations, and the student is trained with a composite loss that includes an external question-answering loss; no parameter is fitted to VSI-Bench and then reported as a prediction. The paper contains no load-bearing self-citation chain: the key baselines (3DRS, VLM-3R) are from other groups, and the relational-distillation prior work is cited contextually rather than invoked as a uniqueness argument. The only notable circularity-adjacent issue is the alignment probe in Appendix A.1: the 42-question subset is selected by conditioning on SFT beating distillation methods, and this same probe is used to support the paper's central claim that MVRD retains vision-language alignment. That is a selection bias that inflates the apparent gap, but it does not invalidate the independent benchmark comparisons or the method's derivation. Other concerns, such as missing error bars on headline accuracy differences, are correctness risks rather than circularity. Overall, no significant circularity in the derivation; the mild probe-conditioning issue warrants a score of 2.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper relies on three unproved premises: VGGT relations faithfully encode 3D proximity (verified on ScanNet but assumed elsewhere), the student can satisfy the relational loss while staying near the pretrained space (invariance argument but no proof), and LoRA fine-tuning on a spatial QA set preserves pretrained behavior (standard, supported by one ablation). The method itself has no fitted physical constants; the listed free parameters are training hyperparameters chosen by ablations.

free parameters (4)
  • Distillation loss weight gamma = 0.5
    Chosen by hand to balance the SFT loss and MVRD loss magnitudes; no sweep over gamma is reported.
  • Distillation layer index = 20 of 28 layers (0-indexed)
    Selected by the ablation in Table 4; early and middle layers underperform while layer 20 gives the best score.
  • LoRA rank and alpha = rank 128, alpha 256
    Standard LoRA configuration chosen without a reported rank sweep, affecting how much of the pretrained space is preserved.
  • Dual Pathway separation layers = 13-20
    Chosen by the separation-layer ablation in Table 5; this range gives the best balance between geometric and semantic objectives.
assumptions (3)
  • domain assumption VGGT cross-view patch-wise cosine similarities are a faithful proxy for 3D Euclidean proximity on training scenes.
    Verified in Appendix A.4 on ScanNet validation with mean row-wise Pearson correlation 0.79, but assumed to hold on ScanNet++ and ARKitScenes used in training. If the teacher's relations do not encode geometry there, the distillation target is wrong.
  • ad hoc to paper A student can satisfy the relational loss while remaining near its pretrained vision-language space because cosine similarities leave features underdetermined.
    Section 4 shows the loss is invariant to rotations and independent scalings, but there is no proof that gradient descent will pick the pretrained solution. Empirical support is indirect and partly rests on a selection-biased probe.
  • domain assumption Fine-tuning via LoRA on a spatial QA dataset does not catastrophically degrade the pretrained VLM behavior.
    Standard assumption in parameter-efficient fine-tuning; supported by Table 3, where LoRA-based feature distillation beats full fine-tuning of 3DRS*.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-View Relational Distillation for Spatial Reasoning with Vision-Language Models." pith.science (2026). https://pith.science/paper/NW52DGDD

@misc{pith2026260810864,
  author       = {Pith},
  title        = {Pith review of: Multi-View Relational Distillation for Spatial Reasoning with Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NW52DGDD}},
  note         = {Machine review of arXiv:2608.10864}
}
read the original abstract

Vision-language models (VLMs) have achieved strong image and video understanding, yet their visual-spatial representations remain geometrically fragile, leading to failures in spatial reasoning needed for embodied AI, robotics, and autonomous driving. Prior approaches to geometry grounding either fine-tune VLMs on spatial question answering, which can perpetuate spurious visual representations, or fuse features from large geometry-grounded vision models, which substantially increases model size at inference. Knowledge distillation from geometry-grounded vision models offers an alternative, but directly matching multi-view teacher features can disrupt the pretrained alignment between visual and textual representations, degrading object- and language-semantic capabilities. We propose multi-view relational distillation (MVRD), which distills patch-wise cosine similarities across views instead of the teacher features themselves. These relations encode geometric correspondences adequate for spatial understanding, while leaving the student representation underdetermined, allowing it to remain close to its pretrained vision- language space. Across representative VLMs, MVRD improves visual-spatial reasoning, outperforming supervised fine-tuning and feature distillation while approaching feature fusion methods with considerably fewer added parameters and lower latency. We show that MVRD makes visual representations more geometric while retaining language alignment, and generalizes to 3D scene understanding tasks such as object grounding, dense captioning, and question answering.

Figures

Figures reproduced from arXiv: 2608.10864 by the authors.

Figure 1
Figure 1. Multi-view relational distillation (MVRD). (a) Spatial reasoning. MVRD with Dual Pathway variant approaches the feature fusion method VLM-3R [1], using the same backbone and dataset, with far fewer added parameters and lower latency. (b) Scene understanding. MVRD transfers to scene understanding, surpassing the prior state-of-the-art 3DRS [2] in all tasks. †Equal advising Preprint. arXiv:2608.10864v1 [cs.CV] 11 Aug … view at source ↗
Figure 2
Figure 2. Failure of feature distillation. (a) Qualitative: At layer 20, 3DRS* fails to align microwave with the corresponding visual patches, unlike the base model LLaVA-Video-7B-Qwen2. (b) Quantitative: A sharp drop in text-based object-detection precision across layers indicates severe degradation in vision-language alignment. overhead of feature fusion. This is done by minimizing the following loss for each input (x1:T , … view at source ↗
Figure 3
Figure 3. An overview of multi-view relational distillation loss and dual pathway. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Vision-language alignment across methods. (a) At distillation layer, MVRD and MVRD + Dual Pathway align microwave with its visual patches better than feature distillation. (b) Both improve vision-language alignment while matching feature distillation in geometric relat…
Figure 5
Figure 5. Figure 5: Row-wise Correlation of cosine distance from VGGT with 3D Euclidean distance on [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Additional text-to-patch similarity maps. [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

82 extracted references · 23 canonical work pages

  1. [1]

    Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction.arXiv preprint arXiv:2505.20279, 2025

    Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang, Huaizhi Qu, Shijie Zhou, Dilin Wang, et al. Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction.arXiv preprint arXiv:2505.20279, 2025

    arXiv 2025

  2. [2]

    3drs: Mllms need 3d-aware representation supervision for scene understanding.Advances in Neural Information Processing Systems, 38: 67961–67988, 2026

    Xiaohu Huang, Jingjing Wu, Qunyi Xie, and Kai Han. 3drs: Mllms need 3d-aware representation supervision for scene understanding.Advances in Neural Information Processing Systems, 38: 67961–67988, 2026

    2026

  3. [3]

    CUP Archive, 1967

    Kenneth James Williams Craik.The nature of explanation, volume 445. CUP Archive, 1967

    1967

  4. [4]

    Henry Holt and Co., Inc., New York, NY , USA, 1982

    David Marr.Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Henry Holt and Co., Inc., New York, NY , USA, 1982. ISBN 0716715678

    work page 1982

  5. [5]

    Number 6

    Philip Nicholas Johnson-Laird.Mental models: Towards a cognitive science of language, inference, and consciousness. Number 6. Harvard University Press, 1983

    1983

  6. [6]

    Separate visual pathways for perception and action

    Melvyn A Goodale and A David Milner. Separate visual pathways for perception and action. Trends in neurosciences, 15(1):20–25, 1992

    1992

  7. [7]

    Mental rotation of three-dimensional objects.Science, 171(3972):701–703, 1971

    Roger N Shepard and Jacqueline Metzler. Mental rotation of three-dimensional objects.Science, 171(3972):701–703, 1971

    1971

  8. [8]

    Oxford university press, 1978

    John O’keefe and Lynn Nadel.The hippocampus as a cognitive map. Oxford university press, 1978

    1978

Show all 82 references
  1. [9]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  2. [10]

    Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    2022 arXiv

  3. [11]

    Palm-e: An embodied multimodal language model.arXiv preprint arXiv:2303.03378, 2023

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: An embodied multimodal language model.arXiv preprint arXiv:2303.03378, 2023

    2023 arXiv

  4. [12]

    Open x- embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0

    Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x- embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In2024 IEEE Internat...

    2024

  5. [13]

    Are vlms ready for autonomous driving? an empirical study from the reliability, data and metric perspectives

    Shaoyuan Xie, Lingdong Kong, Yuhao Dong, Chonghao Sima, Wenwei Zhang, Qi Alfred Chen, Ziwei Liu, and Liang Pan. Are vlms ready for autonomous driving? an empirical study from the reliability, data and metric perspectives. InProceedings of the IEEE/CVF International Conference ...

    2025

  6. [14]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Han...

    2025 arXiv

  7. [15]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    2025 arXiv

  8. [16]

    LLaV A- video: Video instruction tuning with synthetic data.Transactions on Machine Learning Research, 2025

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun MA, Ziwei Liu, and Chunyuan Li. LLaV A- video: Video instruction tuning with synthetic data.Transactions on Machine Learning Research, 2025. ISSN 2835-8856. 10

    2025

  9. [17]

    Probing the 3d awareness of visual foundation models

    Mohamed El Banani, Amit Raj, Kevis-Kokitsi Maninis, Abhishek Kar, Yuanzhen Li, Michael Rubinstein, Deqing Sun, Leonidas Guibas, Justin Johnson, and Varun Jampani. Probing the 3d awareness of visual foundation models. InProceedings of the IEEE/CVF Conference on Computer Vision ...

    2024

  10. [18]

    Sti- bench: Are mllms ready for precise spatial-temporal world understanding? InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5622–5632, 2025

    Yun Li, Yiming Zhang, Tao Lin, XiangRui Liu, Wenxiao Cai, Zheng Liu, and Bo Zhao. Sti- bench: Are mllms ready for precise spatial-temporal world understanding? InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5622–5632, 2025

    2025

  11. [19]

    Thinking in space: How multimodal large language models see, remember, and recall spaces

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10632–10643, 2025

    2025

  12. [20]

    Vlm4d: Towards spatiotem- poral awareness in vision language models

    Shijie Zhou, Alexander Vilesov, Xuehai He, Ziyu Wan, Shuwang Zhang, Aditya Nagachandra, Di Chang, Dongdong Chen, Xin Eric Wang, and Achuta Kadambi. Vlm4d: Towards spatiotem- poral awareness in vision language models. InProceedings of the IEEE/CVF international conference on co...

    2025

  13. [22]

    Cambrian-s: Towards spatial supersensing in video

    Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis L Brown II, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, et al. Cambrian-s: Towards spatial supersensing in video. InThe Fourteenth International Conference on Learning Representations, 2025

    2025

  14. [24]

    Questioning representational optimism in deep learning: The fractured entangled representation hypothesis.arXiv preprint arXiv:2505.11581, 2025

    Akarsh Kumar, Jeff Clune, Joel Lehman, and Kenneth O Stanley. Questioning representational optimism in deep learning: The fractured entangled representation hypothesis.arXiv preprint arXiv:2505.11581, 2025

    2025 arXiv

  15. [26]

    Learning from videos for 3d world: Enhancing mllms with 3d vision geometry priors.arXiv preprint arXiv:2505.24625, 2025

    Duo Zheng, Shijia Huang, Yanyang Li, and Liwei Wang. Learning from videos for 3d world: Enhancing mllms with 3d vision geometry priors.arXiv preprint arXiv:2505.24625, 2025

    2025

  16. [28]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  17. [29]

    Unsupervised natural experience rapidly alters invariant object representation in visual cortex.science, 321(5895):1502–1507, 2008

    Nuo Li and James J DiCarlo. Unsupervised natural experience rapidly alters invariant object representation in visual cortex.science, 321(5895):1502–1507, 2008

    2008

  18. [30]

    Slow feature analysis: Unsupervised learning of invariances.Neural computation, 14(4):715–770, 2002

    Laurenz Wiskott and Terrence J Sejnowski. Slow feature analysis: Unsupervised learning of invariances.Neural computation, 14(4):715–770, 2002

    2002

  19. [31]

    The tolman-eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation.Cell, 183(5):1249– 1263, 2020

    James CR Whittington, Timothy H Muller, Shirley Mark, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy EJ Behrens. The tolman-eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation.Cell, 183(5):1249– 1263, 2020

    2020

  20. [32]

    Psychology of spatial cognition.Wiley Interdisciplinary Reviews: Cognitive Science, 3(6):565–580, 2012

    Luca Tommasi and Bruno Laeng. Psychology of spatial cognition.Wiley Interdisciplinary Reviews: Cognitive Science, 3(6):565–580, 2012. 11

    2012

  21. [33]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabil...

    2025 arXiv

  22. [34]

    Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al

    OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  23. [35]

    Sti- bench: Are mllms ready for precise spatial-temporal world understanding?arXiv preprint arXiv:2503.23765, 2025

    Yun Li, Yiming Zhang, Tao Lin, Xiangrui Liu, Wenxiao Cai, Zheng Liu, and Bo Zhao. Sti- bench: Are mllms ready for precise spatial-temporal world understanding?arXiv preprint arXiv:2503.23765, 2025

    2025 arXiv

  24. [36]

    Vlm4d: Towards spatiotemporal awareness in vision language models.arXiv preprint arXiv:2508.02095, 2025

    Shijie Zhou, Alexander Vilesov, Xuehai He, Ziyu Wan, Shuwang Zhang, Aditya Nagachan- dra, Di Chang, Dongdong Chen, Xin Eric Wang, and Achuta Kadambi. Vlm4d: Towards spatiotemporal awareness in vision language models.arXiv preprint arXiv:2508.02095, 2025

    2025 arXiv

  25. [37]

    Learning transferable visual models from natural language supervision.arXiv preprint arXiv:2103.00020, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision.arXiv preprint arXi...

    2021 arXiv

  26. [38]

    Sigmoid loss for language image pre-training.arXiv preprint arXiv:2303.15343, 2023

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training.arXiv preprint arXiv:2303.15343, 2023

    2023 arXiv

  27. [39]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms.arXiv preprint arXiv:2406.16860, 2024

    Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Ziteng Wang, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal llms.arXiv ...

    2024 arXiv

  28. [40]

    O’Sullivan, Fang Cao, Tahoura Nedaee, Kamyar Rajabalifardi, Fei-Fei Li, Ehsan Adeli, and Euan Ashley

    Mohammad Asadi, Jack W. O’Sullivan, Fang Cao, Tahoura Nedaee, Kamyar Rajabalifardi, Fei-Fei Li, Ehsan Adeli, and Euan Ashley. Mirage: The illusion of visual understanding.arXiv preprint arXiv:2603.21687, 2026

    2026

  29. [41]

    From flatland to space: Teaching vision-language models to perceive and reason in 3d.arXiv preprint arXiv:2503.22976, 2026

    Jiahui Zhang, Yurui Chen, Yanpeng Zhou, Yueming Xu, Ze Huang, Jilin Mei, Junhui Chen, Yu-Jie Yuan, Xinyue Cai, Guowei Huang, Xingyue Quan, Hang Xu, and Li Zhang. From flatland to space: Teaching vision-language models to perceive and reason in 3d.arXiv preprint arXiv:2503.22976, 2026

    2026

  30. [42]

    Visual spatial tuning.arXiv preprint arXiv:2511.05491, 2025

    Rui Yang, Ziyu Zhu, Yanwei Li, Jingjia Huang, Shen Yan, Siyuan Zhou, Zhe Liu, Xiangtai Li, Shuangye Li, Wenqian Wang, Yi Lin, and Hengshuang Zhao. Visual spatial tuning.arXiv preprint arXiv:2511.05491, 2025

    2025

  31. [43]

    Cambrian-s: Towards spatial supersensing in video.arXiv preprint arXiv:2511.04670, 2025

    Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis Brown, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, Daohan Lu, Rob Fergus, Yann LeCun, Li Fei- Fei, and Saining Xie. Cambrian-s: Towards spatial supersensing in video.arXiv preprint arXiv:2511.04670, 2025

    2025 arXiv

  32. [44]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025

    2025

  33. [45]

    Continuous 3d perception model with persistent state

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10510–10522. IEEE, 2025

    2025

  34. [46]

    Spatialstack: Layered geometry-language fusion for 3d vlm spatial reasoning.arXiv preprint arXiv:2603.27437, 2026

    Jian Zhang, Shijie Zhou, Bangya Liu, Achuta Kadambi, and Zhiwen Fan. Spatialstack: Layered geometry-language fusion for 3d vlm spatial reasoning.arXiv preprint arXiv:2603.27437, 2026

    2026 arXiv

  35. [47]

    G2vlm: Geometry grounded vision language model with unified 3d reconstruction and spatial reasoning.arXiv preprint arXiv:2511.21688, 2025

    Wenbo Hu, Jingli Lin, Yilin Long, Yunlong Ran, Lihan Jiang, Yifan Wang, Chenming Zhu, Runsen Xu, Tai Wang, and Jiangmiao Pang. G2vlm: Geometry grounded vision language model with unified 3d reconstruction and spatial reasoning.arXiv preprint arXiv:2511.21688, 2025. 12

    2025

  36. [48]

    Think with 3d: Geometric imagination grounded spatial reasoning from limited views.arXiv preprint arXiv:2510.18632, 2026

    Zhangquan Chen, Manyuan Zhang, Xinlei Yu, Xufang Luo, Mingze Sun, Zihao Pan, Xiang An, Yan Feng, Peng Pei, Xunliang Cai, and Ruqi Huang. Think with 3d: Geometric imagination grounded spatial reasoning from limited views.arXiv preprint arXiv:2510.18632, 2026

  37. [49]

    Vision-aligned latent reasoning for multi-modal large language model.arXiv preprint arXiv:2602.04476, 2026

    Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee, Minsu Cho, and Jinwoo Shin. Vision-aligned latent reasoning for multi-modal large language model.arXiv preprint arXiv:2602.04476, 2026

    2026 arXiv

  38. [50]

    Cambridge University Press, 2001

    Peter Hugoe Matthews.A short history of structural linguistics. Cambridge University Press, 2001

    2001

  39. [51]

    Cambridge University Press, 2014

    Tom Leinster.Basic category theory. Cambridge University Press, 2014

    2014

  40. [52]

    Relational knowledge distillation

    Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3967–3976, 2019

    2019

  41. [53]

    Oriane Siméoni, Huy V . V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, ...

    2025 arXiv

  42. [54]

    Perception encoder: The best visual embeddings are not at the output of the network.arXiv preprint arXiv:2504.13181, 2025

    Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, Junke Wang, Marco Monteiro, Hu Xu, Shiyu Dong, Nikhila Ravi, Daniel Li, Piotr Dollár, and Christoph Feichtenhofer. Perception encoder: ...

    2025 arXiv

  43. [55]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In2017 IEEE conference on computer vision and pattern recognition (CVPR), pages 2432–2443. IEEE, 2017

    2017

  44. [56]

    Videorepa: Learning physics for video generation through relational alignment with foundation models.Advances in Neural Information Processing Systems, 38:122647– 122676, 2026

    Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang, Fanqing Meng, Xiangpeng Wan, Junchi Yan, and Yu Cheng. Videorepa: Learning physics for video generation through relational alignment with foundation models.Advances in Neural Information Processing Systems, 38:122647– 122676, 2026

    2026

  45. [57]

    Moalign: Motion-centric representation alignment for video diffusion models

    Aritra Bhowmik, Denis Korzhenkov, Cees G Snoek, Amirhossein Habibian, and Mohsen Ghafoorian. Moalign: Motion-centric representation alignment for video diffusion models. In International Conference on Learning Representations, volume 2026, pages 122379–122397, 2026

    2026

  46. [58]

    Lever- aging vision-language models for improving domain generalization in image classification

    Sravanti Addepalli, Ashish Ramayee Asokan, Lakshay Sharma, and R Venkatesh Babu. Lever- aging vision-language models for improving domain generalization in image classification. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23922–23932. IEEE, 2024

    2024

  47. [59]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  48. [60]

    Scannet++: A high-fidelity dataset of 3d indoor scenes.arXiv preprint arXiv:2308.11417, 2023

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes.arXiv preprint arXiv:2308.11417, 2023

    2023 arXiv

  49. [61]

    Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data.arXiv preprint arXiv:2111.08897, 2022

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data.arXiv preprint arXiv:...

    2022 arXiv

  50. [62]

    Scaling spatial intelligence with multimodal foundation models.arXiv preprint arXiv:2511.13719, 2026

    Zhongang Cai, Ruisi Wang, Chenyang Gu, Fanyi Pu, Junxiang Xu, Yubo Wang, Wanqi Yin, Zhitao Yang, Chen Wei, Qingping Sun, Tongxi Zhou, Jiaqi Li, Hui En Pang, Oscar Qian, Yukun Wei, Zhiqian Lin, Xuanke Shi, Kewang Deng, Xiaoyang Han, Zukai Chen, Xiangyu 13 Fan, Hanming Deng, Lew...

    2026

  51. [63]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  52. [64]

    Grounded 3d-llm with referent tokens.arXiv preprint arXiv:2405.10370, 2024

    Yilun Chen, Shuai Yang, Haifeng Huang, Tai Wang, Runsen Xu, Ruiyuan Lyu, Dahua Lin, and Jiangmiao Pang. Grounded 3d-llm with referent tokens.arXiv preprint arXiv:2405.10370, 2024

    2024 arXiv

  53. [65]

    Unifying 3d vision-language understanding via promptable queries.arXiv preprint arXiv:2405.11442, 2024

    Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, and Qing Li. Unifying 3d vision-language understanding via promptable queries.arXiv preprint arXiv:2405.11442, 2024

    2024 arXiv

  54. [66]

    Chat-scene: Bridging 3d scene and large language models with object identifiers.arXiv preprint arXiv:2312.08168, 2024

    Haifeng Huang, Yilun Chen, Zehan Wang, Rongjie Huang, Runsen Xu, Tai Wang, Luping Liu, Xize Cheng, Yang Zhao, Jiangmiao Pang, and Zhou Zhao. Chat-scene: Bridging 3d scene and large language models with object identifiers.arXiv preprint arXiv:2312.08168, 2024

    2024 arXiv

  55. [67]

    Inst3d-lmm: Instance-aware 3d scene understanding with multi-modal instruction tuning.arXiv preprint arXiv:2503.00513, 2025

    Hanxun Yu, Wentong Li, Song Wang, Junbo Chen, and Jianke Zhu. Inst3d-lmm: Instance-aware 3d scene understanding with multi-modal instruction tuning.arXiv preprint arXiv:2503.00513, 2025

    2025 arXiv

  56. [68]

    3d-llava: Towards generalist 3d lmms with omni superpoint transformer.arXiv preprint arXiv:2501.01163, 2025

    Jiajun Deng, Tianyu He, Li Jiang, Tianyu Wang, Feras Dayoub, and Ian Reid. 3d-llava: Towards generalist 3d lmms with omni superpoint transformer.arXiv preprint arXiv:2501.01163, 2025

    2025 arXiv

  57. [69]

    Llava-3d: A simple yet effective pathway to empowering lmms with 3d-awareness.arXiv preprint arXiv:2409.18125, 2025

    Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, and Xihui Liu. Llava-3d: A simple yet effective pathway to empowering lmms with 3d-awareness.arXiv preprint arXiv:2409.18125, 2025

    2025 arXiv

  58. [70]

    Video-3d llm: Learning position-aware video representation for 3d scene understanding.arXiv preprint arXiv:2412.00493, 2025

    Duo Zheng, Shijia Huang, and Liwei Wang. Video-3d llm: Learning position-aware video representation for 3d scene understanding.arXiv preprint arXiv:2412.00493, 2025

    2025 arXiv

  59. [71]

    Ross3d: Reconstructive visual instruction tuning with 3d-awareness.arXiv preprint arXiv:2504.01901, 2025

    Haochen Wang, Yucheng Zhao, Tiancai Wang, Haoqiang Fan, Xiangyu Zhang, and Zhaoxiang Zhang. Ross3d: Reconstructive visual instruction tuning with 3d-awareness.arXiv preprint arXiv:2504.01901, 2025

    2025 arXiv

  60. [72]

    Chang, and Matthias Nießner

    Dave Zhenyu Chen, Angel X. Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language.arXiv preprint arXiv:1912.08830, 2020

    1912 arXiv

  61. [73]

    Yiming Zhang, ZeMing Gong, and Angel X. Chang. Multi3drefer: Grounding text description to multiple 3d objects.arXiv preprint arXiv:2309.05251, 2023

    2023 arXiv

  62. [74]

    Dave Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel X. Chang. Scan2cap: Context- aware dense captioning in rgb-d scans.arXiv preprint arXiv:2012.02206, 2020

    2012 arXiv

  63. [75]

    Scanqa: 3d question answering for spatial scene understanding.arXiv preprint arXiv:2112.10482, 2022

    Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding.arXiv preprint arXiv:2112.10482, 2022

    2022 arXiv

  64. [76]

    Sqa3d: Situated question answering in 3d scenes.arXiv preprint arXiv:2210.07474, 2023

    Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes.arXiv preprint arXiv:2210.07474, 2023

    2023 arXiv

  65. [77]

    What is the length of the longest dimension . . . of the <object>. . . ?

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3d: Mask transformer for 3d semantic instance segmentation.arXiv preprint arXiv:2210.03105, 2023. 14 A Implementation Details A.1 Text-based Object Detection Precision We use text-...

    2023 arXiv

  66. [78]

    Frame sampling.Uniformly sample 32 frames from the scene video and resize each to 378×378

  67. [79]

    VGGT features.Pass all 32 frames jointly through VGGT and extract patch-level feature maps of shape32×27×27×d g (patch size 14)

  68. [80]

    Cross-view similarity.Flatten the feature maps to H G ∈R N×d g with N= 32·27· 27 = 23,328, ℓ2-normalize each row to obtain ˜H G, and compute the Gram matrix SG = ˜H G( ˜H G)⊤ ∈R N×N

  69. [81]

    3D back-projection.For each frame, back-project all pixels with valid depth into 3D world coordinates following Appendix A.2, using the depth and camera parameters distributed with ScanNet

  70. [82]

    We retain only patches that contain at least one valid-depth pixel; let V ⊆ {1,

    Patch-level 3D centroids.For each 14×14 patch, average the 3D coordinates of its constituent valid-depth pixels to obtain a single 3D centroid per patch. We retain only patches that contain at least one valid-depth pixel; let V ⊆ {1, . . . , N}denote the index set of retained ...

  71. [83]

    Pairwise 3D distances.Stack the centroids of the retained patches into P∈R |V|×3 and compute the pairwise Euclidean distance matrixD∈R |V|×|V| withD ij =∥P i −P j∥2

  72. [84]

    Correlation.Restrict SG to the same row/column index set V, treat 1−S G V,V as a learned distance, and compute the row-wise Pearson correlation ρrow 1−S G V,V , D as defined in Appendix A.3. Averaged across all ScanNet validation scenes, this procedure yields a mean row-wise P...

  73. [85]

    Dist., Room Size, Rel

    Geometry-centered tasks(Abs. Dist., Room Size, Rel. Dist., Rel. Dir., Route Plan): tasks whose answers are determined by metric or topological properties of the 3D scene and therefore require aggregating observations across frames into a coherent allocentric model

  74. [86]

    Feature. Dist

    Linguistic-grounding-centered tasks(Obj. Count, Obj. Size, Appr. Order): tasks whose answers are primarily determined by reliably grounding object categories in the video, with geometry playing at most a secondary role. Tasks in geometry-centered group. • Absolute Distance.Req...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.