{"id":"83ab88c9-1271-47fc-9b88-ffbb0ba55e4b","arxiv_id":"2506.18385","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A new 12M-pair dataset and accompanying benchmark improve spatial reasoning in vision-language models, with gains of 12.1 points on the proposed benchmark and 10.7 points on VSI-Bench.","lead":"This paper releases InternSpatial, a dataset of roughly 12 million question-answer pairs for spatial reasoning in vision-language models, and InternSpatial-Bench, a 6,008-pair evaluation benchmark. It reports that fine-tuning InternVL2.5-8B on the data raises spatial reasoning scores by 12.1 points on the new benchmark and 10.7 points on the external VSI-Bench.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No control for general-data fine-tuning: the reported +12.1/+10.7 gains may partly reflect LoRA fine-tuning on 291K general samples, not InternSpatial.","rationale":"The reader identified pseudo-ground-truth noise from monocular depth/camera estimation as the weakest assumption. I agree that this matters, but I find a more fundamental confound in the experimental design. InternVL-Spatial-8B is fine-tuned on general data plus InternSpatial, while all reported deltas are relative to the original InternVL2.5-8B checkpoint. The paper never trains a general-data-only control, so the +12.1 and +10.7 deltas cannot be uniquely attributed to InternSpatial. This threatens the abstract's central causal claim regardless of label quality. The VSI-Bench improvement is somewhat independent evidence, but even that result is measured against the same un-fine-tuned baseline; additional fine-tuning on 291K general samples could, in principle, contribute to multi-view performance. The Section 4.4 ablation with 'InternVL-Spatial-Raw-8B' is ambiguously described ('using InternSpatial-Bench without instruction format expansion' - the evaluation benchmark?) and does not include a general-only control. A single matched control run would settle the attribution. Since this is an addressable experimental addition rather than a fundamental rejection of the resource, the appropriate verdict remains CONDITIONAL, and my concern does not move the reader's verdict.","tokens_in":33960,"tokens_out":11320,"duration_ms":109398,"concrete_test":"Train InternVL2.5-8B with the same LoRA recipe (rank 16, LR 2e-5, AdamW, cosine, warmup 0.03, weight decay 0.05, batch 64, 1 epoch) on only the 291K downsampled general samples from InternVL2.5, excluding all InternSpatial data. Evaluate this control on InternSpatial-Bench (same prompts) and VSI-Bench (same 32-frame protocol). If the control's averages are not statistically distinguishable from the original InternVL2.5-8B (58.9 and 41.6), the +12.1/+10.7 deltas are attributable to InternSpatial. If the control rises substantially, the headline gains are overstated; report the control's deltas as the true marginal effect.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim (abstract; Tables 2-3) is that training on InternSpatial yields +12.1 on InternSpatial-Bench and +10.7 on VSI-Bench. However, InternVL-Spatial-8B was fine-tuned on a mix of a downsampled version of InternVL2.5's general datasets plus InternSpatial (Section 4.1), while the baseline in Tables 2-3 is the original, unfine-tuned InternVL2.5-8B. No control model is trained on the general datasets alone, so the reported deltas conflate the effect of InternSpatial with the effect of an additional LoRA fine-tuning pass over 291K general samples. This confound applies to both the single-view and multi-view results and is independent of pseudo-label noise: even if the spatial ground truth were perfect, the causal attribution would remain underdetermined. The format ablation in Section 4.4 does not resolve this; its 'InternVL-Spatial-Raw-8B' is described as trained on 'InternSpatial-Bench' (the evaluation set) without format expansion, which would itself be leakage if taken literally, and in any case lacks a general-data-only control. A matched control training run is therefore the minimal experiment needed to support the abstract's causal claim.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces InternSpatial, a claimed 12M-pair spatial-reasoning dataset for VLMs spanning single-view and multi-view tasks, and InternSpatial-Bench, a 6,008-pair single-view benchmark, together with an extension of VSI-Bench with a rotation-angle prediction task. The dataset is built from COCO, AS-1B, Visual Genome, 3RScan, ScanNet, MultiScan, Cityscapes, Objaverse, and R2R, using a template-based QA generator with 19 instruction formats. The central empirical claim, in the abstract and in Tables 2 and 3, is that an InternVL2.5-8B model fine-tuned with InternSpatial gains 12.1 points on InternSpatial-Bench and 10.7 points on VSI-Bench relative to the untuned baseline while retaining general benchmark performance. The paper also claims a new multi-view rotation angle prediction task and reports that the trained model outperforms GPT-4o and Claude on the proposed benchmark.","tokens_in":34201,"tokens_out":6450,"duration_ms":66431,"significance":"If the causal attribution is correct, InternSpatial would be the largest open spatial-reasoning training resource for VLMs, and the reported VSI-Bench improvement would indicate transfer beyond the authors' own benchmark. The paper has strengths: a large and diverse set of sources, multi-view supervision including a large rotation-estimation component, full templates and prompts in the appendix, and a clearly specified training recipe. The external VSI-Bench gain is the most persuasive evidence for the dataset's utility. However, the internal benchmark is built with the same template and format-expansion machinery as the training data, and the main comparison lacks a general-data-only fine-tuning control, so the headline numbers cannot currently be read as a clean measurement of InternSpatial's effect. The resource's value to the community is real, but the empirical claims need re-grounding with additional controls and validation.","major_comments":[{"comment":"The reported +12.1 and +10.7 improvements compare InternVL-Spatial-8B, which is fine-tuned on a mixture of 291K general samples and InternSpatial samples, against the original untuned InternVL2.5-8B. No control model is trained on the 291K general samples alone. The deltas therefore conflate the effect of InternSpatial with the effect of an additional LoRA fine-tuning pass over general data. A matched general-data-only control is the minimal experiment needed to support the causal claim in the abstract, and it should also be applied to the multi-view VSI-Bench results.","section":"Section 4.1, Tables 2 and 3"},{"comment":"InternSpatial-Bench is constructed with the same process family as the training set: Section 3.2 says instruction formats are expanded 'following the format extension strategy described in subsection 3.1', and Appendix D explicitly states that the benchmark 'reused the processes in the second and third stages of the training dataset pipeline' for answer generation and format expansion. In addition, the benchmark draws images from COCO, ScanNet, Objaverse, and Cityscapes test sets, whose training splits are all used in InternSpatial. The 12.1-point gain on InternSpatial-Bench is therefore partly a within-pipeline, within-distribution effect and should not be presented as an external generalization result. The paper should either construct a benchmark with independent question generation and answer verification, or explicitly reframe the internal result as a format/domain consistency check and place the burden of transfer evidence on VSI-Bench.","section":"Section 3.2 and Appendix D"},{"comment":"For all image-only data sources, spatial ground truth is derived from estimated camera intrinsics, extrinsics, and dense depth using WildCamera, PerspectiveFields, and Metric3Dv2, with no reported validation of these estimates against ground truth and no audit statistics for the resulting QA labels. If these estimates are noisy, answers about near/far, above/below, and size are systematically wrong, and the model may learn the estimation models' biases rather than true spatial relations. The paper should report a validation study, for example comparing the estimated metric properties against known ground truth on a subset of ScanNet or similar data, and provide human-verification statistics for the training and benchmark QAs.","section":"Section 3.1, 'Aligning to View Space'"},{"comment":"The ablation model InternVL-Spatial-Raw-8B is described as trained on 'InternSpatial-Bench' without format expansion. Training on the evaluation benchmark is a form of leakage that invalidates the comparison and the conclusion about cross-format transfer. If this is a typo and the intended training set is InternSpatial (the training set), the text must be corrected; otherwise, the ablation should be redone using a training split that is disjoint from the evaluation benchmark.","section":"Section 4.4"},{"comment":"The VSI-Bench result is the strongest evidence of transfer, but the paper does not report whether any VSI-Bench test videos are drawn from or derived from the multi-view training sources ScanNet, MultiScan, R2R, or Objaverse, all of which are used to build InternSpatial's multi-view training data. A distributional-overlap analysis for VSI-Bench should be reported so that the +10.7 gain can be interpreted as genuinely external.","section":"Section 4.3"}],"minor_comments":[{"comment":"The text first states that InternSpatial-Bench has 'a total of 5,300 high-quality examples', but the later statistics and Table 7 report 6,008 QA pairs; these numbers should be reconciled.","section":"Section 3.2"},{"comment":"The table heading says 'MathVision' while the text refers to 'MathVista [35]'; the benchmark name and reference should be made consistent and correct.","section":"Table 4 and Section 4.5"},{"comment":"The model names 'LLaV A-NeXT-Video-72B' and 'LLaV A-OneVision-72B' contain typographical spacing errors and should be corrected to 'LLaVA-NeXT-Video-72B' and 'LLaVA-OneVision-72B'.","section":"Table 3"},{"comment":"The claim that rotation angle prediction is a 'novel' multi-view task should be substantiated with a comparison to existing object pose and multi-view geometry work, since angle prediction is a standard task in those fields.","section":"Section 1"},{"comment":"The statement that training uses 'approximately 291K general training samples' and '2M samples from InternSpatial, counted with repetition' should clarify whether these are unique samples or include repeated examples, since repetition affects the reported training mix.","section":"Appendix E"},{"comment":"The paper describes InternSpatial as 'open-source' but does not provide a dataset release URL or hosting repository; this should be included for reproducibility.","section":"Abstract and Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"This is a dataset/benchmark paper whose headline result is a 12.1-point gain on the authors' own benchmark. The paper's appendix explicitly states that the benchmark reuses the training pipeline's answer-generation and format-expansion processes, so the internal gain is substantially a same-family effect. The VSI-Bench improvement is the key external evidence, but the missing general-data-only control prevents causal attribution. I recommend requiring: (1) a matched general-data-only fine-tuning control for both evaluated benchmarks, (2) an overlap analysis between InternSpatial training sources and VSI-Bench test videos, (3) a corrected and cleaned ablation in Section 4.4, and (4) a validation of the pseudo-ground-truth metrics. These are all within the scope of a revision, so I do not recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, the short version: this is a serious dataset effort with a genuinely new multi-view rotation task, but the central causal claim — that training on InternSpatial gives +12.1/+10.7 — is underdetermined by the experiments as reported. The paper deserves peer review, but it needs a matched control before I'd trust the headline.\n\nWhat's genuinely new: 12M QA pairs, the largest open spatial-reasoning training set (SpatialVLM's 2B is closed), spanning single- and multi-view settings with 19 instruction formats. The rotation-angle prediction task with 2.46M pairs is new, as is the 1,000-pair extension to VSI-Bench. The external VSI-Bench improvement is the strongest evidence; if it survives a proper ablation, it's an important result for robotics and embodied AI. The general VQA table shows no catastrophic forgetting, which is useful.\n\nThe soft spots are real, and one is load-bearing. Section 4.1 fine-tunes from InternVL2.5-8B on a mix of 291K general samples plus 2M InternSpatial samples, but the baseline in Tables 2-3 is the original, unfine-tuned InternVL2.5-8B. There is no control trained on the general samples alone. So the reported +12.1 and +10.7 conflate the effect of InternSpatial with the effect of any LoRA fine-tuning on general data. This is independent of label noise: even if every spatial ground truth were perfect, the causal attribution remains underdetermined. The format ablation in Section 4.4 does not resolve it; it lacks the same control.\n\nAdditional, smaller concerns: InternSpatial-Bench is built by reusing the training pipeline's template and format-expansion machinery (Section 3.2 and Appendix D), and its images come from sources whose training splits are in InternSpatial — that makes the benchmark same-family. The single-view spatial answers inherit pseudo-ground-truth from WildCamera, PerspectiveFields, and Metric3Dv2; if those are biased, the model can learn the estimators' biases. There are internal inconsistencies (Section 3.2 says 5,300 benchmark pairs, statistics say 6,008). No code or data release accompanies the open-source claim, and no error bars appear anywhere.\n\nWho this is for: people working on spatial-reasoning data pipelines or VLM evaluation methodology will get value, and the rotation task could seed new benchmarks. A serious referee should require the control training run, a cleaner separation between benchmark and training pipeline, and a release commitment. I would send it to review, conditional on those fixes.","headline":"A genuinely large spatial-reasoning dataset with a new rotation-angle task, but the headline +12.1/+10.7 gains are confounded by the missing general-data-only control run.","tokens_in":34814,"tokens_out":2387,"would_cite":false,"duration_ms":21319,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training on InternSpatial, a 12-million-question open dataset with 19 spatial instruction formats, lifts an 8-billion-parameter VLM past proprietary frontier models on spatial benchmarks without hurting general performance.","keywords":["spatial reasoning","vision-language models","dataset","benchmark","multi-view reasoning","rotation angle prediction","instruction formats","3D scene understanding"],"falsifier":"Take a held-out set of images with known 3D ground truth, corrupt the estimated depth and camera parameters by increasing amounts, regenerate the InternSpatial QA labels from the corrupted geometry, and fine-tune identical models on each version; if benchmark gains fall as label noise rises, the claimed spatial learning is carried by the estimated geometry rather than by the dataset's design.","tokens_in":33755,"feed_emoji":"📐","tokens_out":6510,"duration_ms":60334,"temperature":0.7,"pith_summary":"The paper claims that spatial reasoning in vision-language models can be taught at scale by a large, open, automatically generated question-answer resource. It introduces InternSpatial, 12 million QA pairs spanning single-view and multi-view settings, plus InternSpatial-Bench, a diagnostic evaluation set in 19 instruction formats. A model fine-tuned on InternSpatial improves by 12.1 points on the authors' benchmark and by 10.7 points on the external VSI-Bench multi-view benchmark, while general-purpose scores stay roughly flat. If correct, this makes InternSpatial the largest open training resource of its kind and suggests that multi-view supervision and instruction-format diversity are the active ingredients. The authors also introduce a rotation-angle prediction task with 2.46 million pairs that has not appeared in prior spatial benchmarks.","feed_headline":"12M spatial QA pairs lift VLM benchmark scores by 12 points","feed_subtitle":"Training on the open dataset also adds 10.7 points on an external multi-view benchmark while general skills hold.","key_machinery":"The load-bearing mechanism is the canonical view space: a 3D coordinate frame centered at the camera's optical center, with the y-axis along the viewing direction and the z-axis pointing upward, into which every object annotation is lifted. For 3D datasets this is a direct projection; for image-only datasets the pipeline estimates camera intrinsics, camera extrinsics, and dense metric depth, then lifts 2D boxes, masks, and captions into this frame so that spatial answers are computed geometrically. Around that frame, the data engine applies template-based QA generation, expands each QA into up to 19 textual and visual instruction formats, and adds multi-view tasks built from scanned scenes and object renderings, including the new rotation-angle prediction task. The geometry-grounded answers, not the templates themselves, are what the paper credits for the transfer.","core_discovery":"The central claim is that a spatially capable VLM can be produced by supervised fine-tuning on InternSpatial: a fully automated pipeline takes images and 3D scans from diverse sources, lifts 2D object annotations into a canonical 3D view space, and converts them into templated questions and answers. The authors state that the resulting 8-billion-parameter model, trained with roughly 2 million sampled InternSpatial pairs plus general instruction data, outperforms the base model by 12.1 percentage points on InternSpatial-Bench and by 10.7 points on VSI-Bench, and beats the proprietary frontier models they compare against. They interpret this as evidence that scale, scene diversity, and 19 instruction formats teach transferable spatial competence rather than benchmark-specific answers.","pith_inferences":["If the measured gains reflect genuine spatial understanding rather than template patterns, the dataset should transfer to downstream robotics tasks such as navigation and manipulation; the paper reports no such evaluation.","The image-only QA labels inherit errors from monocular depth and camera estimation, so filtering or reweighting samples by estimated label confidence could make training more robust.","A human-written, paraphrase-diverse holdout set would test whether models trained on templated questions generalize to free-form spatial language.","Extending the multi-view pipeline to egocentric video with measured camera trajectories could supply stronger supervision than estimated depth and test the same method's ceiling."],"forward_implications":["Fine-tuning on InternSpatial raises the base 8-billion-parameter model by 12.1 points on InternSpatial-Bench, with the largest jumps in position comparison (+25.0) and size comparison (+20.9).","The same training transfers to the external multi-view benchmark VSI-Bench, lifting the average by 10.7 points with double-digit gains in object counting, object size estimation, and appearance order.","Training on all 19 instruction formats closes most of the format gap and also improves plain natural-language performance beyond training on raw questions alone.","General multimodal ability is preserved: five standard benchmarks change within roughly -1.6 to +1.8 points after spatial training.","The dataset introduces a rotation-angle prediction task with 2.46 million QA pairs, extending multi-view evaluation beyond existing benchmarks.",""],"supporting_citations":[{"why":"Supplies the SpatialBench task definitions and one of the benchmarks InternSpatial-Bench expands.","marker":"[1]"},{"why":"Supplies the camera-and-depth estimation pipeline for image-only data and the SpatialRGPT-Bench being refined.","marker":"[2]"},{"why":"Prior automatic spatial VQA dataset that is not public; InternSpatial is positioned as the open successor.","marker":"[3]"},{"why":"The external multi-view benchmark used to measure transfer.","marker":"[4]"},{"why":"The base 8-billion-parameter model and general instruction data used for fine-tuning.","marker":"[23]"},{"why":"Generates segmentation masks used to create mask-referenced instruction formats and 3D lifting.","marker":"[29]"},{"why":"Estimates camera intrinsics for image-only sources.","marker":"[30]"},{"why":"Estimates camera extrinsics for image-only sources.","marker":"[31]"},{"why":"Predicts dense metric depth maps used to lift 2D annotations into the canonical view space.","marker":"[32]"}],"fun_headline_variants":["InternSpatial: 12M QA pairs boost spatial VLM reasoning by 12%","12M spatial QA pairs lift VLM benchmarks by 12 points","Spatial dataset with 19 instruction formats lifts VLM scores 12%","Open dataset lifts spatial VLMs by 12.1 points, beats frontier models","New spatial reasoning dataset improves VLMs by 12%, diverse scenes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For image-only sources, the ground-truth answers are computed from estimated camera parameters and estimated depth; if those estimates are biased, the model learns the estimators' geometry rather than true spatial relations.","fun_headline_variants_meta":{"raw":{"variants":["InternSpatial: 12M QA pairs boost spatial VLM reasoning by 12%","12M spatial QA pairs lift VLM benchmarks by 12 points","Spatial dataset with 19 instruction formats lifts VLM scores 12%","Open dataset lifts spatial VLMs by 12.1 points, beats frontier models","New spatial reasoning dataset improves VLMs by 12%, diverse scenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000396,"raw_usage":{"total_tokens":2055,"prompt_tokens":904,"completion_tokens":1151,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":1048}},"tokens_in":520,"tokens_out":1151,"duration_ms":10127,"temperature":1.0,"reasoning_tokens":1048,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:49:36.286680+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of images with known 3D ground truth, corrupt the estimated depth and camera parameters by increasing amounts, regenerate the InternSpatial QA labels from the corrupted geometry, and fine-tune identical models on each version; if benchmark gains fall as label noise rises, the claimed spatial learning is carried by the estimated geometry rather than by the dataset's design.","supporting_citations":[{"cited_title":"Spatialbot: Precise spatial understanding with vision language models","cited_arxiv_id":null,"evidence_quote":"Supplies the SpatialBench task definitions and one of the benchmarks InternSpatial-Bench expands."},{"cited_title":"Spatialrgpt: Grounded spatial reasoning in vision-language models","cited_arxiv_id":null,"evidence_quote":"Supplies the camera-and-depth estimation pipeline for image-only data and the SpatialRGPT-Bench being refined."},{"cited_title":"Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie","cited_arxiv_id":null,"evidence_quote":"The external multi-view benchmark used to measure transfer."},{"cited_title":"Tame a wild camera: In-the-wild monocular camera calibration","cited_arxiv_id":null,"evidence_quote":"Estimates camera intrinsics for image-only sources."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Estimates camera extrinsics for image-only sources."}],"review_version":1}