Pith. sign in

REVIEW 4 major objections 6 minor 95 references

VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation

T0 review · 4 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A compact 3D Gaussian scene memory, compressed to 64 tokens, lifts real-robot manipulation success by 22.8 points over 2D and depth baselines, the authors report.

desk verdict A credible 3D-Gaussian VLA interface with a clean compression scheme, but the semantic-grounding attribution isn't isolated by the experiments. read the letter →

arxiv 2607.12356 v2 pith:CXZL3LZM submitted 2026-07-14 cs.RO

Mohan Liu , Zhihao Gu , Xuanyu Chen , Haitian Zhang , Kaimin Mao , Yan Wu , Wei-Yun Yau , Lin Wang This is my paper · ORCID
classification cs.RO
keywords 3DGaussiansplattingvision-language-actionmodelsroboticmanipulationsemanticscenerepresentationtokencompressionspatialgeneralizationmulti-viewfeaturedistillationLIBERObenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VistaVLA argues that vision-language-action (VLA) robots fail at precise manipulation because they see the world as flat 2D images, lacking an explicit, semantics-aware 3D scene model. The paper shows that lifting language-aligned visual features into a multi-view-consistent Gaussian field, then compressing that field to just 64 tokens, improves real-robot success by 22.8 percentage points over a strong 2D baseline across seven tasks. The key evidence that the gains come from the 3D grounding: adding more 2D camera views or raw depth improves the baseline only slightly, while the 3D-token policy is the only one to succeed at all on a position-perturbed task where every baseline scores zero. A reader should care because this is a concrete recipe for giving generalist robot policies genuine 3D spatial reasoning without blowing up their compute budget.

What carries the argument

The load-bearing object is the semantic Gaussian field: each 3D Gaussian primitive carries a 128-dimensional feature vector, trained so that alpha-composited rendering of these features matches a compact teacher representation distilled from language-aligned and self-supervised visual encoders. This binds semantics to concrete 3D positions, visible from any viewpoint. The second mechanism is Merge-then-Query (MtQ), which compresses roughly 10^5 primitives to 64 tokens by Morton-order spatial partitioning, similarity-based merging of redundant tokens, and a two-layer query transformer. MtQ's job is to preserve action-relevant spatial layout while cutting the token count by 99%, making 3D grou

What would settle it

Give the VLA-Adapter baseline the same two calibrated views but replace its 2D patch tokens with 64 tokens produced by the identical teacher-feature distillation and MtQ compression applied to a pseudo-3D input such as depth-composited per-view features; if that baseline then matches VistaVLA's success rates, especially the 3/10 on the position-perturbed task, the Gaussian field itself is not the causal ingredient.

Watch

Extended reading notes

Core claim

The paper's central claim is that a scene-level, geometry-anchored semantic representation—Gaussian primitives whose 128-dimensional latent codes are distilled from foundation visual encoders under multi-view RGB-D supervision—serves as a better context for VLA policy learning than 2D patch tokens or raw depth. Once the dense field (about 10^5 primitives) is summarized by the Merge-then-Query mechanism into 64 query tokens, the policy can plan over spatial layouts and semantic identities together. The authors assert this is why VistaVLA outperforms the VLA-Adapter baseline by 22.8 points across seven real-world tasks, improves depth-perturbed success from 6/10 to 9/10, and is the only method

Load-bearing premise

The load-bearing premise is that the gains come from the 3D semantic Gaussian tokens themselves, not from the extra multi-view reconstruction pipeline; the paper's own limitation note concedes that pipeline depends on fixed calibrated cameras, which complicates a fair comparison against baselines that receive the same two views more naively.

Editorial extensions

If this is right

  • VLA policies can gain explicit 3D spatial reasoning without a large increase in inference cost: 64 summary tokens replace hundreds of thousands of Gaussian primitives (a 99% token reduction) while retaining action-relevant layout and semantics.
  • Robustness to spatial perturbations improves sharply: on a position-shifted task every 2D and depth-based baseline scores 0/10 while VistaVLA scores 3/10, and depth-perturbed success rises from 6/10 to 9/10.
  • The gains are attributable to the 3D-grounded semantic interface, not to extra inputs: ablations show that adding 2D camera views (up to 768 tokens) or raw depth improves the baseline only marginally, while VistaVLA's roughly 32 tokens do far better.
  • The approach transfers to simulation: on LIBERO-Pro-Swap spatial out-of-distribution layouts, VistaVLA raises the baseline from 1.7% to 12.2%, and on standard LIBERO it reaches 96.05% average success without large-scale robot-action pretraining.
  • A 0.5B-parameter policy with the Gaussian context can surpass a 3B generalist policy on spatially demanding real-world tasks, suggesting the spatial interface partly substitutes for raw model scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the Gaussian field is viewpoint-invariant, a natural extension the authors do not pursue is persistent scene memory—updating the same field across time rather than rebuilding it each step—which could give robots temporally stable spatial reasoning under occlusion or camera motion.
  • Editorial inference: the MtQ compression result suggests that action-relevant 3D context is highly redundant; the same merge-then-query principle could be applied to other dense 3D representations (point clouds, voxel grids, depth maps) to test whether the benefit is specific to Gaussian primitives or generalizes.
  • Editorial inference: a cheap control experiment would feed the same distilled teacher features into the baseline as depth-augmented 2D tokens; if the margin narrows, the semantic distillation rather than the 3D field is doing the work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. VistaVLA proposes a two-stage framework that constructs a 3D Gaussian scene representation whose primitives carry distilled 2D foundation-model semantic features, then summarizes the dense primitives with Merge-then-Query (MtQ) into 64 compact tokens injected into a VLA policy. Stage I trains a Gaussian field with RGB-D reconstruction losses and a teacher-feature rendering loss (Sec. 3.1); Stage II performs stride pooling, Morton-order-based merging, and query-based decoding to produce the final context tokens (Sec. 3.2). The method is evaluated on seven real-world tabletop tasks and on LIBERO/LIBERO-Pro-Swap, reporting a 22.8-point average real-world improvement over VLA-Adapter and a 3/10 vs 0/10 success on a position-perturbed test where all baselines fail.

Significance. The paper attacks a timely problem—giving VLA policies an explicit, compact, semantic 3D scene representation—and the proposed MtQ compression is simple, largely parameter-free, and clearly described. The real-world perturbed-task evaluation is a good direction, and the authors are candid about calibration and scope limitations. If the central attribution claim were established, this would be a useful contribution to 3D-grounded VLA. However, that claim is not yet supported: the experiments lack a matched ablation isolating semantic content, the real-world numbers rest on 10-trial point estimates, the baseline comparison may confound representation with pipeline maturity, and no code/data/parameters are released to make the token-reduction and semantic-preservation claims externally checkable.

major comments (4)
  1. [§4.3, Table 3(b); §4.1, Fig. 4] The headline attribution—'gains do not simply come from injecting geometric cues or adding more camera views, but from introducing a structured 3D Gaussian representation as a semantic 3D spatial interface'—is not supported by the experiments. No ablation removes semantic content from the same GS/MtQ pipeline: every row in Table 3(b) uses distilled SigLIP2+DINOv2 features, and VLA-Adapter+Depth is a 2D depth-augmented baseline rather than a GS pipeline without semantics. A variant with random or non-semantic Gaussian features, using the identical multi-view reconstruction and MtQ compression, is needed to show that the learned semantic teacher features, not the 3D geometry/tokenization, are load-bearing.
  2. [§4.1, Fig. 4, Table 1; §4.3, Table 3] All real-world results are point estimates from 10 trials per task, with no per-seed variance, confidence intervals, or statistical tests. Several decisive comparisons are one or two successes apart (e.g., Table 3(a) T1: 9/10 vs 8/10; Table 1 Depth: 9/10 vs 7/10), and the position-perturbation difference is 3/10 vs 0/10. Under a binomial model these differences can be within sampling noise. Reporting more trials, repeated seeds, exact confidence intervals, or at least per-trial logs is necessary to support the 22.8-point and 30-point claims.
  3. [§4.1, Setup; Fig. 4] The fairness of the comparison is under-specified. The baseline is 'equipped with the same number of 2 fixed camera views' in a naive 2D-token manner, while VistaVLA receives calibrated multi-view RGB-D reconstruction, a pretrained DepthSplat encoder, teacher-feature distillation, and an online Gaussian field. No inference latency, reconstruction failure rate, or per-method compute budget is reported. The observed 22.8-point margin could therefore reflect pipeline maturity or additional engineering rather than the 3D semantic representation. A stronger 2D baseline with the same per-view feature encoders and token budget, or a matched 3D geometric baseline, is needed to isolate the claimed cause.
  4. [Abstract; §1; §3.2] The claim of '99% token reduction while preserving action-relevant 3D layouts and semantic context' overstates what is measured. No direct evaluation of semantic fidelity or layout preservation of the compressed tokens is provided; the only evidence is downstream task success. I recommend adding a direct fidelity metric (e.g., reconstruction of the teacher features or semantic segmentation/retrieval consistency from the 64 query tokens) or softening the claim to 'without degrading policy performance in the tested tasks.'
minor comments (6)
  1. [§2] Typo: 'By constrast' should be 'By contrast.' Also, the method name alternates between 'VistaVLA' and 'Vista-VLA'; please use one consistent form.
  2. [Table 3(a)] The row 'Vista-VLA-3Cam 3209/10 7/10 7/10 7/10' is ambiguous: the token count 320 and the success count 9/10 run together. Insert a space or use a separate column.
  3. [Fig. 4] Success rates are shown without confidence intervals or error bars. For 10-trial data, at least add the raw counts or a binomial CI to make the visual comparison informative.
  4. [§4.2, Table 2(a)] The term 'zero-shot transfer on LIBERO-Pro-Swap' is not defined in the setup. Is the same standard-LIBERO checkpoint evaluated directly on Pro-Swap without any adaptation or fine-tuning? Please state the protocol explicitly.
  5. [§3.1] The teacher auto-encoder is described as 'trained once,' but the training data, loss weighting, and schedule are not specified. Please clarify, since the 2176→128 bottleneck is part of the method.
  6. [§3.2] The choice of the compression target N'=1000, the top-50% merge fraction, and the stride-4 pooling are not swept or motivated beyond the main ablations. Even a short sensitivity discussion would help, since these are free design parameters.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the method is an empirical VLA pipeline whose gains are claimed from comparative experiments, not derived from its own fitted quantities.

full rationale

VistaVLA's pipeline is a two-stage empirical system: Stage I distills SigLIP2/DINOv2 features into 3D Gaussian primitives using an RGB-D rendering loss, and Stage II compresses the resulting tokens with Merge-then-Query before injecting them into a VLA policy. The paper's central claim is that this representation improves real-world and simulated manipulation success rates. That claim is supported by external benchmarks, baselines, and ablations rather than by a derivation that reduces to the inputs. There is no equation in which a predicted quantity is defined as the fitted parameter, no fitted input renamed as a prediction, and no uniqueness theorem imported from the authors' prior work to force the design. The closest concerns are (a) the absence of a control that removes only the semantic teacher features from the same 3D-Gaussian/MtQ pipeline, and (b) references [17,19] to the authors' own prior work for the '3D semantic cognitive map' motivation. Neither is circular: (a) is an evidence/fairness gap about whether geometry or semantics drives the improvement, not an internal logical reduction; (b) is a cognitive-science framing citation that is not load-bearing for the quantitative results. The claim that 'the gains do not simply come from injecting geometric cues or adding more camera views, but from introducing a structured 3D Gaussian representation as a semantic 3D spatial interface' is an interpretation of the experimental comparison, not a result forced by construction. Accordingly, the paper does not exhibit the specific reduction required for a circularity finding.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical or ontological entities are introduced. The conceptual invention is the MtQ token pipeline and the 'cognitive map' framing; the free parameters are all standard architectural choices, but several are unablated or unreported (loss weights), which raises the reproduction burden.

free parameters (6)
  • Teacher bottleneck dimension (128-d) = 128
    Chosen to compress SigLIP2+DINOv2 2176-d features; no ablation of this capacity; the 128-d GS token is the main information channel into the policy.
  • Compression target N' = 1000 = 1000
    Intermediate post-merge token count chosen in Sec. 3.2 without a sweep.
  • Query token count Nq = 64 = 64
    Ablated in Table 3(c) on one task; 64 wins, 256 is worst; selected per task-specific data.
  • Top-50% merge fraction = 50%
    Heuristic gate in Morton-guided merging; no sensitivity analysis.
  • Action execution chunk K_exec = 4 = 4
    Inference-frequency choice described in Sec. 3.2; no sweep reported.
  • Loss weights (lambda_rgb, lambda_dep, lambda_feat) = unspecified
    Referenced in Sec. 3.1 but never given numeric values, making the reconstruction/distillation balance unreproducible.
assumptions (5)
  • domain assumption Two calibrated multi-view RGB-D views at training time and calibrated poses at inference are available per control step.
    The entire Gaussian-field construction in Sec. 3.1 and the online scene build at inference depend on posed observations; the paper's own limitation section states this.
  • domain assumption DepthSplat [60] provides a feed-forward Gaussian reconstruction from the posed RGB/D inputs that is stable enough online.
    VistaVLA's Stage I uses DepthSplat as the Gaussian backbone; reconstruction quality is not independently evaluated in the paper.
  • domain assumption SigLIP2 and DINOv2-Large features, compressed to 128-d by a separately trained auto-encoder, retain the semantics needed for manipulation.
    The whole semantic grounding rests on this distillation; no evaluation of feature-reconstruction fidelity is provided.
  • domain assumption Language-conditioned cross-attention ordering [X_vis, X_lang, X_gs, X_act] is a sufficient interface for injecting 3D context.
    Used in Sec. 3.2 without comparison to other injection orders.
  • domain assumption LIBERO-Pro-Swap is a valid zero-shot spatial-transfer benchmark and the reported baseline numbers are comparable given matched training budget.
    The simulation claims depend on this benchmark's fairness; several baselines score 0.0, which is consistent with known benchmark difficulty but also with protocol differences.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation." pith.science (2026). https://pith.science/paper/CXZL3LZM

@misc{pith2026260712356,
  author       = {Pith},
  title        = {Pith review of: VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CXZL3LZM}},
  note         = {Machine review of arXiv:2607.12356}
}
read the original abstract

Vision-Language-Action (VLA) models have emerged as a powerful end-to-end paradigm for robotic manipulation by mapping language instructions and 2D visual inputs directly to actions. However, these models lack an explicit, scene-level 3D representation, limiting their ability to reason over spatial layouts and geometric constraints. While recent efforts incorporate explicit 3D cues, such as depth maps or point clouds, to improve geometric awareness, they primarily capture low-level structures and lack high-level semantic grounding in 3D space. In human cognition, interaction with the physical world relies on a 3D semantic cognitive map - an internal mental model that integrates spatial layouts with semantic context to enable persistent, viewpoint-invariant reasoning. In light of this, we present VistaVLA, a novel two-stage framework that constructs a geometry- and semantics-aware 3D cognitive representation from 3D Gaussian primitives and grounds it as compact context tokens for VLA policy learning. Specifically, VistaVLA lifts multi-view vision-language features into 3D Gaussian primitives, forming geometry-anchored semantic tokens that align view-consistent spatial grounding with 2D visual feature spaces. To make this 3D representation computationally tractable for effective VLA control, we introduce Merge-then-Query (MtQ), a token summarization mechanism. MtQ compresses dense Gaussian primitives into a highly compact set of spatially informative tokens, achieving a 99% token reduction while preserving action-relevant 3D layouts and semantic context. Extensive evaluations in both simulated and real-world environments demonstrate the effectiveness of VistaVLA. Notably, in real-world scenarios, VistaVLA improves success rates by 22.8% across seven real-world tasks and by 30.0% over the VLA-Adapter baseline on challenging out-of-distribution tasks.

Figures

Figures reproduced from arXiv: 2607.12356 by the authors.

Figure 1
Figure 1. VistaVLA bridges 3D geometric structure and semantic grounding for VLA manip￾ulation. (a) We compare current VLA paradigms: 2D-input VLAs lack explicit 3D structure, while existing 3D-input VLAs mainly provide geometric cues without sufficient semantic grounding in 3D space. VistaVLA addresses both aspects by constructing semantically grounded 3D Gaussian scene representations for policy learning. (b) This design im… view at source ↗
Figure 2
Figure 2. Overview of VistaVLA. Stage I: Semantic Gaussian field training distills foundation visual features under RGB-D rendering supervision. Stage II: Merge-then-Query compresses dense Gaussian primitives into compact 3D summary tokens. These tokens are finally injected into the VLA policy with instruction and wrist-image tokens for spatially grounded action prediction. For each training frame, we extract dense SigLIP2 [8… view at source ↗
Figure 3
Figure 3. Illustration of real-world tasks. Each row shows keyframes of the corresponding manip￾ulation trajectory, covering diverse object arrangements, interaction patterns, and temporal horizons. This makes each GS token a multi-view semantic descriptor grounded in 3D space. During train￾ing, we use two calibrated RGB-D views as context and render the third view as a novel target to enforce cross-view consistency. At infer… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Real-world results. All policies are trained or fine-tuned on the same task-specific demon￾strations. Our method consistently achieves the highest success rates across all tasks, with especially large gains on spatially demanding manipulation scenarios. Results are rep…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

95 extracted references · 2 canonical work pages

  1. [1]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, Y . L. Tan, L. Y . Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine. Octo: An open-source generalist robot policy. 2024. URL https://arxiv.org/abs/2405.12213

    arXiv 2024

  2. [2]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. Openvla: An open-source vision-language-action model. 2024. URL https://arxiv.org/abs/2406.09246

    arXiv 2024

  3. [3]

    M. J. Kim, C. Finn, and P. Liang. Fine-tuning vision-language-action models: Optimizing speed and success. 2025. URLhttps://arxiv.org/abs/2502.19645

    arXiv 2025

  4. [4]

    Z. Gu, M. Yang, D. Zou, and D. Xu. Learning diffusion policy from primitive skills for robot manipulation. InProceedings of the AAAI Conference on Artificial Intelligence, page 18262–18270, 2026

    2026

  5. [5]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Haus- man, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky.π 0: A vision-language-action flow model for general robot control, 2026. URLhttps://arxiv. o...

    arXiv 2026

  6. [6]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, M. Y . Galliker, D. Ghosh, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, D. LeBlanc, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, A. Z. Ren, L. X. Shi, L. Smith, J. T. Springenberg, K. Stachowicz, J. Tanner, Q. V...

    arXiv 2025

  7. [7]

    Y . Wang, P. Ding, L. Li, C. Cui, Z. Ge, X. Tong, W. Song, H. Zhao, W. Zhao, P. Hou, S. Huang, Y . Tang, W. Wang, R. Zhang, J. Liu, and D. Wang. Vla-adapter: An effective paradigm for tiny- scale vision-language-action model, 2025. URLhttps://arxiv.org/abs/2509.09372

    arXiv 2025

  8. [8]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Proceedings of The 7th Conference on Robot Learning, volume 229 ofProceedings of Machine Learning Research, pages 2165–2183, 2023

    2023

Show all 95 references
  1. [9]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    2022 arXiv

  2. [10]

    Bhat, Y .-H

    V . Bhat, Y .-H. Lan, P. Krishnamurthy, R. Karri, and F. Khorrami. 3d cavla: Leveraging depth and 3d context to generalize vision language action models for unseen tasks. 2025. URL https://arxiv.org/abs/2505.05800

    2025

  3. [11]

    Y . Li, Y . Chen, M. Zhou, H. Li, Z. Zhang, and D. Zhao. Qdepth-vla: Quantized depth prediction as auxiliary supervision for vision-language-action models, 2025. URLhttps://arxiv. org/abs/2510.14836

    2025 arXiv

  4. [12]

    D. Qu, H. Song, Q. Chen, Y . Yao, X. Ye, Y . Ding, Z. Wang, J. Gu, B. Zhao, D. Wang, and X. Li. Spatialvla: Exploring spatial representations for visual-language-action model. 2025. URLhttps://arxiv.org/abs/2501.15830

    2025 arXiv

  5. [13]

    C. Li, J. Wen, Y . Peng, Y . Peng, F. Feng, and Y . Zhu. Pointvla: Injecting the 3d world into vision-language-action models. 2025. URLhttps://arxiv.org/abs/2503.07511. 9

    2025 arXiv

  6. [14]

    F. Li, W. Song, H. Zhao, J. Wang, P. Ding, D. Wang, L. Zeng, and H. Li. Spatial forcing: Implicit spatial representation alignment for vision-language-action model.arXiv preprint arXiv:2510.12276, 2025

    2025

  7. [15]

    D. H. Clements and M. T. Battista. Geometry and spatial reasoning. 2006

    2006

  8. [16]

    R. M. Byrne and P. N. Johnson-Laird. Spatial reasoning.Journal of memory and language, 28 (5):564–575, 1989

    1989

  9. [17]

    B. D. Manh, S. Debnath, Z. Zhang, S. Damodaran, A. Kumar, Y . Zhang, L. Mi, E. Cambria, and L. Wang. Mind meets space: Rethinking agentic spatial intelligence from a neuroscience- inspired perspective.arXiv preprint arXiv:2509.09154, 2025

    2025 arXiv

  10. [18]

    Zhang, M

    W. Zhang, M. Wang, G. Liu, X. Huixin, Y . Jiang, Y . Shen, G. Hou, Z. Zheng, H. Zhang, X. Li, et al. Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks.arXiv preprint arXiv:2503.21696, 2025

    2025 arXiv

  11. [19]

    S. Ruan, L. Wang, C. Kang, Q. Zhu, S. Liu, X. Wei, and H. Su. From reactive to cognitive: brain-inspired spatial intelligence for embodied agents.intelligence (AGI), 3(9):10, 2025

    2025

  12. [20]

    Kerbl, G

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4):1–14, 2023. doi:10.1145/ 3592433

    2023

  13. [21]

    Dong, K.-H

    P. Dong, K.-H. Hung, T. Gao, D. Sadigh, and C. Finn. Expo-ft: Sample-efficient reinforcement learning finetuning for vision-language-action models, 2026. URLhttps://arxiv.org/ abs/2605.25477

    2026 arXiv

  14. [22]

    Jin and Z

    R. Jin and Z. Zhang. Agentic-vla: Efficient online adaptation for vision-language-action mod- els, 2026. URLhttps://arxiv.org/abs/2605.22896

    2026 arXiv

  15. [23]

    Y . Fu, C. Zhang, S. Gong, Y . Deng, K. Sun, Q. Min, Q. Hou, Y . Tang, J. Wang, and D. Zhou. Stablevla: Towards robust vision-language-action models without extra data, 2026. URL https://arxiv.org/abs/2605.18287

    2026 arXiv

  16. [24]

    X. Guo, B. Xie, W. Chai, X. Deng, T. Wang, Z. Wu, and X. Chen. Priorvla: Prior-preserving adaptation for vision-language-action models, 2026. URLhttps://arxiv.org/abs/2605. 10925

    2026

  17. [25]

    Q. Li, X. Gong, X. Li, P. Li, Q. Zhou, H. Ye, J. Zhou, and Y . Mu. Rotvla: Rotational latent ac- tion for vision-language-action model, 2026. URLhttps://arxiv.org/abs/2605.13403

    2026 arXiv

  18. [26]

    Z. Tang, H. Liu, X. Chang, C. Wu, D. Huo, Y . Yang, B. Liu, Z. Cai, F. Xiong, M. Xu, J. Luo, D. Ma, Z. Ma, and G. Pan. Alam: Algebraically consistent latent action model for vision- language-action models, 2026. URLhttps://arxiv.org/abs/2605.10819

    2026 arXiv

  19. [27]

    Jiang, J

    Y . Jiang, J. Lu, X. Qin, X. Chen, K. Wang, F. Gao, and L. Zhao. Vla-gse: Boosting parameter- efficient fine-tuning in vla with generalized and specialized experts, 2026. URLhttps:// arxiv.org/abs/2605.06175

    2026 arXiv

  20. [28]

    S. Lian, B. Yu, X. Lin, Z. Shen, L. T. Yang, Y . Jin, H. Liu, C. Wu, H. Yuan, C. Huang, and K. Chen. Intentvla: Short-horizon intent modeling for aliased robot manipulation, 2026. URL https://arxiv.org/abs/2605.14712

    2026 arXiv

  21. [29]

    S. Fan, K. Wu, Z. Che, X. Wang, D. Wu, F. Liao, N. Liu, Y . Zhang, Z. Zhao, Z. Xu, M. Li, Q. Liu, S. Zhang, M. Wan, and J. Tang. Xr-1: Towards versatile vision-language-action models via learning unified vision-motion representations, 2025. URLhttps://arxiv.org/abs/ 2511.02776. 10

    2025 arXiv

  22. [30]

    M. Koo, D. Choi, T. Kim, K. Lee, C. Kim, Y . Seo, and J. Shin. Hamlet: Switch your vision- language-action model into a history-aware policy, 2025. URLhttps://arxiv.org/abs/ 2510.00695

    2025 arXiv

  23. [31]

    L. Xiao, J. Li, J. Gao, F. Ye, Y . Jin, J. Qian, J. Zhang, Y . Wu, and X. Yu. Ava-vla: Improving vision-language-action models with active visual attention, 2025. URLhttps://arxiv. org/abs/2511.18960

    2025 arXiv

  24. [32]

    Y . Li, P. Tang, W. Zhang, C. Zhu, Y . Duan, W. Shi, X. Zhang, Z. Yang, J. Ji, and Y . Zhang. Favla: A force-adaptive fast-slow vla model for contact-rich robotic manipulation, 2026. URL https://arxiv.org/abs/2602.23648

    2026

  25. [33]

    Z. Yang, Y . Qi, T. Xie, B. Yu, S. Liu, and M. Li. Dysl-vla: Efficient vision-language-action model inference via dynamic-static layer-skipping for robot manipulation, 2026. URLhttps: //arxiv.org/abs/2602.22896

    2026

  26. [34]

    Zhong, Y

    L. Zhong, Y . Liu, Y . Wei, Z. Xiong, M. Yao, S. Liu, and G. Ren. Acot-vla: Action chain- of-thought for vision-language-action models, 2026. URLhttps://arxiv.org/abs/2601. 11404

    2026

  27. [35]

    H. Zhen, X. Qiu, P. Chen, J. Yang, X. Yan, Y . Du, Y . Hong, and C. Gan. 3d-vla: A 3d vision-language-action generative world model. 2024. URLhttps://arxiv.org/abs/ 2403.09631

    2024 arXiv

  28. [36]

    L. Sun, B. Xie, Y . Liu, H. Shi, T. Wang, and J. Cao. Geovla: Empowering 3d representations in vision-language-action models. 2025. URLhttps://arxiv.org/abs/2508.09071

    2025 arXiv

  29. [37]

    Singh, A

    I. Singh, A. Goyal, S. Birchfield, D. Fox, A. Garg, and V . Blukis. Og-vla: Orthographic image generation for 3d-aware vision-language action model. 2025. URLhttps://arxiv.org/ abs/2506.01196

    2025

  30. [38]

    P. Li, Y . Chen, H. Wu, X. Ma, X. Wu, Y . Huang, L. Wang, T. Kong, and T. Tan. Bridgevla: Input-output alignment for efficient 3d manipulation learning with vision-language models

  31. [39]

    T. Yuan, Y . Liu, C. Lu, Z. Chen, T. Jiang, and H. Zhao. Depthvla: Enhancing vision-language- action models with depth-aware spatial reasoning. 2025. URLhttps://arxiv.org/abs/ 2510.13375

    2025

  32. [40]

    Z. Rao, W. Chen, L. Xie, X. Hua, D. Yin, Z. Tian, and F. R. Yu. Augvla-3d: Depth-driven feature augmentation for vision-language-action models. 2026. URLhttps://arxiv.org/ abs/2602.10698

    2026 arXiv

  33. [41]

    T. Lin, Y . Du, J. Liu, N. Zhu, Y . Li, Y . Fu, Y . Chen, H. Cai, Z. Ye, B. Cheng, K. Ye, Y . Mao, Y . Zhong, M. Dong, J. Yan, G. Li, and B. Zhao. Evo-depth: A lightweight depth-enhanced vision-language-action model, 2026. URLhttps://arxiv.org/abs/2605.14950

    2026 arXiv

  34. [42]

    Song and L

    H. Song and L. Le. Avi: Action from volumetric inference, 2025. URLhttps://arxiv. org/abs/2510.21746

    2025

  35. [43]

    Zhang, H

    Z. Zhang, H. Li, Y . Dai, Z. Zhu, L. Zhou, C. Liu, D. Wang, F. E. H. Tay, S. Chen, Z. Liu, Y . Liu, X. Li, and P. Zhou. From spatial to actions: Grounding vision-language-action model in spatial foundation priors, 2025. URLhttps://arxiv.org/abs/2510.17439

    2025

  36. [44]

    Huang, M

    H. Huang, M. Cen, K. Tan, X. Quan, G. Huang, and H. Zhang. Graphcot-vla: A 3d spatial- aware reasoning vision-language-action model for robotic manipulation with ambiguous in- structions, 2025. URLhttps://arxiv.org/abs/2508.07650. 11

    2025 arXiv

  37. [45]

    J. Koo, T. Cho, H. Kang, E. Pyo, T. G. Oh, T. Kim, and A. J. Choi. Retovla: Reusing register tokens for spatial reasoning in vision-language-action models, 2025. URLhttps://arxiv. org/abs/2509.21243

    2025

  38. [46]

    C. Xu, S. Zhang, Y . Liu, B. Sun, W. Chen, B. Xu, Q. Liu, J. Wang, S. Wang, S. Luo, J. Peters, A. V . Vasilakos, S. Zafeiriou, and J. Deng. An anatomy of vision-language-action models: From modules to milestones and challenges.arXiv preprint arXiv:2512.11362, 2025

    2025

  39. [47]

    Cheng, H

    A.-C. Cheng, H. Yin, Y . Fu, Q. Guo, R. Yang, J. Kautz, X. Wang, and S. Liu. Spatialrgpt: Grounded spatial reasoning in vision language models.arXiv preprint arXiv:2406.01584, 2024

    2024 arXiv

  40. [48]

    G. Sun, T. Du, K. Feng, C. Luo, X. Ding, Z. Shen, Z. Wang, Y . He, and A. Li. Rocket: Residual-oriented multi-layer alignment for spatially-aware vision-language-action models. arXiv preprint arXiv:2602.17951, 2026

    2026

  41. [49]

    Zhang, S

    J. Zhang, S. Wu, X. Luo, H. Wu, L. Gao, H. T. Shen, and J. Song. Inspire: Vision-language- action models with intrinsic spatial reasoning, 2025. URLhttps://arxiv.org/abs/2505. 13888

    2025

  42. [50]

    Q. Li, Y . Deng, Y . Liang, L. Luo, L. Zhou, C. Yao, L. Zeng, Z. Feng, H. Liang, S. Xu, Y . Zhang, X. Chen, H. Chen, L. Sun, D. Chen, J. Yang, and B. Guo. Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos, 2025. URL...

    2025

  43. [51]

    Y . Ling, Q. Lian, J. Li, Q. Jiang, T. Zhang, X. Jiang, C. Liu, J. Liu, and L. Zhang. Guide, think, act: Interactive embodied reasoning in vision-language-action models, 2026. URLhttps: //arxiv.org/abs/2605.13632

    2026 arXiv

  44. [52]

    W. Guo, Y . Wang, R. Zhou, Y . Zhang, R. Fang, Y . Pang, W. Xu, and G. Huang. Action with visual primitives, 2026. URLhttps://arxiv.org/abs/2605.22183

    2026 arXiv

  45. [53]

    H. Liu, S. Ruan, J. Long, J. Wu, J. Hou, H. Tang, T. Jiang, W. Zhou, and W. Yao. Eva-vla: Evaluating vision-language-action models’ robustness under real-world physical variations,

  46. [54]

    Y . Zhai, S. Tong, X. Li, M. Cai, Q. Qu, Y . J. Lee, and Y . Ma. Investigating the catastrophic forgetting in multimodal large language models.arXiv preprint arXiv:2309.10313, 2023

    2023 arXiv

  47. [55]

    URLhttps://arxiv.org/abs/2509.18953

  48. [56]

    S. Li, R. Lin, and S. Pei. Multi-modal preference alignment remedies degradation of visual instruction tuning on language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14188–14200. Association f...

    2024 doi

  49. [57]

    J. Wu, Y . Xiong, X. Li, Y . Xia, R. Wang, Y . Wang, T. Yu, S. Kim, R. A. Rossi, L. Yao, J. Shang, and J. McAuley. Mitigating visual knowledge forgetting in mllm instruction-tuning via modality-decoupled gradient descent. InFindings of the Association for Computational Linguis...

    2025 doi

  50. [58]

    Charatan, S

    D. Charatan, S. Li, A. Tagliasacchi, and V . Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction, 2024. URLhttps://arxiv.org/ abs/2312.12337. 12

    2024 arXiv

  51. [59]

    Yu and S

    Z. Yu and S. Ananiadou. Locate-then-merge: Neuron-level parameter fusion for mitigating catastrophic forgetting in multimodal llms. InFindings of the Association for Computational Linguistics: EMNLP 2025. Association for Computational Linguistics, 2025

    2025

  52. [60]

    H. Xu, S. Peng, F. Wang, H. Blum, D. Barath, A. Geiger, and M. Pollefeys. Depthsplat: Con- necting gaussian splatting and depth, 2025. URLhttps://arxiv.org/abs/2410.13862

    2025 arXiv

  53. [61]

    Y . Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T.-J. Cham, and J. Cai. Mvs- plat: Efficient 3d gaussian splatting from sparse multi-view images. InEuropean conference on computer vision, pages 370–386. Springer, 2024

    2024

  54. [62]

    S. Zhou, H. Chang, S. Jiang, Z. Fan, Z. Zhu, D. Xu, P. Chari, S. You, Z. Wang, and A. Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  55. [63]

    M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister. Langsplat: 3d language gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  56. [64]

    A. Thai, S. Peng, K. Genova, L. Guibas, and T. Funkhouser. Splattalk: 3d VQA with gaussian splatting. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2025

    2025

  57. [65]

    X. Wang, C. Lan, H. Zhu, Z. Chen, and Y . Lu. Gsemsplat: Generalizable semantic 3d gaus- sian splatting from uncalibrated image pairs, 2024. URLhttps://arxiv.org/abs/2412. 16932

    2024

  58. [66]

    Zheng, X

    Y . Zheng, X. Chen, Y . Zheng, S. Gu, R. Yang, B. Jin, P. Li, C. Zhong, Z. Wang, L. Liu, C. Yang, D. Wang, Z. Chen, X. Long, and M. Wang. GaussianGrasper: 3d language gaussian splatting for open-vocabulary robotic grasping.IEEE Robotics and Automation Letters, 9(9): 7827–7834, 2024

    2024

  59. [67]

    Zhang, Y

    Z. Zhang, Y . Jiang, Q. Cheng, S. Liu, D. Zhao, P. Luo, W. Zhou, and H. Yu. Gaussiandream: A feed-forward 3d gaussian world model for robotic manipulation, 2026. URLhttps:// arxiv.org/abs/2605.20752

    2026 arXiv

  60. [68]

    J. Yu, X. Ren, Y . Gu, H. Lin, T. Wang, Y . Zhu, H. Xu, Y .-G. Jiang, X. Xue, and Y . Fu. Sparsegrasp: Robotic grasping via 3d semantic gaussian splatting from sparse multi-view rgb images, 2024. URLhttps://arxiv.org/abs/2412.02140

    2024 arXiv

  61. [69]

    Ji, R.-Z

    M. Ji, R.-Z. Qiu, X. Zou, and X. Wang. GraspSplats: Efficient manipulation with 3d feature splatting. InConference on Robot Learning, 2024

    2024

  62. [70]

    G. Lu, B. Jia, P. Li, Y . Chen, Z. Wang, Y . Tang, and S. Huang. GWM: Towards scalable gaus- sian world models for robotic manipulation. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2025

    2025

  63. [71]

    G. Lu, S. Zhang, Z. Wang, C. Liu, J. Lu, and Y . Tang. ManiGaussian: Dynamic gaussian splatting for multi-task robotic manipulation. InEuropean Conference on Computer Vision, 2024

    2024

  64. [72]

    C. Yan, D. Qu, D. Xu, B. Zhao, Z. Wang, D. Wang, and X. Li. GS-SLAM: Dense visual slam with 3d gaussian splatting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19595–19604, June 2024

    2024

  65. [73]

    Y . Chai, L. Deng, R. Shao, J. Zhang, K. Lv, L. Xing, X. Li, H. Zhang, and Y . Liu. GAF: Gaus- sian action field as a 4d representation for dynamic world modeling in robotic manipulation. arXiv preprint arXiv:2506.14135, 2025. doi:10.48550/arXiv.2506.14135

    2025 doi arXiv

  66. [74]

    Matsuki, R

    H. Matsuki, R. Murai, P. H. Kelly, and A. J. Davison. Gaussian splatting slam. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18039–18048, June 2024

    2024

  67. [75]

    Keetha, J

    N. Keetha, J. Karhade, K. M. Jatavallabhula, G. Yang, S. Scherer, D. Ramanan, and J. Luiten. SplaTAM: Splat, track & map 3d gaussians for dense rgb-d slam. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21357– 21366, June 2024. 13

    2024

  68. [76]

    Y . Rao, W. Zhao, B. Liu, J. Lu, J. Zhou, and C.-J. Hsieh. Dynamicvit: Efficient vision trans- formers with dynamic token sparsification. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  69. [77]

    M. S. Ryoo, A. Piergiovanni, A. Arnab, M. Dehghani, and A. Angelova. Tokenlearner: Adap- tive space-time tokenization for videos. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  70. [78]

    Alayrac, J

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Milli- can, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Mon- teiro, J. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Bi ´nkowski, R....

    2022

  71. [79]

    Bolya, C.-Y

    D. Bolya, C.-Y . Fu, X. Dai, P. Zhang, C. Feichtenhofer, and J. Hoffman. Token merging: Your vit but faster. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  72. [80]

    Z. Fan, K. Wang, K. Wen, Z. Zhu, D. Xu, and Z. Wang. Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ FPS. InAdvances in Neural Information Processing Systems, 2024

    2024

  73. [81]

    J. Li, D. Li, S. Savarese, and S. C. H. Hoi. BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. InInternational Conference on Machine Learning, pages 19730–19742, 2023

    2023

  74. [82]

    Y . Wang, Z. Li, L. Guo, W. Yang, A. C. Kot, and B. Wen. ContextGS: Compact 3d gaussian splatting with anchor level context model. InAdvances in Neural Information Processing Systems, 2024

    2024

  75. [83]

    J. C. Lee, D. Rho, X. Sun, J. H. Ko, and E. Park. Compact 3d gaussian representation for radiance field. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  76. [84]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A...

    2024 arXiv

  77. [85]

    Tschannen, A

    M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdulmohsin, N. Parthasarathy, T. Evans, L. Beyer, Y . Xia, B. Mustafa, O. H´enaff, J. Harmsen, A. Steiner, and X. Zhai. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, a...

    2025 arXiv

  78. [86]

    Shukor, D

    M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zouitine, M. Aractingi, C. Pascal, M. Russi, A. Marafioti, S. Alibert, M. Cord, T. Wolf, and R. Cadene. Smolvla: A vision-language-action model for affordable and efficient robotics, 2025. URLhttps: //arxiv.org/a...

    2025 arXiv

  79. [87]

    G. M. Morton. A computer oriented geodetic data base and a new technique in file sequencing. Technical report, International Business Machines Company, Ottawa, Canada, 1966

    1966

  80. [88]

    X. Zhou, Y . Xu, G. Tie, Y . Chen, G. Zhang, D. Chu, P. Zhou, and L. Sun. Libero-pro: Towards robust and fair evaluation of vision-language-action models beyond memorization, 2025. URL https://arxiv.org/abs/2510.03827

    2025 arXiv

  81. [89]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023. URLhttps://arxiv.org/abs/2306. 03310. 14

    2023

  82. [90]

    J. Lee, J. Duan, H. Fang, Y . Deng, S. Liu, B. Li, B. Fang, J. Zhang, Y . R. Wang, S. Lee, et al. Molmoact: Action reasoning models that can reason in space.arXiv preprint arXiv:2508.07917, 2025

    2025 arXiv

  83. [91]

    Q. Bu, Y . Yang, J. Cai, S. Gao, G. Ren, M. Yao, P. Luo, and H. Li. Univla: Learning to act anywhere with task-centric latent actions, 2025. URLhttps://arxiv.org/abs/2505. 06111

    2025

  84. [92]

    Q. Zhao, Y . Lu, M. J. Kim, Z. Fu, Z. Zhang, Y . Wu, Z. Li, Q. Ma, S. Han, C. Finn, A. Handa, M.-Y . Liu, D. Xiang, G. Wetzstein, and T.-Y . Lin. Cot-vla: Visual chain-of-thought reasoning for vision-language-action models, 2025. URLhttps://arxiv.org/abs/2503.22020

    2025 arXiv

  85. [93]

    Bjorck, F

    NVIDIA, :, J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. J. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, J. Jang, Z. Jiang, J. Kautz, K. Kundalia, L. Lao, Z. Li, Z. Lin, K. Lin, G. Liu, E. Llontop, L. Magne, A. Mandlekar, A. Narayan, S. Nasiriany, S. Reed, Y . L. Tan...

    2025 arXiv

  86. [95]

    J. Cen, C. Yu, H. Yuan, Y . Jiang, S. Huang, J. Guo, X. Li, Y . Song, H. Luo, F. Wang, D. Zhao, and H. Chen. Worldvla: Towards autoregressive action world model, 2025. URLhttps: //arxiv.org/abs/2506.21539. 15

    2025 arXiv

  87. [2025]

    URLhttps://arxiv.org/abs/2506.07961

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.