REVIEW 4 major objections 4 minor 9 cited by
GeoVLA: Empowering 3D Representations in Vision-Language-Action Models
T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Adding 3D point clouds to a vision-language-action policy lifts simulated success to 97.7 percent and improves real-world robustness.
desk verdict GeoVLA is a well-built systems paper with a plausible 3D action-head recipe, but the central attribution to geometry is not yet isolated, and the ManiSkill2 comparison is confounded by extra task-specific inputs. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing components are the Point Embedding Network (PEN) and the 3D-enhanced Action Expert (3DAE). PEN is a dual-path point encoder: one path uses large-kernel convolutions and transformer blocks to produce geometric tokens, while the other uses rotary position embeddings (RoPE) to inject 3D spatial structure; PEN then selects the end-effector token as an anchor feature, $\mathbf{F}_P$. 3DAE is a diffusion-transformer action head that concatenates the vision-language features $\mathbf{F}_{VL}$ with $\mathbf{F}_P$ and uses a static-routing mixture-of-experts, randomly dropping one modality during training, to avoid bias toward the VLM branch. The anchor-token design is what ties the
What would settle it
Train the identical GeoVLA architecture with the same parameter count but feed the point-cloud branch scrambled or depth-free point clouds. If the success rate stays near 97.7%, the 3D geometric content is not doing the work. Second, evaluate ManiSkill2 without proprioceptive state, gripper state, and target-marker position, matching the protocol used for Dita; if the 77% average drops to near the 2D baselines, the geometric pathway is not the cause.
Extended reading notes
Core claim
The central claim is that a vision-language-action model with a parallel 3D pathway can outperform 2D-only VLAs on manipulation benchmarks and generalize better under spatial variation. GeoVLA converts depth maps into point clouds, encodes them with the Point Embedding Network (PEN), selects the token at the end-effector position as an anchor feature, and concatenates this geometric feature with the VLM's vision-language tokens. The 3D-enhanced Action Expert (3DAE), a diffusion transformer with a mixture-of-experts design, then generates action chunks. The paper reports 97.7% average success on LIBERO (beating CogACT's 93.2% and OpenVLA-OFT's 95.3%), 77% on ManiSkill2, and 86.3% in real-worl
Load-bearing premise
The central claim stands or falls on the attribution that the reported gains come from the 3D point-cloud information itself, rather than from the extra parameters, extra input signals, or evaluation protocol.
Editorial extensions
If this is right
- If the central claim holds, a pretrained VLM can be augmented with 3D geometric information through a trainable side pathway, without an expensive 3D instruction-tuning stage.
- The end-effector-anchored point feature becomes a candidate standard representation for precise manipulation, since the paper ties it to gains on long-horizon LIBERO tasks and cluttered ManiSkill2 tasks.
- Static modality routing in the action head appears to be a practical fix for the imbalance that arises when a pretrained visual branch meets a randomly initialized 3D branch.
- The real-world variation experiments suggest 3D conditioning may reduce the need for exhaustive viewpoint/height/scale data augmentation in deployment.
- The larger performance gap on LIBERO-Long and LIBERO-90 suggests 3D information is especially useful when actions must be sequenced over many steps.
Reading between the lines
- The same PEN and anchor-token mechanism could plausibly be grafted onto other action heads, such as flow-matching or autoregressive heads, with similar spatial benefits; the paper only demonstrates the diffusion-based 3DAE.
- A clean test of attribution is a same-capacity control: keep the point-cloud pathway and all added parameters but feed it scrambled or depth-free point clouds. If success stays near 97.7%, the improvement would not come from 3D geometry; the paper does not report this control.
- The anchor token's role could be tested directly by swapping the end-effector anchor for an object-referenced anchor, which would clarify whether the gains come from gripper-centric geometry or from any spatially grounded 3D token.
- Depth noise sensitivity is a natural extension: since PEN must handle noisy depth maps, measuring degradation under simulated sensor noise would predict how well the approach transfers to cheaper depth sensors.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GeoVLA, a vision-language-action model that augments a 2D VLM (Prismatic-7B, initialized from OpenVLA) with a parallel point-cloud pathway. Depth maps are reprojected into an end-effector-centered point cloud, encoded by a new Point Embedding Network (PEN) that selects an end-effector anchor token, and combined with vision-language features in a Mixture-of-Experts diffusion action head (3DAE) with a static routing strategy. The authors report state-of-the-art success rates on LIBERO (97.7% average), ManiSkill2 (77% average), and real-world tasks (86.3% average), with robustness to height, scale, and viewpoint changes. The central claim is that 3D geometric conditioning, rather than 2D appearance alone, enables these gains and generalization.
Significance. If the central claim is established, the paper makes a useful contribution: it demonstrates a practical way to inject 3D geometric information into a frozen/pretrained VLA pipeline without retraining the VLM on large 3D instruction-tuning datasets. The architecture is clearly described, the design choices (anchor token, RoPE, static routing) are motivated, and the evaluation spans two simulation benchmarks plus real-world robustness tests. The paper also provides ablations of point-cloud encoders and MoE routing, and it makes falsifiable empirical predictions. However, the current evidence does not fully support the attribution of the gains to geometric information per se, and some reported numbers are inconsistent with the corresponding text. These issues are load-bearing for the paper's main claims and require additional experiments or substantially qualified conclusions.
major comments (4)
- [Section 4.1 / Table 2] The ManiSkill2 comparison is confounded by input differences. Section 4.1 states that GeoVLA receives proprioceptive state, gripper state, and target marker position for ManiSkill2, while the baselines in Table 2 (Dita, CogACT, OpenVLA) are not described as receiving these signals. In particular, the target marker position directly provides the 3D goal location, substantially reducing the spatial reasoning required. The 77% versus 69%/66% advantage may therefore reflect these extra task-specific inputs rather than the point-cloud pathway. Please either evaluate baselines with the same inputs, or ablate the extra inputs from GeoVLA, or clearly reframe the ManiSkill2 results as a system-level comparison rather than evidence for the 3D representation claim.
- [Section 3.3 / Table 3] The ablations do not isolate the geometric information content. Table 3(a) compares MLP, PointNet, and PEN point encoders, but these differ in architecture and parameter count, and none removes or destroys the geometric signal while keeping the rest of the pipeline fixed. There is no control that keeps PEN/3DAE but feeds a geometry-free input (e.g., shuffled point coordinates, fixed random points, or zeros), nor a no-point-cloud baseline with the same overall capacity. Without such a control, the LIBERO gains attributed to '3D geometric conditioning' could also come from the additional parameters of PEN/3DAE, the end-effector anchor inductive bias, or the static routing regularization. Please add a same-capacity geometry-destroyed ablation or explicitly qualify the claim.
- [Tables 1, 2, 4-8] All success rates are point estimates over 50, 20, or 10 episodes with no variance, confidence intervals, or repeated seeds. This is especially problematic on LIBERO, where the headline average differences are small (e.g., 97.7% vs. 95.3% for OpenVLA-OFT, and 97.7% vs. 95.8% for the MLP encoder in Table 3(a)). With 50 episodes, a difference of roughly 2-4 percentage points is not statistically meaningful without repeated trials or a bootstrap interval. Please report per-seed results, confidence intervals, or at least the number of independent runs, and avoid claiming state-of-the-art margins that may be within noise.
- [Section 5.3 / Table 8] The text describing the Pick Carrot variation contradicts the reported numbers. The paper states that 'our geometric-aware GeoVLA consistently achieves more stable and successful grasps' under removal of the sponge mat, but Table 8 shows GeoVLA at 50% without the mat, while π0 achieves 60%. Either the claim should be corrected, or the comparison needs statistical support (e.g., repeated trials and error bars). As written, this particular robustness example undercuts rather than supports the paper's generalization narrative.
minor comments (4)
- [Throughout] Several typos and inconsistent terms: 'Succuss rate' in the Table 4 caption; 'action export' in Figure 3 and 'export policy' in Section 2 should be 'action expert'/'expert policy'; 'Primatic' in Section 3.2 should be 'Prismatic'; 'F ocused representation learning' in Section 3.3 has an extra space. A careful proofread is needed.
- [Table 1 footnote] The footnote says 'Results of CogACT are reproduced by us' and Table 2 lists OpenVLA* and CogACT* as reproduced, but the reproduction protocol (training steps, input modalities, seeds, hyperparameters) is not described in Section 4.1. If these are literal reproductions, please provide details; otherwise, clarify whether the numbers are taken from prior papers.
- [Section 5.2] The real-world baselines are described as 'π0 (Black et al., 2025) (without pose states and other images as input)' and CogACT, but it is unclear whether they were trained on the same demonstration datasets and number of episodes as GeoVLA. Dataset size and training budget should be reported per method to rule out data/effort confounds.
- [Section 2] The sentence 'For instance, ACT (Dong et al., 2023) mitigates the need for extensive task-specific fine-tuning' appears in a discussion of 3D-VLA and seems out of place or garbled. Please revise the related-work narrative for clarity.
Circularity Check
No significant circularity: GeoVLA's claims are empirical benchmark results with ablations, not quantities derived from their own inputs; the only overlapping-author citations are non-load-bearing related-work examples.
full rationale
GeoVLA does not present a mathematical derivation chain that could collapse into its inputs. Its central claim is empirical: adding a point-cloud pathway (PEN) and a 3D-enhanced action expert (3DAE) improves success rates on LIBERO, ManiSkill2, and real-world tasks. These numbers are measured on held-out episodes and unseen task variations after training, not fitted to the reported outcomes; the design choices (anchor token, RoPE, static routing) are described before evaluation and are ablated rather than derived from the target results. No parameter is estimated from the benchmark numbers and then 'predicted' as a separate quantity. The only overlapping-author citations (Zheng et al. 2025, Zhang et al. 2025) appear in a generic list of 3D foundation models and are not used to justify GeoVLA's correctness, uniqueness, or benchmark performance, so they are not load-bearing. The skeptical concern that LIBERO gains might come from extra parameters or a missing geometry-destroyed control, and that ManiSkill2 comparisons include additional proprioceptive/gripper/target-marker inputs, is an experimental-confound argument, not a demonstration that a claimed prediction is identical by construction to its inputs. Accordingly, no circular step can be quoted, and the correct finding is no significant circularity.
Assumptions & free parameters
free parameters (5)
- action_chunk_length T =
16
- learning rate, batch size, training steps =
2e-5, 256, ~20,000 steps
- point cloud resolution and token count
- static routing modality-drop schedule
- DDIM sampling steps
assumptions (4)
- domain assumption Known camera intrinsics and end-effector pose allow accurate reprojection of depth into an end-effector-centered point cloud.
- domain assumption OpenVLA/Prismatic-7B pretrained weights transfer to the new action head and point-cloud branch without meaningful alignment loss.
- standard math Diffusion denoising and DDIM sampling provide a valid generative model for action chunks.
- domain assumption Success-rate point estimates over 50, 20, or 10 episodes reflect true policy quality without variance or seed dependence.
Cite this review
Pith. "Pith review of GeoVLA: Empowering 3D Representations in Vision-Language-Action Models." pith.science (2026). https://pith.science/paper/NNKI34S7
@misc{pith2026250809071,
author = {Pith},
title = {Pith review of: GeoVLA: Empowering 3D Representations in Vision-Language-Action Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/NNKI34S7}},
note = {Machine review of arXiv:2508.09071}
}
read the original abstract
Vision-Language-Action (VLA) models have emerged as a promising approach for enabling robots to follow language instructions and predict corresponding actions. However, current VLA models mainly rely on 2D visual inputs, neglecting the rich geometric information in the 3D physical world, which limits their spatial awareness and adaptability. In this paper, we present GeoVLA, a novel VLA framework that effectively integrates 3D information to advance robotic manipulation. It uses a vision-language model (VLM) to process images and language instructions,extracting fused vision-language embeddings. In parallel, it converts depth maps into point clouds and employs a customized point encoder, called Point Embedding Network, to generate 3D geometric embeddings independently. These produced embeddings are then concatenated and processed by our proposed spatial-aware action expert, called 3D-enhanced Action Expert, which combines information from different sensor modalities to produce precise action sequences. Through extensive experiments in both simulation and real-world environments, GeoVLA demonstrates superior performance and robustness. It achieves state-of-the-art results in the LIBERO and ManiSkill2 simulation benchmarks and shows remarkable robustness in real-world tasks requiring height adaptability, scale awareness and viewpoint invariance.
Forward citations
Cited by 9 Pith papers
-
DreamWAM: Beyond RGB Future Prediction for World Action Models
Adding motion, depth, and semantic supervision to a world action model's future prediction during training, then removing it at inference, improves robot manipulation robustness under visual perturbations.
-
Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Models
Aligning a VLA's latent features with instruction-selected target-object tri-views (VAE and VGGT) improves manipulation success, especially under target occlusion, with a compact 345M backbone.
-
VR3D: View-Robust 3D Representation Learning for Aerial-Ground Person Re-Identification
VR3D reports state-of-the-art aerial-ground person re-identification by interacting 2D appearance with SAM3D-produced 3D voxels in a canonical space and adaptively fusing the resulting features.
-
See like a Robot: Robot-Centric Pointmaps for Vision-Language-Action Models
Expressing RGB-D observations as end-effector-centered robot-frame pointmaps and adding them element-wise to RGB tokens improves pretrained VLAs under camera viewpoint variation.
-
Learning 4D Geometric Priors for Inference-Efficient World Action Models
Training-only multi-expert co-training with decayed 4D read-mask attention and action-aware temporal geometric distillation improves WAM manipulation success while keeping the original lightweight inference graph.
-
LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization
SOTA VLA models like OpenVLA and pi0 collapse when object positions are perturbed, indicating that standard LIBERO scores reward memorization; LIBERO-PRO provides a systematic perturbed evaluation suite.
-
VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation
VistaVLA lifts multi-view vision-language features into 3D Gaussians, compresses them 99% via Merge-then-Query, and improves real-robot manipulation success by ~23% over baselines.
-
SAM3D-Guided Object-Centric Representation Alignment for Vision-Language-Action Models
Training-time alignment of π0 visual features with object-centric SAM3D 3D features improves VLA manipulation performance while keeping RGB-language-only inference.
-
QDepth-VLA: Quantized Depth Prediction as Auxiliary Supervision for Vision-Language-Action Models
Adding an auxiliary quantized-depth-token prediction task to a VLA policy improves manipulation success rates on LIBERO, Simpler, and real-robot pick-and-place tasks versus the open-pi-zero baseline.
Reference graph
Works this paper leans on
-
[4]
Yueru Jia, Jiaming Liu, Sixiang Chen, Chenyang Gu, Zhilue Wang, Longzan Luo, Lily Lee, Pengwei Wang, Zhongyuan Wang, Renrui Zhang, and Shanghang Zhang. Lift3d foundation policy: Lifting 2d large-scale pretrained models for robust 3d robotic manipulation. arXiv preprint arXiv:2411.18623,
-
[6]
11 Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, and Baining Guo. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXi...
-
[7]
Flow matching for generative modeling
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747,
-
[8]
Libero: Benchmarking knowledge transfer for lifelong robot learning
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems (NeurIPS), pages 44776–44791, 2023a. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Advances in Neural Informat...
-
[9]
Base Tasks. We consider four fundamental manipulation tasks designed to evaluate basic pick-and- place capabilities: (1) Pick Carrot: The robot is required to grasp a slender carrot and place it into a 2 Stack BlockPick Carrot Stack Cup Insert circle Hang Cup Cover Matryoshka Put Basketball Put Hairclip (a) Base tasks (b) 3D-aware tasks Change basket heig...
work page 2025
-
[10]
3d-vla: A 3d vision-language-action generative world model
Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: A 3d vision-language-action generative world model. arXiv preprint arXiv:2403.09631, 2024a. Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: A 3d vision-language-action generative world mo...
-
[11]
Llava-3d: A simple yet effective pathway to empowering lmms with 3d-awareness
Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, and Xihui Liu. Llava-3d: A simple yet effective pathway to empowering lmms with 3d-awareness. arXiv preprint arXiv:2409.18125,
-
[2020]
Dita: Scaling diffusion transformer for generalist vision-language- action policy
Zhi Hou, Tianyi Zhang, Yuwen Xiong, Haonan Duan, Hengjun Pu, Ronglei Tong, Chengyang Zhao, Xizhou Zhu, Yu Qiao, Jifeng Dai, and Yuntao Chen. Dita: Scaling diffusion transformer for generalist vision-language- action policy. arXiv preprint arXiv:2503.19757,
Show all 12 references
-
[2022]
Qwen technical report
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Si...
-
[2023]
Ground- ing beyond detection: Enhancing contextual understanding in embodied 3d grounding
Yani Zhang, Dongming Wu, Hao Shi, Yingfei Liu, Tiancai Wang, Haoqiang Fan, and Xingping Dong. Ground- ing beyond detection: Enhancing contextual understanding in embodied 3d grounding. arXiv preprint arXiv:2506.05199,
-
[2024]
Fine-tuning vision-language-action models: Optimizing speed and success
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success. In Robotics: Science and Systems (RSS) , 2025a. Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Etha...
-
[2025]
Gr00t n1: An open foundation model for generalist humanoid robots
Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Jim Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magn...
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.