Pith. sign in

REVIEW 3 major objections 5 minor 65 references

BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A spatio-temporal memory module lets a heatmap-based 3D robot policy solve memory-dependent tasks near perfectly, without losing data efficiency or generalization.

desk verdict A strong engineering extension with a real confound in the RMBench comparison: the temporal-memory ablation removes the privileged sub-goal gate along with the memory, so MemoryBench is the clean validation. read the letter →

arxiv 2608.05042 v1 pith:VEDDTEHS submitted 2026-08-05 cs.RO

Peiyan Li , Yuze Zhu , Yixiang Chen , Qisen Ma , Yuan Xu , Jiabing Yang , He Guan , Yan Huang
show 5 more authors
Hongtao Wu Xiao Ma Tao Kong Liang Wang Tieniu Tan
This is my paper
classification cs.RO
keywords vision-language-actionmodels3Dmanipulationepisodicmemorytemporalspatialheatmappredictiondata-efficientlearningrobotbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a vision-language-action model for 3D manipulation can be given episodic memory without sacrificing its data efficiency or generalization, and that the memory is what unlocks near-perfect performance on tasks whose current frame is ambiguous. The authors extend their earlier BridgeVLA policy, which predicts robot actions as 2D heatmaps in orthographic projections of a point cloud, with a unified spatio-temporal memory. Temporal memory stores initial anchor views, recent keyframes, and adaptively selected sub-goal frames to decide what to do next; spatial memory re-renders the initial point cloud under the current zoom to expose geometry occluded by the robot. On the RMBench bimanual benchmark and MemoryBench, the memory-augmented model reaches 96.0% and 99.7% success, while matching or improving the memory-free base on RLBench, COLOSSEUM, and GemBench. The paper argues the extension is strictly additive and transfers to two real robot arms.

What carries the argument

The carrying object is the memory-conditioned token interface built on BridgeVLA's input–output alignment: point clouds are rendered into three orthographic views, the pre-trained PaliGemma VLM produces image tokens, and actions are decoded from 2D heatmaps via convex upsampling. Memory is stored as visual tokens rather than raw images: the temporal buffer caches coarse-stage tokens from the initial anchor, neighboring keyframes, and adaptive sub-goal keyframes, and a compact cross-attention module fuses them into the current coarse representation; the spatial memory keeps the initial colored point cloud and re-renders and re-encodes it under each predicted coarse waypoint so that current fine-stage tokens attend to their view-aligned, less-occluded counterparts. The adaptive sub-goal selection gate, trained with per-segment language labels on RMBench, decides which keyframes are retained as sub-goal evidence.

What would settle it

Run the RMBench experiment with the sub-goal gate trained without per-segment language annotations, for example using a self-supervised novelty score or random retention, while keeping the temporal memory mechanism identical; if the score drops far below 96% toward the 21.3% no-temporal-memory ablation, the gate's supervision, not the memory, is the load-bearing part.

Watch

Extended reading notes

Core claim

The central claim is that an explicit scene-level memory, split into temporal and spatial components and injected into the coarse and fine stages of a heatmap-based 3D VLA, lets the policy track completed sub-goals and recover occluded target geometry. The temporal memory (initial anchor views, the two most recent keyframes, and gated sub-goal keyframes) conditions the coarse-stage tokens through cross-attention, determining what to do next; the spatial memory re-renders the initial point cloud under the same zoom as the current fine-stage crop and cross-attends view-aligned tokens, determining where exactly to act. The memory modules add about 270M parameters (9.2% of the 2.92B backbone) and keep the heatmap action interface unchanged. The evidence is that removing temporal memory collapses RMBench success from 96.0% to 21.3%, near the 18.9% of the memory-free base, while removing spatial memory only slightly affects precision-heavy RLBench tasks such as Sort Shape.

Load-bearing premise

The claim that temporal memory, rather than the extra language supervision used to train the sub-goal selector, is the decisive component on RMBench rests on assuming that this supervision did not secretly carry the result.

Editorial extensions

If this is right

  • A keyframe-based 3D VLA can handle tasks that require counting, trial-and-error tracking, and uncovering actions, not just current-frame perception.
  • The memory extension is additive: on RLBench, COLOSSEUM, and GemBench, BridgeVLA++ matches or slightly exceeds the memory-free BridgeVLA, so episodic memory does not trade away data efficiency or out-of-distribution generalization.
  • Bimanual tasks can share one memory and backbone, with only arm-specific action heads duplicated, which keeps the memory-dependent bimanual extension computationally cheap.
  • Real robots can learn memory-dependent tasks from ten demonstrations per instruction and keep near-perfect basic success across distractors, background changes, height, and lighting.
  • The token-space memory injection leaves the heatmap action interface unchanged, so the same memory design can be carried over to other heatmap-based action decoders without retraining the backbone interface.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that the temporal memory architecture would generalize to cross-episode or lifelong learning, because the token-space memory injection is agnostic to timescale and only needs the memory buffer to persist across episodes.
  • The paper's own future-work note suggests that replacing the annotation-dependent sub-goal gate with a self-supervised one is the main barrier to applying the same memory design to benchmarks without per-segment language labels.
  • A testable extension is to run BridgeVLA++ on RMBench with the gate supervised by a learned novelty score instead of language-derived labels; if performance holds, the memory mechanism, not the privileged supervision, is the decisive component.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript extends the authors' earlier BridgeVLA framework with a unified spatio-temporal memory for 3D vision-language-action manipulation. BridgeVLA itself projects point clouds into multi-view orthographic images and predicts 2D heatmaps before translating them into 3D end-effector poses, preserving the input--output alignment of a pre-trained VLM. BridgeVLA++ adds a coarse-stage temporal memory (initial anchor views, recent neighboring keyframes, and adaptively selected sub-goal keyframes) to determine what to do next, and a fine-stage spatial memory that re-renders the initial scene point cloud under the current zoom to recover occluded geometry for precise localization. The paper evaluates the framework on RLBench, COLOSSEUM, GemBench, RMBench, MemoryBench, and two real-robot platforms, claiming state-of-the-art results on the two memory-dependent benchmarks while preserving the data efficiency and out-of-distribution generalization of the base model. Extensive ablations, per-task tables, and implementation details are provided in the main text and appendices.

Significance. If the claims hold, the paper makes a solid contribution: a lightweight memory mechanism (269.77M parameters, 9.2% of the backbone) that plugs into an existing 3D VLA without changing its action interface, a natural bimanual extension, and strong empirical results across simulation and real hardware. The paper is unusually complete in its reporting: per-task success rates, ablation configurations, training schedules, and latency figures are all given. The internal ablations are consistent with the proposed mechanism: removing heatmap decoding collapses RLBench (90.5% to 31.4%, Table I), removing temporal memory collapses RMBench (96.0% to 21.3%, Table XVII), and MemoryBench shows large gains over the memory-free base without using sub-goal annotations. The MemoryBench and real-world memory results are not affected by the main confound I raise below and provide genuine evidence for the memory architecture. However, the headline 'state-of-the-art on two memory-dependent benchmarks' is weakened by a confound in the RMBench experiment and by the best-checkpoint selection protocol, so the central claim needs additional experimental support before it can be fully credited.

major comments (3)
  1. [Sec. V-C, Table II, Appendix D-b, Table XVII] The claim that temporal memory is 'indispensable' on RMBench is not cleanly supported, because the w/oT ablation removes both the temporal memory and the adaptive sub-goal selection module, whose training on RMBench uses per-segment language annotations (Appendix D-b). These annotations constitute privileged task-progress and segmentation information that baselines such as MemoryW AM do not receive. The collapse from 96.0% to 21.3% therefore conflates the memory mechanism with the privileged sub-goal supervision. To isolate the memory contribution, please provide an additional ablation on RMBench in which the gate is trained without the per-segment labels (e.g., a heuristic or self-supervised selection rule, or random keyframe sampling), or report a variant that keeps the neighboring-keyframe memory and anchor views but disables the sub-goal gate entirely. Without such an experiment, the 96.0% result cannot be attributed to temporal memory alone.
  2. [Appendix E-d] The RMBench numbers in Tables II and XVII are obtained by training one model per task and selecting the best-performing checkpoint of that task's training run. This protocol can substantially inflate reported success rates, and the paper does not state whether the quoted baselines (MemoryW AM, Mem-0, etc.) follow the same selection procedure. Please clarify the checkpoint-selection protocol for all methods and, ideally, report results for a fixed checkpoint (e.g., final epoch or a validation-selected single checkpoint) or the mean over checkpoints. Without this, the 13-point margin over MemoryW AM could be an artifact of favorable checkpoint selection.
  3. [Abstract and Sec. V-C] Because of the two issues above, the paper's headline claim that BridgeVLA++ 'achieves state-of-the-art results on two challenging memory-dependent manipulation benchmarks' overstates the evidence. The MemoryBench result (99.7% vs. 94.3% for SAM2Act+, Table XII) is clean and supports a more qualified claim, as does the real-world Dobot evaluation. Please temper the abstract and conclusion to reflect that the RMBench comparison is currently confounded, or strengthen the analysis as requested in the previous comments, so that the stated claim matches the experimental support.
minor comments (5)
  1. [Table II] The grouping of the nine tasks under the 'M(1) tasks' and 'M(n) tasks' headers is visually confusing because the column headers span two levels without a clear separator; consider splitting the table or using a merged-header layout.
  2. [Sec. IV-D and Appendix F] The parameter counts are reported as 'approximately 168M and 84M' for the temporal and spatial memory modules in the main text, while Appendix F gives 83.95M per block and 17.91M for the gate. These numbers are consistent arithmetically but should be reconciled explicitly to avoid the appearance of inconsistency.
  3. [Appendix D-b] The description of the sub-goal labels says positives cover 9--15% of keyframes depending on the task; it would be helpful to state the per-task positive fractions in a small table or figure, since this directly determines how informative the privileged supervision is.
  4. [Sec. V-E and Appendix L] The real-world memory-dependent evaluation uses only three tasks with ten trials per instruction per setting. While the improvements over baselines are large, the small trial counts make the point estimates noisy; a brief note on whether any trials were discarded or repeated, and on the variance across trials, would improve the report.
  5. [Conclusion] The future-work sentence already acknowledges that the sub-goal gate is 'annotation-dependent,' which is commendable; however, given that this is a known limitation, the paper should state it earlier (e.g., in the RMBench section) so that readers can properly interpret the ablation results.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: results are empirical and self-contained; the RMBench sub-goal gate is a confound, not circularity.

full rationale

The paper's central claims are empirical rather than derived: BridgeVLA++ is trained on benchmark demonstrations, the memory modules are trained from scratch (Appendix B), and the headline numbers are measured on held-out episodes against external baselines and benchmarks (RLBench, COLOSSEUM, GemBench, RMBench, MemoryBench, and real-robot suites). No equation equates a predicted quantity to a training input, and the memory-free BridgeVLA results are established by experiments in this paper rather than by self-citation. External references (PaliGemma, RoboPoint, RVT) serve as building blocks, not as the authority for the memory claim. The one notable limitation is the annotation-dependent sub-goal gate on RMBench: Appendix D-b states that the gate is supervised with per-segment language annotations, and the w/oT ablation in Table XVII removes the gate together with temporal memory, so the 96.0% to 21.3% drop cannot be cleanly attributed to the memory mechanism alone. This is a fairness/attribution confound due to privileged sub-goal supervision, not a circular reduction of the result to its inputs. MemoryBench, which does not use the gate, and the real-robot memory tasks provide partially independent, gate-free validation. Overall circularity is minimal.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the transfer of a pre-trained VLM, the validity of success-rate benchmarks as proxies, and the additional sub-goal annotations used to train the memory gate on RMBench. The memory architecture itself introduces tunable hyperparameters (budgets, threshold, weights) rather than fitted physical constants.

free parameters (5)
  • Sub-goal gate threshold = 0.5
    Keyframes are retained when predicted retention probability exceeds 0.5 (Sec. IV-B3, Table VII); chosen by hand, not validated on a separate split.
  • Positive-class weight for sub-goal gate loss = 5.5
    Compensates for 9-15% positive keyframes on RMBench (Table VII, Appendix D-b); tuned on the benchmark.
  • Memory slot budget K = 2 (RLBench/COLOSSEUM/GemBench/MemoryBench) or 12 (RMBench)
    Per-benchmark budget for temporal memory (Table VII, Appendix C-c).
  • Neighboring keyframes n = 2
    Number of recent keyframes stored in H_nbr for all benchmarks (Sec. IV-B2, Table VII).
  • Stage-2 zoom jitter = 0.05 or 0.005 per benchmark
    Perturbs the ground-truth waypoint during fine-stage training (Table VII); chosen by hand.
assumptions (5)
  • domain assumption Calibrated RGB-D cameras allow accurate colored point-cloud reconstruction of the scene.
    Assumed throughout Sec. III-C and real-world setups; errors in calibration would break orthographic projection alignment.
  • domain assumption PaliGemma's pre-trained VLM weights and RoboPoint detection data provide transferable object-grounding knowledge.
    The 2D-heatmap pre-training (Sec. III-B) depends on this transfer; the paper does not prove it, only shows ablation results.
  • domain assumption Benchmark success rate is a valid proxy for manipulation capability.
    Claims of state-of-the-art rely on simulation and real-robot success rates as the metric.
  • domain assumption On RMBench, per-segment language annotations in demonstrations are usable as privileged sub-goal labels.
    The adaptive sub-goal gate is supervised with these labels (Appendix D-b); baselines are not reported to use them.
  • domain assumption The initial point cloud P0 is less occluded and geometrically consistent for re-rendering later in the episode.
    Spatial memory re-renders P0 at each fine stage (Sec. IV-C, Eq. 15); if the scene changes significantly or the robot occludes the initial view, the reference could mislead.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation." pith.science (2026). https://pith.science/paper/VEDDTEHS

@misc{pith2026260805042,
  author       = {Pith},
  title        = {Pith review of: BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VEDDTEHS}},
  note         = {Machine review of arXiv:2608.05042}
}
read the original abstract

Leveraging pre-trained vision-language models (VLMs) to construct vision-language-action (VLA) models has emerged as a promising paradigm for 3D robot manipulation. However, existing 3D VLA methods remain data-hungry, exhibit limited generalization under distribution shifts, and lack explicit memory of past observations. These limitations hinder their application to data-scarce, open-world, and memory-dependent manipulation scenarios. Our previous work, BridgeVLA, improves data efficiency and generalization by preserving the input--output alignment of a pre-trained VLM during 3D action learning: raw point clouds are projected into multi-view images, and intermediate heatmaps are predicted before generating robot actions. In this work, we develop BridgeVLA++ by equipping BridgeVLA with a unified spatio-temporal memory architecture that models persistent spatial context and temporal interaction history. The resulting memory-augmented framework can reason over observation histories while preserving BridgeVLA's data efficiency and generalization capabilities. Extensive experiments show that our framework achieves strong performance on spatial manipulation tasks while exhibiting robust generalization. BridgeVLA++ further achieves state-of-the-art performance on two challenging memory-dependent manipulation benchmarks without sacrificing the data efficiency and generalization of the original BridgeVLA. In addition, BridgeVLA++ performs effectively in bimanual manipulation settings and is validated on an additional real-world robotic platform, demonstrating its scalability across tasks, environments, and robotic platforms. These results establish BridgeVLA++ as a unified 3D vision-language-action framework that simultaneously supports data-efficient learning, robust generalization, and effective memory-aware robot manipulation. Project website: https://bridgevla-plus.github.io/.

Figures

Figures reproduced from arXiv: 2608.05042 by the authors.

Figure 1
Figure 1. Overview. BridgeVLA is a 3D VLA framework that aligns its inputs and outputs in a unified 2D image space. It is pre-trained on object grounding using 2D heatmaps and fine-tuned on action prediction for 3D manipulation. BridgeVLA++ extends BridgeVLA with a unified spatio-temporal memory architecture in which temporal memory preserves interaction history to determine what to do next, whereas spatial memory restores pr… view at source ↗
Figure 2
Figure 2. Model Architecture. Top: BridgeVLA first learns language-conditioned 2D heatmap prediction from detection data and transfers the resulting weights to 3D action fine-tuning. During a policy forward pass, the observed point cloud is rendered into orthographic views and processed with the language instruction by the VLM to produce multi-view visual tokens and heatmaps. The coarse heatmaps localize a 3D waypoint, around… view at source ↗
Figure 3
Figure 3. Occlusion-robust fine localization using spatial memory. Left: the current zoomed observation, partially occluded by the gripper and manipu￾lated object. Right: the stored point cloud P0 re-rendered under the same predicted waypoint and zoom transformation, providing a spatially aligned, less-occluded reference of the same local region. they are geometrically aligned at the view level. Accordingly, we let tokens fro… view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Real-Robot Evaluation Setup. Top left: the general-manipulation platform—a 7-DoF Franka Research 3 arm observed by a static ZED 2i stereo camera. Top right: the memory platform—a 6-DoF Dobot CR5A arm with the same camera configuration. Bottom: the evaluation settings o…
Figure 5
Figure 5. Figure 5: Real-Robot Generalization Results. Average success rate over the 13 Franka tasks in the basic setting and the six generalization settings; the w/o Pre-train bars ablate the 2D-heatmap pre-training (Sec. V-F). TABLE V Per-task success rates on the real Dobot platform in…
Figure 6
Figure 6. Figure 6: The Distractor, Lighting, Background, and Height Settings. Visualization of the four visual-disturbance settings of the real-robot evaluation (Appendix K). Memory-Dependent Memory-Free Setting Cover Blocks Press Button Swap Eggplant Put in Drawer (upper) Put in Drawer …
Figure 7
Figure 7. Figure 7: The Distractor, Lighting, Background, and Height Settings on the Dobot Platform. Initial scene of every instruction of the Dobot suite (columns, named as in Table XIX) under each of the four visual-disturbance settings (rows; Appendix L). Frames in the Lighting row are…
Figure 8
Figure 8. Figure 8: Real-Robot Rollouts (I). BridgeVLA rollouts on the real-robot task suite of Appendix K [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: Real-Robot Rollouts (II). BridgeVLA rollouts on the real-robot task suite of Appendix K [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]
Figure 10
Figure 10. Figure 10: The Combination Setting (I). During training, the manipulated objects and skills are seen, but their combinations are unseen [PITH_FULL_IMAGE:figures/full_fig_p028_10.png]
Figure 11
Figure 11. Figure 11: The Combination Setting (II). During training, the manipulated objects and skills are seen, but their combinations are unseen [PITH_FULL_IMAGE:figures/full_fig_p029_11.png]
Figure 12
Figure 12. Figure 12 [PITH_FULL_IMAGE:figures/full_fig_p030_12.png]
Figure 13
Figure 13. Figure 13: Dobot Rollouts (I): Memory-Dependent Tasks. BridgeVLA++ rollouts on the three memory-dependent instructions of the Dobot suite (Appendix L) in the Basic setting; each strip shows five keyframes of one successful episode. Initial Scene Put the watermelon in the upper d…
Figure 14
Figure 14. Figure 14: Dobot Rollouts (II): Memory-Free Tasks. BridgeVLA++ rollouts on the four memory-free instructions of the Dobot suite in the Basic setting, laid out as in [PITH_FULL_IMAGE:figures/full_fig_p031_14.png]
Figure 15
Figure 15. Figure 15: Ground-Truth Heatmap Construction on Detection Data. For each sample: the original image (left), the bounding boxes of the objects of interest (middle), and the ground-truth heatmap rendered from the box centers (right) [PITH_FULL_IMAGE:figures/full_fig_p032_15.png]
Figure 16
Figure 16. Figure 16: Predictions on Pre-Training Data after Fine-Tuning. Each input image is repeated three times to mimic the multi-view input format of fine-tuning. Rows per sample: input image, predicted heatmaps, ground truth. Samples are not cherry-picked [PITH_FULL_IMAGE:figures/fu…
Figure 17
Figure 17. Figure 17: The 18 RLBench Tasks. Visualization of the 18 RLBench [13] tasks used in Sec. V-A. Battery try Blocks ranking Cover blocks Observe and pickup Press button Put back block Rearrange blocks Swap blocks Swap T [PITH_FULL_IMAGE:figures/full_fig_p034_17.png]
Figure 18
Figure 18. Figure 18: The Nine RMBench Tasks. One evaluation rollout per task of RMBench [16], shown as three frames in temporal order; the dual-arm tasks span the short-term M(1) and long-term M(n) memory regimes [PITH_FULL_IMAGE:figures/full_fig_p034_18.png]
Figure 19
Figure 19. Figure 19: Perturbations in COLOSSEUM [14]. All perturbation axes are shown except the original-RLBench variation setting [PITH_FULL_IMAGE:figures/full_fig_p035_19.png]
Figure 20
Figure 20. Figure 20: The GemBench Task Suite. One representative variation of every task of GemBench [15], shown as the first and final frame of an evaluation rollout. Border colors denote the generalization level: L1 (blue, novel placements), L2 (green, novel rigid objects), L3 (orange, …
Figure 21
Figure 21. Figure 21: The Three MemoryBench Tasks. Two variants of each MemoryBench [17] task, each shown as three rollout frames in which the robot’s own intervention erases the evidence a later step depends on [PITH_FULL_IMAGE:figures/full_fig_p036_21.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

65 extracted references · 38 canonical work pages

  1. [1]

    OpenVLA: An open-source vision-language-action model,

    M. J. Kimet al., “OpenVLA: An open-source vision-language-action model,” inConference on Robot Learning, 2025, pp. 2679–2713

    2025

  2. [2]

    π0.5: a vision-language-action model with open-world generalization,

    Physical Intelligenceet al., “π0.5: a vision-language-action model with open-world generalization,” 2025, arXiv:2504.16054

    arXiv 2025

  3. [3]

    W ALL-OSS-0.5 technical report,

    R. Yuet al., “W ALL-OSS-0.5 technical report,” 2026, arXiv:2605.30877

    arXiv 2026

  4. [4]

    Vision-language foundation models as effective robot imitators,

    X. Liet al., “Vision-language foundation models as effective robot imitators,” inInternational Conference on Learning Representations (ICLR), 2024

    work page 2024

  5. [5]

    RT-2: Vision-language-action models transfer web knowledge to robotic control,

    A. Brohanet al., “RT-2: Vision-language-action models transfer web knowledge to robotic control,” inConference on Robot Learning. PMLR, 2023, pp. 2165–2183

    work page 2023

  6. [6]

    Perceiver-Actor: A multi-task transformer for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Perceiver-Actor: A multi-task transformer for robotic manipulation,” inConference on Robot Learning. PMLR, 2023, pp. 785–799

    work page 2023

  7. [7]

    3D Diffuser Actor: Policy diffusion with 3D scene representations,

    T.-W. Ke, N. Gkanatsios, and K. Fragkiadaki, “3D Diffuser Actor: Policy diffusion with 3D scene representations,” inConference on Robot Learning, 2025, pp. 1949–1974

    work page 2025

  8. [8]

    Act3D: 3D feature field transformers for multi-task robotic manipulation,

    T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki, “Act3D: 3D feature field transformers for multi-task robotic manipulation,” inCon- ference on Robot Learning. PMLR, 2023, pp. 3949–3965

    work page 2023

Show all 65 references
  1. [9]

    RVT: Robotic view transformer for 3D object manipulation,

    A. Goyal, J. Xu, Y . Guo, V . Blukis, Y .-W. Chao, and D. Fox, “RVT: Robotic view transformer for 3D object manipulation,” inConference on Robot Learning. PMLR, 2023, pp. 694–710

    2023

  2. [10]

    RVT-2: Learning precise manipulation from few demonstrations,

    A. Goyal, V . Blukis, J. Xu, Y . Guo, Y .-W. Chao, and D. Fox, “RVT-2: Learning precise manipulation from few demonstrations,” inRobotics: Science and Systems (RSS), 2024

    2024

  3. [11]

    3D-VLA: A 3D vision-language-action generative world model,

    H. Zhenet al., “3D-VLA: A 3D vision-language-action generative world model,” inInternational Conference on Machine Learning. PMLR, 2024, pp. 61 229–61 245

    2024

  4. [12]

    SpatialVLA: Exploring spatial representations for visual- language-action models,

    D. Quet al., “SpatialVLA: Exploring spatial representations for visual- language-action models,” inRobotics: Science and Systems (RSS), 2025

    2025

  5. [13]

    RLBench: The robot learning benchmark & learning environment,

    S. James, Z. Ma, D. R. Arrojo, and A. J. Davison, “RLBench: The robot learning benchmark & learning environment,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3019–3026, 2020

    2020

  6. [14]

    The Colosseum: A benchmark for evaluating generalization for robotic manipulation,

    W. Pumacay, I. Singh, J. Duan, R. Krishna, J. Thomason, and D. Fox, “The Colosseum: A benchmark for evaluating generalization for robotic manipulation,” 2024, arXiv:2402.08191

    2024 arXiv

  7. [15]

    Towards generalizable vision- language robotic manipulation: A benchmark and LLM-guided 3D policy,

    R. Garcia, S. Chen, and C. Schmid, “Towards generalizable vision- language robotic manipulation: A benchmark and LLM-guided 3D policy,” in2025 IEEE International Conference on Robotics and Au- tomation (ICRA). IEEE, 2025, pp. 8996–9002

    2025

  8. [16]

    RMBench: Memory-dependent robotic manipulation benchmark with insights into policy design,

    T. Chenet al., “RMBench: Memory-dependent robotic manipulation benchmark with insights into policy design,” 2026. [Online]. Available: https://arxiv.org/abs/2603.01229

    2026 arXiv

  9. [17]

    SAM2Act: Integrating visual foundation model with a memory architecture for robotic manipulation,

    H. Fanget al., “SAM2Act: Integrating visual foundation model with a memory architecture for robotic manipulation,” inInternational Confer- ence on Machine Learning. PMLR, 2025, pp. 15 925–15 942

    2025

  10. [18]

    BridgeVLA: Input-output alignment for efficient 3D ma- nipulation learning with vision-language models,

    P. Liet al., “BridgeVLA: Input-output alignment for efficient 3D ma- nipulation learning with vision-language models,”Advances in Neural Information Processing Systems, vol. 38, pp. 63 635–63 673, 2025

    2025

  11. [19]

    RT-1: Robotics transformer for real-world control at scale,

    A. Brohanet al., “RT-1: Robotics transformer for real-world control at scale,” inRobotics: Science and Systems (RSS), 2023

    2023

  12. [20]

    π 0: A vision-language-action flow model for general robot control,

    K. Blacket al., “π 0: A vision-language-action flow model for general robot control,” inRobotics: Science and Systems (RSS), 2025

    2025

  13. [21]

    FAST: Efficient action tokenization for vision- language-action models,

    K. Pertschet al., “FAST: Efficient action tokenization for vision- language-action models,” inRobotics: Science and Systems (RSS), 2025

    2025

  14. [22]

    π ∗ 0.6: a vla that learns from experience,

    Physical Intelligenceet al., “π ∗ 0.6: a vla that learns from experience,” 2025, arXiv:2511.14759

    2025 arXiv

  15. [23]

    π 0.7: a steerable generalist robotic foundation model with emergent capabilities,

    ——, “π 0.7: a steerable generalist robotic foundation model with emergent capabilities,” 2026, arXiv:2604.15483

    2026 arXiv

  16. [24]

    GEN-0: Embodied foundation models that scale with physical interaction,

    Generalist Team, “GEN-0: Embodied foundation models that scale with physical interaction,” Generalist AI Blog, 2025. [Online]. Available: https://generalistai.com/blog/gen-0

    2025

  17. [25]

    GEN-1: Scaling embodied foundation models to mastery,

    ——, “GEN-1: Scaling embodied foundation models to mastery,” Generalist AI Blog, 2026. [Online]. Available: https://generalistai.com/ blog/gen-1

    2026

  18. [26]

    GENE-26.5: Advancing robotic manipulation to human level,

    Genesis AI Team, “GENE-26.5: Advancing robotic manipulation to human level,” Genesis AI Blog, May 2026. [Online]. Available: https://genesis.ai/blog/ gene-26-5-advancing-robotic-manipulation-to-human-level

    2026

  19. [27]

    ACT-2 preview: Generalizing reliability,

    Sunday Robotics, “ACT-2 preview: Generalizing reliability,” Sunday Robotics Blog, Jul. 2026. [Online]. Available: https://www.sunday.ai/ blog/act-2-preview

    2026

  20. [28]

    Open X-Embodiment: Robotic learning datasets and RT-X models,

    A. O’Neillet al., “Open X-Embodiment: Robotic learning datasets and RT-X models,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 6892–6903

    2024

  21. [29]

    PolarNet: 3D point clouds for language-guided robotic manipulation,

    S. Chen, R. G. Pinel, C. Schmid, and I. Laptev, “PolarNet: 3D point clouds for language-guided robotic manipulation,” inConference on Robot Learning. PMLR, 2023, pp. 1761–1781

    2023

  22. [30]

    M2T2: Multi-task masked transformer for object-centric pick and place,

    W. Yuan, A. Murali, A. Mousavian, and D. Fox, “M2T2: Multi-task masked transformer for object-centric pick and place,” inConference on Robot Learning. PMLR, 2023, pp. 3619–3630

    2023

  23. [31]

    FP3: A 3D foundation policy for robotic manipulation,

    R. Yang, G. Chen, C. Wen, and Y . Gao, “FP3: A 3D foundation policy for robotic manipulation,” 2025, arXiv:2503.08950

    2025 arXiv

  24. [32]

    Coarse-to-fine Q- attention: Efficient learning for visual robotic manipulation via discreti- sation,

    S. James, K. Wada, T. Laidlow, and A. J. Davison, “Coarse-to-fine Q- attention: Efficient learning for visual robotic manipulation via discreti- sation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 13 739–13 748

    2022

  25. [33]

    PointVLA: Injecting the 3D world into vision-language-action models,

    C. Li, J. Wen, Y . Peng, Y . Peng, and Y . Zhu, “PointVLA: Injecting the 3D world into vision-language-action models,”IEEE Robotics and Automation Letters, vol. 11, no. 3, pp. 2506–2513, 2026

    2026

  26. [34]

    Lift3D policy: Lifting 2D foundation models for robust 3D robotic manipulation,

    Y . Jiaet al., “Lift3D policy: Lifting 2D foundation models for robust 3D robotic manipulation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 17 347–17 358

    2025

  27. [35]

    DINOv2: Learning robust visual features without supervision,

    M. Oquabet al., “DINOv2: Learning robust visual features without supervision,”Transactions on Machine Learning Research, 2024

    2024

  28. [36]

    OG- VLA: Orthographic image generation for 3D-aware vision-language action model,

    I. Singh, A. Goyal, S. Birchfield, D. Fox, A. Garg, and V . Blukis, “OG- VLA: Orthographic image generation for 3D-aware vision-language action model,” 2025, arXiv:2506.01196. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 15

    2025

  29. [37]

    Instruction-driven history-aware policies for robotic manipulations,

    P.-L. Guhur, S. Chen, R. G. Pinel, M. Tapaswi, I. Laptev, and C. Schmid, “Instruction-driven history-aware policies for robotic manipulations,” in Conference on Robot Learning. PMLR, 2023, pp. 175–187

    2023

  30. [38]

    Causal world modeling for robot control,

    L. Liet al., “Causal world modeling for robot control,” 2026, arXiv:2601.21998

    2026 arXiv

  31. [39]

    World-language-action model for unified world model- ing, language reasoning, and action synthesis,

    Y . Yanget al., “World-language-action model for unified world model- ing, language reasoning, and action synthesis,” 2026, arXiv:2606.05979

    2026 arXiv

  32. [40]

    MemoryW AM: Efficient world action modeling with persistent memory,

    S. Yanget al., “MemoryW AM: Efficient world action modeling with persistent memory,” 2026, arXiv:2606.20562

    2026 arXiv

  33. [41]

    TraceVLA: Visual trace prompting enhances spatial- temporal awareness for generalist robotic policies,

    R. Zhenget al., “TraceVLA: Visual trace prompting enhances spatial- temporal awareness for generalist robotic policies,” inInternational Conference on Learning Representations (ICLR), 2025

    2025

  34. [42]

    RoboMemory: A brain-inspired multi-memory agentic framework for interactive environmental learning in physical embodied systems,

    M. Leiet al., “RoboMemory: A brain-inspired multi-memory agentic framework for interactive environmental learning in physical embodied systems,” 2025, arXiv:2508.01415

    2025

  35. [43]

    MemoryVLA: Perceptual-cognitive memory in vision- language-action models for robotic manipulation,

    H. Shiet al., “MemoryVLA: Perceptual-cognitive memory in vision- language-action models for robotic manipulation,” inInternational Con- ference on Learning Representations (ICLR), 2026

    2026

  36. [44]

    Gated memory policy,

    Y . Gao, J. Liu, S. Li, and S. Song, “Gated memory policy,” 2026, arXiv:2604.18933

    2026 arXiv

  37. [45]

    You only scan once: A dynamic scene reconstruction pipeline for 6-DoF robotic grasping of novel objects,

    L. Zhou, H. Wang, Z. Zhang, Z. Liu, F. E. Tay, and M. H. Ang, “You only scan once: A dynamic scene reconstruction pipeline for 6-DoF robotic grasping of novel objects,” inIEEE International Conference on Robotics and Automation (ICRA), 2024

    2024

  38. [46]

    Mem-World: Memory-augmented action- conditioned world models for persistent robot manipulation,

    Z. Zhenget al., “Mem-World: Memory-augmented action- conditioned world models for persistent robot manipulation,” 2026, arXiv:2606.18960

    2026 arXiv

  39. [47]

    Coarse-to-fine imitation learning: Robot manipulation from a single demonstration,

    E. Johns, “Coarse-to-fine imitation learning: Robot manipulation from a single demonstration,” in2021 IEEE international conference on robotics and automation (ICRA). IEEE, 2021, pp. 4613–4619

    2021

  40. [48]

    RoboPoint: A vision-language model for spatial affordance prediction for robotics,

    W. Yuanet al., “RoboPoint: A vision-language model for spatial affordance prediction for robotics,” 2024, arXiv:2406.10721

    2024 arXiv

  41. [49]

    PaliGemma: A versatile 3B VLM for transfer,

    L. Beyeret al., “PaliGemma: A versatile 3B VLM for transfer,” 2024, arXiv:2407.07726

    2024 arXiv

  42. [50]

    Sigmoid loss for language image pre-training,

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, “Sigmoid loss for language image pre-training,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 11 975–11 986

    2023

  43. [51]

    Gemma: Open models based on Gemini research and technology,

    Gemma Teamet al., “Gemma: Open models based on Gemini research and technology,” 2024, arXiv:2403.08295

    2024 arXiv

  44. [52]

    RAFT: Recurrent all-pairs field transforms for optical flow,

    Z. Teed and J. Deng, “RAFT: Recurrent all-pairs field transforms for optical flow,” inComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II

    2020

  45. [53]

    Springer, 2020, pp. 402–419

    2020

  46. [54]

    On the continuity of rotation representations in neural networks,

    Y . Zhou, C. Barnes, J. Lu, J. Yang, and H. Li, “On the continuity of rotation representations in neural networks,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5745–5753

    2019

  47. [55]

    V-REP: A versatile and scalable robot simulation framework,

    E. Rohmer, S. P. Singh, and M. Freese, “V-REP: A versatile and scalable robot simulation framework,” in2013 IEEE/RSJ international conference on intelligent robots and systems. IEEE, 2013, pp. 1321– 1326

    2013

  48. [56]

    Perceiver IO: A general architecture for structured inputs & outputs,

    A. Jaegleet al., “Perceiver IO: A general architecture for structured inputs & outputs,” inInternational Conference on Learning Represen- tations, 2022

    2022

  49. [57]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chiet al., “Diffusion policy: Visuomotor policy learning via action diffusion,”The International Journal of Robotics Research, 2024

    2024

  50. [58]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” inRobotics: Science and Systems (RSS), 2023

    2023

  51. [59]

    X-VLA: Soft-prompted transformer as scalable cross- embodiment vision-language-action model,

    J. Zhenget al., “X-VLA: Soft-prompted transformer as scalable cross- embodiment vision-language-action model,” 2025, arXiv:2510.10274

    2025 arXiv

  52. [60]

    Fast-W AM: Do world action models need test-time future imagination?

    T. Yuan, Z. Dong, Y . Liu, and H. Zhao, “Fast-W AM: Do world action models need test-time future imagination?” 2026, arXiv:2603.16666

    2026 arXiv

  53. [61]

    R3M: A universal visual representation for robot manipulation,

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta, “R3M: A universal visual representation for robot manipulation,” 2022, arXiv:2203.12601

    2022 arXiv

  54. [62]

    Masked visual pre- training for motor control,

    T. Xiao, I. Radosavovic, T. Darrell, and J. Malik, “Masked visual pre- training for motor control,” 2022, arXiv:2203.06173

    2022 arXiv

  55. [63]

    RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipu- lation,

    T. Chenet al., “RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipu- lation,” 2025, arXiv:2506.18088

    2025 arXiv

  56. [64]

    SAPIEN: A simulated part-based interactive environ- ment,

    F. Xianget al., “SAPIEN: A simulated part-based interactive environ- ment,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 097–11 107

    2020

  57. [65]

    Point transformer V3: Simpler faster stronger,

    X. Wuet al., “Point transformer V3: Simpler faster stronger,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4840–4851. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 16 TABLE VII Per-benchmark fine-tuning config...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.