Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A hierarchical Gaussian world model that predicts future scenes in a leader-follower order improves bimanual robot imitation by 20.2 percent in simulation and reaches 60 percent average success on real-world tasks.

desk verdict Clever and clearly specified bimanual extension of ManiGaussian, but the headline numbers contain arithmetic errors and the ablations are too thin to support the 20.2-point gain. read the letter →

arxiv 2506.19842 v1 pith:AYFPN3OE submitted 2025-06-24 cs.RO cs.AI

Tengbo Yu , Guanxing Lu , Zaijia Yang , Haoyuan Deng , Season Si Chen , Jiwen Lu , Wenbo Ding , Guoqiang Hu
show 2 more authors
Yansong Tang Ziwei Wang
This is my paper · ORCID
classification cs.ROcs.AI
keywords bimanualmanipulationGaussianworldmodelSplattingimitationlearningfuturescenepredictionvisualrepresentationleader-followerarchitecture
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the reason bimanual imitation policies fail is that their visual representations do not capture how the two arms interact through the scene over time, and it proposes a fix: a hierarchical Gaussian world model that predicts future scenes in two ordered steps. First a 'leader' predicts how the stabilizing arm deforms a task-oriented 3D Gaussian field, then a 'follower' predicts the physical consequences of the acting arm on top of that deformation. Training the representation with this future-prediction objective, alongside standard behavior cloning, lifts average success from 15.4 percent to 35.6 percent across ten simulated RLBench2 tasks and to a 60 percent average success rate across nine real-robot tasks. The paper's point is that ordered, body-aware future prediction is a representation-level improvement for bimanual manipulation, not just an extra loss term.

What carries the argument

The central object is the hierarchical Gaussian world model built on task-oriented Gaussian Splatting. In this model the scene is a set of Gaussian particles with positions, colors, orientations, scales, opacities, and an instance logit; the instance logit is learned with supervision from a pre-trained open-vocabulary detector prompted with keywords from the human instruction, distinguishing stabilizing arm, acting arm, and target objects. The dynamics are factorized as a leader deformation field $q_{s,\phi}$, which predicts the $\mathrm{SE}(3)$ displacement of Gaussians caused by the stabilizing arm, and a follower deformation field $q_{a,\phi}$, which predicts the acting arm's additional displacement conditioned on the leader's output. Only rigid-body motion of the particles is predicted, with appearance properties held fixed. This factorization is what lets the model separate the question 'what does the stabilizing arm do to the scene?' from 'what does the acting arm do given that?', and it is the inductive bias that the paper argues is missing from prior single-stream world models.

What would settle it

Take a bimanual task with symmetric simultaneous coupling, such as two grippers rotating a single rigid object together where no arm is the clear stabilizer at any point, and train ManiGaussian++ alongside an identical pipeline with a single joint (non-hierarchical) future-prediction model on that task; if the leader-follower version does not beat the joint model in success rate, the causal-ordering assumption is the wrong explanation for the gains reported here.

Watch

Extended reading notes

Core claim

ManiGaussian++ is an extension of the ManiGaussian framework from single-arm to bimanual manipulation. It builds a task-oriented Gaussian Splatting representation in which each Gaussian particle carries an instance logit distilled from a pretrained vision-language model, labelling which arm or object it belongs to, so the two manipulators and the target can be separated. On top of this, a hierarchical Gaussian world model decomposes the future scene prediction into a leader step and a follower step: the leader deformation field predicts the Gaussian Splatting deformation caused by the stabilizing arm's motion, and the follower deformation field, taking the leader's output as well as both actions, predicts the acting arm's physical consequences. Rendered future images are compared with ground truth through photometric and task-mask losses, forcing the intermediate volumetric representation to encode multi-body spatiotemporal dynamics. The resulting representation is fed to a PerceiverIO policy head that predicts discrete action bins for both arms. The authors' claim is that this two-step, role-aware future-prediction objective is what allows a single policy to generalize across ten simulated and nine real-world bimanual tasks, where prior single-stream world models fail.

Load-bearing premise

The load-bearing premise is that bimanual dynamics can be causally ordered: the stabilizing arm's motion deforms the scene first, and the acting arm's effect is generated conditioned on that deformation, so tasks where both arms act simultaneously with mutual coupling would break this inductive bias.

Editorial extensions

If this is right

  • Bimanual policies can be improved without new demonstrations: the future-prediction objective provides free supervision from existing multi-view RGB-D data, and the real-world experiments train on only 30 demonstrations per task.
  • The gains are representation-level rather than policy-architecture-level, so the hierarchical world model can be attached to existing language-conditioned policy heads like PerceiverIO.
  • Task-oriented instance labels that distinguish stabilizing and acting arms are required for the dynamics factorization to work; ablations show that adding them improves success from 20.0 percent to 26.7 percent on selected tasks, and adding the hierarchy then raises it to 60.0 percent.
  • Because the model renders future scenes from explicit Gaussian particles, it also yields interpretable future-prediction visualizations that trace which arm causes which scene change, such as the gripper and broom positions in the sweep-to-dustpan task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence is that the leader-follower ordering will matter most when the two arms' roles are asymmetric and clear; tasks with symmetric, simultaneous, mutually coupled arm motion may require a bidirectional or joint dynamics model, and on such tasks the reported advantage should shrink or invert.
  • Since the instance labels come from a pre-trained vision-language model, the method's ceiling is tied to that model's ability to segment the right objects; in clutter or with novel objects the world model may fail to separate bodies, so robustness could be improved by learning the role assignment from the task rather than distilling it.
  • The world model predicts only rigid-body motion and keeps appearance fixed, so it is likely to underperform on deformable-object or fluid tasks; folding clothes is included in the real tasks, but a stress test on liquids or fabric with visible shape change would clarify the limits of the rigid-Gaussian assumption.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes ManiGaussian++, a bimanual manipulation agent that extends the ManiGaussian framework. The method first builds a task-oriented Gaussian Splatting field from volumetric features, using VLM-generated instance labels to distinguish a stabilizing arm from an acting arm. It then learns a hierarchical Gaussian world model in which a leader deformation model predicts the scene deformation caused by the stabilizing arm, and a follower model predicts the physical consequences of the acting arm conditioned on the leader's output. This future-scene prediction serves as an auxiliary self-supervision for the visual representation used by a PerceiverIO policy. The paper reports a 20.2 percentage-point improvement over PerAct2 on 10 RLBench2 tasks and a 60% success rate across 9 real-world tasks.

Significance. The proposed hierarchy is a reasonable idea: explicitly separating stabilizing and acting arm roles before predicting future Gaussian deformation is more structured than a monolithic world model, and the task-oriented Gaussian Splatting with VLM supervision is a concrete, practical contribution. The paper also provides code, and the experiments cover both simulation and a real dual-arm setup. If the reported results can be reproduced, the method would be a useful step for multi-task bimanual manipulation. However, the quantitative support is weakened by internal numeric inconsistencies in the main tables and by the absence of multiple seeds or error bars, so the central claim of a 20.2% improvement is not currently established at the reported level of reliability.

major comments (4)
  1. [Section IV.B, Table I] The reported average for ManiGaussian++ is 35.6%, but the ten per-task success rates (12, 40, 8, 48, 40, 28, 48, 20, 92, 16) sum to 352, an average of 35.2%. The abstract's 20.2% improvement is computed from the reported averages (35.6 versus 15.4), so the central quantitative claim depends on a number that does not match the per-task evidence. Please correct the table and recompute all relative improvements.
  2. [Section IV.C, Table II] Row 4 reports success rates of 92, 20, and 48 with an average of 60.00%, but the arithmetic mean is 53.33%. This overstates the marginal gain of the hierarchical Gaussian world model relative to Row 3: the true gain is 26.67 percentage points (26.67 to 53.33), not the reported 33.33 percentage points (26.67 to 60.00). The ablation also covers only 3 manually selected tasks out of 10, uses a single evaluation seed, and provides no error bars, so even after correcting the arithmetic the attribution of the performance gain to the hierarchical world model is not statistically supported.
  3. [Section IV.E and abstract] The abstract states a 60% average real-world success rate, but Section IV.E reports 62.22% for ManiGaussian++ (and the relative improvements of 100% and 36.57% are computed from 62.22%). The real-robot evaluation uses 10 episodes per task with no per-task breakdown, variance, or multiple seeds. Please reconcile the numbers and report per-task success rates with confidence intervals or at least a per-task table.
  4. [Section III.D, Eq. (5)] The leader-follower decomposition of bimanual dynamics is a strong causal assumption: the follower receives the leader's deformed Gaussian field and both actions, effectively assuming that the stabilizing arm's motion can be predicted first and the acting arm's effect is conditioned on it. The paper provides no experiment that varies this ordering (e.g., predicting acting-arm deformation first, or predicting both simultaneously), so it is unclear whether the reported gains come from this specific inductive bias or from the auxiliary future-prediction objective in general. A comparison against an alternative ordering, or against a non-hierarchical world model with the same Gaussian backbone, is needed to support the mechanistic claim.
minor comments (5)
  1. [Table II caption] The caption says '12 RLBench2 TASK', but the benchmark described in Section IV.A has 10 tasks; please correct the task count.
  2. [Section IV.E] The phrase 'without any pertaining on the simulation' should read 'without any pre-training on the simulation'.
  3. [Throughout] There are typographical errors: 'Additionaly' should be 'Additionally' and 'underling' should be 'underlining'; a careful proofreading pass would help.
  4. [Table I] The PerAct2 row appears to have only nine numeric entries as typeset, while the table has ten task columns; please verify that all rows have ten per-task values.
  5. [Section III.E, Eq. (8)] The text says predicted future images from multiple views are aligned, but Eq. (8) writes a single-image L2 loss; please clarify whether the loss is summed or averaged over views.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the bimanual world-model claim is evaluated against external benchmarks and ground-truth future images, not against the method's own outputs.

full rationale

The central claim—that a leader-follower hierarchical Gaussian world model improves bimanual manipulation—is supported by behavior cloning on external RLBench 2 tasks (L_BC, Eq. 9) and by auxiliary losses that compare predicted future images to ground-truth future images (L_Pred, Eq. 8; L_Recon, Eq. 6). The world model does not predict its own training labels: future scenes are supervised by actually executed actions and multi-view images. The leader-follower decomposition (Eqs. 4–5) is an inductive bias about causal ordering, not a result derived from the model's own predictions. The manuscript cites ManiGaussian [30], whose authors overlap with the present paper, but only as a prior single-arm baseline and motivation; the bimanual extension and its gains are measured against that baseline externally. Therefore no reduction-by-construction or fitted-input-renamed-as-prediction is present. The reported averages in Table I (35.6% vs. sum of 35.2%) and Table II (60.00% vs. arithmetic mean of 53.33%) are internally inconsistent, and the ablation lacks error bars over a single seed; these are correctness/reliability concerns, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on benchmark and modeling assumptions rather than on any formal derivation. The main load-bearing choices are the leader-follower causal decomposition, the VLM-mask supervision, and the rigid-body SE(3) motion model for Gaussians.

free parameters (3)
  • lambda_Recon (current scene reconstruction loss weight) = not reported
    Loss-balancing weight in Equation 10 is tuned but its value is not given; results depend on it.
  • lambda_Task (task-oriented mask loss weight) = not reported
    Loss-balancing weight in Equation 10 is tuned but its value is not given; results depend on it.
  • lambda_Pred (future scene prediction loss weight) = not reported
    Loss-balancing weight in Equation 10 is tuned but its value is not given; results depend on it.
assumptions (4)
  • domain assumption The dominant Gaussian movement is a rigid-body SE(3) transform following the Newton-Euler equation; color, scale, opacity, and logits stay constant along the transition (Section III.D, Equation 4).
    The world model predicts only SE(3) deltas and keeps other attributes unchanged; this is a modeling choice, not a consequence derived in the paper.
  • domain assumption VLM prompts (GroundedSAM) produce correct ground-truth instance labels for both arms and target objects at every step (Section III.C, Equation 7).
    The task-oriented mask loss treats VLM outputs as ground truth; errors in open-vocabulary detection would propagate into the world-model training.
  • ad hoc to paper Bimanual interaction is causally ordered: the stabilizing arm's motion drives the deformation the follower then refines with the acting arm's action (Section III.D, Equation 5).
    This leader-follower ordering is invented for this method; simultaneous or coupled arm dynamics would violate it.
  • domain assumption RLBench2 oracle demonstrations and success criteria fairly measure multi-task bimanual policy generalization (Section IV.A).
    The benchmark defines task success; the generalization claim is only as strong as the benchmark protocol.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model." pith.science (2026). https://pith.science/paper/AYFPN3OE

@misc{pith2026250619842,
  author       = {Pith},
  title        = {Pith review of: ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AYFPN3OE}},
  note         = {Machine review of arXiv:2506.19842}
}
read the original abstract

Multi-task robotic bimanual manipulation is becoming increasingly popular as it enables sophisticated tasks that require diverse dual-arm collaboration patterns. Compared to unimanual manipulation, bimanual tasks pose challenges to understanding the multi-body spatiotemporal dynamics. An existing method ManiGaussian pioneers encoding the spatiotemporal dynamics into the visual representation via Gaussian world model for single-arm settings, which ignores the interaction of multiple embodiments for dual-arm systems with significant performance drop. In this paper, we propose ManiGaussian++, an extension of ManiGaussian framework that improves multi-task bimanual manipulation by digesting multi-body scene dynamics through a hierarchical Gaussian world model. To be specific, we first generate task-oriented Gaussian Splatting from intermediate visual features, which aims to differentiate acting and stabilizing arms for multi-body spatiotemporal dynamics modeling. We then build a hierarchical Gaussian world model with the leader-follower architecture, where the multi-body spatiotemporal dynamics is mined for intermediate visual representation via future scene prediction. The leader predicts Gaussian Splatting deformation caused by motions of the stabilizing arm, through which the follower generates the physical consequences resulted from the movement of the acting arm. As a result, our method significantly outperforms the current state-of-the-art bimanual manipulation techniques by an improvement of 20.2% in 10 simulated tasks, and achieves 60% success rate on average in 9 challenging real-world tasks. Our code is available at https://github.com/April-Yz/ManiGaussian_Bimanual.

Figures

Figures reproduced from arXiv: 2506.19842 by the authors.

Figure 1
Figure 1. Consider the human instruction ”Bring me the yellow item”, where the task is considered successful if the right arm handover the yellow block to the left arm. The previous method (Peract2 [15]) attempts to pick up the yellow block but fails to do so, while our ManiGaussian++ completes the task successfully by explicitly encoding the scene dynamics via future scene reconstruction in Gaussian embedding space. diverse … view at source ↗
Figure 2
Figure 2. The overall pipeline of ManiGaussian++. The task-oriented Gaussian radiance assigns unique labels to task-relevant agents and objects, and the hierarchical Gaussian world model upon it predicts future scenes in a leader-follower manner to encode the multi-body dynamics for bimanual manipulation. III. APPROACH In this section, we first briefly introduce preliminaries on bimanual manipulation, and then we depict an ov… view at source ↗
Figure 3
Figure 3. Novel View Synthesis Results. Our ManiGaussian++ captures the multi-body spatiotemporal dynamics precisely, while ManiGaussian fails to model it. Note that we turn off the behavior cloning loss for better illustration. Real Robot. The experimental setup consists of two Uni￾versal Robots UR5e arms equipped with Robotiq 2F-85 grippers, controlled via two Xbox controllers to collect demonstration data. Two RGB-D Realse… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Real-world Results. We train and evaluate Peract2 , ManiGaussian and ManiGaussian++ on 9 challenging real-world tasks. (a) Experiment Setups “Play ping pong” “Fold the clothes” (b) Complex Tasks [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Real-World Experiments with two UR5e manipulators. shape cannot be seen, our ManiGaussian++ offers superior detail in modeling buttons and grippers in novel views. Second, our method accurately predicts future states based on the recovered details. For example, in the …

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A geometry-consistent multi-view RGBD 4D world model for robot manipulation, whose actions are recovered by test-time optimization of a learned trajectory latent, outperforming single- and dual-view world-model baseli...

  2. SPGrasp: Spatiotemporal Prompt-driven Grasp Synthesis in Dynamic Scenes

    cs.RO 2025-08 conditional novelty 6.0 of 10

    SPGrasp extends SAM2 with a spatiotemporal memory and object pointer so that one prompt can keep generating grasps for a moving object across later frames at about 73 ms per frame.

  3. The ALMA-QUARKS Survey: III. Clump-to-core fragmentation and search for high-mass starless cores

    astro-ph.GA 2025-08 unverdicted novelty 4.0 of 10

    In 139 infrared-bright massive protoclusters, ALMA resolves 1562 cores whose separations are much smaller than the Jeans length, and finds only two candidate high-mass starless cores.

Reference graph

Works this paper leans on

52 extracted references · 34 canonical work pages · cited by 3 Pith papers

  1. [1]

    Physically embodied gaussian splatting: Embedding physical priors into a visual 3d world model for robotics,

    J. Abou-Chakra, K. Rana, F. Dayoub, and N. S ¨underhauf, “Physically embodied gaussian splatting: Embedding physical priors into a visual 3d world model for robotics,” inCoRL, 2023

    work page 2023

  2. [2]

    A dual-arm collaborative robot system for the smart factories of the future,

    J. F. Buhl, R. Grønhøj, J. K. Jørgensen, G. Mateus, D. Pinto, J. K. Sørensen, S. Bøgh, and D. Chrysostomou, “A dual-arm collaborative robot system for the smart factories of the future,”Procedia manufac- turing, vol. 38, pp. 333–340, 2019

    work page 2019

  3. [3]

    Polarnet: 3d point clouds for language-guided robotic manipulation,

    S. Chen, R. Garcia, C. Schmid, and I. Laptev, “Polarnet: 3d point clouds for language-guided robotic manipulation,”arXiv preprint arXiv:2309.15596, 2023

    arXiv 2023

  4. [4]

    Active vision might be all you need: Exploring active vision in bimanual robotic manipulation,

    I. Chuang, A. Lee, D. Gao, and I. Soltani, “Active vision might be all you need: Exploring active vision in bimanual robotic manipulation,” 2024

    work page 2024

  5. [5]

    Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning,

    R. Ding, Y . Qin, J. Zhu, C. Jia, S. Yang, R. Yang, X. Qi, and X. Wang, “Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning,” 2024

    2024

  6. [6]

    Rein- forcement learning with neural radiance fields,

    D. Driess, I. Schubert, P. Florence, Y . Li, and M. Toussaint, “Rein- forcement learning with neural radiance fields,”NeurIPS, 2022

    work page 2022

  7. [7]

    Learning universal policies via text-guided video generation,

    Y . Du, S. Yang, B. Dai, H. Dai, O. Nachum, J. Tenenbaum, D. Schu- urmans, and P. Abbeel, “Learning universal policies via text-guided video generation,”NeurIPS, vol. 36, 2024

    work page 2024

  8. [8]

    Bi-kvil: Keypoints- based visual imitation learning of bimanual manipulation tasks,

    J. Gao, X. Jin, F. Krebs, N. Jaquier, and T. Asfour, “Bi-kvil: Keypoints- based visual imitation learning of bimanual manipulation tasks,” 2024

    work page 2024

Show all 52 references
  1. [9]

    Enhance sample efficiency and robustness of end-to-end urban autonomous driving via semantic masked world model,

    Z. Gao, Y . Mu, R. Shen, C. Chen, Y . Ren, J. Chen, S. E. Li, P. Luo, and Y . Lu, “Enhance sample efficiency and robustness of end-to-end urban autonomous driving via semantic masked world model,”arXiv preprint arXiv:2210.04017, 2022

    arXiv

  2. [10]

    Bi-vla: Vision-language-action model-based system for bimanual robotic dexterous manipulations,

    K. F. Gbagbe, M. A. Cabrera, A. Alabbas, O. Alyunes, A. Lykov, and D. Tsetserukou, “Bi-vla: Vision-language-action model-based system for bimanual robotic dexterous manipulations,” 2024

    2024

  3. [11]

    Act3d: 3d feature field transformers for multi-task robotic manipulation,

    T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki, “Act3d: 3d feature field transformers for multi-task robotic manipulation,” in CoRL, 2023, pp. 3949–3965

    2023

  4. [12]

    Rvt2: Learning precise manipulation from few demonstrations,

    A. Goyal, V . Blukis, J. Xu, Y . Guo, Y .-W. Chao, and D. Fox, “Rvt2: Learning precise manipulation from few demonstrations,”RSS, 2024

    2024

  5. [13]

    Rvt: Robotic view transformer for 3d object manipulation,

    A. Goyal, J. Xu, Y . Guo, V . Blukis, Y .-W. Chao, and D. Fox, “Rvt: Robotic view transformer for 3d object manipulation,”arXiv preprint arXiv:2306.14896, 2023

    2023 arXiv

  6. [14]

    Stabilize to act: Learning to coordinate for bimanual manipulation,

    J. Grannen, Y . Wu, B. Vu, and D. Sadigh, “Stabilize to act: Learning to coordinate for bimanual manipulation,” inCoRL. PMLR, 2023, pp. 563–576

    2023

  7. [15]

    Peract2: Benchmarking and learning for robotic bimanual manipulation tasks,

    M. Grotz, M. Shridhar, T. Asfour, and D. Fox, “Peract2: Benchmarking and learning for robotic bimanual manipulation tasks,” 2024

    2024

  8. [16]

    Recurrent world models facilitate policy evolution,

    S. Ha, David and Jurgen, “Recurrent world models facilitate policy evolution,”NeurIPS, vol. 31, 2018

    2018

  9. [17]

    Deep hierarchical planning from pixels,

    D. Hafner, K.-H. Lee, I. Fischer, and P. Abbeel, “Deep hierarchical planning from pixels,”NeurIPS, vol. 35, pp. 26 091–26 104, 2022

    2022

  10. [18]

    Dream to con- trol: Learning behaviors by latent imagination,

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to con- trol: Learning behaviors by latent imagination,”arXiv preprint arXiv:1912.01603, 2019

    1912 arXiv

  11. [19]

    Mastering atari with discrete world models,

    D. Hafner, T. Lillicrap, M. Norouzi, and J. Ba, “Mastering atari with discrete world models,”arXiv preprint arXiv:2010.02193, 2020

    2010 arXiv

  12. [20]

    Mastering diverse domains through world models,

    D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap, “Mastering diverse domains through world models,”arXiv preprint arXiv:2301.04104, 2023

    2023 arXiv

  13. [21]

    Td-mpc2: Scalable, robust world models for continuous control,

    N. Hansen, H. Su, and X. Wang, “Td-mpc2: Scalable, robust world models for continuous control,”arXiv preprint arXiv:2310.16828, 2023

    2023 arXiv

  14. [22]

    Generalization in reinforcement learning by soft data augmentation,

    N. Hansen and X. Wang, “Generalization in reinforcement learning by soft data augmentation,” inICRA, 2021

    2021

  15. [23]

    Towards human-robot collaborative surgery: Trajectory and strategy learning in bimanual peg transfer,

    Z. J. Hu, Z. Wang, Y . Huang, A. Sena, F. Rodriguez y Baena, and E. Burdet, “Towards human-robot collaborative surgery: Trajectory and strategy learning in bimanual peg transfer,”RA-L, vol. 8, no. 8, pp. 4553–4560, 2023

    2023

  16. [24]

    Perceiver: General perception with iterative attention,

    A. Jaegle, F. Gimeno, A. Brock, O. Vinyals, A. Zisserman, and J. Carreira, “Perceiver: General perception with iterative attention,” inICML, 2021

    2021

  17. [25]

    Rlbench: The robot learning benchmark & learning environment,

    S. James, Z. Ma, D. Arrojo, David Rovick, and A. J, “Rlbench: The robot learning benchmark & learning environment,”RA-L, 2020

    2020

  18. [26]

    3d gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,”TOG, vol. 42, no. 4, 2023

    2023

  19. [27]

    Surgical robot transformer (srt): Imitation learning for surgical tasks,

    J. W. Kim, T. Z. Zhao, S. Schmidgall, A. Deguet, M. Kobilarov, C. Finn, and A. Krieger, “Surgical robot transformer (srt): Imitation learning for surgical tasks,”arXiv preprint arXiv:2407.12998, 2024

    2024 arXiv

  20. [28]

    V oxact-b: V oxel-based acting and stabilizing policy for bimanual manipulation,

    I.-C. A. Liu, S. He, D. Seita, and G. Sukhatme, “V oxact-b: V oxel-based acting and stabilizing policy for bimanual manipulation,” 2024

    2024

  21. [29]

    Robo-gs: A physics consistent spatial-temporal model for robotic arm with hybrid representation,

    H. Lou, Y . Liu, Y . Pan, Y . Geng, J. Chen, W. Ma, C. Li, L. Wang, H. Feng, L. Shi,et al., “Robo-gs: A physics consistent spatial-temporal model for robotic arm with hybrid representation,”arXiv preprint arXiv:2408.14873, 2024

    2024 arXiv

  22. [30]

    Manigaussian: Dynamic gaussian splatting for multi-task robotic manipulation,

    G. Lu, S. Zhang, Z. Wang, C. Liu, J. Lu, and Y . Tang, “Manigaussian: Dynamic gaussian splatting for multi-task robotic manipulation,” in ECCV. Springer, 2025, pp. 349–366

    2025

  23. [31]

    Structured world models from human videos,

    R. Mendonca, S. Bahl, and D. Pathak, “Structured world models from human videos,”arXiv preprint arXiv:2308.10901, 2023

    2023 arXiv

  24. [32]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,”CACM, vol. 65, no. 1, pp. 99–106, 2021

    2021

  25. [33]

    R3m: A universal visual representation for robot manipulation,

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta, “R3m: A universal visual representation for robot manipulation,”arXiv, 2022

    2022

  26. [34]

    The unsurprising effectiveness of pre-trained vision models for control,

    S. Parisi, A. Rajeswaran, S. Purushwalkam, and A. Gupta, “The unsurprising effectiveness of pre-trained vision models for control,” inICML, 2022

    2022

  27. [35]

    Gaussian splatting to real world flight navigation transfer with liquid networks,

    A. Quach, M. Chahine, A. Amini, R. Hasani, and D. Rus, “Gaussian splatting to real world flight navigation transfer with liquid networks,” arXiv preprint arXiv:2406.15149, 2024

    2024 arXiv

  28. [36]

    Real-world robot learning with masked visual pre-training,

    I. Radosavovic, T. Xiao, S. James, P. Abbeel, J. Malik, and T. Darrell, “Real-world robot learning with masked visual pre-training,” inCoRL, 2023

    2023

  29. [37]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan,et al., “Grounded sam: Assembling open-world models for diverse visual tasks,”arXiv preprint arXiv:2401.14159, 2024

    2024 arXiv

  30. [38]

    Masked world models for visual control,

    Y . Seo, D. Hafner, H. Liu, F. Liu, S. James, K. Lee, and P. Abbeel, “Masked world models for visual control,” inCoRL, 2023, pp. 1332– 1344

    2023

  31. [39]

    Reinforcement learning with action-free pre-training from videos,

    Y . Seo, K. Lee, S. L. James, and P. Abbeel, “Reinforcement learning with action-free pre-training from videos,” inICML, 2022, pp. 19 561– 19 579

    2022

  32. [40]

    Snerl: Semantic-aware neural radiance fields for reinforcement learning,

    D. Shim, S. Lee, and H. J. Kim, “Snerl: Semantic-aware neural radiance fields for reinforcement learning,”ICML, 2023

    2023

  33. [41]

    Splat-mover: multi-stage, open- vocabulary robotic manipulation via editable gaussian splatting,

    O. Shorinwa, J. Tucker, A. Smith, A. Swann, T. Chen, R. Firoozi, M. D. Kennedy, and M. Schwager, “Splat-mover: multi-stage, open- vocabulary robotic manipulation via editable gaussian splatting,” in CoRL, 2024

    2024

  34. [42]

    Perceiver-actor: A multi-task transformer for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Perceiver-actor: A multi-task transformer for robotic manipulation,” inCoRL, 2023

    2023

  35. [43]

    Drivedreamer: Towards real-world-driven world models for autonomous driving,

    X. Wang, Z. Zhu, G. Huang, X. Chen, and J. Lu, “Drivedreamer: Towards real-world-driven world models for autonomous driving,” arXiv preprint arXiv:2309.09777, 2023

    2023 arXiv

  36. [44]

    Day- dreamer: World models for physical robot learning,

    P. Wu, A. Escontrela, D. Hafner, P. Abbeel, and K. Goldberg, “Day- dreamer: World models for physical robot learning,” inCoRL, 2023, pp. 2226–2240

    2023

  37. [45]

    Ace: A cross-platform visual-exoskeletons system for low-cost dexterous teleoperation,

    S. Yang, M. Liu, Y . Qin, R. Ding, J. Li, X. Cheng, R. Yang, S. Yi, and X. Wang, “Ace: A cross-platform visual-exoskeletons system for low-cost dexterous teleoperation,” 2024

    2024

  38. [46]

    Visual reinforce- ment learning with self-supervised 3d representations,

    Y . Ze, N. Hansen, Y . Chen, M. Jain, and X. Wang, “Visual reinforce- ment learning with self-supervised 3d representations,”RA-L, 2023

    2023

  39. [47]

    Gnfactor: Multi-task real robot learning with generalizable neural feature fields,

    Y . Ze, G. Yan, Y .-H. Wu, A. Macaluso, Y . Ge, J. Ye, N. Hansen, L. E. Li, and X. Wang, “Gnfactor: Multi-task real robot learning with generalizable neural feature fields,” inCoRL. PMLR, 2023, pp. 284– 301

    2023

  40. [48]

    Dynamic 3d gaussian tracking for graph-based neural dynamics modeling,

    M. Zhang, K. Zhang, and Y . Li, “Dynamic 3d gaussian tracking for graph-based neural dynamics modeling,” inCoRL, 2024

    2024

  41. [49]

    Empowering embodied manipulation: A bimanual-mobile robot manipulation dataset for household tasks,

    T. Zhang, D. Li, Y . Li, Z. Zeng, L. Zhao, L. Sun, Y . Chen, X. Wei, Y . Zhan, L. Li, and X. He, “Empowering embodied manipulation: A bimanual-mobile robot manipulation dataset for household tasks,” 2024

    2024

  42. [50]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” 2023

    2023

  43. [51]

    Occworld: Learning a 3d occupancy world model for autonomous driving,

    W. Zheng, W. Chen, Y . Huang, B. Zhang, Y . Duan, and J. Lu, “Occworld: Learning a 3d occupancy world model for autonomous driving,” inECCV. Springer, 2025, pp. 55–72

    2025

  44. [52]

    Spa: 3d spatial-awareness enables effective embodied representation,

    H. Zhu, H. Yang, Y . Wang, J. Yang, L. Wang, and T. He, “Spa: 3d spatial-awareness enables effective embodied representation,”arXiv preprint arXiv:2410.08208, 2024

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.