Pith. sign in

REVIEW 3 major objections 5 minor 102 references

RoboPearls: Editable Video Simulation for Robot Manipulation

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RoboPearls claims that demonstration videos can be rebuilt as editable, photo-realistic 3D simulations that make robot manipulation policies substantially more robust.

desk verdict A serious 3DGS simulation-for-robotics system whose headline gains are undercut by an under-specified VLM loop and no data-quantity control. read the letter →

arxiv 2506.22756 v1 pith:YNCPBSEE submitted 2025-06-28 cs.CV cs.RO

Tao Tang , Likui Zhang , Youpeng Wen , Kaidong Zhang , Jia-Wang Bian , xia zhou , Tianyi Yan , Kun Zhan
show 4 more authors
Peng Jia Hefeng Wu Liang Lin Xiaodan Liang
This is my paper
classification cs.CVcs.RO
keywords 3DGaussianSplattingvideosimulationrobotmanipulationeditableLLMagentsvision-languagemodelclosed-looplearningsim-to-real
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RoboPearls is an attempt to turn ordinary demonstration videos into a working simulation environment for robot policies. The paper claims that by reconstructing the scene as dynamic, semantic-labeled 3D Gaussians, one can edit it photorealistically, changing object colors, textures, sizes, removing or inserting objects, and even running physics, and then train manipulation policies on the edited footage. The payoff it reports is quantitative: on the COLOSSEUM robustness benchmark, policies trained with the edited simulations beat RVT and RVT2 by average success gains of 17.5% and 10.8%, and on RLBench it reports state-of-the-art gains of +16.4 and +23.0 on Stack Cups and Put in Cupboard. It also adds LLM agents to turn natural-language commands into edits and a VLM that examines failure cases and proposes new simulations, closing the loop. If correct, this would lower the cost of collecting diverse robot training data and shrink the sim-to-real gap without building a physics simulator for every scene.

What carries the argument

The load-bearing object is the dynamic semantic-enhanced Gaussian: a standard 3D Gaussian Splatting scene extended with a time dimension, following 4D Gaussian Splatting, and with per-Gaussian identity encodings $e_i$ that are trained with cross-entropy on SAM masks plus a KL constraint that keeps nearby Gaussians consistent in 3D. On top of this representation sit the operators: Incremental Semantic Distillation (ISD) for retrieving fine-grained objects that SAM's initial masks missed, and the 3D regularized NNFM loss ($L_{3D\text{-NNFM}} = L_{\text{NNFM}}^{M_{3d}} + L_{gs}^{M_{3d}}$), which applies an image-texture transfer loss only inside the target object's rendered mask while enforcing the reconstruction loss outside it. Multiple LLM agents orchestrate retrieval, editing, asset lookup, refinement, and rendering, and a VLM turns failure cases into new simulation commands. The identity encoding is what makes editing and retrieval possible; the mask-restricted loss is what keeps edits from corrupting the rest of the scene.

What would settle it

Run the full RoboPearls loop twice on the same COLOSSEUM tasks: once with failure cases drawn only from the held-out test episodes, and once with failure cases drawn from the same episodes used to generate training simulations, keeping the number of generated training examples identical across methods. If the second setting reproduces or exceeds the reported 17.5% gain, the central claim would be an artifact of test-distribution leakage rather than of the simulation framework.

Watch

Extended reading notes

Core claim

At the paper's center is a claim about representation: a video of a manipulation scene can be rebuilt as a set of 3D Gaussian primitives that carry not only position, color, opacity, and temporal dynamics, but also a learnable identity encoding $e_i$ per primitive, supervised by SAM's 2D masks. Once the scene is grouped this way, individual objects can be retrieved, deleted, inserted, recolored in CIELAB space, or retextured with a 3D regularized nearest-neighbor feature-matching loss that updates only the target object's spherical-harmonic coefficients. The paper's central reported discovery is empirical: feeding the resulting edited simulations back into manipulation policies produces consistent success-rate gains across every perturbation in COLOSSEUM and new state-of-the-art numbers on RLBench, and the authors attribute these gains to view-consistent, temporally coherent edits rather than to 2D image-space augmentation such as InstructPix2Pix. The VLM closed loop is the mechanism that lets the framework propose its own training scenarios from failure keyframes.

Load-bearing premise

The reported test improvements rest on the assumption that the failure cases fed to the VLM come from a held-out set and are not used to create the training simulations, and that RoboPearls is trained with the same amount of data as the baselines; the paper does not specify the failure-case source or include a data-quantity control.

Editorial extensions

If this is right

  • If the reported gains hold, demonstration video becomes a scalable data source: one recorded scene can yield many training variations without new robot time.
  • Closed-loop failure analysis with a VLM would let a policy propose its own corrective training data, reducing human involvement in data cleaning.
  • Because edits are made on 3D Gaussians rather than 2D frames, the same simulated variations remain view-consistent and can be used for multi-camera policies such as RVT and SAM2Act.
  • The operator set covers object-level changes, scene-level changes such as background texture and lighting, and physical simulation, so robustness tests like COLOSSEUM's 13 perturbations could be generated from a single demonstration video.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not run this control, but a testable extension would be to measure whether the VLM loop's improvement comes from the diagnosis or just from adding more training examples; a control that appends random edits of equal quantity would separate the two.
  • The same mask-restricted 3D NNFM loss could serve as a general object-centric editing primitive for Gaussian scenes outside robotics, for example in product visualization or film previsualization, where multi-view consistency matters.
  • If failure cases are deliberately held out and never used to build training simulations, the framework would become a systematic stress-test generator for manipulation policies, not just a data augmenter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes RoboPearls, a 3D Gaussian Splatting-based editable video simulation framework for robotic manipulation. It reconstructs dynamic, semantically segmented Gaussian scenes from demonstration videos and supports a broad set of editing operators (object retrieval, removal, insertion, color/texture/size/position modification, and physics simulation), orchestrated by multiple LLM agents that interpret natural-language commands. A vision-language model (VLM) is then used to analyze failure keyframes, infer possible causes, and issue simulation demands that are fed back into the framework to generate targeted training simulations. The authors evaluate the approach by training RVT, RVT2, and SAM2Act policies on the generated simulations and testing on COLOSSEUM, RLBench, a real-world Kinova robot, Ego4D, and Open X-Embodiment. They report average success gains of +17.5 and +10.8 over RVT and RVT2 on COLOSSEUM, gains of +16.4 and +23.0 on the RLBench Stack Cups and Put in Cupboard tasks, improved real-world success counts, and visual reconstruction metrics (PSNR 40.6, LPIPS 0.08).

Significance. If the central claims are supported, RoboPearls would be a meaningful step toward automated, editable simulation generation for robot policy training, with the potential to reduce human data-collection effort and to close the sim-to-real gap. The paper's strengths include the breadth of the operator suite, the ISD and 3D-NNFM modules, the LLM-agent automation, the VLM closed-loop idea, and the unusually wide evaluation across benchmarks, real-world datasets, and a physical robot. The visual reconstruction numbers are strong. However, the current experimental protocol does not yet establish the causal contribution of the editing operators or of the VLM: the paper does not specify the provenance of the VLM's failure cases, and no condition controls for the quantity of added training simulations. The significance of the headline performance gains is therefore conditional on additional controlled experiments.

major comments (3)
  1. [Sec. 3.5 and Sec. 4.1] The provenance of the VLM failure cases is unspecified. Section 3.5 states that a VLM analyzes keyframes of failure cases and generates simulation demands that are used to retrain the policy, but Section 4.1 describes only 96 training demonstrations and 25 unseen test demonstrations per task and does not define a validation split. If the failure cases are drawn from those 25 unseen demonstrations, then the policy is trained on simulations targeted at the exact test instances on which it is later scored, and the gains in Tables 1, 2, and 4 would be contaminated by test distribution leakage. The authors must specify where the failure cases come from, for example a separate validation set or a held-out subset of training failures, and ideally report a variant in which the VLM is conditioned only on validation failures that are never used for evaluation.
  2. [Sec. 4.3, Table 4; Sec. 4.2, Tables 1-2] The reported gains do not control for training-data quantity. In Table 4, RoboPearls (Ours) reflects both the simulation operators and the VLM-selected additional simulations, while the 'w/o VLM' condition and the RVT baseline do not receive a matched number of additional demonstrations; the paper does not state how many simulations each condition generated or how the non-VLM condition selected them. Consequently, the +17.5/+10.8 COLOSSEUM gains in Table 1 and the +16.4/+23.0 RLBench gains in Table 2 could be caused simply by adding more demonstrations rather than by the editing operators or the VLM's semantic selection. Please add data-matched baselines, such as equal counts of randomly selected or non-semantic simulations, and report the simulation count for every condition.
  3. [Sec. 4.2, Table 3] The real-world comparison with RDT is not data-matched and is statistically thin: each condition is evaluated on 20 trials with no variance reported, and the RoboPearls condition includes additional VLM-generated simulations while the RDT baseline does not. This experiment cannot, as reported, attribute the seen/unseen gains to the simulation framework rather than to extra data. A matched-data control, or an explicit statement that this is an uncontrolled case study, is needed.
minor comments (5)
  1. [Tables 2 and 4] The RVT baseline scores in Table 4 (14.5, 40.4, 11.0) do not match the Table 2 RVT scores for the same tasks (17.6, 50.4, 16.4); please clarify whether the evaluation protocol, random seed, or task subset differs.
  2. [Sec. 3.2, Sec. 3.4, Sec. 4.3, Appendix A.1.2] There are several typographical errors: 'Optimazition' in Sec. 3.2, 'Oveall Workflow' in Sec. 3.4, 'photo-photorealistic' in Sec. 4.3, and 'Limitaions' in the appendix heading A.1.2.
  3. [Sec. 3.5 and Appendix A.3.4] Please specify the VLM model used (e.g., GPT-4V) and the concrete protocol: how many keyframes are used per failure case, how many simulation demands are generated per failure, and what budget of generated simulations is used per training run.
  4. [Table 6] The 'Simulation (I2I-CLIP)' metric is not defined; please describe the reference images used and the scoring procedure.
  5. [Abstract] The abstract quotes gains of +16.4 and +23.0 on Stack Cups and Put in Cupboard without naming the baseline; in Table 2 these values correspond to comparisons against RVT2, and the text should state this to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No demonstrated circularity: the VLM closed loop is underspecified, but the paper contains no equation or protocol that makes its headline gains equal to its own inputs.

full rationale

RoboPearls' derivation chain is largely self-contained. Dynamic scene reconstruction uses MSE, semantic cross-entropy, and KL losses (Eq. 4) on SAM/G-DINO-derived labels and external tools (LAMA, libcom, ARF, PhysGaussian); the editing operators are evaluated with visual metrics (Table 6) and task success against external baselines, so no headline result is defined as the output of the system's own components. The only candidate for circularity is the closed-loop VLM in Sec. 3.5/A.3.4, which analyzes 'failure cases' and generates 'simulation demands' to retrain the policy. The paper does not state the provenance of these failure cases or a disjoint validation split, and Sec. A.3.5 only says 'We use 96 demonstrations per task for training and 25 unseen demonstrations for testing.' If the failure cases were drawn from those 25 test demonstrations, the reported gains in Tables 1, 2, and 4 could reflect test-distribution leakage. However, that is a conditional risk, not a by-construction reduction: no sentence in the paper asserts that the VLM's failure cases come from the test split, and the VLM output is a data-augmentation suggestion rather than a fitted parameter whose value is then reported as 'success.' Similarly, Table 4's w/o-VLM versus Ours comparison may be confounded by training-data quantity, but confounding is not circularity. The paper's self-citations (e.g., [74], [56]) appear only in related-work or component context and are not load-bearing for the central claim. I therefore find no significant circularity, while flagging the unspecified failure-case provenance as a separate correctness/reproducibility concern.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities. The time dimension and identity encoding are extensions of existing representations in 4DGS and Semantic Gaussians.

free parameters (3)
  • loss weights lambda_2d, lambda_sem, lambda_3d
    Weights in the combined optimization objective (Eq. 4) are chosen by hand; their values are not reported in the paper.
  • K (nearest neighbors in KL loss)
    Top-K nearest Gaussians used in the 3D consistency KL loss; the value of K is not specified.
  • physical parameters (density, Young's modulus, Poisson's ratio)
    Assigned manually or by GPT-4V for physics simulation; not reported and not used in the main policy experiments.
assumptions (5)
  • standard math 3D Gaussian Splatting and 4DGS dynamic reconstruction render photorealistic, view-consistent frames (Eqs. 1-2).
    Borrowed from Kerbl et al. and Yang et al.; assumed correct for the pipeline.
  • domain assumption SAM masks provide accurate enough object segmentation to supervise identity encodings.
    Used in Section 3.2 to train semantic Gaussians; if masks are noisy, grouping and editing fail.
  • domain assumption Identity encodings are time-invariant and inherited by new Gaussians during densification.
    Stated in A.3.1; needed for spatiotemporal consistency of object identities.
  • domain assumption G-DINO reliably detects objects and blurry holes for retrieval and inpainting.
    Used in ISD and object removal; failures would degrade editing quality.
  • domain assumption LLM and VLM agents correctly parse natural language commands and failure analyses.
    The whole automation and closed-loop enhancement rests on the competence of GPT-based models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RoboPearls: Editable Video Simulation for Robot Manipulation." pith.science (2026). https://pith.science/paper/YNCPBSEE

@misc{pith2026250622756,
  author       = {Pith},
  title        = {Pith review of: RoboPearls: Editable Video Simulation for Robot Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YNCPBSEE}},
  note         = {Machine review of arXiv:2506.22756}
}
read the original abstract

The development of generalist robot manipulation policies has seen significant progress, driven by large-scale demonstration data across diverse environments. However, the high cost and inefficiency of collecting real-world demonstrations hinder the scalability of data acquisition. While existing simulation platforms enable controlled environments for robotic learning, the challenge of bridging the sim-to-real gap remains. To address these challenges, we propose RoboPearls, an editable video simulation framework for robotic manipulation. Built on 3D Gaussian Splatting (3DGS), RoboPearls enables the construction of photo-realistic, view-consistent simulations from demonstration videos, and supports a wide range of simulation operators, including various object manipulations, powered by advanced modules like Incremental Semantic Distillation (ISD) and 3D regularized NNFM Loss (3D-NNFM). Moreover, by incorporating large language models (LLMs), RoboPearls automates the simulation production process in a user-friendly manner through flexible command interpretation and execution. Furthermore, RoboPearls employs a vision-language model (VLM) to analyze robotic learning issues to close the simulation loop for performance enhancement. To demonstrate the effectiveness of RoboPearls, we conduct extensive experiments on multiple datasets and scenes, including RLBench, COLOSSEUM, Ego4D, Open X-Embodiment, and a real-world robot, which demonstrate our satisfactory simulation performance.

Figures

Figures reproduced from arXiv: 2506.22756 by the authors.

Figure 1
Figure 1. RoboPearls, an editable video simulation framework for robotic manipulation. RoboPearls reconstructs photo-realistic scenes with semantic features from demonstration videos. Then, with various simulation operators, RoboPearls leverages multiple LLM agents to process user commands into specific editing functions. Furthermore, RoboPearls utilizes a VLM to analyze learning issues and generate corresponding simulation d… view at source ↗
Figure 2
Figure 2. (a) RoboPearls extends the Gaussian representation to reconstruct dynamic scenes with semantic features from demonstration videos. (b) RoboPearls includes and refines various simulation operators. (c) RoboPearls leverages multiple LLM agents to automate and streamline the simulation production process following user natural language commands. has demonstrated impressive open-world segmentation ca￾pability. Similar t… view at source ↗
Figure 3
Figure 3. The demonstrations for manipulation tasks in simulation (a) and the real world (b) (zoom-in for the best of views). the agent is instructed to “stack the other cups on top of the red cup”. The results indicate that the previous agent struggles to complete the task, whereas our RoboPearls pre￾cisely identifies each cup and successfully stacks them onto the red one. This improvement is attributed to our method’s effic… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: The photo-realistic simulations on in-the-wild datasets. Our RoboPearls supports various simulations. 4.3. Ablation Study We conduct quantitative and qualitative ablations to com￾prehensively evaluate our designed modules’ effects [PITH_FULL_IMAGE:figures/full_fig_p00…
Figure 5
Figure 5. Figure 5: Qualitative ablations on the proposed modules Quantitative ablations. In Tab. 4, we first validate the view-consistent 3D simulation capability of RoboPearls by comparing it with IP2P [5], which can be regarded as sim￾ulators on 2D image space. While IP2P provides limi…
Figure 6
Figure 6. Figure 6: After retrieving the desired object Gaussians, we render the 2D object mask and use G-DINO to verify whether it corresponds to the desired object. If the target [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: The object removal pipeline. Object Removal. The pipeline is shown in [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: The 3D asset management pipeline. 3D Asset Management. The pipeline is shown in [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: The texture modification pipeline (zoom-in for the best of views). Texture Modification. The pipeline is shown in [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: The prompts and examples of the LLM agents (zoom-in for the best of views). end-effector control, position and velocity control, and even wheeled locomotion. For our task, we primarily adjust the RDT chunk size to 4 and fill its actions to the right-arm portion of the…
Figure 11
Figure 11. Figure 11: The prompts and examples of the VLM (zoom-in for the best of views). A.5. Additional Visualizations A.5.1. Spatial-temporal Consistency [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: The detailed simulations (a) and more tasks (b) on RLBench (zoom-in for the best of views) [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: The visualizations of the spatial-temporal consis￾tency (zoom-in for the best of views) [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: The visualizations of the learned feature vectors [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

102 extracted references · 37 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Unsupervised pixel- level domain adaptation with generative adversarial net- works

    Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. Unsupervised pixel- level domain adaptation with generative adversarial net- works. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3722–3731, 2017. 3

    2017

  3. [3]

    Robocat: A self- improving foundation agent for robotic manipulation

    Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao, Co- line Devin, Alex X Lee, Maria Bauza, Todor Davchev, Yuxi- ang Zhou, Agrim Gupta, Akhil Raju, et al. Robocat: A self- improving foundation agent for robotic manipulation. arXiv preprint arXiv:2306.11706, 2023. 2

    arXiv 2023

  4. [4]

    Rt-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakr- ishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. 2

    arXiv 2022

  5. [5]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions. In CVPR, 2023. 5, 8, 16

    2023

  6. [6]

    Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulation

    Peng Chang and Taskin Padir. Sim2real2sim: Bridging the gap between simulation and real-world in flexible object ma- nipulation. arXiv preprint arXiv: Arxiv-2002.02538 , 2020. 3

    work page Pith review arXiv 2002

  7. [7]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19457–19467, 2024. 13

    2024

  8. [8]

    A survey on 3d gaussian splatting

    Guikun Chen and Wenguan Wang. A survey on 3d gaussian splatting. arXiv preprint arXiv:2401.03890, 2024. 2

    arXiv 2024

Show all 102 references
  1. [9]

    Gaussianeditor: Swift and controllable 3d editing with gaussian splatting, 2024

    Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting, 2024. 3, 5

    2024

  2. [10]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In European Conference on Computer Vision, pages 370–386. Springer, 2024. 13

    2024

  3. [11]

    Dynamic gaussian splatting from markerless motion capture reconstruct infants movements

    R James Cotton and Colleen Peyton. Dynamic gaussian splatting from markerless motion capture reconstruct infants movements. In Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision , pages 60–68,

  4. [12]

    Pybullet, a python mod- ule for physics simulation for games, robotics and machine learning

    Erwin Coumans and Yunfei Bai. Pybullet, a python mod- ule for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016. 2

    2016

  5. [13]

    A brief survey of sim2real methods for robot learning

    Konstantinos Dimitropoulos, Ioannis Hatzilygeroudis, and Konstantinos Chatzilygeroudis. A brief survey of sim2real methods for robot learning. In International Conference on Robotics in Alpe-Adria Danube Region , pages 133–140. Springer, 2022. 3

    2022

  6. [14]

    Vision-language models as success detectors

    Yuqing Du, Ksenia Konyushkova, Misha Denil, Akhil Raju, Jessica Landon, Felix Hill, Nando de Freitas, and Serkan Cabi. Vision-language models as success detectors. arXiv preprint arXiv:2303.07280, 2023. 6

    2023 arXiv

  7. [15]

    Policy transfer via kinematic domain randomization and adaptation

    Ioannis Exarchos, Yifeng Jiang, Wenhao Yu, and C Karen Liu. Policy transfer via kinematic domain randomization and adaptation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 45–51. IEEE, 2021. 3

    2021

  8. [16]

    Sam2act: Integrating visual foundation model with a mem- ory architecture for robotic manipulation

    Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox, Ranjay Krishna, and Jiafei Duan. Sam2act: Integrating visual foundation model with a mem- ory architecture for robotic manipulation. arXiv preprint arXiv:2501.18564, 2025. 7, 13, 15

    2025 arXiv

  9. [17]

    Gaussianeditor: Editing 3d gaussians delicately with text instructions

    Jiemin Fang, Junjie Wang, Xiaopeng Zhang, Lingxi Xie, and Qi Tian. Gaussianeditor: Editing 3d gaussians delicately with text instructions. 2024. 3, 5

    2024

  10. [18]

    Helix: A vision-language-action model for gener- alist humanoid control, 2025

    FigureAI. Helix: A vision-language-action model for gener- alist humanoid control, 2025. https://www.figure. ai/news/helix. 13

    2025

  11. [19]

    Act3d: 3d feature field transformers for multi-task robotic manipulation

    Theophile Gervet, Zhou Xian, Nikolaos Gkanatsios, and Ka- terina Fragkiadaki. Act3d: 3d feature field transformers for multi-task robotic manipulation. CoRL, 2023. 2, 13, 15

    2023

  12. [20]

    Rvt: Robotic view transformer for 3d object manipulation

    Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, and Dieter Fox. Rvt: Robotic view transformer for 3d object manipulation. In Conference on Robot Learning, pages 694–

  13. [21]

    Rvt-2: Learning precise manipulation from few demonstrations

    Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, and Dieter Fox. Rvt-2: Learning precise manipulation from few demonstrations. arXiv preprint arXiv:2406.08545, 2024. 7, 15

    2024 arXiv

  14. [22]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. InPro- ceedings of the IEEE/CVF Conference on Computer Vision...

    2022

  15. [23]

    Maniskill2: A unified bench- mark for generalizable manipulation skills

    Jiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yunchao Yao, et al. Maniskill2: A unified bench- mark for generalizable manipulation skills. arXiv preprint arXiv:2302.04659, 2023. 2, 3

    2023 arXiv

  16. [24]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. 14

    2025 arXiv

  17. [25]

    Semantic gaussians: Open-vocabulary scene understanding with 3d gaussian splatting, 2024

    Jun Guo, Xiaojian Ma, Yue Fan, Huaping Liu, and Qing Li. Semantic gaussians: Open-vocabulary scene understanding with 3d gaussian splatting, 2024. 3

    2024

  18. [26]

    Ankur Handa, Arthur Allshire, Viktor Makoviychuk, Aleksei Petrenko, Ritvik Singh, Jingzhou Liu, Denys Makoviichuk, Karl Van Wyk, Alexander Zhurkevich, Balakumar Sundar- alingam, and Yashraj S. Narang. Dextreme: Transfer of agile in-hand manipulation from simulation to reality....

    2023

  19. [27]

    A moving least squares material point method with displacement disconti- nuity and two-way rigid body coupling

    Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. A moving least squares material point method with displacement disconti- nuity and two-way rigid body coupling. ACM Transactions on Graphics (TOG), 37(4):1–14, 2018. 5

    2018

  20. [28]

    Domain ran- domization for sim2real transfer of automatically generated grasping datasets

    Johann Huber, Franc ¸ois H ´el´enon, Hippolyte Watrelot, Fa¨ız Ben Amar, and St ´ephane Doncieux. Domain ran- domization for sim2real transfer of automatically generated grasping datasets. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 4112–4118. IEEE,

    2024

  21. [29]

    Rlbench: The robot learning benchmark & learning environment

    Stephen James, Zicong Ma, David Rovick Arrojo, and An- drew J Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Let- ters, 5(2):3019–3026, 2020. 2, 3, 6, 15

    2020

  22. [30]

    Graspsplats: Efficient manipulation with 3d feature splatting

    Mazeyu Ji, Ri-Zhao Qiu, Xueyan Zou, and Xiaolong Wang. Graspsplats: Efficient manipulation with 3d feature splatting. arXiv preprint arXiv:2409.02084, 2024. 3, 13

    2024 arXiv

  23. [31]

    Seg- ment any 4d gaussians, 2024

    Shengxiang Ji, Guanjun Wu, Jiemin Fang, Jiazhong Cen, Taoran Yi, Wenyu Liu, Qi Tian, and Xinggang Wang. Seg- ment any 4d gaussians, 2024. 2

    2024

  24. [32]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4), 2023. 2, 3

    2023

  25. [33]

    Droid: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ash- win Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yun- liang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:24...

    2024 arXiv

  26. [34]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024. 2

    2024 arXiv

  27. [35]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 2, 3

    2023

  28. [36]

    Behavior-1k: A benchmark for embodied ai with 1,000 ev- eryday activities and realistic simulation

    Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Mart ´ın-Mart´ın, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior-1k: A benchmark for embodied ai with 1,000 ev- eryday activities and realistic simulation. In Conferenc...

    2023

  29. [37]

    Langsurf: Language-embedded surface gaussians for 3d scene under- standing

    Hao Li, Roy Qin, Zhengyu Zou, Diqi He, Bohan Li, Bingquan Dai, Dingewn Zhang, and Junwei Han. Langsurf: Language-embedded surface gaussians for 3d scene under- standing. arXiv preprint arXiv:2412.17635, 2024. 4

    2024

  30. [38]

    Libero: Benchmarking knowl- edge transfer for lifelong robot learning

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowl- edge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  31. [39]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 4

    2023 arXiv

  32. [40]

    Rdt-1b: a diffusion foundation model for bimanual manipu- lation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipu- lation. arXiv preprint arXiv:2410.07864, 2024. 2, 7, 15

    2024 arXiv

  33. [41]

    Uncom- mon objects in 3d

    Xingchen Liu, Piyush Tayal, Jianyuan Wang, Jesus Zarzar, Tom Monnier, Konstantinos Tertikas, Jiali Duan, Antoine Toisoul, Jason Y Zhang, Natalia Neverova, et al. Uncom- mon objects in 3d. arXiv preprint arXiv:2501.07574, 2025. 5

    2025 arXiv

  34. [42]

    Robo-gs: A physics consistent spatial- temporal model for robotic arm with hybrid representation

    Haozhe Lou, Yurong Liu, Yike Pan, Yiran Geng, Jianteng Chen, Wenlong Ma, Chenglong Li, Lin Wang, Hengzhen Feng, Lu Shi, et al. Robo-gs: A physics consistent spatial- temporal model for robotic arm with hybrid representation. arXiv preprint arXiv:2408.14873, 2024. 3

    2024 arXiv

  35. [43]

    Turbo-gs: Accelerating 3d gaussian fitting for high- quality radiance fields

    Tao Lu, Ankit Dhiman, R Srinath, Emre Arslan, Angela Xing, Yuanbo Xiangli, R Venkatesh Babu, and Srinath Srid- har. Turbo-gs: Accelerating 3d gaussian fitting for high- quality radiance fields. arXiv preprint arXiv:2412.13547 ,

    arXiv

  36. [44]

    Shapes- plat: A large-scale dataset of gaussian splats and their self- supervised pretraining

    Qi Ma, Yue Li, Bin Ren, Nicu Sebe, Ender Konukoglu, Theo Gevers, Luc Van Gool, and Danda Pani Paudel. Shapes- plat: A large-scale dataset of gaussian splats and their self- supervised pretraining. arXiv preprint arXiv:2408.10906 ,

    arXiv

  37. [45]

    What mat- ters in learning from offline human demonstrations for robot manipulation

    Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiri- any, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Mart´ın-Mart´ın. What mat- ters in learning from offline human demonstrations for robot manipulation. arXiv preprint arXiv:2108.03298, 2021. 2

    2021 arXiv

  38. [46]

    A language agent for autonomous driving

    Jiageng Mao, Junjie Ye, Yuxi Qian, Marco Pavone, and Yue Wang. A language agent for autonomous driving. arXiv preprint arXiv:2311.10813, 2023. 14

    2023 arXiv

  39. [47]

    Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manip- ulation tasks

    Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wol- fram Burgard. Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manip- ulation tasks. IEEE Robotics and Automation Letters , 7(3): 7327–7334, 2022. 2

    2022

  40. [48]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 2021. 3

    2021

  41. [49]

    20 years of reality gap: a few thoughts about simulators in evo- lutionary robotics

    Jean-Baptiste Mouret and Konstantinos Chatzilygeroudis. 20 years of reality gap: a few thoughts about simulators in evo- lutionary robotics. In Proceedings of the genetic and evo- lutionary computation conference companion , pages 1121– 1124, 2017. 3

    2017

  42. [50]

    Making images real again: A comprehensive survey on deep image composition

    Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, and Liqing Zhang. Making images real again: A comprehensive survey on deep image composition. arXiv preprint arXiv:2106.14490, 2021. 5

    2021

  43. [51]

    Nvidia isaac sim, 2021

    NVIDIA. Nvidia isaac sim, 2021. 2

    2021

  44. [52]

    Open x-embodiment: Robotic learning datasets and rt-x models

    Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, et al. Open x-embodiment: Robotic learning datasets and rt-x models. arXiv preprint arXiv:2310.08864, 2023. 2, 6

    arXiv

  45. [53]

    Gpt-4v(ision) system card, 2023

    OpenAI. Gpt-4v(ision) system card, 2023. https:// openai.com/research/gpt-4v-vision. 5

    2023

  46. [54]

    Gags: Granularity-aware feature distillation for language gaussian splatting

    Yuning Peng, Haiping Wang, Yuan Liu, Chenglu Wen, Zhen Dong, and Bisheng Yang. Gags: Granularity-aware feature distillation for language gaussian splatting. arXiv preprint arXiv:2412.13654, 2024. 4

    2024 arXiv

  47. [55]

    The colosseum: A bench- mark for evaluating generalization for robotic manipulation

    Wilbert Pumacay, Ishika Singh, Jiafei Duan, Ranjay Krishna, Jesse Thomason, and Dieter Fox. The colosseum: A bench- mark for evaluating generalization for robotic manipulation. arXiv preprint arXiv:2402.08191, 2024. 2, 6, 15

    2024 arXiv

  48. [56]

    Diffusiongpt: Llm-driven text-to-image generation system

    Jie Qin, Jie Wu, Weifeng Chen, Yuxi Ren, Huixia Li, Hefeng Wu, Xuefeng Xiao, Rui Wang, and Shilei Wen. Diffusiongpt: Llm-driven text-to-image generation system. arXiv preprint arXiv:2401.10061, 2024. 14

    2024

  49. [57]

    Feature splatting: Language-driven physics-based scene syn- thesis and editing, 2024

    Ri-Zhao Qiu, Ge Yang, Weijia Zeng, and Xiaolong Wang. Feature splatting: Language-driven physics-based scene syn- thesis and editing, 2024. 3

    2024

  50. [58]

    V-rep: A versatile and scalable robot simulation framework

    Eric Rohmer, Surya PN Singh, and Marc Freese. V-rep: A versatile and scalable robot simulation framework. In 2013 IEEE/RSJ international conference on intelligent robots and systems, pages 1321–1326. IEEE, 2013. 15

    2013

  51. [59]

    Robofail: Ana- lyzing failures in robot learning policies

    Som Sagar and Ransalu Senanayake. Robofail: Ana- lyzing failures in robot learning policies. arXiv preprint arXiv:2412.02818, 2024. 6

    2024 arXiv

  52. [60]

    Toolflownet: Robotic manipulation with tools via predicting tool flow from point clouds

    Daniel Seita, Yufei Wang, Sarthak J Shetty, Edward Yao Li, Zackory Erickson, and David Held. Toolflownet: Robotic manipulation with tools via predicting tool flow from point clouds. In Conference on Robot Learning, pages 1038–1049. PMLR, 2023. 2

    2023

  53. [61]

    Hi robot: Open-ended instruction following with hierarchical vision- language-action models

    Lucy Xiaoyang Shi, Brian Ichter, Michael Equi, Liyim- ing Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, et al. Hi robot: Open-ended instruction following with hierarchical vision- language-action models. arXiv preprint arXiv:2502.19417,

    arXiv

  54. [62]

    Perceiver- actor: A multi-task transformer for robotic manipulation

    Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver- actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning , pages 785–799. PMLR,

  55. [63]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. In WACV, 2022. 4

    2022

  56. [64]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. In European Conference on Computer Vision, pages 1–18. Springer, 2024. 5

    2024

  57. [65]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024. 2

    2024 arXiv

  58. [66]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems , pages 5026–5033. IEEE, 2012. 2

    2012

  59. [67]

    Cy- berdemo: Augmenting simulated human demonstration for real-world dexterous manipulation

    Jun Wang, Yuzhe Qin, Kaiming Kuang, Yigit Korkmaz, Akhilan Gurumoorthy, Hao Su, and Xiaolong Wang. Cy- berdemo: Augmenting simulated human demonstration for real-world dexterous manipulation. arXiv preprint arXiv: Arxiv-2402.14795, 2024. 3

    2024 arXiv

  60. [68]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. In CVPR, 2024. 15

    2024

  61. [69]

    Tenenbaum, and Chuang Gan

    Tsun-Hsuan Wang, Andrew Everett Spielberg, Pingchuan Ma, Zhou Xian, Hao Zhang, Joshua B. Tenenbaum, and Chuang Gan. Softzoo: A soft robot co-design benchmark for locomotion in diverse environments. In International Con- ference on Learning Representations, 2023. 2

    2023

  62. [70]

    Robogen: Towards unleashing infi- nite data for automated robot learning via generative simula- tion

    Yufei Wang, Zhou Xian, Feng Chen, Tsun-Hsuan Wang, Yian Wang, Katerina Fragkiadaki, Zackory Erickson, David Held, and Chuang Gan. Robogen: Towards unleashing infi- nite data for automated robot learning via generative simula- tion. arXiv preprint arXiv:2311.01455, 2023. 14

    2023 arXiv

  63. [71]

    Editable scene simulation for autonomous driving via collaborative llm-agents

    Yuxi Wei, Zi Wang, Yifan Lu, Chenxin Xu, Changxing Liu, Hao Zhao, Siheng Chen, and Yanfeng Wang. Editable scene simulation for autonomous driving via collaborative llm-agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15077– 15...

    2024

  64. [72]

    Fabricflownet: Bimanual cloth manipulation with a flow-based policy

    Thomas Weng, Sujay Man Bajracharya, Yufei Wang, Khush Agrawal, and David Held. Fabricflownet: Bimanual cloth manipulation with a flow-based policy. In Conference on Robot Learning, pages 192–202. PMLR, 2022. 2

    2022

  65. [73]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering

  66. [74]

    Gaussctrl: Multi-view consistent text-driven 3d gaussian splatting edit- ing, 2024

    Jing Wu, Jia-Wang Bian, Xinghui Li, Guangrun Wang, Ian Reid, Philip Torr, and Victor Adrian Prisacariu. Gaussctrl: Multi-view consistent text-driven 3d gaussian splatting edit- ing, 2024. 3, 5

    2024

  67. [75]

    Autogen: Enabling next-gen llm ap- plications via multi-agent conversation framework

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen llm ap- plications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155, 2023. 14

    2023 arXiv

  68. [76]

    Recent advances in 3d gaussian splatting

    Tong Wu, Yu-Jie Yuan, Ling-Xiao Zhang, Jie Yang, Yan- Pei Cao, Ling-Qi Yan, and Lin Gao. Recent advances in 3d gaussian splatting. Computational Visual Media, 10(4):613– 642, 2024. 2

    2024

  69. [77]

    Chaineddiffuser: Uni- fying trajectory diffusion and keypose prediction for robotic manipulation

    Zhou Xian, Nikolaos Gkanatsios, Theophile Gervet, Tsung- wei Ke, and Katerina Fragkiadaki. Chaineddiffuser: Uni- fying trajectory diffusion and keypose prediction for robotic manipulation. Conference on Robot Learning, 2023. 2

    2023

  70. [78]

    Fluid- lab: A differentiable environment for benchmarking com- plex fluid manipulation

    Zhou Xian, Bo Zhu, Zhenjia Xu, Hsiao-Yu Tung, Antonio Torralba, Katerina Fragkiadaki, and Chuang Gan. Fluid- lab: A differentiable environment for benchmarking com- plex fluid manipulation. arXiv preprint arXiv:2303.02346 ,

    arXiv

  71. [79]

    Sapien: A simulated part-based interactive environment

    Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, et al. Sapien: A simulated part-based interactive environment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11...

    2020

  72. [80]

    Physgaussian: Physics- integrated 3d gaussians for generative dynamics

    Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Physgaussian: Physics- integrated 3d gaussians for generative dynamics. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4389–4398, 2024. 3, 5

    2024

  73. [81]

    Gaussianproperty: Integrat- ing physical properties to 3d gaussians with lmms

    Xinli Xu, Wenhang Ge, Dicong Qiu, ZhiFei Chen, Dongyu Yan, Zhuoyun Liu, Haoyu Zhao, Hanfeng Zhao, Shunsi Zhang, Junwei Liang, et al. Gaussianproperty: Integrat- ing physical properties to 3d gaussians with lmms. arXiv preprint arXiv:2412.11258, 2024. 5

    2024 arXiv

  74. [82]

    Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation. In European Con- ference on Computer Vision, pages 1–20. Springer, 2024. 5

    2024

  75. [83]

    Fouhey, and Joyce Chai

    Jianing Yang, Xuweiyi Chen, Shengyi Qian, Nikhil Madaan, Madhavan Iyengar, David F. Fouhey, and Joyce Chai. Llm- grounder: Open-vocabulary 3d visual grounding with large language model as an agent, 2023. 14

    2023

  76. [84]

    Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. arXiv preprint arXiv:2310.10642, 2023. 3

    2023 arXiv

  77. [85]

    Gaussian grouping: Segment and edit anything in 3d scenes

    Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaussian grouping: Segment and edit anything in 3d scenes. In European Conference on Computer Vision , pages 162–

  78. [86]

    Large batch optimiza- tion for deep learning: Training bert in 76 minutes

    Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Dem- mel, Kurt Keutzer, and Cho-Jui Hsieh. Large batch optimiza- tion for deep learning: Training bert in 76 minutes. arXiv preprint arXiv:1904.00962, 2019. 15

    1904 arXiv

  79. [87]

    Meta- world: A benchmark and evaluation for multi-task and meta reinforcement learning

    Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta- world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning , pages 1094–1100. PMLR, 2020. 3

    2020

  80. [88]

    Learning to manipulate any- where: A visual generalizable framework for reinforcement learning

    Zhecheng Yuan, Tianming Wei, Shuiqi Cheng, Gu Zhang, Yuanpei Chen, and Huazhe Xu. Learning to manipulate any- where: A visual generalizable framework for reinforcement learning. arXiv preprint arXiv:2407.15815, 2024. 3

    2024 arXiv

  81. [89]

    Gnfactor: Multi-task real robot learn- ing with generalizable neural feature fields

    Yanjie Ze, Ge Yan, Yueh-Hua Wu, Annabella Macaluso, Yuying Ge, Jianglong Ye, Nicklas Hansen, Li Erran Li, and Xiaolong Wang. Gnfactor: Multi-task real robot learn- ing with generalizable neural feature fields. arXiv preprint arXiv:2308.16891, 2023. 15

    2023 arXiv

  82. [90]

    Arf: Artistic radiance fields

    Kai Zhang, Nick Kolkin, Sai Bi, Fujun Luan, Zexiang Xu, Eli Shechtman, and Noah Snavely. Arf: Artistic radiance fields. In European Conference on Computer Vision, pages 717–733. Springer, 2022. 5

    2022

  83. [91]

    On scaling up 3d gaussian splatting training

    Hexu Zhao, Haoyang Weng, Daohan Lu, Ang Li, Jinyang Li, Aurojit Panda, and Saining Xie. On scaling up 3d gaussian splatting training. arXiv preprint arXiv:2406.18533 , 2024. 13

    2024

  84. [92]

    Gpdan: Grasp pose domain adaptation network for sim-to-real 6-dof object grasping

    Liming Zheng, Wenxuan Ma, Yinghao Cai, Tao Lu, and Shuo Wang. Gpdan: Grasp pose domain adaptation network for sim-to-real 6-dof object grasping. IEEE Robotics and Automation Letters, 8(8):4585–4592, 2023. 3

    2023

  85. [93]

    Gaussiangrasper: 3d language gaussian splatting for open-vocabulary robotic grasping, 2024

    Yuhang Zheng, Xiangyu Chen, Yupeng Zheng, Songen Gu, Runyi Yang, Bu Jin, Pengfei Li, Chengliang Zhong, Zeng- mao Wang, Lina Liu, Chao Yang, Dawei Wang, Zhen Chen, Xiaoxiao Long, and Meiqing Wang. Gaussiangrasper: 3d language gaussian splatting for open-vocabulary robotic grasp...

    2024

  86. [94]

    Robot parkour learning

    Ziwen Zhuang, Zipeng Fu, Jianren Wang, Christopher Atke- son, Soeren Schwertfeger, Chelsea Finn, and Hang Zhao. Robot parkour learning. arXiv preprint arXiv:2309.05665 ,

    arXiv

  87. [98]

    The training data is summarized in a thumbnail that displays the learned demonstrations

  88. [99]

    Task: Your task is to analyze the failure cases and provide a reasonable explanation for the failure, along with suggested improvements

    The failure cases are provided in several keyframe images, which visually indicate the cause of the failure. Task: Your task is to analyze the failure cases and provide a reasonable explanation for the failure, along with suggested improvements. You should focus on factors suc...

  89. [100]

    Failure Reason: <chosen reason>

  90. [101]

    Explanation: <explanation of why the failure occurred, based on the images and training data provided>

  91. [102]

    Explanation: In the training data provided (first image), the cups are predominantly in shades of red, purple, and green

    Solution: <suggested improvement for the training data to fix the issue> Failure Reason: The colors of the objects in the training data were not varied enough, leading to poor generalization by the model. Explanation: In the training data provided (first image), the cups are pre...

  92. [179]

    Springer, 2024. 2, 3, 4

    2024

  93. [710]

    2, 7, 8, 13, 15

    PMLR, 2023. 2, 7, 8, 13, 15

    2023

  94. [2023]

    blurry hole

    3 RoboPearls: Editable Video Simulation for Robot Manipulation Supplementary Material Contents A.1. Social Impact and Limitations. . . . . . . . . 13 A.1.1. Future Work . . . . . . . . . . . . . . . . . 13 A.1.2. Limitaions . . . . . . . . . . . . . . . . . . 13 A.1.3. Scope ....

    1918

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.