Pith. sign in

REVIEW 4 major objections 8 minor 41 references

Learning Generalizable 3D Manipulation With 10 Demonstrations

T0 review · 4 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a 3D manipulation policy trained on just 10 demonstrations can generalize to new object poses and camera viewpoints, beating current point-cloud diffusion policies by 60–70% in success rate.

desk verdict The striking performance margin over DP3 is carried by a rotation augmentation whose coordinate frames are unspecified; the literal reading of Eq. 4 would generate physically invalid training data. read the letter →

arxiv 2411.10203 v1 pith:OACAWTEI submitted 2024-11-15 cs.CV cs.RO

Yu Ren , Yang Cong , Ronghan Chen , Jiahao Long This is my paper · ORCID
classification cs.CVcs.RO
keywords 3Dmanipulationimitationlearningdiffusionpolicypointcloudfew-shotspatialgeneralizationequivariantaugmentationrobot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that generalization in few-shot imitation learning comes from encoding the spatial relationship between gripper and objects, not from memorizing trajectories. It proposes a perception module that turns RGB-D frames into a task-focused point cloud expressed in a moving gripper-centered frame, and a diffusion decision module trained while rotating the point cloud, gripper pose, and demonstrated actions together. If the claim holds, a robot could be taught a new manipulation skill from a handful of human demonstrations and then handle objects placed elsewhere or observed from another camera angle. The authors report that the method keeps above 80% success on all tested benchmark tasks while previous methods fall below 30% on several of them.

What carries the argument

The load-bearing mechanism is the spatially equivariant training strategy expressed in Eq. (4): a random rotation $T$ is applied jointly to the point cloud $p$, the gripper pose $h$, and the action $a$, so that the relative gripper–object relation is preserved while the absolute pose changes. This is carried by Semantic Guided Perception, which builds the point cloud in a dynamically selected gripper-base reference frame and balances point sampling across semantic categories, and by Spatial Generalized Decision, a diffusion denoiser that predicts action noise conditioned on point-cloud and gripper features.

What would settle it

Train the same framework with and without spatial augmentation on a pouring or oriented-insertion task whose success depends on absolute object or gripper orientation; if the augmented policy fails where the unaugmented one succeeds, the rotation-invariance premise is false for that task. Alternatively, evaluate the trained policy on objects rotated 90 degrees about each of the three axes; a collapse under non-z rotations would show the learned invariance is narrower than claimed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that spatial generalization with very few demonstrations is learnable if the policy is trained on the relative geometry between the gripper and the manipulated objects. The key move is to express the scene in the gripper base frame, sample point clouds by semantic category, and augment each trajectory by applying one random rotation simultaneously to the point cloud, the gripper pose, and the action sequence. This makes the diffusion policy see many spatial variants of the same relative pattern, so at test time it generates actions from observed object poses instead of reproducing a memorized path. The paper reports an average success rate of 97.6% across the ten hardest tasks compared with 55.9% for the strongest baseline, and reports graceful degradation, not collapse, when the object start region is enlarged sixfold or the camera is moved along or rotated around all six degrees of freedom.

Load-bearing premise

The tasks must be invariant under the random rotation applied simultaneously to the scene, gripper, and actions; if success depends on an absolute direction such as gravity or a fixed object orientation, the augmented examples teach the wrong behavior.

Editorial extensions

If this is right

  • Training on ten demonstrations per task yields at least 80% success on every one of the ten selected benchmark tasks, where the strongest comparison method drops below 30% on several tasks.
  • The reported average improvement in success rate over the point-cloud diffusion baselines is 60–70%, under the same evaluation protocol of sixty trials per task.
  • When the object initialization region is expanded to two, four, and six times its training size, the method retains most of its success while the leading baseline falls to near zero on some tasks.
  • The policy tolerates camera translation and rotation in all six degrees of freedom, a setting in which the comparison method often scores zero.
  • Since the augmentation acts on the input representation rather than the network architecture, the training strategy can be grafted onto any policy that consumes point clouds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the symmetry assumption is as general as the paper suggests, the same joint-rotation augmentation could be combined with SO(3)-equivariant network backbones to push the demonstration count below ten, and with canonical orientation frames to handle tasks that are not rotation-invariant.
  • The category-balanced sampling makes performance depend on the quality of the video object segmentation; mask drift or a missed object category would likely degrade success in ways the reported aggregate numbers do not expose.
  • A natural stress test is to rotate objects out of the plane used for augmentation: if the policy fails under 90-degree out-of-plane rotations, the effective equivariance is a single-axis group rather than full 3D spatial generalization.
  • The claimed 60–70% gain is measured against three specific baselines on ten tasks; a broader set of tasks, seeds, and real-world trials would show how much of the margin is due to the augmentation versus the perception changes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes a framework for learning manipulation policies from 10 demonstrations that generalizes to spatial variations (object initial poses, camera viewpoints). The framework combines Semantic Guided Perception (SGP), which builds task-focused point clouds in a gripper-base reference frame, with Spatial Generalized Decision (SGD), a diffusion-based action generator. The key novelty is a "spatially equivariant training strategy" that jointly rotates point cloud, gripper pose, and action data as data augmentation. The authors report simulation results on 10 MetaWorld tasks, showing success rates over 80% and a 60-70% relative improvement over DP3, ManiCM, and Diffusion Policy, plus ablation studies attributing much of the gain to the augmentation, and a brief real-world demonstration.

Significance. If the empirical claims are correct, this is a practically important result: it suggests that a simple, task-agnostic rotation augmentation, combined with a gripper-frame point cloud, can make diffusion policies generalize substantially from only 10 demonstrations. The SGP perception pipeline (SAM + video segmentation + category-balanced sampling) is a reasonable engineering contribution, and the augmentation strategy is easy to integrate into other point-cloud policies. The authors ship a code link and report experiments across many MetaWorld tasks and a real robot, which are strengths. However, the main empirical claims currently lack statistical quantification, and the validity of the central augmentation depends on coordinate-frame details that are not specified; if those details are wrong or inconsistent, the reported gains could be an artifact. The significance therefore hinges on the clarifications requested in the major comments.

major comments (4)
  1. [Section III-C, Eq. (4)] The coordinate frames for p, h, and a are unspecified, and this is load-bearing for the central claim. Section III-B states that the point cloud p is aligned to the gripper base at each timestep, while Section III defines the action a only as "relative movement" of the gripper. In the MetaWorld action space used by the baselines, actions are typically world-frame end-effector deltas. If that is the case, applying the same random rotation T to p, h, and a in Eq. (4) does not correspond to a rigid rotation of the physical scene: rotating p about the gripper frame while rotating a about the world origin produces training trajectories that the physical robot cannot execute. The ambiguity is not resolved by Figure 4, which only displays z-angle values. Since Table II shows that removing this augmentation drops average success from 97.6 to 43.2, the entire performance gain depends on this step. Please specify the rotation group, the frame in which T is applied, the coordinate frame of a, and how actions and gripper poses are transformed, and justify that the transformation is a valid symmetry for each selected task.
  2. [Table I and Section IV-B] The reported success rates are only means over 60 trials with three seeds, with no variance, per-seed breakdown, or statistical comparison against baselines. The abstract claims a 60-70% improvement and "consistently above 80%" success, but with no confidence intervals or significance tests it is impossible to know whether the differences are robust or whether they reflect a few favorable seeds. Please provide standard deviations or per-seed results, and, if possible, paired evaluations over the same initial states with the baselines.
  3. [Section IV-A] The 10 evaluation tasks are selected as "the 10 most challenging tasks" with baseline success rates below 50%. This selection criterion can exaggerate the reported improvement because it discards tasks where baselines already perform well, and it may introduce selection bias. Please clarify whether this task list was fixed before experiments were run, and report results on a broader or standard MetaWorld suite, or at least discuss how the improvement varies across easier tasks.
  4. [Sections IV-D and IV-E] The generalization experiments lack the actual perturbation magnitudes: the initial-region expansion is reported only as multiples of the original size without metric units, and Table IV reports success rates per camera motion direction (δtx, δty, δtz, δrx, δry, δrz) without stating the applied translation or rotation amounts. Figure 5 shows settings but no scale. Without these values, the claim of "substantial" spatial variations and strong viewpoint generalization cannot be assessed quantitatively. Please report the exact perturbation ranges and camera poses.
minor comments (8)
  1. [Section IV-F] The real-world experiments section states only that tasks were "successfully executed" without any quantitative success rates, number of trials, or task details; if the supplementary material contains such data, a summary should appear in the main text, otherwise the real-world validation claim in the abstract is unsupported.
  2. [Conclusion] The claim that this is "the first work to apply trajectory augmentation in 3D manipulation learning" is too strong and not substantiated by the related-work discussion; prior work on rotation augmentation and equivariant policies should be cited or the claim should be qualified.
  3. [Section III-C and Figure 4] If the augmentation is restricted to rotations about the gripper z-axis, that restriction should be stated explicitly, along with the rationale for why this subgroup is sufficient for the tested tasks; otherwise the reader cannot tell whether the method assumes full SO(3) or planar rotation symmetry.
  4. [Table IV] Table IV is difficult to read because the column headers δtx, δty, ..., δrz appear to mix perturbation directions and possibly perturbation magnitudes; please separate the input perturbation values from the output success rates and label units clearly.
  5. [Section IV-B] The text says "60 trials for each task, using different random seeds (0, 1, 2)"; please clarify how the 60 trials are allocated across the three seeds (e.g., 20 trials per seed) so that the amount of data per seed is unambiguous.
  6. [Equation (2)] The notation α̂k and β̂k is used without definition; please define these quantities or reference the exact noise scheduler from [10] or [11].
  7. [References] Reference [7] duplicates reference [2]; one should be removed or both should be cited appropriately.
  8. [Section I] There is a typo in the introduction: "Spatial Generalized Decesion" should be "Spatial Generalized Decision."

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the framework is validated by held-out empirical evaluation, and no claim reduces to its own inputs.

full rationale

The paper makes an empirical claim: with 10 demonstrations and a spatially equivariant data-augmentation strategy, a point-cloud diffusion policy achieves high success rates and generalizes to spatial and viewpoint variations. This is not a derivation-from-first-principles claim, so the circularity analysis focuses on whether any reported result is forced by construction, by fitted inputs relabeled as predictions, or by load-bearing self-citation. No such reduction appears. The training objective in Eq. (4) is a standard diffusion denoising loss applied to augmented inputs; the augmentation T(p), T(h), T(a) is a training-time input transformation, and the reported generalization is measured on held-out initial-region and camera-viewpoint settings (Tables III and IV), not on the training distribution. The ablation in Table II attributes performance to the augmentation and reference-frame choices, which is a component analysis rather than a circular validation. The only self-citation is reference [3] in a background sentence about service robots performing housework; it plays no role in the method, equations, or experimental claims. The under-specification of the rotation group in the augmentation is a reproducibility and correctness concern, but it is not circularity: even if the literal reading of Eq. (4) produces invalid trajectories, that would be an implementation or modeling error, not an equivalence between the claimed generalization and the training input. Accordingly, the honest finding is no significant circularity, with score 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework depends on the unstated symmetry assumption behind the rotation augmentation, the reliability of SAM/Cutie segmentation at test time, the consistency of the gripper-base frame, and standard diffusion training. These are domain assumptions, not derived guarantees.

free parameters (4)
  • Point count and category sampling ratio = 1024 total points, equal across categories
    Chosen by hand; affects representation fidelity and background interference. No ablation on point count.
  • Diffusion training horizon, observation steps, action steps = 4, 2, 4
    Hyperparameters taken from diffusion policy literature; no sensitivity analysis in this paper.
  • Training epochs = 3000
    Fixed for all tasks without justification or ablation on convergence.
  • Rotation augmentation distribution = Not reported
    The paper applies a random rotation T to point cloud, gripper pose, and actions, but does not specify the distribution (e.g., uniform over [0,2pi) about the z-axis). This choice directly affects generalization and is a free parameter of the method.
assumptions (4)
  • domain assumption Tasks are invariant under rotations in the augmentation group, so jointly rotating point cloud, gripper pose, and actions preserves valid demonstrations.
    Spatially Equivariant Training, Section III-C and Figure 4. The paper does not analyze which tasks break this symmetry.
  • domain assumption SAM and Cutie segmentation reliably identify task-relevant objects in new scenes.
    Section III-B and Implementation. The method depends on accurate masks at test time, but segmentation failures are not quantified.
  • domain assumption The gripper-base frame provides consistent relative geometry across different trajectories.
    Section III-B, Figure 3. The assumption is that gripper-object relative poses converge at the end of manipulation, reducing learning complexity.
  • standard math The diffusion training objective (Eq. 2) learns a conditional action distribution that can be denoised into effective actions.
    Standard diffusion policy formulation; no proof of convergence or optimality is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Generalizable 3D Manipulation With 10 Demonstrations." pith.science (2026). https://pith.science/paper/OACAWTEI

@misc{pith2026241110203,
  author       = {Pith},
  title        = {Pith review of: Learning Generalizable 3D Manipulation With 10 Demonstrations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OACAWTEI}},
  note         = {Machine review of arXiv:2411.10203}
}
read the original abstract

Learning robust and generalizable manipulation skills from demonstrations remains a key challenge in robotics, with broad applications in industrial automation and service robotics. While recent imitation learning methods have achieved impressive results, they often require large amounts of demonstration data and struggle to generalize across different spatial variants. In this work, we present a novel framework that learns manipulation skills from as few as 10 demonstrations, yet still generalizes to spatial variants such as different initial object positions and camera viewpoints. Our framework consists of two key modules: Semantic Guided Perception (SGP), which constructs task-focused, spatially aware 3D point cloud representations from RGB-D inputs; and Spatial Generalized Decision (SGD), an efficient diffusion-based decision-making module that generates actions via denoising. To effectively learn generalization ability from limited data, we introduce a critical spatially equivariant training strategy that captures the spatial knowledge embedded in expert demonstrations. We validate our framework through extensive experiments on both simulation benchmarks and real-world robotic systems. Our method demonstrates a 60 percent improvement in success rates over state-of-the-art approaches on a series of challenging tasks, even with substantial variations in object poses and camera viewpoints. This work shows significant potential for advancing efficient, generalizable manipulation skill learning in real-world applications.

Figures

Figures reproduced from arXiv: 2411.10203 by the authors.

Figure 1
Figure 1. In a), we report the average success rates of the two methods [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall workflow of our developed framework, which consists of two key modules: Semantic Guided Perception (SGP) and Spatial Generalized [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Given the original point cloud p, gripper pose h, action a, we apply a random rotation T to get the augmented data T(p), T(h), and corresponding T(a). positioning rather than absolute poses, the model learns to adapt to variations in object orientation and positioning. This approach significantly increases the training data while preserving spatial consistency. Given the augmented inputs T(p), T(h), and T(a), the tr… view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: We compared two choices of coordinate reference. When using [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 5
Figure 5. Figure 5: Here we visualize the different camera viewpoint settings used in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 20 canonical work pages

  1. [1]

    Catgrasp: Learning category-level task-relevant grasping in clutter from simulation,

    B. Wen, W. Lian, K. Bekris, and S. Schaal, “Catgrasp: Learning category-level task-relevant grasping in clutter from simulation,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 6401–6408

    2022

  2. [2]

    You only demonstrate once: Category-level manipulation from single visual demonstration,

    B. Wen, W. Lian, K. E. Bekris, and S. Schaal, “You only demonstrate once: Category-level manipulation from single visual demonstration,” in Robotics: Science and Systems XVIII, New York City, NY, USA, June 27 - July 1, 2022 , K. Hauser, D. A. Shell, and S. Huang, Eds., 2022

    work page 2022

  3. [3]

    Autonomous manipulation learning for similar deformable objects via only one demonstration,

    Y . Ren, R. Chen, and Y . Cong, “Autonomous manipulation learning for similar deformable objects via only one demonstration,” in Computer Vision and Pattern Recognition(CVPR) , 2023, pp. 17 069–17 078

    work page 2023

  4. [4]

    Instructing robots by sketching: Learning from demonstration via probabilistic diagram- matic teaching,

    W. Zhi, T. Zhang, and M. Johnson-Roberson, “Instructing robots by sketching: Learning from demonstration via probabilistic diagram- matic teaching,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 15 047–15 053

    work page 2024

  5. [5]

    Vision-driven compliant manipulation for reliable, high- precision assembly tasks,

    A. S. Morgan, B. Wen, J. Liang, A. Boularias, A. M. Dollar, and K. Bekris, “Vision-driven compliant manipulation for reliable, high- precision assembly tasks,” arXiv preprint arXiv:2106.14070 , 2021

    arXiv 2021

  6. [6]

    Multi-level reasoning for robotic assembly: From sequence inference to contact selection,

    X. Zhu, D. K. Jha, D. Romeres, L. Sun, M. Tomizuka, and A. Cherian, “Multi-level reasoning for robotic assembly: From sequence inference to contact selection,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 816–823

    work page 2024

  7. [7]

    You only demonstrate once: Category-level manipulation from single visual demonstration,

    B. Wen, W. Lian, K. Bekris, and S. Schaal, “You only demonstrate once: Category-level manipulation from single visual demonstration,” arXiv preprint arXiv:2201.12716 , 2022

    arXiv 2022

  8. [8]

    Miles: Making imitation learning easy with self-supervision,

    G. Papagiannis and E. Johns, “Miles: Making imitation learning easy with self-supervision,” in 8th Annual Conference on Robot Learning , 2024

    work page 2024

Show all 41 references
  1. [9]

    Mail: Improving im- itation learning with selective state space models,

    X. Jia, Q. Wang, A. Donat, B. Xing, G. Li, H. Zhou, O. Celik, D. Blessing, R. Lioutikov, and G. Neumann, “Mail: Improving im- itation learning with selective state space models,” in 8th Annual Conference on Robot Learning , 2024

    2024

  2. [10]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in Proceedings of Robotics: Science and Systems (RSS) , 2023

    2023

  3. [11]

    3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,” in Proceedings of Robotics: Science and Systems (RSS), 2024

    2024

  4. [12]

    Manicm: Real-time 3d diffusion policy via consistency model for robotic manipulation,

    G. Lu, Z. Gao, T. Chen, W. Dai, Z. Wang, and Y . Tang, “Manicm: Real-time 3d diffusion policy via consistency model for robotic manipulation,” arXiv preprint arXiv:2406.01586 , 2024

    2024 arXiv

  5. [13]

    Commonsense spatial knowledge-aware 3-d human motion and object interaction prediction,

    S. U. Lee, “Commonsense spatial knowledge-aware 3-d human motion and object interaction prediction,” in IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13- 17, 2024 . IEEE, 2024, pp. 3057–3063

    2024

  6. [14]

    Grasp manipulation relationship detection based on graph sample and aggregation,

    J. Luo, Y . Liu, H. Wang, M. Ding, and X. Lan, “Grasp manipulation relationship detection based on graph sample and aggregation,” in IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13-17, 2024 . IEEE, 2024, pp. 4098– 4104

    2024

  7. [15]

    CYCLO: cyclic graph transformer approach to multi-object relationship modeling in aerial videos,

    T. Nguyen, P. A. Nguyen, X. Li, J. D. Cothren, A. Yilmaz, and K. Luu, “CYCLO: cyclic graph transformer approach to multi-object relationship modeling in aerial videos,” CoRR, vol. abs/2406.01029, 2024

    2024 arXiv

  8. [16]

    Cliport: What and where pathways for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Cliport: What and where pathways for robotic manipulation,” in Conference on robot learning . PMLR, 2022, pp. 894–906

    2022

  9. [17]

    Mechanical intelligence for prehensile in-hand manipulation of spatial trajectories,

    Q. Lu, Z. Gan, X. Wang, G. Bai, Z. Zhang, and N. Rojas, “Mechanical intelligence for prehensile in-hand manipulation of spatial trajectories,” in IEEE International Conference on Robotics and Automation, ICRA 2023, London, UK, May 29 - June 2, 2023 . IEEE, 2023, pp. 8075– 8081

    2023

  10. [18]

    Dall-e-bot: Introducing web- scale diffusion models to robotics,

    I. Kapelyukh, V . V osylius, and E. Johns, “Dall-e-bot: Introducing web- scale diffusion models to robotics,” IEEE Robotics and Automation Letters, vol. 8, no. 7, pp. 3956–3963, 2023

    2023

  11. [19]

    Playfusion: Skill acquisition via diffusion from language-annotated play,

    L. Chen, S. Bahl, and D. Pathak, “Playfusion: Skill acquisition via diffusion from language-annotated play,” in Conference on Robot Learning. PMLR, 2023, pp. 2012–2029

    2023

  12. [20]

    Generative skill chaining: Long-horizon skill planning with diffusion models,

    U. A. Mishra, S. Xue, Y . Chen, and D. Xu, “Generative skill chaining: Long-horizon skill planning with diffusion models,” in Conference on Robot Learning . PMLR, 2023, pp. 2905–2925

    2023

  13. [21]

    Movement primitive diffusion: Learning gentle robotic manipulation of deformable objects,

    P. M. Scheikl, N. Schreiber, C. Haas, N. Freymuth, G. Neumann, R. Lioutikov, and F. Mathis-Ullrich, “Movement primitive diffusion: Learning gentle robotic manipulation of deformable objects,” IEEE Robotics Autom. Lett. , vol. 9, no. 6, pp. 5338–5345, 2024

    2024

  14. [22]

    Crossway diffusion: Im- proving diffusion-based visuomotor policy via self-supervised learn- ing,

    X. Li, V . Belagali, J. Shang, and M. S. Ryoo, “Crossway diffusion: Im- proving diffusion-based visuomotor policy via self-supervised learn- ing,” in IEEE International Conference on Robotics and Automation, ICRA 2024 . IEEE, 2024, pp. 16 841–16 849

    2024

  15. [23]

    Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,

    Z. Fu, T. Z. Zhao, and C. Finn, “Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,” in Conference on Robot Learning (CoRL) , 2024

    2024

  16. [24]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots,

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song, “Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots,” in Proceedings of Robotics: Science and Systems (RSS) , 2024

    2024

  17. [25]

    Scaling up and distilling down: Language-guided robot skill acquisition,

    H. Ha, P. Florence, and S. Song, “Scaling up and distilling down: Language-guided robot skill acquisition,” in Conference on Robot Learning. PMLR, 2023, pp. 3766–3777

    2023

  18. [26]

    Xskill: Cross em- bodiment skill discovery,

    M. Xu, Z. Xu, C. Chi, M. Veloso, and S. Song, “Xskill: Cross em- bodiment skill discovery,” in Conference on Robot Learning . PMLR, 2023, pp. 3536–3555

    2023

  19. [27]

    Chaineddiffuser: Unifying trajectory diffusion and keypose predic- tion for robotic manipulation,

    Z. Xian, N. Gkanatsios, T. Gervet, T.-W. Ke, and K. Fragkiadaki, “Chaineddiffuser: Unifying trajectory diffusion and keypose predic- tion for robotic manipulation,” in 7th Annual Conference on Robot Learning, 2023

    2023

  20. [28]

    Act3d: 3d feature field transformers for multi-task robotic manipulation,

    T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki, “Act3d: 3d feature field transformers for multi-task robotic manipulation,” in 7th Annual Conference on Robot Learning , 2023

    2023

  21. [29]

    Learning generalizable manipulation policies with object-centric 3d representations,

    Y . Zhu, Z. Jiang, P. Stone, and Y . Zhu, “Learning generalizable manipulation policies with object-centric 3d representations,” in 7th Annual Conference on Robot Learning , 2024

    2024

  22. [30]

    Hierarchical diffu- sion policy for kinematics-aware multi-task robotic manipulation,

    X. Ma, S. Patidar, I. Haughton, and S. James, “Hierarchical diffu- sion policy for kinematics-aware multi-task robotic manipulation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 18 081–18 090

    2024

  23. [31]

    Vision-based manipulation from single human video with open-world object graphs,

    Y . Zhu, A. Lim, P. Stone, and Y . Zhu, “Vision-based manipulation from single human video with open-world object graphs,” arXiv preprint arXiv:2405.20321, 2024

    2024 arXiv

  24. [32]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 3836–3847

    2023

  25. [33]

    Emergent correspondence from image diffusion,

    L. Tang, M. Jia, Q. Wang, C. P. Phoo, and B. Hariharan, “Emergent correspondence from image diffusion,” Advances in Neural Informa- tion Processing Systems , vol. 36, pp. 1363–1389, 2023

    2023

  26. [34]

    Plug-and-play diffusion features for text-driven image-to-image translation,

    N. Tumanyan, M. Geyer, S. Bagon, and T. Dekel, “Plug-and-play diffusion features for text-driven image-to-image translation,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1921–1930

    2023

  27. [35]

    Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,

    N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 22 500–22 510

    2023

  28. [36]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. , “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 4015–4026

    2023

  29. [37]

    Putting the object back into video object segmentation,

    H. K. Cheng, S. W. Oh, B. Price, J.-Y . Lee, and A. Schwing, “Putting the object back into video object segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 3151–3161

    2024

  30. [38]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

    2021

  31. [39]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R¨adle, C. Rolland, L. Gustafson, et al., “Sam 2: Segment anything in images and videos,” arXiv preprint arXiv:2408.00714 , 2024

    2024 arXiv

  32. [40]

    Mask dino: Towards a unified transformer-based framework for object detection and segmentation,

    F. Li, H. Zhang, H. Xu, S. Liu, and e. a. Zhang, “Mask dino: Towards a unified transformer-based framework for object detection and segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 3041–3050

    2023

  33. [41]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,

    T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine, “Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,” in Conference on robot learning . PMLR, 2020, pp. 1094–1100

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.