Pith. sign in

REVIEW 1 major objections 5 minor 64 references

ClutterDexGrasp: A Sim-to-Real System for General Dexterous Grasping in Cluttered Scenes

T0 review · 1 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a policy trained entirely in simulation can close the loop on target-oriented dexterous grasping in cluttered real-world scenes, with no real-world demonstrations.

desk verdict Solid sim-to-real system with strong simulation ablations, but the real-world success metric (torque threshold before lift verification) likely inflates the headline 83.9% and needs fixing before publication. read the letter →

arxiv 2506.14317 v3 pith:5WJT3J3D submitted 2025-06-17 cs.RO

Zeyuan Chen , Qiyang Yan , Yuanpei Chen , Tianhao Wu , Jiyao Zhang , Zihan Ding , Jinzhou Li , Yaodong Yang
show 1 more author
Hao Dong
This is my paper
classification cs.RO
keywords DexterousgraspingClutteredscenesSim-to-realtransferTeacher-studentdistillation3DdiffusionpolicyCurriculumlearningPointcloudClosed-loopmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a dexterous hand-arm system can learn to grasp a specified target object in dense clutter entirely in simulation, then transfer to the real world with no real-world demonstrations. The proposed ClutterDexGrasp trains a teacher policy with privileged geometric information and staged curricula, distills it into a point-cloud-based 3D diffusion policy, and reports an 83.9 percent success rate over 167 real-world attempts. The relevance is that closed-loop, target-oriented dexterous grasping in occlusion-heavy scenes has previously required either open-loop pose prediction or large amounts of real human demonstrations; this work claims a path that avoids both. It also reports that the student policy loses under five percentage points of success relative to the teacher in simulation, including on unseen objects and ultra-dense layouts.

What carries the argument

The load-bearing mechanism is the teacher-student distillation pipeline. The teacher is an RL policy trained with PPO on a geometry-and-spatial representation: for each of 11 selected finger links, the 3D distance vector to the nearest sampled points on the target object ($d_{\text{pos}}$) and on non-target objects ($d_{\text{neg}}$). These feed a reward that encourages approach to the target and penalizes risky closeness to clutter. Three curricula shape the behavior: single-object grasping first, then cluttered-scene fine-tuning, then a safety stage that tightens a fingertip force threshold and terminates episodes on high force or table penetration. The student is a 3D diffusion policy (DP3) that acts on a 4096-point partial point cloud with a target mask, augmented with synthetic robot and ground point clouds, and is trained offline on 20,000 teacher rollouts. System identification calibrates arm and hand dynamics so that the student transfers without real-world fine-tuning.

What would settle it

Run the same real-world scenes with an independent success criterion, for example a vision system or human judge confirming that the target object remains in the hand and is lifted 0.1 meters after the robot stops, without contact support from the table or other objects, and compare the resulting success rate with the torque-threshold-based 83.9 percent. A large gap would show the reported metric overcounts.

Watch

Extended reading notes

Core claim

The central claim is that the combination of a privileged geometry-and-spatial representation, a clutter-density curriculum, and an interaction-safety curriculum makes reinforcement learning for dexterous cluttered-scene grasping stable enough to be distilled into a real-world-ready student. The teacher observes, for each finger link, the 3D distance to the nearest sampled surface points of the target object and of non-target objects; this compact representation lets the policy learn collision-aware, human-like strategies such as gently nudging occluders aside before grasping. The student replaces privileged state with a partial point cloud and is trained by imitation on 20,000 successful teacher trajectories, using a 3D diffusion policy. The paper claims this is the first zero-shot sim-to-real closed-loop system for target-oriented dexterous grasping in cluttered scenes, with real-world success at 83.9 percent over 167 attempts and robust generalization to unseen objects and layouts.

Load-bearing premise

The load-bearing premise is that the real-world success signal, a finger-torque threshold that triggers a lift-up, correctly identifies genuine, stable grasps; if that threshold counts partial or unstable contacts as successes, the reported 83.9 percent rate would overstate the policy's real ability.

Editorial extensions

If this is right

  • If the central claim holds, real-world dexterous grasping in clutter no longer requires collecting human demonstrations for every new scene; a simulation-trained policy can be deployed directly.
  • The under-5% teacher-student success drop suggests that point-cloud distillation preserves the strategic behaviors, such as occluder clearing and side approaches, learned by the privileged teacher.
  • Because the policy generalizes to ultra-dense scenes it never saw in training, the clutter-density curriculum appears to teach a scalable skill rather than a memorized layout.
  • The explicit safety curriculum implies that sim-to-real dexterous policies can be made gentle enough for contact-rich interaction, not just high-scoring in simulation.
  • A practical consequence is that target selection can be interactive: the system uses SAM2 to mask the target object in the camera view, so a user can choose which object to grasp.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same teacher-student pattern, privileged distance representation plus staged curricula plus diffusion distillation, could transfer to other contact-rich dexterous tasks such as in-hand reorientation or tool use, provided a comparable privileged signal exists in simulation.
  • The reported success metric depends on a finger-torque threshold; an independent test in which a human or vision system confirms the object stays lifted after the robot raises the arm would settle whether the 83.9 percent figure reflects stable grasps.
  • The failure analysis points to a concrete extension: multi-view or active perception, rather than a single side camera, should reduce failures on tiny or heavily occluded objects without changing the policy-learning pipeline.
  • The data-scaling experiment suggests the student's generalization improves with trajectory count, implying the method's ceiling is partly a data-generation question, not only an algorithmic one.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper proposes ClutterDexGrasp, a two-stage teacher-student framework for target-oriented dexterous grasping in cluttered tabletop scenes. A privileged teacher policy is trained with PPO using a geometry-and-spatial distance representation, a clutter-density curriculum, and an interaction-safety curriculum; a student 3D diffusion policy (DP3) is then distilled offline from teacher rollouts on partial point-cloud observations. Simulation evaluation covers 88 training objects, 2,029 unseen test objects, three clutter densities, and three seeds, reporting success rates around 87--92% for the teacher and 74--91% for the student, with ablations of the representation, curriculum, and safety components. Real-world evaluation reports 83.9% success over 167 attempts on 41 unseen objects without real-world demonstrations, and the paper claims this is the first zero-shot sim-to-real closed-loop system for target-oriented dexterous grasping in cluttered scenes.

Significance. If the central claims hold, this is a strong and practically relevant systems contribution: it demonstrates that an RL-trained teacher and a point-cloud diffusion student can transfer to a real 7-DoF arm and 12-DoF hand in cluttered scenes without real-world demonstrations. The simulation results are carefully reported with three seeds, a large unseen-object evaluation, and informative ablations; the appendices give substantial implementation detail, and the authors provide a project website with videos. The main weakness is the real-world evidence: the success criterion is an internal finger-torque threshold rather than an independently verified lift, no real-world baselines or error bars are reported, and the "zero-shot" label is weakened by system identification and several real-world-specific adjustments. These issues bear directly on the headline 83.9% success rate and on the novelty claim of zero-shot transfer, so the verdict is major revision rather than acceptance.

major comments (1)
  1. [Table 1, Section 5.1.1] The text says the student "shows less than a 5% average success rate drop across all combinations of seen and unseen objects and layouts," but the table shows a 7.4 percentage-point drop for seen objects with unseen layouts (80.9 to 73.5) and 7.4 for unseen objects with unseen ultra-dense layouts (81.6 to 74.2). The average may be below 5%, but the claim as written is misleading for individual cells. Please state the per-cell drops explicitly or rephrase to "average drop" with the individual maximum drop disclosed.
minor comments (5)
  1. [Appendix A/References] References [22] and [23] are the same paper (Ross, Gordon, Bagnoli, "A reduction of imitation learning and structured prediction to no-regret online learning") and are cited with identical text; please merge them or remove the duplicate.
  2. [Figure 2] The framework diagram is dense and some text is very small (e.g., "Force Penalty2" and "Table Contact Penalty1"); consider enlarging or separating the three curriculum stages into distinct panels for readability.
  3. [Section 3] The action space description says the hand has 6-DoF actuated finger joints and 6-DoF underactuated joints, but the action space is defined as 7D arm joint changes plus 6D absolute hand joint positions. Please clarify which joints are actually commanded and how the underactuated joints are modeled in both simulation and real hardware.
  4. [Appendix C.1] The point cloud observation is described as O_pc in R^{4x3584} plus R^{4x512} and R^{4x1024}, totaling 5120 points, but the student observation dimension is given as R^{4109} in Appendix C. Please reconcile the dimension (joints 13 + 4096 points + masks?).
  5. [Section 7] The limitation paragraph mentions tiny objects but does not mention the failure mode identified in Appendix F where objects fall outside the arm's working range; including that in the main text would strengthen the honesty of the failure analysis.

Circularity Check

2 steps flagged · score 2.0 of 10

Central sim-to-real derivation is not circular; minor self-defined evaluation and safety-force metrics account for the low score.

  1. self definitional [Section 4.1.3 (Eqs. 3-4), Appendix B.4 (Eq. 12), Table 2]
    "Exceeding this force threshold incurs a sparse penalty term rforce in the reward and leads to early termination. ... With the safety curriculum, the policy performs more gentle interactions, significantly reduces the force, and shows lower variance (Tab. 2)."

    The average maximum contact force reported in Table 2 is the exact quantity minimized by the reward term rforce in Eq. (12), with early termination whenever any fingertip force exceeds the threshold. The comparison 'Ours 43.2 vs w/o Safety 80.6' therefore measures the direct effect of the safety penalty itself; the force reduction is partly enforced by construction rather than an emergent property. This does not undermine the central zero-shot sim-to-real claim, but it makes the safety-curriculum effectiveness result weaker than presented.

  2. self definitional [Appendix E.1]
    "we use finger torque as a success signal: Once it exceeds a threshold, the grasp is considered successful and a predefined lift-up action is executed."

    The real-world Success Rate (83.9% over 167 attempts) is computed from this torque-threshold event, not from the simulation success criterion (target lifted 0.1 m) or from any independent verification. The threshold value is not reported, and no external criterion is applied, so the headline number is conditional on an unvalidated self-defined detector. This makes the empirical evaluation mildly self-referential, although the policy itself is not derived from the threshold.

full rationale

The core derivation chain — teacher RL trained in simulation with a geometric-spatial representation and curricula, student distilled via DP3 on point clouds, and zero-shot real-world deployment with perception and system identification — is not circular. The teacher's reward (Eq. 7) and simulation success definition (0.1 m lift) are external to the real-world evaluation; the student is trained offline on teacher rollouts; and the real-world policy uses no demonstrations. The cited prior works (DP3, GraspXL/RobustDexGrasp, etc.) are not by the present authors and are not load-bearing self-citations. The mild circularity resides only in two self-defined evaluation quantities: the force metric is the optimized safety objective, and the real-world success signal is the authors' own unreported torque threshold. These affect the strength of the safety ablation and the absolute real-world success number, but they do not reduce the central generalization or sim-to-real claim to its inputs.

Assumptions & free parameters 8 free parameters · 7 assumptions · 0 invented entities

The central claim rests on a large number of hand-set training choices and domain assumptions about physics fidelity, perception, and evaluation. No new physical entities are introduced. The counts above show that the paper's contribution is an engineered integration of known algorithms, not a derivation.

free parameters (8)
  • Reward weighting coefficients c1 to c6 = not reported
    Hand-selected weights in Eqs. (3), (7), (8), (9), (11) balance approach, grasp, and safety reward terms; no values or sensitivity analysis are given.
  • Distance decay coefficients alpha_pos, alpha_neg, alpha_mid = not reported
    Exponential decay rates in Eqs. (5), (6), (8) convert distances into rewards; choices affect how strongly the policy avoids clutter.
  • Safety force threshold schedule f0, fbar, delta-f = 200, 50, 5 (sim units)
    Appendix B.4 sets initial threshold 200, final threshold 50, decrement 5; this schedule directly produces the reported force reduction.
  • Safety curriculum success-rate threshold wbar and minimum interval DeltaTmin = not reported
    Control when the force threshold is tightened; values are not given, so the curriculum dynamics are not fully specified.
  • Coarse-to-fine activation distance dbar = 0.08
    Appendix B.5 enables hand DoFs only when hand-object distance is below 0.08, a hand-set threshold affecting dexterous control.
  • Clutter density bins = sparse 4 to 8, dense 9 to 15, ultra-dense 16 to 25 objects
    Arbitrary scene difficulty definitions in Sec. 5.1; the generalization claim is relative to these bins.
  • Real-world success torque threshold = not reported
    Appendix E.1 uses finger torque as the success signal; the threshold value and its calibration are not reported, which is the weakest evaluation link.
  • Point cloud component sizes = 3584 real points, 512 synthetic ground, 1024 synthetic robot points
    Appendix C.1 defines the student observation; these sizes are manual design choices for perception.
assumptions (7)
  • domain assumption Isaac Gym physics, after system identification, accurately models real-world arm and hand dynamics for the tested objects.
    Section 4.2.3 and Appendix C.4 assume the calibrated simulation transfers; uncalibrated friction, soft contacts, or sensor latency would break zero-shot transfer.
  • domain assumption Randomly dropping objects and keeping only stable scenes produces clutter representative of real-world target-oriented grasping.
    Section D describes the scene generation pipeline; the generalization claim depends on this distributional match.
  • domain assumption A compact distance representation using 200 target-surface points and 50 non-target points per link is sufficient for clutter-aware grasping.
    Appendix B.1 uses these sample counts as privileged observations; if the sampling misses critical local geometry, the teacher policy cannot learn robust strategies.
  • domain assumption SAM2 segmentation of the target object in real-world point clouds is correct enough for masking.
    Appendix E.1 relies on SAM2 masks to select the target; mis-segmentation would misdirect the policy.
  • ad hoc to paper Disabling hand-table collisions in simulation and replacing the real table with a synthetic ground plane does not distort learned grasping safety.
    Section 4.1.3 and Appendix C.1 make this unusual modeling choice; it is specific to this paper and not independently justified.
  • standard math PPO converges and DP3 distillation preserves teacher behavior under the reported hyperparameters.
    Training relies on these prior algorithmic results (Refs. [57], [35]) without new convergence guarantees.
  • domain assumption Selecting targets only from visible masks is a fair way to evaluate cluttered-scene grasping.
    Sec 5.2 and Appendix E.1 choose target from visible masks, which excludes fully occluded targets and may overstate performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ClutterDexGrasp: A Sim-to-Real System for General Dexterous Grasping in Cluttered Scenes." pith.science (2026). https://pith.science/paper/5WJT3J3D

@misc{pith2026250614317,
  author       = {Pith},
  title        = {Pith review of: ClutterDexGrasp: A Sim-to-Real System for General Dexterous Grasping in Cluttered Scenes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5WJT3J3D}},
  note         = {Machine review of arXiv:2506.14317}
}
read the original abstract

Dexterous grasping in cluttered scenes presents significant challenges due to diverse object geometries, occlusions, and potential collisions. Existing methods primarily focus on single-object grasping or grasp-pose prediction without interaction, which are insufficient for complex, cluttered scenes. Recent vision-language-action models offer a potential solution but require extensive real-world demonstrations, making them costly and difficult to scale. To address these limitations, we revisit the sim-to-real transfer pipeline and develop key techniques that enable zero-shot deployment in reality while maintaining robust generalization. We propose ClutterDexGrasp, a two-stage teacher-student framework for closed-loop target-oriented dexterous grasping in cluttered scenes. The framework features a teacher policy trained in simulation using clutter density curriculum learning, incorporating both a geometry and spatially-embedded scene representation and a novel comprehensive safety curriculum, enabling general, dynamic, and safe grasping behaviors. Through imitation learning, we distill the teacher's knowledge into a student 3D diffusion policy (DP3) that operates on partial point cloud observations. To the best of our knowledge, this represents the first zero-shot sim-to-real closed-loop system for target-oriented dexterous grasping in cluttered scenes, demonstrating robust performance across diverse objects and layouts. More details and videos are available at https://clutterdexgrasp.github.io/.

Figures

Figures reproduced from arXiv: 2506.14317 by the authors.

Figure 1
Figure 1. ClutterDexGrasp achieves zero-shot sim-to-real transfer for closed-loop target-oriented [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Training Framework ing the discounted return Eπ [∑ ∞ t=0 γ t r(ot ,at)]. For IL, we learn a student policy π S by minimizing E(o,a)∼DE [−logπ(a|o)] on expert demonstrations DE from π E . State and Action Spaces We consider a tabletop scenario with a 7-DoF robotic arm J a ∈ R 7 and a 12-DoF dexterous hand J h ∈ R 12. The hand joints consist of 6-DoF actuated finger joints J f ∈ R 6 , and 6-DoF underactuated finger jo… view at source ↗
Figure 3
Figure 3. Visualization of Human-like Grasping Strategy: (a) Efficient grasping in simple scenes. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Real-world Objects and Example of Cluttered Scenes: (a): real-world object datasets, (b): [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Real-world Experiment Method SR@20s SR@40s SR@60s AUC Ours 61.8 79.5 83.9 0.617 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Learning curves of the cluttered-scene policies with or without the Geometric and Spatial [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Cluttered Scene Policy Strategy Comparison: (a) Policy trained with GS-Representation, [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Learning curves of the cluttered-scene policies (1) [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Visualization of the Geometric and Spatial Representation. For each finger joint, distances [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Point-cloud comparison between simulation(Left) and real world(Right). [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Real-World Setup To improve deployment stability, we introduce several practical adjustments. Since the system oc￾casionally failed to detect successful grasps, leading to redundant actions, we use finger torque as a success signal: Once it exceeds a threshold, the gr…
Figure 12
Figure 12. Figure 12: Data Scaling for Generalization G Data Scaling for Scene-Level Generalization In this section, we investigate how the performance of student policy scales with the number of collected trajectories. Specifically, we use a consistent teacher policy to collect trajectori…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 31 canonical work pages

  1. [1]

    I. M. Bullock and A. M. Dollar. Classifying human manipulation behavior. In 2011 IEEE international conference on rehabilitation robotics, pages 1–6. IEEE, 2011

    work page 2011

  2. [2]

    A. I. Weinberg, A. Shirizly, O. Azulay, and A. Sintov. Survey of learning approaches for robotic in-hand manipulation. arXiv preprint arXiv:2401.07915, 2024

    work page Pith review arXiv 2024

  3. [3]

    T. Chen, J. Xu, and P. Agrawal. A system for general in-hand object re-orientation. CoRR, abs/2111.03043, 2021. URL https://arxiv.org/abs/2111.03043

    work page Pith review arXiv 2021

  4. [4]

    Xu, Y .-L

    G.-H. Xu, Y .-L. Wei, D. Zheng, X.-M. Wu, and W.-S. Zheng. Dexterous grasp transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17933–17942, 2024

    2024

  5. [5]

    J. Lu, H. Kang, H. Li, B. Liu, Y . Yang, Q. Huang, and G. Hua. Ugg: Unified generative grasping. In European Conference on Computer Vision, pages 414–433. Springer, 2024

    work page 2024

  6. [6]

    Zhong, Q

    Y . Zhong, Q. Jiang, J. Yu, and Y . Ma. Dexgrasp anything: Towards universal robotic dexterous grasping with physics awareness. arXiv preprint arXiv:2503.08257, 2025

    arXiv 2025

  7. [7]

    Y . Qin, B. Huang, Z.-H. Yin, H. Su, and X. Wang. Dexpoint: Generalizable point cloud reinforcement learning for sim-to-real dexterous manipulation. Conference on Robot Learning (CoRL), 2022

    work page 2022

  8. [8]

    Y .-H. Wu, J. Wang, and X. Wang. Learning generalizable dexterous manipulation from human grasp affordance. In Conference on Robot Learning, pages 618–629. PMLR, 2023

    work page 2023

Show all 64 references
  1. [9]

    W. Wan, H. Geng, Y . Liu, Z. Shan, Y . Yang, L. Yi, and H. Wang. Unidexgrasp++: Improving dexterous grasping policy learning via geometry-aware curriculum and iterative generalist- specialist learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision,...

    2023

  2. [10]

    Huang, H

    Z. Huang, H. Yuan, Y . Fu, and Z. Lu. Efficient residual learning with mixture-of-experts for universal dexterous grasping. arXiv preprint arXiv:2410.02475, 2024

    2024 arXiv

  3. [11]

    T. G. W. Lum, M. Matak, V . Makoviychuk, A. Handa, A. Allshire, T. Hermans, N. D. Ratliff, and K. Van Wyk. Dextrah-g: Pixels-to-action dexterous arm-hand grasping with geometric fabrics. In 8th Annual Conference on Robot Learning

  4. [12]

    Singh, A

    R. Singh, A. Allshire, A. Handa, N. Ratliff, and K. Van Wyk. Dextrah-rgb: Visuomotor policies to grasp anything with dexterous hands. arXiv preprint arXiv:2412.01791, 2024. 9

    2024 arXiv

  5. [13]

    H.-S. Fang, H. Yan, Z. Tang, H. Fang, C. Wang, and C. Lu. Anydexgrasp: Learning general dexterous grasping for any hands with human-level learning efficiency. In 7th Robot Learning Workshop: Towards Robots with Human-Level Abilities

  6. [14]

    Zhang, H

    J. Zhang, H. Liu, D. Li, X. Yu, H. Geng, Y . Ding, J. Chen, and H. Wang. Dexgraspnet 2.0: Learning generative dexterous grasping in large-scale synthetic cluttered scenes. In8th Annual Conference on Robot Learning, 2024

    2024

  7. [15]

    W. Wei, D. Li, P. Wang, Y . Li, W. Li, Y . Luo, and J. Zhong. Dvgg: Deep variational grasp generation for dextrous manipulation. IEEE Robotics and Automation Letters , 7(2):1659– 1666, 2022

    2022

  8. [16]

    Zhong, X

    Y . Zhong, X. Huang, R. Li, C. Zhang, Y . Liang, Y . Yang, and Y . Chen. Dexgraspvla: A vision-language-action framework towards general dexterous grasping. arXiv preprint arXiv:2502.20900, 2025

    2025

  9. [17]

    M. Zare, P. M. Kebria, A. Khosravi, and S. Nahavandi. A survey of imitation learning: Al- gorithms, recent developments, and challenges, 2023. URL https://arxiv.org/abs/ 2309.02473

    2023 arXiv

  10. [18]

    Z. Si, K. L. Zhang, Z. Temel, and O. Kroemer. Tilde: Teleoperation for dexterous in-hand manipulation learning with a deltahand. arXiv preprint arXiv:2405.18804, 2024

    2024 arXiv

  11. [19]

    Rajeswaran, V

    A. Rajeswaran, V . Kumar, A. Gupta, J. Schulman, E. Todorov, and S. Levine. Learning com- plex dexterous manipulation with deep reinforcement learning and demonstrations. CoRR, abs/1709.10087, 2017. URL http://arxiv.org/abs/1709.10087

    2017 arXiv

  12. [20]

    O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, et al. Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1):3–20, 2020

    2020

  13. [21]

    H. Zhu, A. Gupta, A. Rajeswaran, S. Levine, and V . Kumar. Dexterous manipulation with deep reinforcement learning: Efficient, general, and low-cost. CoRR, abs/1810.06045, 2018. URL http://arxiv.org/abs/1810.06045

    2018 arXiv

  14. [23]

    S. Ross, G. Gordon, and D. Bagnell. A reduction of imitation learning and structured predic- tion to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627–635. JMLR Workshop and Conference Pro-...

    2011

  15. [24]

    Nagabandi, K

    A. Nagabandi, K. Konolige, S. Levine, and V . Kumar. Deep dynamics models for learning dexterous manipulation. CoRR, abs/1909.11652, 2019. URL http://arxiv.org/abs/ 1909.11652

    1909 arXiv

  16. [25]

    Kumar, E

    V . Kumar, E. Todorov, and S. Levine. Optimal control with learned local models: Application to dexterous manipulation. In 2016 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 378–383, 2016. doi:10.1109/ICRA.2016.7487156

    2016

  17. [26]

    Zhang, S

    H. Zhang, S. Christen, Z. Fan, O. Hilliges, and J. Song. Graspxl: Generating grasping motions for diverse objects at scale. In European Conference on Computer Vision , pages 386–403. Springer, 2024

    2024

  18. [27]

    Zhang, Z

    H. Zhang, Z. Wu, L. Huang, S. Christen, and J. Song. Robustdexgrasp: Robust dexterous grasping of general objects. arXiv preprint arXiv:2504.05287, 2025. 10

    2025 arXiv

  19. [28]

    Andrychowicz, F

    M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. To- bin, O. Pieter Abbeel, and W. Zaremba. Hindsight experience replay. Advances in neural information processing systems, 30, 2017

    2017

  20. [30]

    Tobin, R

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 23–30. IEEE, 2017

    2017

  21. [31]

    L. Ljung. Perspectives on system identification. Annual Reviews in Control , 34(1):1–12,

  22. [32]

    Maddukuri, Z

    A. Maddukuri, Z. Jiang, L. Y . Chen, S. Nasiriany, Y . Xie, Y . Fang, W. Huang, Z. Wang, Z. Xu, N. Chernyadev, et al. Sim-and-real co-training: A simple recipe for vision-based robotic ma- nipulation. arXiv preprint arXiv:2503.24361, 2025

    2025 arXiv

  23. [33]

    T. Lin, K. Sachdev, L. Fan, J. Malik, and Y . Zhu. Sim-to-real reinforcement learning for vision-based dexterous manipulation on humanoids. arXiv preprint arXiv:2502.20396, 2025

    2025 arXiv

  24. [34]

    Valassakis, Z

    E. Valassakis, Z. Ding, and E. Johns. Crossing the gap: A deep dive into zero-shot sim-to-real transfer for dynamics. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5372–5379. IEEE, 2020

    2020

  25. [35]

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu. 3d diffusion policy. arXiv e-prints, pages arXiv–2403, 2024

    2024

  26. [36]

    Hinton, O

    G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network, 2015. URL https://arxiv.org/abs/1503.02531

    2015 arXiv

  27. [37]

    Z. Ding, Y . Chen, A. Z. Ren, S. S. Gu, Q. Wang, H. Dong, and C. Jin. Learning a universal human prior for dexterous manipulation from human preference. arXiv preprint arXiv:2304.04602, 2023

    2023 arXiv

  28. [38]

    Akkaya, M

    OpenAI, I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, W. Zaremba, and L. Zhang. Solving rubik’s cube with a robot hand.CoRR, abs/1910.07113,...

    1910 arXiv

  29. [39]

    Y . Qin, H. Su, and X. Wang. From one hand to multiple hands: Imitation learning for dexterous manipulation from single-camera teleoperation. IEEE Robotics and Automation Letters, 7(4): 10873–10881, 2022

    2022

  30. [40]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. pi 0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    2024 arXiv

  31. [41]

    R. Ding, Y . Qin, J. Zhu, C. Jia, S. Yang, R. Yang, X. Qi, and X. Wang. Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning, 2024. URL https:// arxiv.org/abs/2407.03162

    2024 arXiv

  32. [42]

    Q. Yan, Z. Ding, X. Zhou, and A. J. Spiers. Variable-friction in-hand manipulation for arbitrary objects via diffusion-based imitation learning. arXiv preprint arXiv:2503.02738, 2025

    2025 arXiv

  33. [43]

    Abbeel and A

    P. Abbeel and A. Y . Ng. Exploration and apprenticeship learning in reinforcement learning. In Proceedings of the 22nd international conference on Machine learning, pages 1–8, 2005. 11

    2005

  34. [44]

    Ding, Y .-Y

    Z. Ding, Y .-Y . Tsai, W. W. Lee, and B. Huang. Sim-to-real transfer for robotic manipulation with tactile sensory. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6778–6785. IEEE, 2021

    2021

  35. [45]

    Huang, H

    L. Huang, H. Zhang, Z. Wu, S. Christen, and J. Song. Fungrasp: Functional grasping for diverse dexterous hands. arXiv preprint arXiv:2411.16755, 2024

    2024 arXiv

  36. [46]

    Sievers, J

    L. Sievers, J. Pitz, and B. B ¨auml. Learning purely tactile in-hand manipulation with a torque- controlled hand. In 2022 International Conference on Robotics and Automation (ICRA), pages 2745–2751, 2022. doi:10.1109/ICRA46639.2022.9812093

    2022

  37. [47]

    Narayanan, J

    G. Narayanan, J. A. Raj, A. Gandhi, A. A. Gupte, A. J. Spiers, and B. Calli. Within-hand manipulation planning and control for variable friction hands. In Experimental Robotics: The 17th International Symposium, pages 600–610. Springer, 2021

  38. [48]

    Sahin, A

    A. Sahin, A. J. Spiers, and B. Calli. Region-based planning for 3d within-hand-manipulation via variable friction robot fingers and extrinsic contacts. In 2021 IEEE International Confer- ence on Robotics and Automation (ICRA), pages 6549–6555. IEEE, 2021

    2021

  39. [49]

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137, 2023

    2023 arXiv

  40. [50]

    Z. Wang, J. J. Hunt, and M. Zhou. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.06193, 2022

    2022 arXiv

  41. [51]

    Pearce, T

    T. Pearce, T. Rashid, A. Kanervisto, D. Bignell, M. Sun, R. Georgescu, S. V . Macua, S. Z. Tan, I. Momennejad, K. Hofmann, et al. Imitating human behaviour with diffusion models. arXiv preprint arXiv:2301.10677, 2023

    2023 arXiv

  42. [52]

    Reuss, M

    M. Reuss, M. Li, X. Jia, and R. Lioutikov. Goal-conditioned imitation learning using score- based diffusion policies. arXiv preprint arXiv:2304.02532, 2023

    2023 arXiv

  43. [53]

    A. Ajay, Y . Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal. Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657, 2022

    2022 arXiv

  44. [54]

    Janner, Y

    M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991, 2022

    2022 arXiv

  45. [55]

    Ding and C

    Z. Ding and C. Jin. Consistency models as a rich and efficient policy class for reinforcement learning. arXiv preprint arXiv:2309.16984, 2023

    2023 arXiv

  46. [56]

    Huang, Y

    B. Huang, Y . Chen, T. Wang, Y . Qin, Y . Yang, N. Atanasov, and X. Wang. Dynamic handover: Throw and catch with bimanual hands, 2023. URL https://arxiv.org/abs/2309. 05655

    2023

  47. [57]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  48. [58]

    T. Wu, M. Wu, J. Zhang, Y . Gan, and H. Dong. Learning score-based grasping primitive for human-assisting dexterous grasping. Advances in Neural Information Processing Systems, 36: 22132–22150, 2023

    2023

  49. [59]

    T. Wu, Y . Gan, M. Wu, J. Cheng, Y . Yang, Y . Zhu, and H. Dong. Unidexfpm: Universal dexterous functional pre-grasp manipulation via diffusion policy. arXiv e-prints, pages arXiv– 2403, 2024

    2024

  50. [60]

    T. Chen, M. Tippur, S. Wu, V . Kumar, E. Adelson, and P. Agrawal. Visual dexterity: In-hand reorientation of novel and complex object shapes. Science Robotics, 8(84):eadc9244, 2023. 12

    2023

  51. [61]

    H.-S. Fang, C. Wang, M. Gou, and C. Lu. Graspnet-1billion: A large-scale benchmark for general object grasping. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11444–11453, 2020

    2020

  52. [62]

    Zhang, W

    J. Zhang, W. Huang, B. Peng, M. Wu, F. Hu, Z. Chen, B. Zhao, and H. Dong. Omni6dpose: A benchmark and model for universal 6d object pose estimation and tracking. In European Conference on Computer Vision, pages 199–216. Springer, 2024

    2024

  53. [63]

    Makoviychuk, L

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021

    2021 arXiv

  54. [64]

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Doll ´ar, and C. Feichtenhofer. Sam 2: Segment anything in images and videos, 2024. URL https: //arxi...

    2024 arXiv

  55. [2010]

    doi:https://doi.org/10.1016/j.arcontrol.2009.12.001

    ISSN 1367-5788. doi:https://doi.org/10.1016/j.arcontrol.2009.12.001. URL https: //www.sciencedirect.com/science/article/pii/S1367578810000027

    2009 doi

  56. [2017]

    URL http://arxiv.org/abs/1703.06907

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.