REVIEW 15 cited by
DextrAH-RGB: Visuomotor Policies to Grasp Anything with Dexterous Hands
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
One of the most important, yet challenging, skills for a dexterous robot is grasping a diverse range of objects. Much of the prior work has been limited by speed, generality, or reliance on depth maps and object poses. In this paper, we introduce DextrAH-RGB, a system that can perform dexterous arm-hand grasping end-to-end from RGB image input. We train a privileged fabric-guided policy (FGP) in simulation through reinforcement learning that acts on a geometric fabric controller to dexterously grasp a wide variety of objects. We then distill this privileged FGP into a RGB-based FGP strictly in simulation using photorealistic tiled rendering. To our knowledge, this is the first work that is able to demonstrate robust sim2real transfer of an end2end RGB-based policy for complex, dynamic, contact-rich tasks such as dexterous grasping. DextrAH-RGB is competitive with depth-based dexterous grasping policies, and generalizes to novel objects with unseen geometry, texture, and lighting conditions in the real world. Videos of our system grasping a diverse range of unseen objects are available at \url{https://dextrah-rgb.github.io/}.
Forward citations
Cited by 15 Pith papers
-
SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routing
SILO enables the first reported zero-shot sim-to-real RL transfer for multi-stage cable routing by approximating cables as articulated rigid links and executing policy actions inside a synchronized digital twin.
-
HUGS: Guiding Unified Dexterous Grasp Synthesis Across Modes and Scales via Learned Human Priors
An object-conditioned human prior over contact modes and wrists guides force-closure optimization to synthesize diverse multi-mode dexterous grasps across object scales more efficiently than heuristics.
-
MuJoCo Playground
An open-source, MJX-based robot learning framework with integrated batch rendering that provides fast training and demonstrates sim-to-real transfer on six robot platforms.
-
World Translation: Minimizing Sim-to-Real Gap with Backward Dynamics Extraction and Unpaired Domain Translation
World Translation predicts a robot's next state by encoding hidden dynamics from the observed transition and cycle-translating that latent code from simulation to reality.
-
Cross-Embodiment Robot Manipulation via a Unified Hand Action Space
UHAS maps hand actions to deformations of a shared unit sphere and recovers joint commands via cascade IK, enabling multi-hand RL, zero-shot transfer, and modest real-world cube reorientation on LEAP and Allegro.
-
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Task-agnostic RL play pretraining on diverse objects yields a reusable dexterous prior that makes sparse-reward assembly learning ~33× more sample-efficient and enables zero-shot sim-to-real transfer on tight insertio...
-
PTLD: Sim-to-real Privileged Tactile Latent Distillation for Dexterous Manipulation
PTLD distills real privileged tactile data into a state estimator to boost sim-to-real performance of proprioceptive dexterous manipulation policies, yielding 182% improvement on in-hand rotation and 57% on reorientat...
-
Grasp to Act: Dexterous Grasping for Tool Use in Dynamic Settings
Combining wrench-tested grasp optimization with real-time RL finger adjustments lets a 16-DoF robot hand keep tools stable during hammering, sawing, cutting, stirring, and scooping.
-
StereoVLA: Enhancing Vision-Language-Action Models with Stereo Vision
A vision-language-action model that fuses stereo-derived geometric features with semantic features improves real-world grasping success and camera-pose robustness over single-view baselines.
-
Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration
Dexplore learns dexterous robotic hand control from human MoCap demonstrations by treating them as soft, adaptively shrinking spatial references, then distills the policy into a vision-based controller.
-
Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control
A value-guided MPC policy trained on 2 million synthetic trajectories improves closed-loop 6-DoF grasping in clutter and adapts to object perturbations.
-
ClutterDexGrasp: A Sim-to-Real System for General Dexterous Grasping in Cluttered Scenes
A simulation-trained teacher-student policy achieves zero-shot sim-to-real closed-loop target-oriented dexterous grasping in cluttered scenes, with 83.9 percent real-world success.
-
HERMES: Human-to-Robot Embodied Learning from Multi-Source Motion Data for Mobile Dexterous Manipulation
HERMES converts a single human motion demonstration into a deployable mobile bimanual dexterous manipulation policy, using RL, depth-image distillation, and closed-loop PnP pose refinement.
-
SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training
Simulation-pretrained policies, with digital-twin demos for critic bootstrapping and action proposals, cut real-world RL training time while reaching near-perfect success on three manipulation tasks.
-
A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
Embodied intelligence learning is reviewed through the complementary lenses of physical simulators and world models, with a proposed IR-L0 to IR-L4 robot capability taxonomy.
Discussion (0). Continue with ORCID to comment.