REVIEW 14 cited by
Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. Human hands and robot hands differ in shape, size, and joint structure, and performing this translation from a single uncalibrated camera is a highly underconstrained problem. Moreover, the retargeted trajectories must effectively execute tasks on a physical robot, which requires them to be temporally smooth and free of self-collisions. Our key insight is that while paired human-robot correspondence data is expensive to collect, the internet contains a massive corpus of rich and diverse human hand videos. We leverage this data to train a system that understands human hands and retargets a human video stream into a robot hand-arm trajectory that is smooth, swift, safe, and semantically similar to the guiding demonstration. We demonstrate that it enables previously untrained people to teleoperate a robot on various dexterous manipulation tasks. Our low-cost, glove-free, marker-free remote teleoperation system makes robot teaching more accessible and we hope that it can aid robots in learning to act autonomously in the real world. Videos at https://robotic-telekinesis.github.io/
Forward citations
Cited by 14 Pith papers
-
High Fidelity Capture, Reconstruction, and Transfer of Human Demonstrations for Robot-Assisted Bathing
A contact-centered pipeline yields the first multimodal dataset of clinician-performed bathing and transfers the demonstrations to a soft robotic hand and arm.
-
Teleopit: A Full-Embodiment Humanoid Teleoperation System
Teleopit combines VR body, hand, and head tracking with a learned whole-body tracker and a cross-hand retargeter, and teleop-collected demos train ACT and GR00T policies to around 90 to 95 percent success on a humanoi...
-
DexDirect: Direct Kinesthetic Arm Guidance for Efficient Dexterous Demonstration Collection
A hybrid kinesthetic-arm-plus-webcam-hand teleoperation interface achieved 17x/3x higher demonstration throughput than vision baselines and trained a 90%-success pick-and-place policy in a ten-person study.
-
Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting
Sampling-Based Retargeter (SBR) delivers lower-jitter real-time kinematic hand retargeting and higher task success with less operator fatigue than gradient-based baselines in an 18-person study.
-
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Task-agnostic RL play pretraining on diverse objects yields a reusable dexterous prior that makes sparse-reward assembly learning ~33× more sample-efficient and enables zero-shot sim-to-real transfer on tight insertio...
-
SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows
SERNF fine-tunes dexterous manipulation policies on real hardware by pairing normalizing-flow policies with action-chunked critics and conservative off-policy RL.
-
DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation
A dual-phase diffusion planner with dynamics-consistency and LLM-written guidance achieves strong success on goal-adaptive dexterous manipulation in simulation.
-
Bimanual Dexterity for Complex Tasks
BiDex combines Manus mocap gloves for finger tracking with GELLO-style teacher arms for wrist tracking to enable low-cost, portable, bimanual dexterous teleoperation that outperforms VR and SteamVR baselines on most t...
-
DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control
A dual-arm teleoperation system combines a graph-based motion retargeting network with VLM-informed MPC force control to achieve cross-platform motion mapping and adaptive grasping across multiple robots and objects.
-
ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling
Decoupled arm–hand retargeting with anthropomorphic arm-plane constraints and polyhedral contact invariants raises real-robot dexterous-task success to 75.8% versus 61.6% and 50.8% for OKAMI and ORION.
-
CRAFT: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance
Hybrid hard-soft tendon hand with rolling joints improves strength and fragile teleop over a rigid baseline, covers 33/33 Feix grasps, and costs under $600 open-source.
-
TelePreview: A User-Friendly Teleoperation System with Virtual Arm Assistance for Enhanced Effectiveness
TelePreview adds a physically aligned augmented-reality preview and a preview/execute foot-pedal switch to low-cost glove-and-IMU teleoperation, and reports higher success rates and shorter execution times in a five-t...
-
Arm Robot: AR-Enhanced Embodied Control and Visualization for Intuitive Robot Arm Manipulation
An AR-embodied teleoperation interface with freeze, scale, and mirror mapping plus a zero-delay virtual robot preview, evaluated with 18 users on cube and daily-object tasks.
-
Modality-Driven Design for Multi-Step Dexterous Manipulation: Insights from Neuroscience
A neuroscience-inspired, modality-driven pipeline with classical control, a vision-language-action model, and force-feedback RL performs pick-and-rotate on a real robot, but only 5 of 35 trials complete all steps.
Discussion (0). Continue with ORCID to comment.