Pith. sign in

REVIEW 5 major objections 4 minor 3 cited by

Learning to Transfer Human Hand Skills for Robot Manipulations

T0 review · 5 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A learned motion manifold over human hand, robot action, and object trajectory lets a robot infer executable manipulation commands from human motion-capture demonstrations, outperforming kinematic fingertip matching in real-world bottle…

desk verdict Novel pseudo-supervision triplet synthesis, but the Eq. 3 inference objective ignores human hand motion except for initialization, so the 'hand skill transfer' claim is over-sold. read the letter →

arxiv 2501.04169 v1 pith:OFWRPUK4 submitted 2025-01-07 cs.RO cs.AIcs.LG

Sungjae Park , Seungho Lee , Mingi Choi , Jiye Lee , Jeonghwan Kim , Jisoo Kim , Hanbyul Joo This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords learning-basedretargetingdexterousmanipulationhumanmotioncapturemanifoldpseudo-supervisionlatentcodeoptimizationteleoperationobjecttrajectorytransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a robot can learn to perform a manipulation task from a human hand motion-capture demonstration without any paired teleoperation of the same task. The proposed solution learns a joint motion manifold over human hand motion, robot action, and object trajectory, then, given a new human demonstration, optimizes a latent code to recover a robot action that should reproduce the demonstrated object motion. Because truly paired human and robot demonstrations of the same object motion cannot be collected, the authors synthesize pseudo-ground-truth triplets by generating plausible human hand motions from teleoperation data, using a regressor plus a human-motion manifold refinement. Real-world tests on bottle, bowl, and book tasks show lower object-trajectory error and higher overall success than kinematic fingertip matching, with the book task as the clear exception. If correct, the method offers a route to scale robot skill learning from casual human demonstrations rather than costly teleoperation.

What carries the argument

The load-bearing mechanism is a joint spatio-temporal manifold learned by a 1-D temporal-convolutional autoencoder $\Psi$ over the concatenated triplet of object trajectory $O$, human hand motion $H$, and robot action $R$. The manifold latent code $L = \Psi_{\text{enc}}(O,H,R)$ models correlations among the three motion components; given a new human demonstration, the robot action is recovered by optimizing $L$ to minimize the decoded object-trajectory error $\|\Psi^O_{\text{dec}}(L)-O\|_2$ and then reading off $R=\Psi^R_{\text{dec}}(L^*)$. Because paired $(O,H,R)$ data do not exist, the paper generates pseudo-supervision with a second manifold over human hands and objects: a neural regressor $\Omega$ maps each teleoperated robot action to an initial human hand pose, and latent optimization over the human-object manifold refines that pose to be consistent with the object trajectory. The refined synthetic triplets train $\Psi$, so the entire pipeline rests on the assumption that these synthesized human motions live on the same manifold as real human demonstrations.

What would settle it

Measure the model's success and object-trajectory error on human demonstrations recorded from a person or grasp style not represented in the training mocap set, with ground-truth object motion tracked by the camera rig; if success rate drops to the level of fingertip matching, the pseudo-GT triplets did not capture the human motion distribution.

Watch

Extended reading notes

Core claim

The central claim is that a retargeting model $F(O,H) \to R$, trained with synthetic pseudo-ground-truth triplets $\{O_i,H_i,R_i\}$, can infer plausible robot actions from human hand motion capture such that executing those actions reproduces the demonstrated object trajectory. The authors argue this addresses the embodiment gap more directly than conventional retargeting, which matches fingertip positions and implicitly assumes the same kinematics produce the same object interaction. The evidence is real-world execution on three tasks: the learned model achieves 0.7, 0.8, and 0.27 success rates on bottle, bowl, and book versus 0.2, 0.5, and 0.45 for fingertip matching, with lower overall center-of-mass and orientation error. The book result is the exception, and the paper reports no instance where a baseline beats it on both accuracy and success simultaneously.

Load-bearing premise

The whole pipeline depends on the synthetic human hand motions $H_i=E(O_i,R_i)$ being representative enough of real human manipulation that a model trained on them transfers to genuine motion-capture demonstrations.

Editorial extensions

If this is right

  • Human motion capture can serve as a scalable data source for dexterous robot manipulation, since the learned model converts each demonstration into an executable robot command without task-specific teleoperation for every example.
  • Robot actions inferred through the manifold are conditioned on the object trajectory, so the method generalizes to unseen object trajectories for the same object, as validated on held-out mocap sequences.
  • The object-aware manifold makes the retargeting robust to small noise in human hand pose estimates: adding Gaussian noise to the bowl-task mocap dropped the fingertip baseline from 0.5 to 0.3 success while the learned model improved from 0.8 to 0.9.
  • Per-object models limit the current scope: generalization across objects and object representations is left as future work by the authors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same manifold argument could extend to a unified retargeting model across objects if the object representation includes local geometry or hand-object proximity rather than only pose, a direction the paper itself points to as future work.
  • Editorial inference: the book-task failure suggests the pseudo-GT synthetic data may not capture fine, sustained contact patterns well; a testable extension would be to condition synthetic hand generation on contact regions or add contact-aware losses.
  • Editorial inference: the robustness-to-noise result hints that the latent manifold acts as a denoiser, so deployment with noisier vision-based hand estimators (rather than the multi-camera rig) is a plausible path the paper does not explicitly claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes a learning-based retargeting framework that takes a human hand motion demonstration and an object trajectory as input and outputs a plausible robot hand action sequence for dexterous manipulation. The method learns a joint temporal-convolutional autoencoder over triplets (O, H, R) of object trajectory, human hand motion, and robot action, trained on synthetic pseudo-ground-truth triplets generated from unpaired human mocap demonstrations and robot teleoperation data. At inference, a latent code is optimized against the target object trajectory and the robot action is decoded from the manifold. Real-world experiments on Bottle, Bowl, and Book tasks with an Allegro hand and xArm arm compare the method against kinematic fingertip-matching baselines. The central claim is that the learned retargeting significantly outperforms conventional kinematic retargeting and effectively transfers human hand skills to the robot.

Significance. If the claims held, the paper would make a useful contribution to dexterous manipulation by reducing dependence on paired teleoperation data and addressing the embodiment gap through a learned joint manifold. The idea of synthesizing pseudo-ground-truth triplets from separately collected human mocap and robot teleoperation data is interesting, and the authors attempt a real-world evaluation rather than simulation-only results. The paper also includes ablations of the hand-motion refinement and the regression-based initializer, which are informative. However, the current formulation does not actually use the observed human hand motion in the inference objective, and the empirical evidence is too thin and partly negative to support the stated 'significantly outperforms' claim. The contribution is potentially salvageable, but the central methodology and evaluation need substantial revision.

major comments (5)
  1. [Section IV-C, Table III] The inference objective optimizes only the object-trajectory reconstruction term, with no term involving the observed human hand motion H. After initialization, the latent code L is free to move anywhere that reconstructs O, so the final robot action R is essentially determined by O alone (modulo the local optimum reached from the initializer). This contradicts Eq. (1), which defines R = F(O, H), and undercuts the abstract's claim that the method infers robot actions from human motion demonstrations. The stress-test concern is valid: the method as written does not transfer human hand skills; it transfers object trajectories. The authors should add an explicit H-consistency term such as lambda * ||Psi^H_dec(L) - H||^2, or provide an ablation showing that the optimized latent code also reconstructs H.
  2. [Table II] The success-rate evaluation uses only 10 to 11 episodes per condition with no confidence intervals, no significance tests, and no repeated-seed variation. In the Book task, the proposed method achieves a lower success rate (0.27) than the Fingertip baseline (0.45), which directly contradicts the abstract's statement that the method 'significantly outperforms conventional retargeting techniques.' The claim of overall superiority rests on a post hoc combined criterion across tasks and metrics; without statistical evidence, the empirical central claim is not supported.
  3. [Section III-A, Eq. (7)] COM and Orientation errors are computed only within succeeded trajectories. Because the set of succeeded episodes differs across methods, these error comparisons are confounded: a method may appear more accurate simply because it succeeded on different or easier episodes. The authors should report errors on all episodes or use paired per-episode comparisons over identical demonstration inputs, and should also report the number of succeeded episodes used for each error average.
  4. [Section IV-C] The synthetic pseudo-GT generation pipeline has the same object-only optimization issue as the main inference. In Eq. (7), the latent code for human hand synthesis is optimized only against the object trajectory, with the robot action R_i affecting the result only through the initialization H_init_i = Omega(R_i). Consequently, the synthesized human hand H_i is not actually anchored to R_i except through initialization, and the triplet database may not encode a consistent human-robot correspondence. This weakens the supervised training signal for F and further separates the method from the human-skill-transfer claim. The authors should verify that the generated H_i retains information from R_i, for example by measuring whether different R_i with the same O_i produce distinguishable H_i, or by comparing the synthetic H_i distribution against real human mocap demonstrations.
  5. [Section IV-C] The statement that 'our method consistently achieved lower COM and Ori errors' is not true for the Book task: Ours (0.052 m, 0.22 rad) is nearly identical to Fingertip (0.055 m, 0.24 rad), and the success rate is lower. The paper's summary of its own results in Section IV-C is therefore misleading and needs to be rewritten to acknowledge the negative Book result and the absence of statistical significance.
minor comments (4)
  1. [Section IV-B] The text says 'the latent code optimization Eq. 2 is performed via a gradient decent method,' but Eq. (2) is the autoencoder reconstruction objective; the optimization being referred to is Eq. (3). Also, 'decent' should be 'descent.'
  2. [Section IV-C] In the robustness experiment, the noise is described only as 'mean=0.001 at all dimension of human hand,' with no standard deviation and no distinction between positional and rotational dimensions. Since wrist rotation uses a 6D representation and finger joints use 3D positions, the noise level should be specified separately for each type of quantity, and the standard deviation should be reported.
  3. [Section IV-C] The term 'total loss' in the discussion of Table II is not defined. If it is the sum of COM and Ori errors, state that explicitly; otherwise define the aggregation rule.
  4. [Section IV-C] The 'noisy mocap' result for the Bowl task shows the proposed method improving from 0.8 to 0.9 success with added noise. With 10 episodes this is a one-episode change and should not be interpreted as evidence of robustness without additional trials.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: the central retargeting claim is benchmarked externally with real-world robot episodes; only the synthetic-hand refinement ablation is self-referential in a minor way.

  1. other [Section III-A (Eq. 7) and Section IV-B (Table I)]
    "li* = arg min_l ||ψ^O_dec(l) − Oi||2 ... All metrics are computed on the synthetic dataset we generated, {Oi, Hiinit_i} and {Oi, Hi}, each referring to before and after applying hand motion refinement ... based on the generated synthetic hand motion and the ground truth object trajectory."

    The refined hand motion Hi is produced by Eq. 7, an optimization that reconstructs the object trajectory Oi through the learned manifold. Section IV-B then evaluates the quality of Hi with contact and penetration metrics computed relative to the same Oi, calling it the 'ground truth object trajectory.' This makes the improvement of Hi over the initial estimate partly a consistency check of the optimizer, not an independent test that the synthesized human hand is realistic. The step is internal to the pseudo-GT data-generation module, however; the main human-to-robot retargeting claim is evaluated on held-out real mocap demonstrations against kinematic baselines, so the circularity does not propagate to the central result.

full rationale

The paper's principal derivation chain is data-driven and externally checked: a joint manifold Ψ is trained on synthetic pseudo-GT triplets (Oi, Hi, Ri), and at inference F optimizes a latent code against a real held-out object trajectory O, then executes the decoded robot action in the real world. The real-robot success rates (Table III) and COM/Ori errors (Table II) are compared against two kinematic baselines on previously unseen human mocap data, so the central claim is not forced by construction. Self-citations to the authors' prior systems [34], [36] supply architecture and capture hardware, but neither is load-bearing in the sense of proving uniqueness or forbidding alternatives. The skeptic's point that Eq. 3 only uses H for initialization is a modeling limitation that the negative Book result may expose, but it is not circular: the method could fail, and does fail on Book. The only mild self-referentiality is the Table I ablation, in which synthetic hand motions are scored against the same object trajectory used to generate them; this is an internal consistency check rather than the paper's headline evaluation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced; the joint motion manifold and latent code are internal learned representations rather than independently testable entities. The main assumptions are about the validity of synthetic human hand motions, the sufficiency of object pose, and the causal quality of teleoperation labels. The free parameters are undisclosed training and optimization hyperparameters plus the ambiguously specified noise level in the robustness experiment.

free parameters (3)
  • Manifold latent code dimension and temporal window size = Not disclosed
    Architecture hyperparameters for the convolutional autoencoders Psi and psi are not reported. The bottleneck size and temporal window directly affect the learned correlation structure and retargeting quality.
  • Latent code optimization iterations and step size = Not disclosed
    Equation 3 is solved by gradient descent, but the number of iterations, learning rate, and convergence criteria are not specified, which matters for reproducing the inference procedure.
  • Noise level in robustness experiment = mean=0.001, standard deviation not stated
    The robustness test adds Gaussian noise with mean 0.001 to human hand dimensions. Without the standard deviation or multiple seeds, the reported improvement under noise cannot be interpreted or replicated.
assumptions (4)
  • domain assumption Synthesized human hand motions H_i = E(O_i,R_i) are representative enough of real human demonstrations to train the retargeting model F
    The pseudo-GT triplet DB pairs teleoperation object trajectories and robot actions with generated human hand motions. If the synthetic H_i distribution diverges from real mocap, F will inherit the bias. This enters in Section III-A.
  • domain assumption Object pose trajectory is a sufficient representation of the manipulation interaction for retargeting
    The method uses only 3D object position and rotation as the object representation in Equations 1 and 3. Contact forces, object slip, and task-specific constraints are not modeled, so the object pose must carry enough information about the task. See Section III, Equation 1.
  • domain assumption Teleoperation robot actions R_i actually produced the recorded object trajectory O_i
    The teleoperation data are treated as valid labels of robot actions that achieve the observed object motion, despite the noted discrepancy between desired and actual joint values (Section IV-A, Figure 5). If the recorded actions are not causally responsible for O_i, the pseudo-GT triplets are misaligned.
  • domain assumption Latent code optimization with only the object-trajectory loss recovers robot actions consistent with the human hand input
    Equation 3 optimizes only the decoded object trajectory component and uses the human hand only for initialization. The correctness of the decoded robot action depends on the learned manifold correlating H and R well enough. See Section III, Equations 3 and 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Transfer Human Hand Skills for Robot Manipulations." pith.science (2026). https://pith.science/paper/OFWRPUK4

@misc{pith2026250104169,
  author       = {Pith},
  title        = {Pith review of: Learning to Transfer Human Hand Skills for Robot Manipulations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OFWRPUK4}},
  note         = {Machine review of arXiv:2501.04169}
}
read the original abstract

We present a method for teaching dexterous manipulation tasks to robots from human hand motion demonstrations. Unlike existing approaches that solely rely on kinematics information without taking into account the plausibility of robot and object interaction, our method directly infers plausible robot manipulation actions from human motion demonstrations. To address the embodiment gap between the human hand and the robot system, our approach learns a joint motion manifold that maps human hand movements, robot hand actions, and object movements in 3D, enabling us to infer one motion component from others. Our key idea is the generation of pseudo-supervision triplets, which pair human, object, and robot motion trajectories synthetically. Through real-world experiments with robot hand manipulation, we demonstrate that our data-driven retargeting method significantly outperforms conventional retargeting techniques, effectively bridging the embodiment gap between human and robotic hands. Website at https://rureadyo.github.io/MocapRobot/.

Figures

Figures reproduced from arXiv: 2501.04169 by the authors.

Figure 1
Figure 1. Our model learns a human-to-robot retargeting model using an unpaired (i.e., object may move differently) human mocap and robot teleoperation dataset. demonstrations, and also output physically plausible actions for the robot hand under such embodiment gap. In this paper, we propose a novel approach to inferring plausible robot hand actions from human motion demonstra￾tions, through a learning-based retargeting meth… view at source ↗
Figure 2
Figure 2. Overview of the Proposed Framework. We first synthesize the paired triplet dataset consisting of robot action and human motion achieving the same object trajectory, followed by learning a retargeting module. The retargeting model is evaluated in real world, and we use IsaacGym simulator for visualization only. code L as follows: L * = arg min L ∥ΨO dec(L) − O∥2, (3) where ΨO dec is the object trajectory component de… view at source ↗
Figure 3
Figure 3. System Overview: Our system consists of 16 syn￾chronized cameras, an xArm6 robot arm, and a 16-DoF Allegro robot hand [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Objects used in the experiment and a marker system [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Visualization of robot teleoperation dataset. Yellow: desired robot joint values. White: actual robot joint values. The dataset is collected in real world, and Isaac Gym simulator is only used for rendering [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 7
Figure 7. Figure 7: Ablation on different initial estimate before refine [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video

    cs.RO 2026-08 conditional novelty 7.0 of 10

    C2Dex converts monocular human videos into executable dexterous robot manipulation trajectories by using stable object-side contacts as a shared representation for reconstruction and retargeting, achieving 57.78% and ...

  2. DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation

    cs.RO 2025-05 conditional novelty 6.0 of 10

    DexMachina uses decaying virtual object controllers as a curriculum to train bimanual dexterous policies that track demonstrated object states, and reports large gains over baselines on a new six-hand benchmark.

  3. Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Research

    cs.RO 2025-06 accept novelty 1.0 of 10

    A perspective article reviews the state of using foundation models for laboratory automation and proposes a roadmap for fully autonomous experiments.

Reference graph

Works this paper leans on

37 extracted references · 31 canonical work pages · cited by 3 Pith papers

  1. [1]

    Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system

    Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan Ratliff, and Dieter Fox. Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system. In ICRA, 2020

    work page 2020

  2. [2]

    Robotic telekinesis: Learning a robotic hand imitator by watching humans on youtube

    Aravind Sivakumar, Kenneth Shaw, and Deepak Pathak. Robotic telekinesis: Learning a robotic hand imitator by watching humans on youtube. In RSS, 2022

    work page 2022

  3. [3]

    Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system

    Yuzhe Qin, Wei Yang, Binghao Huang, Karl Van Wyk, Hao Su, Xiaolong Wang, Yu-Wei Chao, and Dieter Fox. Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system. In RSS, 2023

    work page 2023

  4. [4]

    Dexterous imitation made easy: A learning-based framework for efficient dexterous manipulation

    Sridhar Pandian Arunachalam, Sneha Silwal, Ben Evans, and Lerrel Pinto. Dexterous imitation made easy: A learning-based framework for efficient dexterous manipulation. In ICRA, 2023

    work page 2023

  5. [5]

    Dexmv: Imitation learning for dexterous manipulation from human videos

    Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu, Hanwen Jiang, Ruihan Yang, Yang Fu, and Xiaolong Wang. Dexmv: Imitation learning for dexterous manipulation from human videos. In ECCV, 2022

    work page 2022

  6. [6]

    Reconstructing hands in 3d with transformers

    Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Reconstructing hands in 3d with transformers. In CVPR, 2024

    work page 2024

  7. [7]

    Frankmocap: A monocular 3d whole-body pose estimation system via regression and integration

    Yu Rong, Takaaki Shiratori, and Hanbyul Joo. Frankmocap: A monocular 3d whole-body pose estimation system via regression and integration. In CVPR, 2021

    work page 2021

  8. [8]

    Mocap everyone everywhere: Lightweight motion capture with smartwatches and a head-mounted camera

    Jiye Lee and Hanbyul Joo. Mocap everyone everywhere: Lightweight motion capture with smartwatches and a head-mounted camera. In CVPR, 2024

    work page 2024

Show all 37 references
  1. [9]

    Dexcap: Scalable and portable mocap data collection system for dexterous manipulation

    Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, and C Karen Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation. In RSS, 2024

    2024

  2. [10]

    Telemoma: A modular and versatile teleoperation system for mobile manipulation

    Shivin Dass, Wensi Ai, Yuqian Jiang, Samik Singh, Jiaheng Hu, Ruohan Zhang, Peter Stone, Ben Abbatematteo, and Roberto Martin- Martin. Telemoma: A modular and versatile teleoperation system for mobile manipulation. arXiv preprint arXiv:2403.07869 , 2024

    2024 arXiv

  3. [11]

    Open-television: teleoperation with immersive active visual feedback

    Xuxin Cheng, Jialong Li, Shiqi Yang, Ge Yang, and Xiaolong Wang. Open-television: teleoperation with immersive active visual feedback. In CoRL, 2024

    2024

  4. [12]

    Ace: A cross-platform visual-exoskeletons system for low-cost dexterous teleoperation

    Shiqi Yang, Minghuan Liu, Yuzhe Qin, Runyu Ding, Jialong Li, Xuxin Cheng, Ruihan Yang, Sha Yi, and Xiaolong Wang. Ace: A cross-platform visual-exoskeletons system for low-cost dexterous teleoperation. In CoRL, 2024

    2024

  5. [13]

    Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning

    Runyu Ding, Yuzhe Qin, Jiyue Zhu, Chengzhe Jia, Shiqi Yang, Ruihan Yang, Xiaojuan Qi, and Xiaolong Wang. Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning. arXiv preprint arXiv:2407.03162, 2024

    2024 arXiv

  6. [14]

    Bimanual dexterity for complex tasks

    Kenneth Shaw, Yulong Li, Jiahui Yang, Mohan Kumar Srirama, Ray Liu, Haoyu Xiong, Russell Mendonca, and Deepak Pathak. Bimanual dexterity for complex tasks. In CoRL, 2024

    2024

  7. [15]

    Omnih2o: Univer- sal and dexterous human-to-humanoid whole-body teleoperation and learning

    Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Univer- sal and dexterous human-to-humanoid whole-body teleoperation and learning. In CoRL, 2024

    2024

  8. [16]

    Learning human-to-humanoid real- time whole-body teleoperation

    Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning human-to-humanoid real- time whole-body teleoperation. In IROS, 2024

    2024

  9. [17]

    Humanplus: Humanoid shadowing and imitation from humans

    Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. Humanplus: Humanoid shadowing and imitation from humans. In CoRL, 2024

    2024

  10. [18]

    Holo-dex: Teaching dexterity with immersive mixed reality

    Sridhar Pandian Arunachalam, Irmak G ¨uzey, Soumith Chintala, and Lerrel Pinto. Holo-dex: Teaching dexterity with immersive mixed reality. In ICRA, 2023

    2023

  11. [19]

    Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators

    Philipp Wu, Yide Shentu, Zhongke Yi, Xingyu Lin, and Pieter Abbeel. Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators. arXiv preprint arXiv:2309.13037 , 2023

    2023 arXiv

  12. [20]

    Ace: Adversarial correspondence embedding for cross morphology motion retargeting from human to nonhuman characters

    Tianyu Li, Jungdam Won, Alexander Clegg, Jeonghwan Kim, Akshara Rai, and Sehoon Ha. Ace: Adversarial correspondence embedding for cross morphology motion retargeting from human to nonhuman characters. In SIGGRAPH Asia , 2023

    2023

  13. [21]

    Time- contrastive networks: Self-supervised learning from video

    Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine, and Google Brain. Time- contrastive networks: Self-supervised learning from video. In ICRA, 2018

    2018

  14. [22]

    Human- conquad: human motion control of quadrupedal robots using deep reinforcement learning

    Sunwoo Kim, Maks Sorokin, Jehee Lee, and Sehoon Ha. Human- conquad: human motion control of quadrupedal robots using deep reinforcement learning. In SIGGRAPH Asia , 2022

    2022

  15. [23]

    Crossloco: Human motion driven control of legged robots via guided unsupervised reinforcement learning

    Tianyu Li, Hyunyoung Jung, Matthew Gombolay, Yong Kwon Cho, and Sehoon Ha. Crossloco: Human motion driven control of legged robots via guided unsupervised reinforcement learning. In ICLR, 2024

    2024

  16. [24]

    One-shot transfer of long-horizon extrinsic manipulation through contact retargeting

    Albert Wu, Ruocheng Wang, Sirui Chen, Clemens Eppner, and C Karen Liu. One-shot transfer of long-horizon extrinsic manipulation through contact retargeting. In IROS, 2024

    2024

  17. [25]

    Multi-finger grasping like humans

    Yuming Du, Philippe Weinzaepfel, Vincent Lepetit, and Romain Br´egier. Multi-finger grasping like humans. In IROS, 2022

    2022

  18. [26]

    Contact transfer: A direct, user-driven method for human to robot transfer of grasps and manipulations

    Arjun Lakshmipathy, Dominik Bauer, Cornelia Bauer, and Nancy S Pollard. Contact transfer: A direct, user-driven method for human to robot transfer of grasps and manipulations. In ICRA, 2022

    2022

  19. [27]

    Kinematic motion retargeting for contact-rich anthropomorphic ma- nipulations

    Arjun S Lakshmipathy, Jessica K Hodgins, and Nancy S Pollard. Kinematic motion retargeting for contact-rich anthropomorphic ma- nipulations. arXiv preprint arXiv:2402.04820 , 2024

    2024 arXiv

  20. [28]

    Videodex: Learning dexterity from internet videos

    Kenneth Shaw, Shikhar Bahl, and Deepak Pathak. Videodex: Learning dexterity from internet videos. In CoRL, 2023

    2023

  21. [29]

    Hrp: Human affordances for robotic pre-training

    Mohan Kumar Srirama, Sudeep Dasari, Shikhar Bahl, and Abhinav Gupta. Hrp: Human affordances for robotic pre-training. In RSS, 2024

    2024

  22. [30]

    Scaling egocentric vision: The epic-kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Scaling egocentric vision: The epic-kitchens dataset. In ECCV, 2018

    2018

  23. [31]

    Quasisim: Parameterized quasi-physical simulators for dexterous manipulations transfer

    Xueyi Liu, Kangbo Lyu, Jieqiong Zhang, Tao Du, and Li Yi. Quasisim: Parameterized quasi-physical simulators for dexterous manipulations transfer. In ECCV, 2024

    2024

  24. [32]

    On the continuity of rotation representations in neural networks

    Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neural networks. In CVPR, 2019

    2019

  25. [33]

    A deep learning framework for character motion synthesis and editing

    Daniel Holden, Jun Saito, and Taku Komura. A deep learning framework for character motion synthesis and editing. In ACM Transa. Graph., 2016

    2016

  26. [34]

    Locomotion-action-manipulation: Syn- thesizing human-scene interactions in complex 3d environments

    Jiye Lee and Hanbyul Joo. Locomotion-action-manipulation: Syn- thesizing human-scene interactions in complex 3d environments. In ICCV, 2023

    2023

  27. [35]

    Attention is all you need

    A Vaswani. Attention is all you need. In Neurips, 2017

    2017

  28. [36]

    Parahome: Parameterizing everyday home activities towards 3d gen- erative modeling of human-object interactions

    Jeonghwan Kim, Jisoo Kim, Jeonghyeon Na, and Hanbyul Joo. Parahome: Parameterizing everyday home activities towards 3d gen- erative modeling of human-object interactions. arXiv preprint arXiv:2401.10232, 2024

    2024 arXiv

  29. [37]

    Learning dexter- ous manipulation from exemplar object trajectories and pre-grasps

    Sudeep Dasari, Abhinav Gupta, and Vikash Kumar. Learning dexter- ous manipulation from exemplar object trajectories and pre-grasps. In ICRA, 2023

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.