pith. sign in

arxiv: 2606.06033 · v2 · pith:AEVUSAYYnew · submitted 2026-06-04 · 💻 cs.RO

RealDexUMI: A Wearable Universal Manipulation Interface for Dexterous Robot Learning

Chaoyi Xu , Yixuan Jiang , Jiahui Huan , Yuhui Fu , Haoyu Zhou , Weitian Yuan , Jiayi Yu , Wanpeng Zhang
show 2 more authors
Haoqi Yuan Zongqing Lu
This is my paper

Pith reviewed 2026-06-28 01:41 UTC · model grok-4.3

classification 💻 cs.RO
keywords dexterous manipulationteleoperationwearable interfacetactile sensingin-hand visionembodiment transferimitation learningrobot learning
0
0 comments X

The pith

A wearable shared-hand interface collects dexterous demonstrations that transfer directly to robots without retargeting losses.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces RealDexUMI, a wearable device centered on a shared dexterous hand module that combines a lightweight hand, in-hand vision, and fingertip tactile sensors. A palm-side isomorphic teleoperation glove maps human finger motions straight to robot joint commands, producing matched observations, contacts, and actions between collection and deployment. Policies trained on the collected data reach an average success rate of 88.75 percent across eight real-robot tasks that include fine-grained, contact-rich, long-horizon, and bimanual manipulation. These policies also generalize to unseen initial poses and transfer across three different robot embodiments.

Core claim

RealDexUMI uses a shared dexterous end-effector module and isomorphic teleoperation glove to generate zero-gap end-effector data, with identical in-hand observations, tactile signals, contacts, and hand actions between human collection and robot deployment. Imitation policies trained on this data achieve an average success rate of 88.75 percent on eight tasks, generalize to unseen initial poses, and transfer across three embodiments.

What carries the argument

The shared dexterous end-effector module integrating a lightweight hand, in-hand vision, and fingertip tactile sensing, paired with the palm-side isomorphic teleoperation glove for retargeting-free joint mapping.

If this is right

  • Policies trained on RealDexUMI data achieve 88.75 percent average success on eight real-robot tasks.
  • The policies generalize to unseen initial poses.
  • The policies transfer across three different robot embodiments.
  • The interface supports data collection for fine-grained, contact-rich, long-horizon, and bimanual manipulation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same shared-module approach could support collection of larger-scale datasets by lowering the expertise needed for teleoperation.
  • If the zero-gap property holds, the design might combine with other modalities such as audio or force sensing.
  • Data collected this way could serve as a starting point for reinforcement learning fine-tuning on new tasks.
  • The interface might extend beyond humanoid robots if the hand module can be mounted on different arm types.

Load-bearing premise

The shared hand and sensing modules produce identical end-effector observations and actions during human data collection and robot deployment.

What would settle it

Deploy the trained policies on robots whose hand, camera, and tactile sensors differ from those used in collection and check whether success rates fall substantially below 88.75 percent.

Figures

Figures reproduced from arXiv: 2606.06033 by Chaoyi Xu, Haoqi Yuan, Haoyu Zhou, Jiahui Huan, Jiayi Yu, Wanpeng Zhang, Weitian Yuan, Yixuan Jiang, Yuhui Fu, Zongqing Lu.

Figure 1
Figure 1. Figure 1: RealDexUMI turns a dexterous hand into a shared interface for zero-gap wearable demonstration and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Hardware system overview. The wearable device combines a reusable dexterous end-effector module, a 6-DoF tracker, and a palm-side isomorphic teleoperation glove. The end-effector module consists of the lightweight dexterous hand, in-hand camera, and fingertip tactile sensors, and is mounted on robot bodies during deployment. by a human-worn exoskeleton rather than the deployed robot hand, the recorded sign… view at source ↗
Figure 3
Figure 3. Figure 3: Lightweight dexterous hand module. The hand uses compact finger actuation, integrated fingertip tactile sensing, and a lightweight structural shell. tactile array. These sensors provide explicit contact observations at the same fingertip surfaces used during robot execution, reducing reliance on vision-only contact inference. 3.3 Palm-Side Isomorphic Teleoperation Glove The palm-side teleoperation glove is… view at source ↗
Figure 4
Figure 4. Figure 4: Action–state correspondence. By learning from paired executable hand actions and states, the policy receives direct supervision for contact-aware corrections in contact-rich manipulation, which state-only supervision cannot provide. 4 Policy Learning and Deployment 4.1 Policy Interface From the recorded demonstration streams, the policy observation at timestep t is ot = [PITH_FULL_IMAGE:figures/full_fig_p… view at source ↗
Figure 5
Figure 5. Figure 5: Policy rollouts. Policies trained from RealDexUMI demonstrations execute representative tasks across multi-object grasping, precision insertion, tool use, twisting, articulated-object interaction, long-horizon execution, and bimanual operation. The policy predicts a chunk of future actions: Aˆ t = πθ(ot) = {aˆt,1, . . . , aˆt,C }. (4) We instantiate πθ with ACT [51] for all main experiments and train it on… view at source ↗
Figure 6
Figure 6. Figure 6: Initial-pose robustness. A cube pick-and-place policy is evaluated under unseen initial robot poses. Initial-pose variation [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Cross-embodiment deployment. The same drawer-stowing checkpoint runs on Franka FR3, RealMan RM65, and PND Adam-U without retraining. 5.2 Cross-Embodiment Deployment [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Teleoperation comparison. Time is averaged over successful trials. Trials exceeding 5 min are counted as failures. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Glove embedded interface. Six AS5600L magnetic encoders measure the actuated glove DoFs. The ESP32-S3 controller reads encoder values through I2C and streams the resulting 6-D command vector to the host computer through USB serial. A.1 Glove Sensing Interface The glove measures six actuated command DoFs using magnetic encoders. Each measured DoF corresponds to one actuated DoF of the RealDexUMI hand, produ… view at source ↗
Figure 10
Figure 10. Figure 10: illustrates the single-joint encoder design. For each sensed DoF, a diametric magnet is aligned with the joint rotation axis and placed above an AS5600L sensor. The sensor provides an absolute angular reading by measuring the magnetic field direction of the rotating magnet. This non-contact measurement avoids mechanical friction in the sensing path and allows the glove to recover the current joint reading… view at source ↗
Figure 11
Figure 11. Figure 11: Representative task props. We use simple fixtures or 3D-printed props for tasks where controlled geometry is useful: a cup and 2.5 cm cube for cube pick-and-place, a ridged cap fixture for lid twisting, a tweezer-and-cup setup for tea picking, and a plug fixture for insertion. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Policy training loss. Action-prediction L1 loss during training. D.1 Diffusion Policy Baseline We additionally train Diffusion Policy on the same RealDexUMI demonstrations using the same observation and action interface as the ACT policies in the main experiments. This evaluation tests whether RealDexUMI data can support another common imitation-learning policy backend [PITH_FULL_IMAGE:figures/full_fig_p… view at source ↗
Figure 13
Figure 13. Figure 13: Survey form for perceived teleoperation complexity. Evaluators rate the perceived setup and operation complexity of each demonstration interface using a three-level scale. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
read the original abstract

Learning dexterous manipulation requires demonstrations that preserve fine hand-object interactions while remaining executable at deployment. Existing pipelines either lose deployable dexterity through retargeting or embodiment conversion, or rely on robot-specific teleoperation that is costly to scale and often lacks intuitive, contact-aware control for dexterous data collection. We present RealDexUMI, a wearable universal manipulation interface built around a shared dexterous end-effector module that integrates a lightweight dexterous hand, in-hand vision, and fingertip tactile sensing. A palm-side isomorphic teleoperation glove maps human finger inputs to robot-hand joint commands, enabling real-time, retargeting-free, intuitive, and precise hand control. The shared hand and sensing modules yield zero-gap end-effector data, with matched in-hand observations, tactile signals, contacts, and hand actions between collection and deployment. Across eight real-robot tasks spanning fine-grained, contact-rich, long-horizon, and bimanual manipulation, policies trained on RealDexUMI data achieve an average success rate of 88.75%, generalize to unseen initial poses, and transfer across three embodiments. Website: https://research.beingbeyond.com/realdexumi

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript introduces RealDexUMI, a wearable universal manipulation interface centered on a shared dexterous hand module with in-hand vision and fingertip tactile sensing. A palm-side isomorphic teleoperation glove enables retargeting-free data collection. The core claim is that the shared hardware produces zero-gap end-effector observations and actions, allowing policies trained on the collected data to achieve an average success rate of 88.75% across eight real-robot tasks (fine-grained, contact-rich, long-horizon, bimanual), generalize to unseen initial poses, and transfer across three embodiments.

Significance. If the zero-gap property and reported success rates hold under rigorous evaluation, the work would be significant for dexterous robot learning. It directly tackles the data-collection bottleneck by providing scalable, intuitive, contact-aware demonstrations that avoid retargeting losses and embodiment gaps, potentially enabling more efficient policy training than existing teleoperation or conversion pipelines.

major comments (1)
  1. [Abstract, §4] Abstract and §4 (Results): The central performance claim of 88.75% average success rate, generalization, and cross-embodiment transfer is load-bearing, yet the abstract supplies no trial counts per task, variance, exclusion criteria, or baseline comparisons. Without these in the results section, the empirical support for the zero-gap advantage cannot be fully assessed.
minor comments (2)
  1. [§3] §3 (Hardware/Method): The description of how the isomorphic glove maps human finger inputs to robot joint commands should include explicit equations or pseudocode for the mapping to support reproducibility.
  2. [Figure 1, §2] Figure 1 and §2: The diagram of the shared hand module would benefit from clearer labeling of the tactile sensor locations and in-hand camera field of view to illustrate the matched observations.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on the empirical claims. We address the major comment point-by-point below and will revise the manuscript to strengthen the presentation of results.

read point-by-point responses

  1. Referee: [Abstract, §4] Abstract and §4 (Results): The central performance claim of 88.75% average success rate, generalization, and cross-embodiment transfer is load-bearing, yet the abstract supplies no trial counts per task, variance, exclusion criteria, or baseline comparisons. Without these in the results section, the empirical support for the zero-gap advantage cannot be fully assessed.

    Authors: We agree that the abstract and §4 would benefit from greater transparency on the evaluation protocol. In the revised version we will expand §4 to report: (i) the exact number of trials per task (20–40 trials depending on task complexity), (ii) standard deviations or success-rate ranges across trials, (iii) any exclusion criteria (e.g., hardware resets or sensor failures), and (iv) quantitative comparisons against at least one baseline (retargeted teleoperation and/or direct robot teleoperation). The abstract will be updated to note that these details appear in §4. These additions will make the support for the zero-gap advantage fully assessable. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper introduces a wearable hardware interface (RealDexUMI) for data collection and reports empirical policy success rates (88.75% average) on real-robot tasks. No mathematical derivation chain, equations, fitted parameters renamed as predictions, or self-citation load-bearing steps appear in the provided text. Claims rest on experimental outcomes from training on collected data rather than definitional equivalence or imported uniqueness results. The central premise of zero-gap data via shared modules is a hardware design claim, not a circular reduction to inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides insufficient technical detail to enumerate free parameters, axioms, or invented entities; no equations or modeling choices are visible.

pith-pipeline@v0.9.1-grok · 5771 in / 902 out tokens · 26749 ms · 2026-06-28T01:41:58.580906+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 10 linked inside Pith

  1. [1]

    Dexvitac: Collecting human visuo-tactile-kinematic demonstrations for contact-rich dexterous manipulation.arXiv preprint arXiv:2603.17851, 2026

    Xitong Chen, Yifeng Pan, Min Li, and Xiaotian Ding. Dexvitac: Collecting human visuo-tactile-kinematic demonstrations for contact-rich dexterous manipulation.arXiv preprint arXiv:2603.17851, 2026

    arXiv 2026

  2. [2]

    Dexumi: Using human hand as the universal manipulation interface for dexterous manipulation

    Mengda Xu, Han Zhang, Yifan Hou, Zhenjia Xu, Linxi Fan, Manuela Veloso, and Shuran Song. Dexumi: Using human hand as the universal manipulation interface for dexterous manipulation. InConference on Robot Learning, pages 437–459. PMLR, 2025

    2025

  3. [3]

    Robocoin: An open-sourced bimanual robotic data collection for integrated manipulation

    Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, et al. Robocoin: An open-sourced bimanual robotic data collection for integrated manipulation. arXiv preprint arXiv:2511.17441, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Ace: A cross-platform visual-exoskeletons system for low-cost dexterous teleoperation.arXiv preprint arXiv:2408.11805, 2024

    Shiqi Yang, Minghuan Liu, Yuzhe Qin, Runyu Ding, Jialong Li, Xuxin Cheng, Ruihan Yang, Sha Yi, and Xiaolong Wang. Ace: A cross-platform visual-exoskeletons system for low-cost dexterous teleoperation.arXiv preprint arXiv:2408.11805, 2024

    arXiv 2024

  5. [5]

    Airexo-2: Scaling up generalizable robotic imitation learning with low-cost exoskeletons.arXiv preprint arXiv:03081, 2025

    Hongjie Fang, Chenxi Wang, Yiming Wang, Jingjing Chen, Shangning Xia, Jun Lv, Zihao He, Xiyan Yi, Yunhan Guo, Xinyu Zhan, Lixin Yang, Weiming Wang, Cewu Lu, and Hao-Shu Fang. Airexo-2: Scaling up generalizable robotic imitation learning with low-cost exoskeletons.arXiv preprint arXiv:03081, 2025

    2025

  6. [6]

    Being-h0: Vision-language-action pretraining from large-scale human videos.arXiv preprint arXiv:2507.15597, 2025

    Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, Haoqi Yuan, Jiazheng Liu, Chaoyi Xu, Qin Jin, and Zongqing Lu. Being-h0: Vision-language-action pretraining from large-scale human videos.arXiv preprint arXiv:2507.15597, 2025

    arXiv 2025

  7. [7]

    Being-h0

    Hao Luo, Ye Wang, Wanpeng Zhang, Sipeng Zheng, Ziheng Xi, Chaoyi Xu, Haiweng Xu, Haoqi Yuan, Chi Zhang, Yiqing Wang, et al. Being-h0. 5: Scaling human-centric robot learning for cross-embodiment generalization.arXiv preprint arXiv:2601.12993, 2026

    arXiv 2026

  8. [8]

    Dexwild: Dexterous human interactions for in-the-wild robot policies.arXiv preprint arXiv:2505.07813, 2025

    Tony Tao, Mohan Kumar Srirama, Jason Jingzhou Liu, Kenneth Shaw, and Deepak Pathak. Dexwild: Dexterous human interactions for in-the-wild robot policies.arXiv preprint arXiv:2505.07813, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    World in your hands: A large-scale and open-source ecosystem for learning human-centric manipulation in the wild.arXiv preprint arXiv:2512.24310, 2025

    Yupeng Zheng, Jichao Peng, Weize Li, Yuhang Zheng, Xiang Li, Yujie Jin, Julong Wei, Guanhua Zhang, Ruiling Zheng, Ming Cao, et al. World in your hands: A large-scale and open-source ecosystem for learning human-centric manipulation in the wild.arXiv preprint arXiv:2512.24310, 2025

    arXiv 2025

  10. [10]

    Open-television: Teleoperation with immersive active visual feedback.arXiv preprint arXiv:2407.01512, 2024

    Xuxin Cheng, Jialong Li, Shiqi Yang, Ge Yang, and Xiaolong Wang. Open-television: Teleoperation with immersive active visual feedback.arXiv preprint arXiv:2407.01512, 2024

    arXiv 2024

  11. [11]

    Open teach: A versatile teleoperation system for robotic manipulation.arXiv preprint arXiv:2403.07870, 2024

    Aadhithya Iyer, Zhuoran Peng, Yinlong Dai, Irmak Guzey, Siddhant Haldar, Soumith Chintala, and Lerrel Pinto. Open teach: A versatile teleoperation system for robotic manipulation.arXiv preprint arXiv:2403.07870, 2024

    arXiv 2024

  12. [12]

    Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system

    Yuzhe Qin, Wei Yang, Binghao Huang, Karl Van Wyk, Hao Su, Xiaolong Wang, Yu-Wei Chao, and Dieter Fox. Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system. InRobotics: Science and Systems, 2023

    2023

  13. [13]

    Unitachand: Unified spatio-tactile represen- tation for human to robotic hand skill transfer.arXiv preprint arXiv:2512.21233, 2025

    Chi Zhang, Penglin Cai, Haoqi Yuan, Chaoyi Xu, and Zongqing Lu. Unitachand: Unified spatio-tactile represen- tation for human to robotic hand skill transfer.arXiv preprint arXiv:2512.21233, 2025

    arXiv 2025

  14. [14]

    Zhao, and Chelsea Finn

    Zipeng Fu, Tony Z. Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. InConference on Robot Learning (CoRL), 2024

    2024

  15. [15]

    Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators, 2023

    Philipp Wu, Yide Shentu, Zhongke Yi, Xingyu Lin, and Pieter Abbeel. Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators, 2023

    2023

  16. [16]

    Dexterous teleoperation of 20-dof bytedexter hand via human motion retargeting, 2025

    Ruoshi Wen, Jiajun Zhang, Guangzeng Chen, Zhongren Cui, Min Du, Yang Gou, Zhigang Han, Junkai Hu, Liqun Huang, Hao Niu, Wei Xu, Haoxiang Zhang, Zhengming Zhu, Hang Li, and Zeyu Ren. Dexterous teleoperation of 20-dof bytedexter hand via human motion retargeting, 2025. URLhttps://arxiv.org/abs/2507.03227

    arXiv 2025

  17. [17]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  18. [18]

    Dexop: A device for robotic transfer of dexterous human manipulation.arXiv preprint arXiv:2509.04441, 2025

    Hao-Shu Fang, Branden Romero, Yichen Xie, Arthur Hu, Bo-Ruei Huang, Juan Alvarez, Matthew Kim, Gabriel Margolis, Kavya Anbarasu, Masayoshi Tomizuka, et al. Dexop: A device for robotic transfer of dexterous human manipulation.arXiv preprint arXiv:2509.04441, 2025. 11

    arXiv 2025

  19. [19]

    Dexexo: A wearability-first dexterous exoskeleton for operator-agnostic demonstration and learning.arXiv preprint arXiv:2603.17323, 2026

    Alvin Zhu, Mingzhang Zhu, Beom Jun Kim, Jose Victor SH Ramos, Yike Shi, Yufeng Wu, Raayan Dhar, Fuyi Yang, Ruochen Hou, Hanzhang Fang, et al. Dexexo: A wearability-first dexterous exoskeleton for operator-agnostic demonstration and learning.arXiv preprint arXiv:2603.17323, 2026

    arXiv 2026

  20. [20]

    Dex-mouse: A low-cost portable and universal interface with force feedback for data collection of dexterous robotic hands.arXiv preprint arXiv:2604.15013, 2026

    Joonho Koh, Haechan Jung, Nayoung Kim, Wook Ko, and Changjoo Nam. Dex-mouse: A low-cost portable and universal interface with force feedback for data collection of dexterous robotic hands.arXiv preprint arXiv:2604.15013, 2026

    Pith/arXiv arXiv 2026

  21. [21]

    Exo-viha: A cross-platform exoskeleton system with visual and haptic feedback for efficient dexterous skill learning

    Xintao Chao, Shilong Mu, Yushan Liu, Shoujie Li, Chuqiao Lyu, Xiao-Ping Zhang, and Wenbo Ding. Exo-viha: A cross-platform exoskeleton system with visual and haptic feedback for efficient dexterous skill learning. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 18732–18739. IEEE, 2025

    2025

  22. [22]

    Andy Park, Shenli Yuan, Yuke Zhu, and Luis Sentis

    Mingyo Seo, H. Andy Park, Shenli Yuan, Yuke Zhu, and Luis Sentis. Legato: Cross-embodiment imitation using a grasping tool.IEEE Robotics and Automation Letters (RA-L), 2025

    2025

  23. [23]

    Data scaling laws in imitation learning for robotic manipulation

    Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data scaling laws in imitation learning for robotic manipulation. InInternational Conference on Learning Representations, volume 2025, pages 54877–54910, 2025

    2025

  24. [24]

    Fastumi: A scalable and hardware-independent universal manipulation interface with dataset.arXiv preprint arXiv:2409.19499, 2024

    Kehui Liu, Chuyue Guan, Zhongjie Jia, Ziniu Wu, Xin Liu, Tianyu Wang, Shuai Liang, Pengan Chen, Pingrui Zhang, Haoming Song, et al. Fastumi: A scalable and hardware-independent universal manipulation interface with dataset.arXiv preprint arXiv:2409.19499, 2024

    arXiv 2024

  25. [25]

    Fastumi-100k: Advancing data-driven robotic manipulation with a large-scale umi-style dataset.arXiv preprint arXiv:2510.08022, 2025

    Kehui Liu, Zhongjie Jia, Yang Li, Pengan Chen, Song Liu, Xin Liu, Pingrui Zhang, Haoming Song, Xinyi Ye, Nieqing Cao, et al. Fastumi-100k: Advancing data-driven robotic manipulation with a large-scale umi-style dataset.arXiv preprint arXiv:2510.08022, 2025

    arXiv 2025

  26. [26]

    3d-vitac: Learning fine-grained manipulation with visuo-tactile sensing.arXiv preprint arXiv:2410.24091, 2024

    Binghao Huang, Yixuan Wang, Xinyi Yang, Yiyue Luo, and Yunzhu Li. 3d-vitac: Learning fine-grained manipulation with visuo-tactile sensing.arXiv preprint arXiv:2410.24091, 2024

    arXiv 2024

  27. [27]

    Touch in the wild: Learning fine-grained manipulation with a portable visuo-tactile gripper

    Xinyue Zhu, Binghao Huang, and Yunzhu Li. Touch in the wild: Learning fine-grained manipulation with a portable visuo-tactile gripper. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems,

  28. [28]

    URLhttps://openreview.net/forum?id=WabVVQKTUF

  29. [29]

    exumi: Extensible robot teaching system with action-aware task-agnostic tactile representation.arXiv preprint arXiv:2509.14688, 2025

    Yue Xu, Litao Wei, Pengyu An, Qingyu Zhang, and Yong-Lu Li. exumi: Extensible robot teaching system with action-aware task-agnostic tactile representation.arXiv preprint arXiv:2509.14688, 2025

    arXiv 2025

  30. [30]

    Tacumi: A multi-modal universal manipulation interface for contact-rich tasks

    Tailai Cheng, Kejia Chen, Lingyun Chen, Liding Zhang, Yue Zhang, Yao Ling, Mahdi Hamad, Zhenshan Bing, Fan Wu, Karan Sharma, et al. Tacumi: A multi-modal universal manipulation interface for contact-rich tasks. arXiv preprint arXiv:2601.14550, 2026

    arXiv 2026

  31. [31]

    Omniumi: Towards physically grounded robot learning via human-aligned multimodal interaction.arXiv preprint arXiv:2604.10647, 2026

    Shaqi Luo, Yuanyuan Li, Youhao Hu, Chenhao Yu, Chaoran Xu, Jiachen Zhang, Guocai Yao, Tiejun Huang, Ran He, and Zhongyuan Wang. Omniumi: Towards physically grounded robot learning via human-aligned multimodal interaction.arXiv preprint arXiv:2604.10647, 2026

    Pith/arXiv arXiv 2026

  32. [32]

    Hommi: Learning whole-body mobile manipulation from human demonstrations.arXiv preprint arXiv:2603.03243, 2026

    Xiaomeng Xu, Jisang Park, Han Zhang, Eric Cousineau, Aditya Bhat, Jose Barreiros, Dian Wang, and Shuran Song. Hommi: Learning whole-body mobile manipulation from human demonstrations.arXiv preprint arXiv:2603.03243, 2026

    Pith/arXiv arXiv 2026

  33. [33]

    Activeumi: Robotic manipulation with active perception from robot-free human demonstrations.arXiv preprint arXiv:2510.01607, 2025

    Qiyuan Zeng, Chengmeng Li, Jude St John, Zhongyi Zhou, Junjie Wen, Guorui Feng, Yichen Zhu, and Yi Xu. Activeumi: Robotic manipulation with active perception from robot-free human demonstrations.arXiv preprint arXiv:2510.01607, 2025

    arXiv 2025

  34. [34]

    Xrzero-g0: Pushing the frontier of dexterous robotic manipulation with interfaces, quality and ratios.arXiv preprint arXiv:2604.13001, 2026

    Junming Wang, Teng Pu, Wingmun Fung, Jindong Wang, Shanchang Wang, Yuan Deng, Shuyuan Wang, Ziwei Liu, Kunhao Pan, Ping Yang, et al. Xrzero-g0: Pushing the frontier of dexterous robotic manipulation with interfaces, quality and ratios.arXiv preprint arXiv:2604.13001, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    Vitamin: Learning contact-rich tasks through robot-free visuo-tactile manipulation interface.arXiv preprint arXiv:2504.06156, 2025

    Fangchen Liu, Chuanyu Li, Yihua Qin, Jing Xu, Pieter Abbeel, and Rui Chen. Vitamin: Learning contact-rich tasks through robot-free visuo-tactile manipulation interface.arXiv preprint arXiv:2504.06156, 2025

    arXiv 2025

  36. [36]

    Maniwav: Learning robot manipulation from in-the-wild audio-visual data.arXiv preprint arXiv:2406.19464, 2024

    Zeyi Liu, Cheng Chi, Eric Cousineau, Naveen Kuppuswamy, Benjamin Burchfiel, and Shuran Song. Maniwav: Learning robot manipulation from in-the-wild audio-visual data.arXiv preprint arXiv:2406.19464, 2024

    arXiv 2024

  37. [37]

    UMI on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers

    Huy Ha, Yihuai Gao, Zipeng Fu, Jie Tan, and Shuran Song. UMI on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers. InProceedings of the 2024 Conference on Robot Learning, 2024. 12

    2024

  38. [38]

    Bifrostumi: Bridging robot-free demonstrations and humanoid whole-body manipulation.arXiv preprint arXiv:2605.03452, 2026

    Chenhao Yu, Hongwu Wang, Youhao Hu, Jiachen Zhang, Yuanyuan Li, and Shaqi Luo. Bifrostumi: Bridging robot-free demonstrations and humanoid whole-body manipulation.arXiv preprint arXiv:2605.03452, 2026

    Pith/arXiv arXiv 2026

  39. [39]

    Mobile umi: Cross-view diffusion policy with decoupled kinematics for mobile manipulation.arXiv preprint arXiv:2605.20894, 2026

    Haoran Huang, Haonan Dong, and Huixu Dong. Mobile umi: Cross-view diffusion policy with decoupled kinematics for mobile manipulation.arXiv preprint arXiv:2605.20894, 2026

    Pith/arXiv arXiv 2026

  40. [40]

    Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning

    Runyu Ding, Yuzhe Qin, Jiyue Zhu, Chengzhe Jia, Shiqi Yang, Ruihan Yang, Xiaojuan Qi, and Xiaolong Wang. Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning. 2024. URL https://arxiv.org/abs/2407.03162

    arXiv 2024

  41. [41]

    Dexcap: Scalable and portable mocap data collection system for dexterous manipulation.arXiv preprint arXiv:2403.07788, 2024

    Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, and C Karen Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation.arXiv preprint arXiv:2403.07788, 2024

    arXiv 2024

  42. [42]

    Arcap: Collecting high-quality human demonstrations for robot learning with augmented reality feedback.arXiv preprint arXiv:2410.08464, 2024

    Sirui Chen, Chen Wang, Kaden Nguyen, Li Fei-Fei, and C Karen Liu. Arcap: Collecting high-quality human demonstrations for robot learning with augmented reality feedback.arXiv preprint arXiv:2410.08464, 2024

    arXiv 2024

  43. [43]

    Glovity: Learning dexterous contact-rich manipulation via spatial wrench feedback teleoperation system.arXiv preprint arXiv:2510.09229, 2025

    Yuyang Gao, Haofei Ma, and Pai Zheng. Glovity: Learning dexterous contact-rich manipulation via spatial wrench feedback teleoperation system.arXiv preprint arXiv:2510.09229, 2025

    arXiv 2025

  44. [44]

    A wearable robotic hand for hand-over-hand imitation learning

    Dehao Wei and Huazhe Xu. A wearable robotic hand for hand-over-hand imitation learning. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 18113–18119. IEEE, 2024

    2024

  45. [45]

    Doglove: Dexterous manipulation with a low-cost open-source haptic force feedback glove.arXiv preprint arXiv:2502.07730, 2025

    Han Zhang, Songbo Hu, Zhecheng Yuan, and Huazhe Xu. Doglove: Dexterous manipulation with a low-cost open-source haptic force feedback glove.arXiv preprint arXiv:2502.07730, 2025

    arXiv 2025

  46. [46]

    Cdf-glove: A cable-driven force feedback glove for dexterous teleoperation.arXiv preprint arXiv:2603.05804, 2026

    Huayue Liang, Ruochong Li, Yaodong Yang, Long Zeng, Yuanpei Chen, and Xueqian Wang. Cdf-glove: A cable-driven force feedback glove for dexterous teleoperation.arXiv preprint arXiv:2603.05804, 2026

    arXiv 2026

  47. [47]

    Eyesight hand: Design of a fully-actuated dexterous robot hand with integrated vision-based tactile sensors and compliant actuation

    Branden Romero, Hao-Shu Fang, Pulkit Agrawal, and Edward Adelson. Eyesight hand: Design of a fully-actuated dexterous robot hand with integrated vision-based tactile sensors and compliant actuation. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1853–1860. IEEE, 2024

    2024

  48. [48]

    Feel robot feels: Tactile feedback array glove for dexterous manipulation.arXiv preprint arXiv:2603.28542, 2026

    Feiyu Jia, Xiaojie Niu, Sizhe Yang, Qingwei Ben, Tao Huang, Jingbo Wang, Jiangmiao Pang, et al. Feel robot feels: Tactile feedback array glove for dexterous manipulation.arXiv preprint arXiv:2603.28542, 2026

    arXiv 2026

  49. [49]

    Mile: A mechanically isomorphic exoskeleton data collection system with fingertip visuotactile sensing for dexterous manipulation.arXiv preprint arXiv:2512.00324, 2025

    Jinda Du, Jieji Ren, Qiaojun Yu, Ningbin Zhang, Yu Deng, Xingyu Wei, Yufei Liu, Guoying Gu, and Xiangyang Zhu. Mile: A mechanically isomorphic exoskeleton data collection system with fingertip visuotactile sensing for dexterous manipulation.arXiv preprint arXiv:2512.00324, 2025

    Pith/arXiv arXiv 2025

  50. [50]

    Zilin Si, Jose Enrique Chen, M. Emre Karagozler, Antonia Bronars, Jonathan Hutchinson, Thomas Lampe, Nimrod Gileadi, Taylor Howell, Stefano Saliceti, Lukasz Barczyk, Ilan Olivarez Correa, Tom Erez, Mohit Shridhar, Murilo Fernandes Martins, Konstantinos Bousmalis, Nicolas Heess, Francesco Nori, and Maria Bauza. Exostart: Efficient learning for dexterous ma...

    arXiv 2025

  51. [51]

    Touchguide: Inference-time steering of visuomotor policies via touch guidance.arXiv preprint arXiv:2601.20239, 2026

    Zhemeng Zhang, Jiahua Ma, Xincheng Yang, Xin Wen, Yuzhi Zhang, Boyan Li, Yiran Qin, Jin Liu, Can Zhao, Li Kang, et al. Touchguide: Inference-time steering of visuomotor policies via touch guidance.arXiv preprint arXiv:2601.20239, 2026

    Pith/arXiv arXiv 2026

  52. [52]

    Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn

    Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. InICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems,

  53. [53]

    URLhttps://openreview.net/forum?id=e8Eu1lqLaf

  54. [54]

    Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 2024

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 2024

    2024

  55. [55]

    Using apple vision pro to train and control robots, 2024

    Younghyo Park and Pulkit Agrawal. Using apple vision pro to train and control robots, 2024. URLhttps: //github.com/Improbable-AI/VisionProTeleop

    2024

  56. [56]

    Analyzing key objectives in human-to-robot retargeting for dexterous manipulation.IEEE Robotics and Automation Practice, 2026

    Chendong Xin, Mingrui Yu, Yongpeng Jiang, Zhefeng Zhang, and Xiang Li. Analyzing key objectives in human-to-robot retargeting for dexterous manipulation.IEEE Robotics and Automation Practice, 2026. 13 Appendix A Glove Hardware and Embedded Interface This section provides implementation details of the palm-side teleoperation glove used for RealDexUMI data ...

    2026