Pith. sign in

REVIEW 4 major objections 7 minor 3 cited by

DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human References

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A data flywheel turns human references into a generalizable dexterous tracking controller that outperforms baselines by more than 10%.

desk verdict Solid simulation pipeline with a novel data flywheel and homotopy path generator, but the real-world evaluation is simulator-in-the-loop, so the robustness claims are thinner than advertised. read the letter →

arxiv 2502.09614 v1 pith:WWKC5WCG submitted 2025-02-13 cs.RO cs.AIcs.CVcs.LG

Xueyi Liu , Jianibieke Adalibieke , Qianwei Han , Yuzhe Qin , Li Yi This is my paper · ORCID
classification cs.ROcs.AIcs.CVcs.LG
keywords dexterousmanipulationtrackingcontrolimitationlearningreinforcementhomotopyoptimizationdataflywheelhumanreferencessim-to-realtransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a single neural controller can track a wide range of dexterous manipulation references—including novel objects, intricate in-hand re-orientations, and thin tools—if it is trained on large numbers of successful robot tracking demonstrations, and that such demonstrations can be generated automatically by the controller itself. DexTrack alternately trains the controller with a combined reinforcement-learning and imitation objective and uses that controller, through a learned homotopy path generator, to mine higher-quality and more diverse demonstrations. On two public hand-object interaction datasets, the method reports success rates more than 10 percentage points above the strongest prior baselines, both in simulation and on a real dexterous hand. A sympathetic reader would care because it attacks the bottleneck that limits human-reference dexterous manipulation: the absence of large, high-quality paired datasets of references and successful robot actions.

What carries the argument

The central mechanism is a data flywheel that alternates two stages. In the learning stage, a neural tracking policy is trained with a reward for matching reference hand and object states plus an imitation loss that pulls its actions toward expert action sequences from mined demonstrations; the policy acts in a residual action space, predicting a small delta added to a baseline hand trajectory, which keeps exploration near the reference and makes demonstrated actions easy to imitate. In the mining stage, the current controller supplies the baseline for per-trajectory RL trackers, and a learned homotopy path generator (a conditional diffusion model) proposes a chain of progressively easier tracking tasks, so that solving each task provides a better warm start for the next; the best tracked trajectories become new demonstrations. The homotopy path generator is what converts the controller's prior into demonstration diversity, and the residual action space is what makes both imitation and task-to-task transfer tractable.

What would settle it

Ablate the simulator-in-the-loop hand feedforward in the real-world setup: command the LEAP hand directly from the policy's output instead of first simulating the hand and using the simulated state as the target. If success rates on the apple and flute tasks fall to the level of the PPO baseline (e.g., below 25% at the highest threshold where DexTrack reports 25–50%), then the simulator's assumed fidelity, not the controller itself, is carrying the real-world result.

Watch

Extended reading notes

Core claim

On its own terms, the central claim is that the pairing of a bootstrapping data flywheel with a controller trained jointly by reinforcement learning and imitation learning yields a dexterous tracking policy that generalizes across objects and skills. The flywheel works because the controller's current policy supplies a 'tracking prior' that makes individual trajectory-optimization problems easier: a homotopy generator proposes intermediate reference motions of increasing difficulty, and solving those in sequence produces demonstrations of higher quality and diversity than optimizing each trajectory from scratch. The paper reports that this design raises success rates by over 10 percentage points over DGrasp, OmniGrasp-reward PPO, and PPO with the paper's own tracking reward on the GRAB and TACO benchmarks, and that the resulting controller transfers to a real hand for tasks like lifting a round apple and manipulating a thin flute.

Load-bearing premise

The claim that the controller works in the real world rests on the assumption that the simulator used for training and for generating the hand's commanded positions—where hand gravity is ignored—faithfully matches the real hand's contact and actuation dynamics; if the real hand cannot closely reach the simulated states, the reported real-world success would drop.

Editorial extensions

If this is right

  • Success rates rise by over 10 percentage points over the strongest baselines on both benchmarks, with the gain holding under two different success thresholds.
  • The controller generalizes to novel manipulation sequences—unseen object geometries, thin objects, and subtle finger-driven re-orientations—and tolerates large noise and unreachable states in the kinematic references.
  • Performance scales with the amount of high-quality demonstration data in the tested range and has not plateaued, so further data collection should continue improving the controller.
  • Per-trajectory tracking quality improves when the current controller's tracking prior is transferred into the baseline, and the homotopy generator's proposed paths solve previously unsolvable tracking problems, though its generalization to out-of-distribution tasks improves when training coverage is broadened.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the paper's own S3 results (new object categories and unseen interaction triplets) show a steep drop, which suggests the flywheel's ceiling is the homotopy generator's coverage; adding online adaptation or filtering references for reachability before deployment would likely recover much of that gap.
  • If demonstration quality, not just quantity, drives the gain, then a cheaper proxy—ranking references by predicted tracking error and mining only the hard ones—should reproduce most of the homotopy curve's benefit at lower compute.
  • The simulator-in-the-loop hand command strategy implies the controller's real-world transfer would be even more direct on a hand with high-bandwidth, low-latency position control; a testable prediction is that such a hand would not need the simulated hand state feedforward.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes DexTrack, a neural tracking controller for dexterous hand-object manipulation that takes retargeted kinematic human references as goals. The method alternates between training a generalizable controller with a combined reinforcement-learning and imitation-learning objective on a growing set of robot tracking demonstrations, and mining new demonstrations with a learned homotopy path generator that reuses the current controller as a tracking prior. The authors evaluate the approach in Isaac Gym on the GRAB and TACO datasets, report success-rate improvements of more than 10% over prior baselines, and include real-world experiments with a LEAP hand and Franka arm. The paper also provides ablations on demonstration quality, scaling behavior, and generalization of the homotopy path generator.

Significance. If the results hold, DexTrack would be a useful contribution to generalizable dexterous manipulation tracking: the bootstrap loop is a plausible way to scale demonstration data, the homotopy path generator is an interesting mechanism for turning hard per-trajectory optimization problems into easier curricula, and the simulation evaluation is broad, with held-out subjects, difficulty levels, and several baselines. The paper also honestly reports the homotopy generator's limited out-of-distribution generalization in Table 9 and the time cost of data collection. However, the real-world evaluation as described in Appendix C is not closed-loop with respect to real sensor feedback, and this directly affects the claim of real-world robustness. The self-referential nature of the data flywheel is another point that needs quantitative isolation before the central generalizability claim is fully supported.

major comments (4)
  1. [Appendix C (real-world setup), §4.2, Table 2] The real-world control loop described in Appendix C is not closed-loop with respect to the real hand or real object state. The policy's commands are applied to a simulated LEAP hand, the simulated state is read out, and the text states that 'in practice, we directly use the state obtained from the simulator as the positional target command fed to the real LEAP controller.' As written, the policy observes the simulated hand state; the real hand only tracks the simulated joint trajectories. This makes Table 2 a measure of how well the real LEAP hand can replay simulated commands, not a measure of the controller's robustness to real joint-state estimation errors, real object-pose noise, or unmodeled contact dynamics. The sentence in §4.2 claiming that the real-world evaluation assesses 'robustness of the state-based controller against noise in the state estimator' is therefore unsupported. Please clarify exactly which components of the observation in Eq. (8) are computed from real sensors in the real-world runs. If the observation is simulated, either run a genuinely closed-loop real-world evaluation using real joint encoders and real FoundationPose object poses, or reframe the real-world results as open-loop trajectory transfer and remove the state-estimator-robustness claim.
  2. [§3.3, §5, Appendix C (data filtering)] The data-flywheel loop is self-referential in a way that is not quantitatively isolated. The controller trained at each iteration is used both to mine new demonstrations (via the tracking prior and homotopy paths) and to filter them, since Appendix C states that 'only trajectories with a reward above the threshold' provide supervision and §3.3 samples trajectories weighted by the controller's tracking error. Consequently, part of the measured improvement across iterations could come from the controller selecting demonstrations that are already within its current competence, rather than from genuinely better tracking of a fixed distribution. The held-out test sets mitigate circularity, but the paper does not quantify this effect. Please add an ablation that trains on an equal-size demonstration set mined by the first-iteration controller only, or report the overlap and novelty of demonstrations added in each iteration, so that the flywheel itself, rather than simply the number of demonstrations, is shown to drive the gains in Table 1.
  3. [Table 2 and Appendix B.2] The real-world success rates are reported as percentages such as 25.0, 50.0, and 75.0 without trial counts, standard errors, or per-trial data. The values are quantized to multiples of 25%, which suggests very small sample sizes and makes the real-world comparison statistically fragile. Please report the number of trials per object and per success level, and provide per-trial outcomes or confidence intervals, so that the real-world component of the 'over 10% improvement' claim can be assessed.
  4. [Appendix C (simulation settings)] The statement 'The hand's gravity is ignored in the simulation' is a significant simplifying assumption for a contact-rich manipulation task. Because the real-world control strategy replays simulated hand states on the real LEAP hand, the effect of hand weight on required joint torques and on grasp stability is never exercised in training or in the real-world evaluation. Please either train with hand gravity enabled and report the effect on the Table 1 results, or provide a quantitative analysis—for example, torque margins on the LEAP hand or a comparison on a subset of trajectories with gravity enabled—to justify that ignoring hand gravity does not change the conclusions.
minor comments (7)
  1. [Eq. (12)] There is a parenthesis typo in the definition of ro,q: 'np.pi − Diff Angle(qo n − ˆqo n))' has an extra closing parenthesis.
  2. [Tables 13 and 14] The caption of Table 13 reads 'Generalization score (GRAB dataset)' while the table content concerns the adaptivity score; Table 14 repeats the same error. The captions should be corrected.
  3. [§4.2 heading] The heading contains a typo: 'DEXTEROOOUS MANIPULATION' should be 'DEXTEROUS MANIPULATION'.
  4. [Reproducibility] The paper repeatedly refers to 'code in the supplementary materials,' but the arXiv version and project website do not appear to provide the code or retargeted datasets. Please make the code and data publicly available or state clearly where the supplementary code can be found.
  5. [Section 5] The names 'Ours (w/o data)' and 'Ours (w/o data, w/o homotopy)' are confusing because both variants are trained on demonstrations; 'data' here appears to refer to the iteratively mined high-quality data. Please choose names that reflect the ablated component, such as 'w/o flywheel' or 'w/o data-quality curation'.
  6. [Appendix C, Eq. (22)] The smoothness measure in Eq. (22) is written as a difference of finite differences without a norm or absolute value, and the weight vector ws is multiplied by a state difference in a way that is not dimensionally transparent. Please rewrite the formula with an explicit norm and define the units.
  7. [Appendix C (datasets)] The sentence beginning 'we interpolate the phone pass sequence of the subject s2 from the GRAB dataset with such TACO sequences as the final modified sequence' is garbled and hard to parse; please rewrite it to describe the data-modification procedure clearly.

Circularity Check

1 steps flagged · score 4.0 of 10

Core held-out simulation evaluation is not circular, but the real-world robustness claim reduces to simulator-in-the-loop tracking by construction.

  1. other [Appendix C (Real world experiment setup) and Sec. 4.2 (Real-world evaluations and comparisons)]
    "Instead of directly applying the control signal to the LEAP hand and the Franka arm, we set up a simulator with physical and control-related parameters same as our simulation settings during training. Then, in each timestep, we first apply the control commands to the simulated LEAP hand... Thereby, in practice, we directly use the state obtained from the simulator as the positional target command fed to the real LEAP controller."

    Sec. 4.2 states that the real-world test evaluates 'the robustness of the state-based controller against noise in the state estimator.' However, Appendix C's control loop feeds the policy's commands to a simulated LEAP hand and then uses the simulated state as the real hand's command. The policy therefore observes simulated hand/object states rather than real joint encoders or FoundationPose object estimates, so real state-estimation noise is excluded from the loop by construction. The reported real-world success is thus a measure of the simulator-in-the-loop controller plus the real hand's low-level position tracking, not of robustness to real state-estimation noise. The claimed real-world robustness prediction reduces to the simulator's own tracking behavior by construction.

full rationale

The core method is a bootstrapped data flywheel: demonstrations are mined with the current controller and homotopy generator, then used to retrain the controller. This loop is self-referential but not definitionally circular, because the final simulation evaluation uses held-out trajectories (GRAB subject s1; TACO test sets) that are excluded from demonstration mining and training, and the success thresholds are fixed kinematic error criteria. The homotopy generator distills search results into a diffusion model; training and testing it with the same tracking-error comparator is supervised distillation, not a forced prediction. The only construction-level reduction found is the real-world robustness evaluation, which is closed around the simulator as described above. No load-bearing self-citation was found: the authors' TACO benchmark is paired with the independent GRAB dataset, and no uniqueness theorem or authority is imported to force the design. The real-world issue is therefore a partial circularity in one evaluation claim while the central simulation claims retain independent content.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The method relies on standard learning assumptions plus three domain assumptions: trackable retargeted references, faithful simulation, and convergence of combined RL+IL. Free parameters include reward weights, demonstration filtering threshold, and curriculum hyperparameters.

free parameters (4)
  • Reward weights (wo,p, wo,q, wwrist*wtrans, wwrist*wornt, wfinger) = 1.0, 0.33, 0.3, 0.05, 0.05
    Hand-set weights in Eq. 10 (Table 3) balance object position, orientation, wrist, finger, and affinity rewards; they affect the learned policy and the quality of mined demonstrations.
  • Demonstration filtering reward threshold = 50
    Appendix C states that only trajectories with reward above 50 are used to supervise the tracking controller, a manually chosen cut that controls dataset quality.
  • Homotopy path length K and neighbor count Knei = K=3, Knei=10
    Appendix A.3 sets these to balance time cost and effectiveness; they govern the depth of the curriculum and the search space for parent tasks.
  • PD gains (position gain and damping) = 20 and 1 per finger joint
    Section 4.1 fixes these for all joints; they determine how the simulated hand tracks the commanded joint positions.
assumptions (4)
  • domain assumption Retargeted kinematic references from human-object interactions are trackable by the Allegro and LEAP hands
    The controller is trained and evaluated on retargeted sequences produced by manual mesh correspondences and L-BFGS optimization (Eq. 5). The method assumes these references are physically realizable or at least trackable enough for the controller to succeed; the paper acknowledges noisy references but relies on them.
  • domain assumption Isaac Gym simulation with gravity ignored and PD control faithfully models real contact dynamics for sim-to-real transfer
    Section 4.1 and Appendix C: the policy is trained with gravity ignored and the real-world transfer uses a simulator-in-the-loop strategy that assumes the simulated LEAP hand nearly matches the real hand commands.
  • domain assumption PPO with residual action space and IL supervision converges to a policy that maximizes the tracking reward
    The RL objective (Eq. 1) and IL loss (Eq. 4) are optimized with standard PPO; the paper assumes this combined training yields a generalizable controller rather than overfitting.
  • ad hoc to paper Sampling training trajectories weighted by tracking error improves demonstration diversity and avoids self-confirmation bias
    Section 3.3: the data flywheel samples trajectories proportionally to the controller's tracking error. The paper assumes this prevents the bootstrapping loop from collapsing onto trajectories the controller already handles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human References." pith.science (2026). https://pith.science/paper/WWKC5WCG

@misc{pith2026250209614,
  author       = {Pith},
  title        = {Pith review of: DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human References},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WWKC5WCG}},
  note         = {Machine review of arXiv:2502.09614}
}
read the original abstract

We address the challenge of developing a generalizable neural tracking controller for dexterous manipulation from human references. This controller aims to manage a dexterous robot hand to manipulate diverse objects for various purposes defined by kinematic human-object interactions. Developing such a controller is complicated by the intricate contact dynamics of dexterous manipulation and the need for adaptivity, generalizability, and robustness. Current reinforcement learning and trajectory optimization methods often fall short due to their dependence on task-specific rewards or precise system models. We introduce an approach that curates large-scale successful robot tracking demonstrations, comprising pairs of human references and robot actions, to train a neural controller. Utilizing a data flywheel, we iteratively enhance the controller's performance, as well as the number and quality of successful tracking demonstrations. We exploit available tracking demonstrations and carefully integrate reinforcement learning and imitation learning to boost the controller's performance in dynamic environments. At the same time, to obtain high-quality tracking demonstrations, we individually optimize per-trajectory tracking by leveraging the learned tracking controller in a homotopy optimization method. The homotopy optimization, mimicking chain-of-thought, aids in solving challenging trajectory tracking problems to increase demonstration diversity. We showcase our success by training a generalizable neural controller and evaluating it in both simulation and real world. Our method achieves over a 10% improvement in success rates compared to leading baselines. The project website with animated results is available at https://meowuu7.github.io/DexTrack/.

Figures

Figures reproduced from arXiv: 2502.09614 by the authors.

Figure 1
Figure 1. DexTrack learns a generalizable neural tracking controller for dexterous manipulation from human references. It generates hand action commands from kinematic references, ensuring close tracking of input trajectories (Fig. (a)), generalizes to novel and challenging tasks involving thin objects, complex movements and intricate in-hand manipulations (Fig. (b)), and demonstrates robustness to large kinematics noise and … view at source ↗
Figure 2
Figure 2. DexTrack learns a generalizable neural tracking controller for dexterous manipulation from human references. It alternates between training the tracking controller using abundant and high-quality robot tracking demonstrations and improving the data via the tracking controller through a homotopy optimization scheme. achieve a state sequence {sn} N n=0 aligned with {ˆsn} N n=0. A “robust” controller can tolerate distu… view at source ↗
Figure 3
Figure 3. Robustness w.r.t. unreasonable states. Please check our website and video for animated results. We demonstrate the generalization ability and robustness of our tracking controller on unseen trajec￾tories involving challenging manipulations and novel, thin objects. Our controller has no difficulty in handling intricate motions, subtle in-hand re-orientations, and expressive functional manipulations, even when dealing… view at source ↗
Figures from the paper (12 more)
Figure 5
Figure 5. Figure 5: Scaling the amount of demonstrations. Scaling the number of demonstrations. To in￾vestigate the relationship between the tracking con￾troller’s performance and the number of demonstra￾tions, we vary the size of the demonstration dataset during training and tested perfo…
Figure 4
Figure 4. Figure 4: Qualitative comparisons. Please check our website and the accompanying video for animated results. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 6
Figure 6. Figure 6: DexTrack learns a generalizable neural tracking controller for dexterous manipulation from human references. It alternates between training the tracking controller using abundant and high-quality robot tracking demonstrations, and improving the data via the tracking co…
Figure 7
Figure 7. Figure 7: Robustness towards out-of-distribution objects and manipulations. Please refer to our website and the accompanying video for animated results. Further generalization ability evaluations on TACO dataset. We further evaluate the model’s generalization ability across vari…
Figure 8
Figure 8. Figure 8: Additional qualitative comparisons. Please refer to our website and the accompany￾ing video for animated results [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Additional real-world qualitative results. Please refer to our website and the accom￾panying video for animated results. Success thresholds. We define three levels of success rates. The first level of success is defined as reaching the object, finding a good grasp pose…
Figure 10
Figure 10. Figure 10: Failure cases in real-world experiments. Please refer to our website for animated results. Failure cases. A typical fail mode is that the object tends to drop from the hand as contact varies when attempting the in-hand manipulations, as shown in [PITH_FULL_IMAGE:figu…
Figure 11
Figure 11. Figure 11: Effectiveness of the homotopy optimization scheme. Please refer to our website and the accompanying video for animated results. We conduct further analysis of the proposed homotopy optimization scheme and the homotopy path generator to demonstrate their effectiveness.…
Figure 12
Figure 12. Figure 12: Failure Cases. Please refer to our website and the accompanying video for animated results. B.4 FAILURE CASES Our method may fail to perform well in some cases where the object is from a brand new category with challenging thin geometry, as demonstrated in [PITH_FULL…
Figure 13
Figure 13. Figure 13: Examples of novel objects from the seen object category (TACO). [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Examples of objects from new object categories (TACO). [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 15
Figure 15. Figure 15: Real-world experiment setup. Real world experiment setup. We use the Franka arm and LEAP hand to conduct real-world evaluations ( [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Human-level Dexterous Teleoperation

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A single-stage RL co-tracking controller trained on consecutive human-derived hand–object subgoals achieves ~75% real-robot success on long-horizon dexterous teleoperation where baselines fail.

  2. DexVLG: Dexterous Vision-Language-Grasp Model at Scale

    cs.CV 2025-07 conditional novelty 6.0 of 10

    DexVLG is a vision-language model trained on 170 million simulated dexterous grasps that generates hand poses aligned with language instructions about which part of an object to grasp.

  3. HERMES: Human-to-Robot Embodied Learning from Multi-Source Motion Data for Mobile Dexterous Manipulation

    cs.RO 2025-08 conditional novelty 5.0 of 10

    HERMES converts a single human motion demonstration into a deployable mobile bimanual dexterous manipulation policy, using RL, depth-image distillation, and closed-loop PnP pose refinement.

Reference graph

Works this paper leans on

47 extracted references · 19 canonical work pages · cited by 3 Pith papers

  1. [1]

    Solving rubik's cube with a robot hand

    Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. Solving rubik's cube with a robot hand. arXiv preprint arXiv:1910.07113, 2019

    arXiv 1910

  2. [2]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenhang Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, K. Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, ...

    arXiv 2023

  3. [3]

    Cimrl: Combining imitation and reinforcement learning for safe autonomous driving

    Jonathan Booher, Khashayar Rohanimanesh, Junhong Xu, Vladislav Isenbaev, Ashwin Balakrishna, Ishan Gupta, Wei Liu, and Aleksandr Petiushko. Cimrl: Combining imitation and reinforcement learning for safe autonomous driving. ArXiv, abs/2406.08878, 2024. URL https://api.semanticscholar.org/CorpusID:270440413

    arXiv 2024

  4. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Ma teusz Litwin, ...

    arXiv 2005

  5. [5]

    A system for general in-hand object re-orientation

    Tao Chen, Jie Xu, and Pulkit Agrawal. A system for general in-hand object re-orientation. Conference on Robot Learning, 2021

    2021

  6. [6]

    Visual dexterity: In-hand reorientation of novel and complex object shapes

    Tao Chen, Megha Tippur, Siyang Wu, Vikash Kumar, Edward Adelson, and Pulkit Agrawal. Visual dexterity: In-hand reorientation of novel and complex object shapes. Science Robotics, 8 0 (84): 0 eadc9244, 2023. doi:10.1126/scirobotics.adc9244. URL https://www.science.org/doi/abs/10.1126/scirobotics.adc9244

    doi:10.1126/scirobotics.adc9244 2023

  7. [7]

    Vividex: Learning vision-based dexterous manipulation from human videos

    Zerui Chen, Shizhe Chen, Cordelia Schmid, and Ivan Laptev. Vividex: Learning vision-based dexterous manipulation from human videos. ArXiv, abs/2404.15709, 2024. URL https://api.semanticscholar.org/CorpusID:269330215

    arXiv 2024

  8. [8]

    Gonzalez, and Ion Stoica

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. Chatbot arena: An open platform for evaluating llms by human preference. ArXiv, abs/2403.04132, 2024. URL https://api.semanticscholar.org/CorpusID:268264163

    arXiv 2024

Show all 47 references
  1. [9]

    D-grasp: Physically plausible dynamic grasp synthesis for hand-object interactions

    Sammy Christen, Muhammed Kocabas, Emre Aksan, Jemin Hwangbo, Jie Song, and Otmar Hilliges. D-grasp: Physically plausible dynamic grasp synthesis for hand-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 20577--20586, 2022

    2022

  2. [10]

    Learning dexterous manipulation for a soft robotic hand from human demonstrations

    Abhishek Gupta, Clemens Eppner, Sergey Levine, and Pieter Abbeel. Learning dexterous manipulation for a soft robotic hand from human demonstrations. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 3786--3793. IEEE, 2016

    2016

  3. [11]

    Agapiou, Joel Z

    Todd Hester, Matej Vecer \'i k, Olivier Pietquin, Marc Lanctot, Tom Schaul, Bilal Piot, Dan Horgan, John Quan, Andrew Sendonaris, Ian Osband, Gabriel Dulac-Arnold, John P. Agapiou, Joel Z. Leibo, and Audrunas Gruslys. Deep q-learning from demonstrations. In AAAI Conference on ...

    2017

  4. [12]

    Per-contact iteration method for solving contact dynamics

    Jemin Hwangbo, Joonho Lee, and Marco Hutter. Per-contact iteration method for solving contact dynamics. IEEE Robotics and Automation Letters, 3 0 (2): 0 895--902, 2018

    2018

  5. [13]

    Dtc: Deep tracking control

    Fabian Jenelten, Junzhe He, Farbod Farshidian, and Marco Hutter. Dtc: Deep tracking control. Science Robotics, 9, 2023. URL https://api.semanticscholar.org/CorpusID:263152143

    2023

  6. [14]

    Complementarity-free multi-contact modeling and optimization for dexterous manipulation

    Wanxin Jin. Complementarity-free multi-contact modeling and optimization for dexterous manipulation. 2024. URL https://api.semanticscholar.org/CorpusID:271874325

    2024

  7. [15]

    Dart: Noise injection for robust imitation learning

    Michael Laskey, Jonathan Lee, Roy Fox, Anca Dragan, and Ken Goldberg. Dart: Noise injection for robust imitation learning. In Conference on robot learning, pp.\ 143--156. PMLR, 2017

    2017

  8. [16]

    Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control

    Zhongyu Li, Xue Bin Peng, Pieter Abbeel, Sergey Levine, Glen Berseth, and Koushil Sreenath. Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control. ArXiv, abs/2401.16889, 2024. URL https://api.semanticscholar.org/CorpusID:267320454

    2024 arXiv

  9. [17]

    Herd: Continuous human-to-robot evolution for learning from human demonstration

    Xingyu Liu, Deepak Pathak, and Kris M Kitani. Herd: Continuous human-to-robot evolution for learning from human demonstration. arXiv preprint arXiv:2212.04359, 2022

    2022 arXiv

  10. [18]

    Stevens, Matthew R

    Xuefeng Liu, Takuma Yoneda, Rick L. Stevens, Matthew R. Walter, and Yuxin Chen. Blending imitation and reinforcement learning for robust policy improvement. ArXiv, abs/2310.01737, 2023. URL https://api.semanticscholar.org/CorpusID:263609068

    2023 arXiv

  11. [19]

    Quasisim: Parameterized quasi-physical simulators for dexterous manipulations transfer

    Xueyi Liu, Kangbo Lyu, Jieqiong Zhang, Tao Du, and Li Yi. Quasisim: Parameterized quasi-physical simulators for dexterous manipulations transfer. arXiv preprint arXiv:2404.07988, 2024 a

    2024 arXiv

  12. [20]

    Taco: Benchmarking generalizable bimanual tool-action-object understanding

    Yun Liu, Haolin Yang, Xu Si, Ling Liu, Zipeng Li, Yuxiang Zhang, Yebin Liu, and Li Yi. Taco: Benchmarking generalizable bimanual tool-action-object understanding. arXiv preprint arXiv:2401.08399, 2024 b

    2024 arXiv

  13. [21]

    Universal humanoid motion representations for physics-based control

    Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, and Weipeng Xu. Universal humanoid motion representations for physics-based control. ArXiv, abs/2310.04582, 2023 a . URL https://api.semanticscholar.org/CorpusID:263829555

    2023 arXiv

  14. [22]

    Winkler, Kris Kitani, and Weipeng Xu

    Zhengyi Luo, Jinkun Cao, Alexander W. Winkler, Kris Kitani, and Weipeng Xu. Perpetual humanoid control for real-time simulated avatars. In International Conference on Computer Vision (ICCV), 2023 b

    2023

  15. [23]

    Grasping diverse objects with simulated humanoids

    Zhengyi Luo, Jinkun Cao, Sammy Joe Christen, Alexander Winkler, Kris Kitani, and Weipeng Xu. Grasping diverse objects with simulated humanoids. ArXiv, abs/2407.11385, 2024. URL https://api.semanticscholar.org/CorpusID:271217823

    2024 arXiv

  16. [24]

    rl-games: A high-performance framework for reinforcement learning

    Denys Makoviichuk and Viktor Makoviychuk. rl-games: A high-performance framework for reinforcement learning. https://github.com/Denys88/rl_games, May 2021

    2021

  17. [25]

    Isaac gym: High performance gpu-based physics simulation for robot learning

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021

    2021 arXiv

  18. [26]

    Contact-invariant optimization for hand manipulation

    Igor Mordatch, Zoran Popovi \'c , and Emanuel Todorov. Contact-invariant optimization for hand manipulation. In Proceedings of the ACM SIGGRAPH/Eurographics symposium on computer animation, pp.\ 137--144, 2012

    2012

  19. [27]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. 2023. URL https://api.semanticscholar.org/CorpusID:257532815

    2023

  20. [28]

    A convex quasistatic time-stepping scheme for rigid multibody systems with contact and friction

    Tao Pang and Russ Tedrake. A convex quasistatic time-stepping scheme for rigid multibody systems with contact and friction. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 6614--6620. IEEE, 2021

    2021

  21. [29]

    Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models

    Tao Pang, HJ Terry Suh, Lujie Yang, and Russ Tedrake. Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models. IEEE Transactions on Robotics, 2023

    2023

  22. [30]

    Dexmv: Imitation learning for dexterous manipulation from human videos

    Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu, Hanwen Jiang, Ruihan Yang, Yang Fu, and Xiaolong Wang. Dexmv: Imitation learning for dexterous manipulation from human videos. In European Conference on Computer Vision, pp.\ 570--587. Springer, 2022

    2022

  23. [31]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations

    Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087, 2017

    2017 arXiv

  24. [32]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. ArXiv, abs/1707.06347, 2017. URL https://api.semanticscholar.org/CorpusID:28695052

    2017 arXiv

  25. [33]

    Leap hand: Low-cost, efficient, and anthropomorphic hand for robot learning

    Kenneth Shaw, Ananye Agarwal, and Deepak Pathak. Leap hand: Low-cost, efficient, and anthropomorphic hand for robot learning. ArXiv, abs/2309.06440, 2023. URL https://api.semanticscholar.org/CorpusID:259327055

    2023 arXiv

  26. [34]

    Hand-object interaction pretraining from videos

    Himanshu Gaurav Singh, Antonio Loquercio, Carmelo Sferrazza, Jane Wu, Haozhi Qi, Pieter Abbeel, and Jitendra Malik. Hand-object interaction pretraining from videos. 2024. URL https://api.semanticscholar.org/CorpusID:272600324

    2024

  27. [35]

    Andrew Bagnell, and Byron Boots

    Wen Sun, J. Andrew Bagnell, and Byron Boots. Truncated horizon policy search: Combining reinforcement learning & imitation learning. ArXiv, abs/1805.11240, 2018. URL https://api.semanticscholar.org/CorpusID:3533333

    2018 arXiv

  28. [36]

    Grab: A dataset of whole-body human grasping of objects

    Omid Taheri, Nima Ghorbani, Michael J Black, and Dimitrios Tzionas. Grab: A dataset of whole-body human grasping of objects. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part IV 16, pp.\ 581--600. Springer, 2020

    2020

  29. [37]

    Physhoi: Physics-based imitation of dynamic human-object interaction

    Yinhuai Wang, Jing Lin, Ailing Zeng, Zhengyi Luo, Jian Zhang, and Lei Zhang. Physhoi: Physics-based imitation of dynamic human-object interaction. arXiv preprint arXiv:2312.04393, 2023

    2023 arXiv

  30. [38]

    Xia, Quoc Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Huai hsin Chi, F. Xia, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. ArXiv, abs/2201.11903, 2022. URL https://api.semanticscholar.org/CorpusID:246411621

    2022 arXiv

  31. [39]

    Birchfield

    Bowen Wen, Wei Yang, Jan Kautz, and Stanley T. Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 17868--17879, 2023. URL https://api.semanticscholar.org/CorpusI...

    2024

  32. [40]

    Learning generalizable dexterous manipulation from human grasp affordance

    Yueh-Hua Wu, Jiashun Wang, and Xiaolong Wang. Learning generalizable dexterous manipulation from human grasp affordance. In Conference on Robot Learning, pp.\ 618--629. PMLR, 2023

    2023

  33. [41]

    Unidexgrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy

    Yinzhen Xu, Weikang Wan, Jialiang Zhang, Haoran Liu, Zikang Shan, Hao Shen, Ruicheng Wang, Haoran Geng, Yijia Weng, Jiayi Chen, et al. Unidexgrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy. arXiv preprint arXiv:2...

    2023 arXiv

  34. [42]

    Artigrasp: Physically plausible synthesis of bi-manual dexterous grasping and articulation

    Hui Zhang, Sammy Christen, Zicong Fan, Luocheng Zheng, Jemin Hwangbo, Jie Song, and Otmar Hilliges. Artigrasp: Physically plausible synthesis of bi-manual dexterous grasping and articulation. arXiv preprint arXiv:2309.03891, 2023

    2023 arXiv

  35. [43]

    PyTorch Kinematics , February 2024

    Sheng Zhong, Thomas Power, Ashwin Gupta, and Peter Mitrano. PyTorch Kinematics , February 2024

    2024

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  37. [45]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  38. [46]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  39. [47]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.