Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Dynamic Reconstruction of Hand-Object Interaction with Distributed Force-aware Contact Representation

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Tactile force readings sharpen hand-object reconstruction

desk verdict Useful deformable-object dataset, but the force-aware energy has a barrier bug that guts the penetration-prevention claim; the optimization gains are likely geometric, not physical. read the letter →

arxiv 2411.09572 v2 pith:ET5W75YF submitted 2024-11-14 cs.CV

Zhenjun Yu , Wenqiang Xu , Pengfei Xie , Yutong Li , Brian W. Anthony , Zhuorui Zhang , Cewu Lu This is my paper · ORCID
classification cs.CV
keywords hand-objectreconstructionvisual-tactilesensingdistributedtactilesensorsforce-awarecontactrepresentationdeformableobjectdynamictrackingsigneddistancefieldsoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that adding distributed tactile force readings to visual point-cloud tracking makes hand-object reconstruction more accurate, especially where vision is occluded. It introduces DF-Field, a contact representation that treats contact as a relative-potential energy between hand keypoints and object vertices plus a barrier energy that prevents penetration. On that basis the proposed ViTaM-D pipeline first tracks hand and object from depth point clouds with a learned contact constraint, then refines the hand pose by minimizing the DF-Field energy using measured region forces. Evaluations on the rigid-object DexYCB benchmark and a new deformable-object dataset (HOT) report improvements in overlap, Chamfer distance, hand-joint error, penetration depth, and contact-mask IoU over earlier methods. The paper also contributes the HOT dataset, 600 simulated hand-object sequences with tactile readings and deformation ground truth, for benchmarking deformable manipulation.

What carries the argument

The central object is DF-Field, a distributed force-aware contact representation. It defines, for each hand keypoint–object vertex pair $(i,j)$ at distance $l_{ij}$, a relative potential energy $E_{ij}=\kappa l_{ij}^2$ and an IPC-style barrier energy $B_{ij}$ that vanishes at distances above a threshold $\hat{l}$; the total energy $E=\sum_{i,j}(E_{ij}+B_{ij})$ is zero at proper contact and is minimized during hand-pose refinement. The force-awareness comes from dividing the 22 hand regions' average tactile readings by the pair distance to set $\kappa_{ij}\sim M_j/l_{ij}$, so stronger measured force produces stiffer, deeper contact. The visual stage is carried by a flow prediction module plus transformer fusion feeding SDF-based object decoding and MANO-based hand decoding, with a contact constraint $L_C$ that pins object SDF values to zero at predicted contact points.

What would settle it

Keep the visual stage fixed and run the force-aware optimization on the HOT dataset with the measured force maps replaced by random permutations across regions and by a constant force. If penetration depth and contact IoU barely change—or if the random maps do as well as the real ones—then the reported gains come from the energy barrier and pose regularization, not from the force readings.

Watch

Extended reading notes

Core claim

The paper's central claim is that a spatially distributed, force-aware contact representation—one that encodes both where the hand touches the object and how hard—resolves contact details that visual-only reconstruction misses. In ViTaM-D, the visual stage reconstructs hand and object geometry from point clouds under a contact constraint, and the force-aware stage optimizes the MANO hand pose against the reconstructed object by minimizing the sum of a relative potential energy $E_{ij}=\kappa l_{ij}^2$ and a barrier energy $B_{ij}$ over hand-region–object-vertex pairs. The stiffness $\kappa$ is set per region from the averaged tactile reading divided by the pair distance ($\kappa_{ij}\sim M_j/l_{ij}$), which is what makes the optimization force-aware. On DexYCB the authors report that the full method beats gSDF, HOTrack, and a depth-based tracker in object IoU and Chamfer distance; on HOT it beats ViTaM and, with force-aware optimization, substantially lowers penetration depth and raises contact-mask IoU (from 29.8 to 40.3). The reported comparisons position the tactile-derived force field, rather than the energy formulation alone, as the driver of these gains.

Load-bearing premise

The entire refinement gain rests on the assumption that a per-region average force reading can be converted into a contact stiffness by $\kappa_{ij}\sim M_j/l_{ij}$, so that minimizing the resulting energy gives a physically correct contact state.

Editorial extensions

If this is right

  • Visual-only hand-object trackers can be upgraded with a contact-refinement stage that uses force readings, without retraining the visual network.
  • On the HOT dataset, force-aware optimization raises contact-mask IoU from 29.8 to 40.3 and lowers penetration depth from 10.7 to 7.3 mm, indicating that measured forces restore contact details that vision misses.
  • Even a fixed, non-tactile force setting improves rigid-object results on DexYCB, suggesting the energy formulation itself regularizes contact.
  • The HOT dataset with 600 sequences and 30 deformable objects provides a benchmark for deformable hand-object reconstruction with tactile ground truth.
  • The two-stage design lets the visual tracker and the force-based optimizer be improved or replaced independently, so better visual trackers can be plugged in without redesigning the tactile refinement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the force-to-stiffness mapping is right, the same refinement could be applied to any hand pose tracker that outputs MANO parameters, making tactile refinement a drop-in post-processing step rather than part of the learned model.
  • The fixed-force ablation hints that even crude force priors may help, so a learned mapping from tactile arrays to stiffness could replace the hand-designed division by distance.
  • The FEM-based HOT dataset could support a stronger test: train a contact model to predict object deformation from forces and check whether DF-Field's optimized poses reduce simulation-to-ground-truth mismatch, not just mask IoU.
  • A direct comparison of optimized poses against independently measured contact forces would show whether the energy minimum actually corresponds to the physical contact state, separate from reconstruction-quality metrics.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes ViTaM-D, a two-stage visual-tactile framework for dynamic hand-object reconstruction. A visual dynamic tracking network predicts MANO hand parameters and object SDFs from point cloud sequences under a contact constraint, and a second stage optimizes the hand pose against a proposed DF-Field energy that combines a relative potential energy and a barrier term. The per-region stiffness in this energy is derived from distributed tactile readings. Experiments are reported on DexYCB (rigid) and on a new simulated HOT dataset (deformable objects), with ablations on contact sources and force representations, plus a single real-world example. The central claims are that DF-Field improves contact modeling and penetration resolution and that ViTaM-D outperforms gSDF, HOTrack, ViTaM, CPF, and TOCH.

Significance. The paper addresses an important problem and provides useful artifacts: the HOT dataset (600 simulated sequences with FEM deformation, tactile readings, and 8 views) is a potentially valuable benchmark, and the visual tracking network alone is competitive on DexYCB object metrics. The authors also commit to releasing code, models, and data. However, the central DF-Field contact model, as written, has a mathematical inconsistency: substituting Eq. (4) into Eq. (2) makes the barrier vanish at contact, so the optimization does not prevent penetration. In addition, the force-aware gains are evaluated on metrics that are directly aligned with the optimized energy. If the energy model and evaluation are repaired, the framework could be significant; in its current form the force-awareness claim is not established.

major comments (4)
  1. [Sec. 3.2, Eq. (4) vs. Eq. (2)] Substituting kappa_ij ~ M_j/l_ij into the barrier B_ij makes the barrier tend to zero as l_ij approaches 0 rather than diverge: for fixed M_j>0, B_ij is approximately -exp(-M_j * hat_l^2 / l_ij) * log(l_ij/hat_l), which tends to 0 because the exponential dominates the logarithm. The standard IPC barrier's penetration resistance is therefore lost, so the optimization in Sec. 4.3 does not, on the basis of the stated energy, prevent penetration. The remaining term E_ij = M_j * l_ij is a constant-force attractive potential, so the reported PD and CIoU improvements can be produced by attraction to the object surface alone. This undermines the claim in Sec. 6.4 that DF-Field 'solves most of the penetration problems'.
  2. [Sec. 3.2, Eq. (4)] The identification kappa_ij ~ M_j/l_ij is not physically justified. Dividing a scalar force reading by the current distance makes kappa distance-dependent, and the resulting potential E_ij = M_j * l_ij is linear in l, not a spring potential; the text nonetheless calls kappa a stiffness or interaction parameter. The mapping from averaged regional tactile readings to point-pair stiffness is an ad-hoc free parameter and is not validated against measured force/displacement behavior. As written, the tactile magnitude only rescales an attractive geometry prior, so the 'force-aware' characterization is not demonstrated.
  3. [Sec. 6.4, Table 3 and Sec. 4.3] The PD and CIoU metrics are directly entangled with the optimization objective. The energy in Eq. (3) is minimized by reducing distances between hand keypoints and object vertices within the ball-query radius; PD measures penetration depth and CIoU uses a 3 mm distance threshold. The Table 3 improvements are therefore partly a restatement of the objective, not independent evidence of contact accuracy. The asterisked object metrics in Table 1 mean that the force-optimized rows have no object IoU/CD values, so the only non-distance-based metrics are absent for the force-optimized model. Please report object-space metrics with the force-optimized hand or provide independent contact/force accuracy measures.
  4. [Sec. 6.4 and Sec. 6.6] No error bars or multiple seeds are reported, and the headline differences in Table 1 are modest (for example, DexYCB IoU 90.1 vs 88.2 and MPJPE 13.2 vs 14.4). The real-world validation is a single qualitative reconstruction in Fig. 6 with no quantitative evaluation. These omissions make it difficult to assess the significance of the comparisons. Please add repeated-run statistics and a quantitative real-world evaluation, or explicitly scope the claims to the simulated setting.
minor comments (5)
  1. [Table 1] The asterisk footnote explains why object metrics are omitted for the refinement rows, but no object-space values are given; please provide at least the object IoU and CD values for the force-optimized rows in the supplementary material.
  2. [Eq. (2)] Equation (2) has a notational issue: the condition '0<l_ij < hat_l' and the formula's 'l_ij hat_l' should be cleaned up so the intended multiplication is explicit.
  3. [Sec. 4.2] The contact constraint L_C in Eq. (7) penalizes only the SDF values at contact points; please clarify whether non-contact sample positions are constrained by other terms in L_SDF or left unconstrained.
  4. [General] The paper repeatedly refers to the supplementary for point-pair establishment, flow prediction, keypoint-versus-all-vertices optimization, and tactile array integration; these details should be summarized in the main text for reproducibility.
  5. [Figure 5] Figure 5 qualitatively shows progress at 50 and 100 optimization steps, but no quantitative per-iteration curve is provided; please add one to support the claim that the optimization converges to a better contact state.

Circularity Check

1 steps flagged · score 6.0 of 10

Reported PD and CIoU gains for force-aware optimization are partially by construction, as the optimized energy is defined over the same point-pair distances as these metrics; IoU/CD and MPJPE remain independent.

  1. self definitional [Sec. 6.4 Results, Sec. 4.3 Eq. (10), Sec. 6.3 metrics]
    "On the HOT dataset, our visual dynamic tracking net matches ViTaM's performance, while force-based optimization significantly improves MPJPE, PD, and CIoU, confirming its effectiveness."

    The optimization in Eq. (10) minimizes E = sum_i sum_j (E_ij + B_ij), with E_ij = kappa l_ij^2 (Eq. 1) and B_ij defined for l_ij < l_hat = 2 mm (Eq. 2), using point pairs selected by ball-query radius R = 5 mm. PD is the maximum hand penetration into the object and CIoU is a contact mask at distance < 3 mm, both direct functions of the same l_ij that the energy drives toward zero. Thus reporting PD/CIoU gains after minimization is reporting that the optimizer moved hand keypoints toward its own objective's attractor, not an independent validation of force-aware contact modeling. With kappa_ij ~ M_j / l_ij (Eq. 4), the tactile magnitude only rescales this geometric attraction, so the claimed confirmation is partially by construction.

full rationale

The visual tracking stage is evaluated on independent metrics: object IoU and Chamfer distance improve on DexYCB and HOT without force optimization, and the network is trained with standard losses (Eq. 8). MPJPE is partly protected by the L_o and L_r regularizers, so it is not a direct minimization target. The circular core is the force-aware refinement evaluation: PD and CIoU are defined on point-pair distances (penetration, <3 mm contact mask) that the optimized energy E explicitly minimizes, with pairs selected within 5 mm and a barrier threshold of 2 mm. Eq. (4) makes kappa = M_j / l_ij, so the tactile readings only rescale this distance pull, meaning the PD/CIoU improvements largely restate the optimization objective rather than validate a physical force model. This is partial circularity, not full circularity, because object IoU, CD, and to a degree MPJPE supply independent signal. The self-citations (ViTaM [15], ZeMa [8]) are not load-bearing theorems; the barrier is attributed to IPC [17]. Separately, substituting Eq. (4) into Eq. (2) makes the barrier decay to zero as l approaches 0, so the claim that DF-Field prevents penetration is a correctness risk rather than a circularity, and is not counted beyond the entanglement above.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework is not parameter-free: kappa turns tactile readings into a compliance coefficient, several distance thresholds are hand-selected, and the simulation is trusted as ground truth. No new physical entities are invented.

free parameters (4)
  • kappa (per-point-pair stiffness) = M_j / l_ij (tactile force divided by distance); M_j=0.5 in fixed-force ablations
    Eqs. 1 and 4 define the contact energy through kappa, but kappa is estimated from tactile readings or set empirically, not derived from first principles; the central force-aware optimization depends on it.
  • ball-query radius R = 5 mm
    Sec 6.1 sets the radius for pairing object vertices to hand region keypoints; this choice determines which contacts contribute to the energy.
  • barrier threshold l_hat = 2 mm
    Sec 6.1 sets the threshold below which the barrier energy activates, directly controlling the penetration penalty strength.
  • contact distance threshold l_c = 3 mm
    Sec 4.2 and Sec 6.3 define contact states for both the contact constraint and the CIoU evaluation metric, so the same threshold affects training and reported contact accuracy.
assumptions (4)
  • domain assumption The IPC barrier energy of Li et al. [17] is a valid and sufficient contact model for hand-object optimization.
    Sec 3.1 Eq. 2 adopts the barrier function unchanged to prevent penetration.
  • domain assumption MANO provides an adequate parametric hand model whose pose parameters can be optimized against contact energies.
    Sec 4.2 uses MANO for hand mesh reconstruction and Sec 4.3 optimizes its pose theta.
  • domain assumption ZeMa simulation with FEM and frictional contact generates ground-truth deformations and tactile readings that transfer to real hand-object reconstruction.
    The HOT dataset and the real-world experiment in Sec 6.6 assume simulation-to-real transfer without quantitative validation.
  • ad hoc to paper Averaging tactile readings over 22 hand regions and dividing by distance gives a usable local stiffness kappa for the energy.
    Eq. 4 is asserted without physical or empirical justification beyond matching energy units.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dynamic Reconstruction of Hand-Object Interaction with Distributed Force-aware Contact Representation." pith.science (2026). https://pith.science/paper/ET5W75YF

@misc{pith2026241109572,
  author       = {Pith},
  title        = {Pith review of: Dynamic Reconstruction of Hand-Object Interaction with Distributed Force-aware Contact Representation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ET5W75YF}},
  note         = {Machine review of arXiv:2411.09572}
}
read the original abstract

We present ViTaM-D, a novel visual-tactile framework for reconstructing dynamic hand-object interaction with distributed tactile sensing to enhance contact modeling. Existing methods, relying solely on visual inputs, often fail to capture occluded interactions and object deformation. To address this, we introduce DF-Field, a distributed force-aware contact representation leveraging kinetic and potential energy in hand-object interactions. ViTaM-D first reconstructs interactions using a visual network with contact constraint, then refines contact details through force-aware optimization, improving object deformation modeling. To evaluate deformable object reconstruction, we introduce the HOT dataset, featuring 600 hand-object interaction sequences in a high-precision simulation environment. Experiments on DexYCB and HOT datasets show that ViTaM-D outperforms state-of-the-art methods in reconstruction accuracy for both rigid and deformable objects. DF-Field also proves more effective in refining hand poses and enhancing contact modeling than previous refinement methods. The code, models, and datasets are available at https://sites.google.com/view/vitam-d/.

Figures

Figures reproduced from arXiv: 2411.09572 by the authors.

Figure 1
Figure 1. (a): The relationship between tactile information and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a): The 22 regions and a typical distributed tactile sensor [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. ViTaM-D pipeline. (a): The overview of our pipeline, including the Flow Prediction module for visual feature extraction and flow estimation, Hand and Object Decoders, the Contact Constraint, and Force-aware hand-pose Optimization. (b): The Object Decoder to reconstruct the object mesh. (c): The Hand Decoder for estimating hand parameters and reconstructing based on the MANO model. and network prediction for acquirin… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative results on both DexYCB and HOT datasets. The upper shows our better performances on rigid objects compared to [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results with force-aware optimization. As iteration steps rise, the penetration problem decreases, and the contact map [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Real-world reconstruction example of an unseen stuffed [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation

    cs.RO 2025-04 conditional novelty 5.0 of 10

    Taccel combines Incremental Potential Contact and Affine Body Dynamics on GPUs to run parallel vision-based tactile simulations, with measured sim-to-real transfer in object classification and articulated manipulation.

  2. Detecting Reading-Induced Confusion Using EEG and Eye Tracking

    cs.HC 2025-08 unverdicted novelty 4.0 of 10

    Multimodal EEG plus eye tracking classifies reading-induced confusion at 77.3% average weighted accuracy, beating unimodal models by 4-22%, in an 11-participant study.

Reference graph

Works this paper leans on

43 extracted references · 40 canonical work pages · cited by 2 Pith papers

  1. [1]

    The ycb object and model set: Towards common benchmarks for manipula- tion research

    Berk Calli, Arjun Singh, Aaron Walsman, Siddhartha Srini- vasa, Pieter Abbeel, and Aaron M Dollar. The ycb object and model set: Towards common benchmarks for manipula- tion research. In2015 international conference on advanced robotics (ICAR), pages 510–517. IEEE, 2015. 6

    work page 2015

  2. [2]

    Reconstructing hand-object interactions in the wild

    Zhe Cao, Ilija Radosavovic, Angjoo Kanazawa, and Jitendra Malik. Reconstructing hand-object interactions in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12417–12426, 2021. 2

    work page 2021

  3. [3]

    Dexycb: A benchmark for capturing hand grasping of objects

    Yu-WeiChao,WeiYang,YuXiang,PavloMolchanov,Ankur Handa,JonathanTremblay,YashrajSNarang,KarlVanWyk, Umar Iqbal, Stan Birchfield, et al. Dexycb: A benchmark for capturing hand grasping of objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9044–9053, 2021. 2, 5

    work page 2021

  4. [4]

    Tracking and reconstructing hand object interactions from point cloud sequencesinthewild.In ProceedingsoftheAAAIConference on Artificial Intelligence, pages 304–312, 2023

    Jiayi Chen, Mi Yan, Jiazhao Zhang, Yinzhen Xu, Xiaolong Li, YijiaWeng, LiYi, ShuranSong, andHeWang. Tracking and reconstructing hand object interactions from point cloud sequencesinthewild.In ProceedingsoftheAAAIConference on Artificial Intelligence, pages 304–312, 2023. 1, 2, 6

    work page 2023

  5. [5]

    Alignsdf: Pose-alignedsigneddistancefieldsforhand-object reconstruction.In EuropeanConferenceonComputerVision , pages 231–248

    ZeruiChen,YanaHasson,CordeliaSchmid,andIvanLaptev. Alignsdf: Pose-alignedsigneddistancefieldsforhand-object reconstruction.In EuropeanConferenceonComputerVision , pages 231–248. Springer, 2022. 2

    work page 2022

  6. [6]

    gsdf: Geometry-drivensigneddistancefunctionsfor3dhand- objectreconstruction

    ZeruiChen,ShizheChen,CordeliaSchmid,andIvanLaptev. gsdf: Geometry-drivensigneddistancefunctionsfor3dhand- objectreconstruction. InProceedingsoftheIEEE/CVFCon- ference on Computer Vision and Pattern Recognition, pages 12890–12900, 2023. 2, 6

    work page 2023

  7. [7]

    Hope-net: A graph-based model for hand-object poseestimation

    Bardia Doosti, Shujon Naha, Majid Mirbagheri, and David J Crandall. Hope-net: A graph-based model for hand-object poseestimation. InProceedingsoftheIEEE/CVFconference on computer vision and pattern recognition, pages 6608– 6617, 2020. 2

    work page 2020

  8. [8]

    Intersection-freerobotmanipulation with soft-rigid coupled incremental potential contact.IEEE Robotics and Automation Letters, 2024

    Wenxin Du, Siqiong Yao, Xinlei Wang, Yuhang Xu, Wen- qiangXu,andCewuLu. Intersection-freerobotmanipulation with soft-rigid coupled incremental potential contact.IEEE Robotics and Automation Letters, 2024. 2, 5

    work page 2024

Show all 43 references
  1. [9]

    Arctic: A dataset for dexterous bimanual hand- object manipulation

    Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J Black, and Otmar Hilliges. Arctic: A dataset for dexterous bimanual hand- object manipulation. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages ...

    2023

  2. [10]

    Hold: Category-agnostic 3d reconstruction of in- teracting hands and objects from video

    Zicong Fan, Maria Parelli, Maria Eleni Kadoglou, Xu Chen, Muhammed Kocabas, Michael J Black, and Otmar Hilliges. Hold: Category-agnostic 3d reconstruction of in- teracting hands and objects from video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  3. [11]

    In ProceedingsoftheIEEE/CVFconferenceoncomputervision and pattern recognition, pages 11807–11816, 2019

    YanaHasson,GulVarol,DimitriosTzionas,IgorKalevatykh, MichaelJBlack,IvanLaptev,andCordeliaSchmid.Learning joint reconstruction of hands and manipulated objects. In ProceedingsoftheIEEE/CVFconferenceoncomputervision and pattern recognition, pages 11807–11816, 2019. 2

    2019

  4. [12]

    In ProceedingsoftheIEEE/CVFconferenceoncomputervision and pattern recognition, pages 11807–11816, 2019

    YanaHasson,GulVarol,DimitriosTzionas,IgorKalevatykh, MichaelJBlack,IvanLaptev,andCordeliaSchmid.Learning joint reconstruction of hands and manipulated objects. In ProceedingsoftheIEEE/CVFconferenceoncomputervision and pattern recognition, pages 11807–11816, 2019. 1, 2

    2019

  5. [13]

    Leveraging photometric consistency over time for sparsely supervised hand-object reconstruction

    YanaHasson,BugraTekin,FedericaBogo,IvanLaptev,Marc Pollefeys, and Cordelia Schmid. Leveraging photometric consistency over time for sparsely supervised hand-object reconstruction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 571–580,

  6. [14]

    Pvn3d: A deep point-wise 3d keypoints voting network for 6dof pose estimation

    Yisheng He, Wei Sun, Haibin Huang, Jianran Liu, Haoqiang Fan, and Jian Sun. Pvn3d: A deep point-wise 3d keypoints voting network for 6dof pose estimation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11632–11641, 2020. 4

    2020

  7. [15]

    Capturing forceful interac- tion with deformable objects using a deep learning-powered stretchable tactile array

    Chunpeng Jiang, Wenqiang Xu, Yutong Li, Zhenjun Yu, Longchun Wang, Xiaotong Hu, Zhengyi Xie, Qingkun Liu, Bin Yang, Xiaolin Wang, et al. Capturing forceful interac- tion with deformable objects using a deep learning-powered stretchable tactile array. Nature Communications, 15(...

    2024

  8. [16]

    Digit: Anoveldesignforalow-costcompacthigh-resolution tactilesensorwithapplicationtoin-handmanipulation

    Mike Lambeta, Po-Wei Chou, Stephen Tian, Brian Yang, Benjamin Maloon, Victoria Rose Most, Dave Stroud, Ray- mond Santos, Ahmad Byagowi, Gregg Kammerer, et al. Digit: Anoveldesignforalow-costcompacthigh-resolution tactilesensorwithapplicationtoin-handmanipulation. IEEE Robotics...

    2020

  9. [17]

    Minchen Li, Zachary Ferguson, Teseo Schneider, Timothy Langlois, Denis Zorin, Daniele Panozzo, Chenfanfu Jiang, and Danny M. Kaufman. Incremental potential contact: Intersection-andinversion-freelargedeformationdynamics. ACM Trans. Graph. (SIGGRAPH), 39(4), 2020. 3

    2020

  10. [18]

    A glove-based system for studying hand- objectmanipulationviajointposeandforcesensing

    Hangxin Liu, Xu Xie, Matt Millar, Mark Edmonds, Feng Gao, Yixin Zhu, Veronica J Santos, Brandon Rothrock, and Song-Chun Zhu. A glove-based system for studying hand- objectmanipulationviajointposeandforcesensing. In 2017 IEEE/RSJInternationalConferenceonIntelligentRobotsand Sys...

    2017

  11. [19]

    Marching cubes: A high resolution 3d surface construction algorithm.ACM TOG, 21(4):163–169, 1987

    William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm.ACM TOG, 21(4):163–169, 1987. 4

    1987

  12. [20]

    Graspit! a versatile simulatorforroboticgrasping

    Andrew T Miller and Peter K Allen. Graspit! a versatile simulatorforroboticgrasping. IEEERobotics&Automation Magazine, 11(4):110–122, 2004. 2

    2004

  13. [21]

    Convolutional occupancy networks

    Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. Convolutional occupancy networks. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part III 16, pages 523–540. Springer, 2020. 4

    2020

  14. [22]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017. 4

    2017

  15. [23]

    Mc-Tac: Modular camera-based tactile sensor for robot gripper

    Jieji Ren, Jiang Zou, and Guoying Gu. Mc-Tac: Modular camera-based tactile sensor for robot gripper. InThe 16th International Conference on Intelligent Robotics and Appli- cations (ICIRA), 2023. 1

    2023

  16. [24]

    Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: Modeling and capturing hands and bodies together.ACMTransactionsonGraphics,(Proc.SIGGRAPH Asia), 2017. 4

    2017

  17. [25]

    3d shape reconstruction from vision and touch.Advances in Neural Information Processing Systems, 33:14193–14206,

    Edward Smith, Roberto Calandra, Adriana Romero, Georgia Gkioxari,DavidMeger,JitendraMalik,andMichalDrozdzal. 3d shape reconstruction from vision and touch.Advances in Neural Information Processing Systems, 33:14193–14206,

  18. [26]

    Learning the signatures of the human grasp using a scalable tactile glove

    SubramanianSundaram,PetrKellnhofer,YunzhuLi,Jun-Yan Zhu, Antonio Torralba, and Wojciech Matusik. Learning the signatures of the human grasp using a scalable tactile glove. Nature, 569(7758):698–702, 2019. 1, 3

    2019

  19. [27]

    Gelslim 3.0: High-resolutionmeasurementofshape,forceandslipin a compact tactile-sensing finger

    Ian H Taylor, Siyuan Dong, and Alberto Rodriguez. Gelslim 3.0: High-resolutionmeasurementofshape,forceandslipin a compact tactile-sensing finger. In2022 International Con- ference on Robotics and Automation (ICRA), pages 10781– 10787. IEEE, 2022. 1

    2022

  20. [28]

    H+ o: Unified egocentric recognition of 3d hand-object poses and interactions

    Bugra Tekin, Federica Bogo, and Marc Pollefeys. H+ o: Unified egocentric recognition of 3d hand-object poses and interactions. InProceedingsoftheIEEE/CVFconferenceon computer vision and pattern recognition, pages 4511–4520,

  21. [29]

    Joint3dtrackingofa deformableobjectininteractionwithahand

    AggelikiTsoliandAntonisAArgyros. Joint3dtrackingofa deformableobjectininteractionwithahand. In Proceedings of the European Conference on Computer Vision (ECCV), pages 484–500, 2018. 1, 2

    2018

  22. [30]

    3dshapeperceptionfrommonocularvision,touch, andshapepriors.In 2018IEEE/RSJInternationalConference onIntelligentRobotsandSystems(IROS) ,pages1606–1613

    ShaoxiongWang,JiajunWu,XingyuanSun,WenzhenYuan, William T Freeman, Joshua B Tenenbaum, and Edward H Adelson. 3dshapeperceptionfrommonocularvision,touch, andshapepriors.In 2018IEEE/RSJInternationalConference onIntelligentRobotsandSystems(IROS) ,pages1606–1613. IEEE, 2018. 1, 3

    2018

  23. [31]

    Visual-tactile sensing for in-hand object reconstruction

    Wenqiang Xu, Zhenjun Yu, Han Xue, Ruolin Ye, Siqiong Yao, and Cewu Lu. Visual-tactile sensing for in-hand object reconstruction. In CVPR IEEE Conference on Computer Vision and Pattern Recognition, pages 8803–8812, 2023. 1, 3

    2023

  24. [32]

    Dipgrasp: Parallel local searching for efficient differentiable grasp planning

    WenqiangXu,JieyiZhang,TutianTang,ZhenjunYu,Yutong Li, and Cewu Lu. Dipgrasp: Parallel local searching for efficient differentiable grasp planning. IEEE Robotics and Automation Letters, 2024. 5

    2024

  25. [33]

    Cpf: Learning a contact potential field to model the hand-object interaction

    Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, and Cewu Lu. Cpf: Learning a contact potential field to model the hand-object interaction. InICCV IEEE/CVF InternationalConferenceonComputerVision ,pages11097– 11106, 2021. 1, 2, 6

    2021

  26. [34]

    Artiboost: Boosting articulated 3d hand-object pose estimation via online exploration and synthesis

    Lixin Yang, Kailin Li, Xinyu Zhan, Jun Lv, Wenqiang Xu, Jiefeng Li, and Cewu Lu. Artiboost: Boosting articulated 3d hand-object pose estimation via online exploration and synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2750–2760,

  27. [35]

    Oakink: A large-scale knowledge repos- itory for understanding hand-object interaction

    Lixin Yang, Kailin Li, Xinyu Zhan, Fei Wu, Anran Xu, Liu Liu, and Cewu Lu. Oakink: A large-scale knowledge repos- itory for understanding hand-object interaction. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20953–20962, 2022. 2, 5

    2022

  28. [36]

    Learningacontactpotentialfieldformodeling the hand-object interaction

    LixinYang,XinyuZhan,KailinLi,WenqiangXu,JiefengLi, andCewuLu. Learningacontactpotentialfieldformodeling the hand-object interaction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 1, 2

    2024

  29. [37]

    Diffusion-guided reconstruction of everyday hand- object interaction clips

    YufeiYe, PoorviHebbar, AbhinavGupta, andShubhamTul- siani. Diffusion-guided reconstruction of everyday hand- object interaction clips. In Proceedings of the IEEE/CVF InternationalConferenceonComputerVision ,pages19717– 19728, 2023. 2

    2023

  30. [38]

    Rotating without seeing: To- wards in-hand dexterity through touch

    Zhao-Heng Yin, Binghao Huang, Yuzhe Qin, Qifeng Chen, and Xiaolong Wang. Rotating without seeing: To- wards in-hand dexterity through touch. arXiv preprint arXiv:2303.10880, 2023. 1

    2023 arXiv

  31. [39]

    Gel- sight: High-resolution robot tactile sensors for estimating geometry and force.Sensors, 17(12):2762, 2017

    Wenzhen Yuan, Siyuan Dong, and Edward H Adelson. Gel- sight: High-resolution robot tactile sensors for estimating geometry and force.Sensors, 17(12):2762, 2017. 1

    2017

  32. [40]

    Single depth view based real-time reconstruction of hand-object interactions.ACM Transactions on Graphics (TOG), 40(3):1–12, 2021

    Hao Zhang, Yuxiao Zhou, Yifei Tian, Jun-Hai Yong, and Feng Xu. Single depth view based real-time reconstruction of hand-object interactions.ACM Transactions on Graphics (TOG), 40(3):1–12, 2021. 6

    2021

  33. [41]

    Dynamic modeling of hand- object interactions via tactile sensing

    Qiang Zhang, Yunzhu Li, Yiyue Luo, Wan Shou, Michael Foshey, Junchi Yan, Joshua B Tenenbaum, Wojciech Ma- tusik, and Antonio Torralba. Dynamic modeling of hand- object interactions via tactile sensing. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems...

    2021

  34. [42]

    Toch: Spatio-temporal object-to-hand correspondence for motion refinement

    Keyang Zhou, Bharat Lal Bhatnagar, Jan Eric Lenssen, and Gerard Pons-Moll. Toch: Spatio-temporal object-to-hand correspondence for motion refinement. InEuropean Confer- ence on Computer Vision, pages 1–19. Springer, 2022. 6

    2022

  35. [43]

    Monocular real-time hand shape and motion capture using multi-modal data

    YuxiaoZhou,MarcHabermann,WeipengXu,IkhsanulHabi- bie, Christian Theobalt, and Feng Xu. Monocular real-time hand shape and motion capture using multi-modal data. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 5346–5355, 2020. 4

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.