Pith. sign in

REVIEW 4 major objections 6 minor 98 references

Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgery

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces OphNet-3D, a 7.1-million-frame RGB-D dataset of real ophthalmic surgery annotated with 3D hand meshes and instrument poses, and reports models that outperform prior methods on two new benchmarks.

desk verdict A genuinely new large-scale surgical hand-instrument dataset whose benchmark numbers rest on auto-generated, unvalidated 3D labels; deserves review with an external validation condition. read the letter →

arxiv 2505.17677 v2 pith:2YUQSNL4 submitted 2025-05-23 cs.CV

Ming Hu , Zhengdi Yu , Feilong Tang , Kaiwen Chen , Yulong Li , Imran Razzak , Junjun He , Tolga Birdal
show 2 more authors
Kaijing Zhou Zongyuan Ge
This is my paper
classification cs.CV
keywords OphNet-3DRGB-Ddataset3Dhandposeestimationhand-instrumentinteractionophthalmicmicrosurgeryMANOmodel6-DoFinstrumentsurgicalskillassessment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that realistic 3D reconstruction of surgeon hands and instruments in ophthalmic microsurgery is now possible at scale, and to provide the data and models to make it a research problem. Its vehicle is OphNet-3D, a claimed first-of-its-kind RGB-D dataset of real cataract surgery: 41 sequences from 40 surgeons, 7.1 million frames, annotated with 12 surgical phases, 10 instrument categories, dense MANO hand meshes, and full 6-DoF instrument poses. To produce those labels without motion capture, the authors design a multi-stage automatic annotation pipeline, then define two benchmarks—bimanual hand pose estimation and two-hand-two-instrument interaction reconstruction—and report two architectures, H-Net and OH-Net, that outperform prior methods by over 2 mm in MPJPE and up to 23% in ADD-S. A sympathetic reader would care because these are the ingredients missing from objective, contactless surgical skill assessment: quantitative kinematic measures of grip, tool orientation, and phase-by-phase technique.

What carries the argument

The machinery runs on MANO, a parametric hand model that turns pose and shape parameters into a 3D mesh by linear blend skinning, used both to label the dataset and to supervise the learned models. Labels come from a multi-stage optimization: instance masks from SAM2 and per-view hand initializations from a transformer-based hand tracker feed a joint hand-instrument fitting that aligns MANO meshes and laser-scanned articulated instrument models to merged multi-view point clouds, under a 2D re-projection loss, silhouette loss, point-to-mesh distance, a data-driven hand motion prior (HMP), biomechanical joint constraints, an attraction/repulsion interaction loss, and a signed-distance-field penetration penalty. For prediction, H-Net and OH-Net use a collision-aware center-based representation: center heatmaps with repulsion separate the two hands and two instruments, weak-perspective camera modeling recovers translation, and a T-Net refines the relative translation between the two hands and instruments using segmentation-derived spatial features.

What would settle it

Record a handful of ophthalmic surgery sequences while also capturing independent ground truth, for example retroreflective markers on the gloved hands and instruments tracked by an optical motion-capture system, or careful manual 3D landmarking by surgeons on a subset of frames, then run the paper's annotation pipeline on the same sequences. If the fitted meshes and poses systematically deviate from the independent measurements, for instance with errors that grow in rare hand poses or correlate with the motion prior's confidence, the benchmark scores are measuring agreement with the auto-labeler, not absolute accuracy.

Watch

Extended reading notes

Core claim

The paper's central claim is that hand and instrument motion in real ophthalmic surgery can be captured, labeled, and reconstructed in 3D at a scale and fidelity that general-purpose hand datasets cannot provide. OphNet-3D is presented as the first extensive RGB-D dynamic 3D reconstruction dataset for ophthalmic surgery, roughly 2.5 times the size of the largest general 3D hand reconstruction dataset and 70 times the largest prior surgical one, offering video-level coverage of complete procedures through 12 annotated phases. On this foundation, the paper claims two benchmarks and two dedicated baselines: H-Net recovers bimanual hand meshes from a monocular image, and OH-Net jointly reconstructs two hands with two interacting instruments, modeling articulation and relative spatial relations of all four bodies. The paper reports that both architectures outperform existing methods by substantial margins—improvements of over 2 mm in MPJPE and up to 23% in ADD-S—and that the RGB-D variants improve further on interaction-specific metrics such as penetration volume and relative hand-tool translation.

Load-bearing premise

The load-bearing premise is that the automatically fitted MANO hand meshes and instrument poses are accurate enough to serve as ground truth; if the optimization pipeline biases the labels, for instance by pulling hand poses toward the motion prior's training distribution, every benchmark number measures agreement with the auto-labeler rather than true reconstruction accuracy.

Editorial extensions

If this is right

  • Future work on surgical hand and instrument reconstruction gains two concrete benchmarks with subject-disjoint splits, phase balance, and standardized metrics (MPJPE, MPVPE, MRRTE, ADD-S, MAE, penetration volume), so methods can be compared directly.
  • The reported results imply that domain-specific design matters: the collision-aware center representation and T-Net relative-translation refinement outperform generic two-hand and hand-object methods on surgical imagery, and adding depth (the '-D' variants) improves accuracy further.
  • Video-length sequences with phase annotations open the door to temporal and workflow-level modeling, including linking hand and tool kinematics to each of the 12 cataract-surgery phases for skill evaluation.
  • Kinematic proxies such as grip-dependent hand pose, instrument articulation, and hand-tool penetration become quantifiable from a single monocular RGB camera, which is the contactless setup required for sterile surgical environments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My reading of the benchmark numbers: they measure agreement with the auto-annotation pipeline, so an independent validation against motion capture or manual 3D landmarking is needed before they are read as absolute reconstruction accuracy, even though the dataset itself may be exactly as described.
  • My extension: the same recipe of multi-view RGB-D plus a data-driven motion prior plus biomechanical and collision constraints should transfer to other microsurgical settings, and OphNet-3D could serve as pretraining data for gloved-hand reconstruction in general.
  • Testable extension: cross-evaluate models trained on existing synthetic surgical hand datasets against OphNet-3D and the reverse, to quantify how much of the reported gain comes from real data versus from the specific architecture choices.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces OphNet-3D, a claimed first large-scale real-world RGB-D dataset for dynamic 3D reconstruction of hand-instrument interaction in ophthalmic surgery, with 41 sequences from 40 surgeons and 7.1M frames, 12 phase annotations, 10 instrument categories, MANO hand meshes, and 6-DoF instrument poses. The authors propose a multi-stage automatic annotation pipeline combining multi-view RGB-D observations, a data-driven hand motion prior, biomechanical constraints, and hand-instrument interaction losses. They then define two benchmarks and propose H-Net and OH-Net for bimanual hand pose and two-hand-two-instrument reconstruction, reporting state-of-the-art results on both.

Significance. If the dataset and labels are valid, OphNet-3D would be a substantial contribution: it is an order of magnitude larger than prior surgical hand datasets, it captures real clinical procedures rather than synthetic scenes, and it includes scanned CAD models of ten instruments, multi-view RGB-D data, and fine-grained phase annotations. The annotation pipeline is described in unusual detail with explicit losses and weight settings, and the benchmark definitions are reasonably standard. The proposed H-Net and OH-Net designs, especially the collision-aware center-based representation and the T-Net for relative translation refinement, are interesting and potentially reusable. However, the significance of the quantitative claims is conditional on whether the auto-generated 3D labels can be trusted as ground truth, which the paper does not currently establish.

major comments (4)
  1. [Section 3, Eq. (3)-(8); Tables 2-3] The 3D ground truth used in both benchmarks is produced entirely by the authors' own optimization pipeline, which includes a data-driven hand motion prior (Lprior), biomechanical constraints, silhouette and point-cloud losses, and hand-set weights. No independent validation is reported: there is no motion capture, no manual 3D landmarking, no alternative sensor modality, and the Limitations section (Section 5) does not list label validation as a limitation. Because Lprior penalizes deviations from the HMP prior distribution, systematic drift of the labels toward prior-typical poses is a concrete risk, especially for gloved, heavily occluded surgical hands. This makes every absolute number in Tables 2 and 3, including the claimed over-2mm MPJPE improvement, potentially a measure of agreement with the auto-labeler rather than of true reconstruction accuracy. The authors should validate a held-out subset of the labels against manual 3D landmarks or motion capture, report per-label bias, and discuss what the reported metrics do and do not certify.
  2. [Tables 2 and 3; Section 4.3; Abstract] The stated claim that the architectures 'outperform existing methods by substantial margins' and 'achieve state-of-the-art performance on all metrics' is not supported by the tables for the RGB-only models. In Table 3, HOISDF has val MPJPE 17.05 versus OH-Net's 17.12 (OH-Net worse) and val MPVPE 18.22 versus 18.43 (worse); on test, OH-Net's MPVPE 18.36 is worse than HOISDF's 17.91. In Table 2, H-Net's val MPVPE 18.72 is worse than IntagHand's 17.96, and the test MPJPE gap over the best RGB baseline is 1.32mm rather than 'over 2mm'. The over-2mm improvement is driven by the RGB-D variants H-Net-D and OH-Net-D, which are not directly comparable to the RGB-only baselines unless the modality difference is clearly marked. These claims need to be reworded to state which model and which metric improves over which baseline.
  3. [Tables 2 and 3] No error bars, confidence intervals, or significance tests are reported for any of the benchmark numbers. Point estimates alone are insufficient to support comparative claims such as 'H-Net outperforms ACR' (Table 2, test MPJPE 17.66 vs. 18.86) or 'OH-Net outperforms HOISDF' (Table 3, test MPJPE 17.34 vs. 17.36), where differences are small relative to typical run-to-run variation. The authors should report mean and standard deviation over multiple training seeds, or bootstrap confidence intervals, and a paired significance test on the test set.
  4. [Table 5] The frame counts in Table 5 are internally inconsistent. Summing the per-phase 'all' column gives 7,141,720 frames, matching the abstract's 7.1M claim, and the val and test totals (468,240 and 1,511,312) match the sums of their phase rows. However, the 'train' total row reports 4,955,272, whereas the phase rows sum to 5,162,168, a discrepancy of 206,896 frames; furthermore, the row totals give train+val+test = 6,934,824, not 7,141,720. Since the dataset scale is a headline contribution, this arithmetic error must be corrected and the split statistics rechecked.
minor comments (6)
  1. [D.3] The text contains 'CoordConv [?]' with a missing citation; provide the reference for CoordConv.
  2. [D.1] There is a typo in 'wong handedness' in the filtering description; this should read 'wrong handedness'.
  3. [Author affiliations] The affiliation for Wenzhou Medical University is spelled 'Wenzhou Medical Univeristy'; please correct the typo.
  4. [Section 3.2] The reference to instrument scans appears as 'Please see 16 for the detailed visualization'; this should be 'Fig. 16', and figure numbering in the appendix should be checked for consistency.
  5. [Section 4.1] The evaluation protocol should state explicitly which camera views are used at test time and whether metrics are averaged per image, per view, or per sequence; training on all eight views while evaluating per-image makes the effective test set size and independence assumptions unclear.
  6. [Equation (2)] The camera extrinsics are written with a time index (R_t, tau_t) although calibration is described as static; using a fixed camera index would be clearer.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the unvalidated auto-annotation benchmark is a correctness risk, not a circular derivation.

full rationale

The paper's central contribution is a new dataset with auto-generated 3D labels and two baselines trained and evaluated on those labels. Walking the claimed derivation chain, I find no step where a prediction is equivalent by construction to an input. The annotation pipeline (Eq. 3) fits MANO to multi-view RGB-D observations plus an external motion prior [14] and biomechanical constraints [71]; the resulting MANO parameters are then used as supervision targets for H-Net and OH-Net (D.3 explicitly says "θ∗ and β∗ denote pseudo ground-truth values from the annotation pipeline") and as the reference for MPJPE and ADD-S in Tables 2 and 3. This is standard supervised benchmark construction: the models are trained on a training split and evaluated on a held-out test split against the same label-generation protocol. The test labels are not a function of the model's predictions, so the reported improvements over baselines are not forced by construction. The lack of independent validation of the auto-labels (no motion capture, no manual 3D landmarking, no external sensor modality) is a genuine correctness risk for the absolute accuracy claims such as "high-fidelity labels" and "improvements of over 2mm," but that is a benchmark-validity concern, not circularity. Self-citations ([33], [34], [91]) appear in related work and as "inspired by" DynHaMR; they are not load-bearing because the optimization is fully specified in the paper and rests on external components such as HMP [14] and biomechanical constraints [71]. No circular step can be exhibited with the paper's own equations, so the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central benchmark rests on auto-generated 3D labels whose accuracy is not independently verified. The many hand-set optimization weights and thresholds directly determine those labels, and the MANO model plus learned motion prior are assumptions about gloved, microsurgical hands. Little of the pipeline is derived from first principles or externally validated data, so these parameters and assumptions carry the burden of the paper's empirical claims.

free parameters (6)
  • Stage II optimization weights (lambda_2d, lambda_smooth, lambda_theta, lambda_beta) = 0.001, 10, 0.04, 0.05
    Hand-set weights in the hand pose optimization objective (Eq. 3, D.2); no sensitivity analysis, and they control the produced pseudo-ground-truth meshes.
  • Stage III joint optimization weights (lambda_z, lambda_phi, lambda_gamma, lambda_pen, lambda_ja, lambda_palm, lambda_bl) = 200, 2, 10, 10, 1, 1, 1
    Hand-set weights for biomechanical and prior terms in the interaction optimization (D.2); directly shape the final labels used as ground truth.
  • 2D keypoint confidence threshold epsilon_j = 0.5
    ViTPose keypoints below 0.5 are discarded (D.1); affects which observations enter the optimization and thus the labels.
  • Bounding box IoU thresholds (0.9 keep, 0.1 invalid) = 0.9 / 0.1
    Used for false-positive suppression and invalid-frame detection in hand tracking (D.1); these choices filter data that become annotations.
  • Contact distance threshold for hand-object contact vertices = 5 mm
    Vertices within 5 mm of the object are treated as contact and drive the attraction loss (D.2), influencing generated interaction labels.
  • Network training loss weights (lambda_focal, lambda_pj2d, lambda_3d, lambda_sil, lambda_theta, lambda_beta, lambda_seg) = 80, 400, 300, 50, 80, 10, 160
    Hand-set weights in Eq. 9/10/22 for H-Net and OH-Net training (D.3); no sensitivity analysis, and they determine the reported performance.
assumptions (5)
  • domain assumption MANO hand model, including its shape and pose parameters, can accurately represent hands wearing surgical gloves.
    The annotation pipeline fits MANO to gloved hands (Sec. 3.1); MANO is learned from bare-hand data and may not capture glove-induced deformation, so label accuracy depends on this assumption.
  • domain assumption The data-driven hand motion prior (HMP) and biomechanical constraints used in the optimization are valid for ophthalmic microsurgical motions.
    The prior (ref [14]) is trained on general hand motion; applying it to the constrained, small-scale motions of cataract surgery (Sec. 3.1) may bias the optimized labels toward the prior's distribution.
  • standard math Multi-view RGB-D fusion and PnP calibration produce metric-scale, globally consistent 3D point clouds and camera poses.
    The whole annotation pipeline relies on this standard geometry (Sec. 2, B.2); errors here propagate directly into all 3D annotations.
  • domain assumption The pig-eye simulator setup is representative of real ophthalmic surgery hand-instrument interactions.
    Data collected on pig eyes with standardized cataract protocol (Sec. 2, Limitations); the benchmark's clinical relevance and generalizability depend on this transfer.
  • domain assumption ICP and RANSAC-based registration converge to the correct instrument pose when initialized from the scanned CAD models.
    Instrument pose initialization (Sec. 3.2) assumes the global registration over the segmented point cloud will find the right minimum; overexposed instrument tips noted in Limitations may break this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgery." pith.science (2026). https://pith.science/paper/2YUQSNL4

@misc{pith2026250517677,
  author       = {Pith},
  title        = {Pith review of: Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2YUQSNL4}},
  note         = {Machine review of arXiv:2505.17677}
}
read the original abstract

Accurate 3D reconstruction of hands and instruments is critical for vision-based analysis of ophthalmic microsurgery, yet progress has been hampered by the lack of realistic, large-scale datasets and reliable annotation tools. In this work, we introduce OphNet-3D, the first extensive RGB-D dynamic 3D reconstruction dataset for ophthalmic surgery, comprising 41 sequences from 40 surgeons and totaling 7.1 million frames, with fine-grained annotations of 12 surgical phases, 10 instrument categories, dense MANO hand meshes, and full 6-DoF instrument poses. To scalably produce high-fidelity labels, we design a multi-stage automatic annotation pipeline that integrates multi-view data observation, data-driven motion prior with cross-view geometric consistency and biomechanical constraints, along with a combination of collision-aware interaction constraints for instrument interactions. Building upon OphNet-3D, we establish two challenging benchmarks-bimanual hand pose estimation and hand-instrument interaction reconstruction-and propose two dedicated architectures: H-Net for dual-hand mesh recovery and OH-Net for joint reconstruction of two-hand-two-instrument interactions. These models leverage a novel spatial reasoning module with weak-perspective camera modeling and collision-aware center-based representation. Both architectures outperform existing methods by substantial margins, achieving improvements of over 2mm in Mean Per Joint Position Error (MPJPE) and up to 23% in ADD-S metrics for hand and instrument reconstruction, respectively.

Figures

Figures reproduced from arXiv: 2505.17677 by the authors.

Figure 1
Figure 1. OphNet-3D’s acquisition framework, comparisons with other datasets, and phase–frame distributions. Left: (a) a synchronized multi-camera rig with 8 calibrated RGB￾D cameras and 3 directional LED lights; (b) participants perform standardized cataract surgery maneuvers on pig-eye simulators under an ophthalmic microscope; (c) boxplots of pixel errors for eight cameras across three calibration runs. Upper Right: compar… view at source ↗
Figure 2
Figure 2. Our automatic annotation pipeline. Given a multi-view RGB-D video sequence as input, our pipeline reconstructs the 3D hand mesh and 6D instrument pose in a multi-stage manner. H and W represent the initialization network for hands [61] and instance segmentation masks [67]. In the first stage, we use the 3D CAD models scanned as described in Sec. 2 to track per-frame 6D instrument poses. Moreover, we leverage the sta… view at source ↗
Figure 3
Figure 3. Overview of the OH-Net. The backbone image encoder outputs the image feature, which is then used to decode the hand/instrument centre heatmap and segmentation mask. MANO decoders predict their cor￾responding weak-perspective camera parameters. De￾coupling the instrument branch forms H-Net. training, validation, and test sets by sub￾jects, which have 30, 3, 8 subjects sep￾arately. Based on the data split, biman￾ual h… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Qualitative results on the Hand￾instrument interaction benchmark. Each row shows a sample from the test set, with columns displaying: (1) input image, (2) mesh prediction, (3) rendered mesh from a side view, and (4) bird’s-eye view. Results. We evaluate the proposed OH…
Figure 5
Figure 5. Figure 5: Synchronized calibration of 8 cameras. B.2 Synchronized Calibration In our multi-camera calibration pipeline, each Intel RealSense D435’s intrinsic parameters—including focal lengths, principal point coordinates and lens distortion coefficients—are retrieved at runtime…
Figure 6
Figure 6. Figure 6: 10 different instruments. From left to right they are: capsulorhexis forceps, viscoelastic [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results on the hand pose estimation benchmark. Each image is an overlay from each camera view. D.4 More Qualitative Results In this section, we provide qualitative visualizations of our model predictions on the Hand-Instrument Interaction benchmark for diff…
Figure 8
Figure 8. Figure 8: Qualitative results on the hand-instrument interaction benchmark. Each row shows a sample from the test set, with columns displaying: (1) input RGB image, (2) mesh overlay prediction, and (3)(4) for alternative view. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: Qualitative results on the hand-instrument interaction benchmark. Each row shows a sample from the test set, with columns displaying: (1) input RGB image, (2) mesh overlay prediction, and (3)(4) for alternative view. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: 12 phases from 2 different views. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 11
Figure 11. Figure 11: Instance mask examples for phacoemulsification and viscoelastic injection. [PITH_FULL_IMAGE:figures/full_fig_p028_11.png]
Figure 16
Figure 16. Figure 16: Scanned model files for 10 types of instruments. [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

98 extracted references · 67 canonical work pages

  1. [1]

    Banerjee, S

    P. Banerjee, S. Shkodrani, P. Moulon, S. Hampali, S. Han, F. Zhang, L. Zhang, J. Fountain, E. Miller, S. Basol, R. Newcombe, R. Wang, J. J. Engel, and T. Hodan. HOT3D: Hand and object tracking in 3D from egocentric multi-view videos. CVPR, 2025

    2025

  2. [2]

    R. A. Bartholomew, H. Zhou, M. Boreel, K. Suresh, S. Gupta, M. B. Mitchell, C. Hong, S. E. Lee, T. R. Smith, J. P. Guenette, et al. Surgical navigation in the anterior skull base using 3-dimensional endoscopy and surface reconstruction. JAMA Otolaryngology–Head & Neck Surgery, 150(4):318–326, 2024

    2024

  3. [3]

    HUP-3D: A 3D multi-view synthetic dataset for assisted-egocentric hand-ultrasound pose estimation

    M. Birlo, R. Caramalau, P. J. Edwards, B. Dromey, M. J. Clarkson, D. Stoyanov, et al. Hup-3d: A 3d multi-view synthetic dataset for assisted-egocentric hand-ultrasound pose estimation. arXiv preprint arXiv:2407.09215, 2024

    work page Pith review arXiv 2024

  4. [4]

    Borgli, V

    H. Borgli, V . Thambawita, P. H. Smedsrud, S. Hicks, D. Jha, S. L. Eskeland, K. R. Randel, K. Pogorelov, M. Lux, D. T. D. Nguyen, et al. Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Scientific data, 7(1):283, 2020

    2020

  5. [5]

    Brahmbhatt, C

    S. Brahmbhatt, C. Tang, C. D. Twigg, C. C. Kemp, and J. Hays. Contactpose: A dataset of grasps with object contact and hand pose. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16, pages 361–378. Springer, 2020

    2020

  6. [6]

    Z. Cao, I. Radosavovic, A. Kanazawa, and J. Malik. Reconstructing hand-object interactions in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12417–12426, 2021

    2021

  7. [7]

    Y .-W. Chao, W. Yang, Y . Xiang, P. Molchanov, A. Handa, J. Tremblay, Y . S. Narang, K. Van Wyk, U. Iqbal, S. Birchfield, et al. Dexycb: A benchmark for capturing hand grasping of objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9044–9053, 2021

    2021

  8. [8]

    P. Chen, W. Li, N. Gunderson, J. Ruthberg, R. Bly, Z. Sun, W. M. Abuzeid, and E. J. Seibel. Endoperfect: A hybrid nerf-stereo vision approach pioneering monocular depth estimation and 3d reconstruction in endoscopy, 2025

    2025

Show all 98 references
  1. [9]

    Y . Chen, Z. Tu, D. Kang, R. Chen, L. Bao, Z. Zhang, and J. Yuan. Joint hand-object 3d reconstruction from a single image with cross-branch feature fusion. IEEE Transactions on Image Processing, 30:4008–4021, 2021

    2021

  2. [10]

    Corona, A

    E. Corona, A. Pumarola, G. Alenya, F. Moreno-Noguer, and G. Rogez. Ganhand: Predicting human grasp affordances in multi-object scenes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5031–5041, 2020

    2020

  3. [11]

    Cotsoglou, S

    C. Cotsoglou, S. Granieri, S. Bassetto, V . Bagnardi, R. Pugliese, G. L. Grazi, A. Guglielmi, A. Ruzzenente, L. Aldrighetti, F. Ratti, et al. Dynamic surgical anatomy using 3d reconstruction technology in complex hepato-biliary surgery with vascular involvement. results from a...

    2024

  4. [12]

    B. Cui, M. Islam, L. Bai, and H. Ren. Surgical-dino: adapter learning of foundation models for depth estimation in endoscopic surgery. International Journal of Computer Assisted Radiology and Surgery, pages 1–8, 2024

    2024

  5. [13]

    H. Dong, A. Chharia, W. Gou, F. V . Carrasco, and F. De la Torre. Hamba: Single-view 3d hand reconstruc- tion with graph-guided bi-scanning mamba. arXiv preprint arXiv:2407.09646, 2024

    2024

  6. [14]

    Duran, M

    E. Duran, M. Kocabas, V . Choutas, Z. Fan, and M. J. Black. Hmp: Hand motion priors for pose and shape estimation from video. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6353–6363, 2024

    2024

  7. [15]

    Entezami, L

    P. Entezami, L. E. Franzblau, and K. C. Chung. Mentorship in surgical training: a systematic review.Hand, 7(1):30–36, 2012

    2012

  8. [16]

    Z. Fan, O. Taheri, D. Tzionas, M. Kocabas, M. Kaufmann, M. J. Black, and O. Hilliges. ARCTIC: A dataset for dexterous bimanual hand-object manipulation. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  9. [17]

    Fujii, R

    R. Fujii, R. Hachiuma, H. Kajita, and H. Saito. Surgical tool detection in open surgery videos. Applied Sciences, 12(20):10473, 2022

    2022

  10. [18]

    Garcia-Hernando, S

    G. Garcia-Hernando, S. Yuan, S. Baek, and T.-K. Kim. First-person hand action benchmark with rgb-d videos and 3d hand pose annotations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 409–419, 2018. 10

    2018

  11. [19]

    S. Geman. Statistical methods for tomographic image restoration. Bull. Internat. Statist. Inst., 52:5–21, 1987

    1987

  12. [20]

    Gkioxari, R

    G. Gkioxari, R. Girshick, P. Dollár, and K. He. Detecting and recognizing human-object interactions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8359–8367, 2018

    2018

  13. [21]

    E. D. Goodman, K. K. Patel, Y . Zhang, W. Locke, C. J. Kennedy, R. Mehrotra, S. Ren, M. Guan, O. Zohar, M. Downing, et al. Analyzing surgical technique in diverse open surgical videos with multitask machine learning. JAMA surgery, 159(2):185–192, 2024

    2024

  14. [22]

    Grady, C

    P. Grady, C. Tang, C. D. Twigg, M. V o, S. Brahmbhatt, and C. C. Kemp. Contactopt: Optimizing contact to improve grasps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1471–1481, 2021

    2021

  15. [23]

    Grammatikopoulou, E

    M. Grammatikopoulou, E. Flouty, A. Kadkhodamohammadi, G. Quellec, A. Chow, J. Nehme, I. Luengo, and D. Stoyanov. Cadis: Cataract dataset for surgical rgb-image segmentation. Medical Image Analysis, 71:102053, 2021

    2021

  16. [24]

    Hamer, J

    H. Hamer, J. Gall, T. Weise, and L. Van Gool. An object-dependent hand pose prior from sparse training data. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 671–678. IEEE, 2010

    2010

  17. [25]

    Hampali, M

    S. Hampali, M. Rad, M. Oberweger, and V . Lepetit. Honnotate: A method for 3d annotation of hand and object poses. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3196–3206, 2020

    2020

  18. [26]

    Hasson, B

    Y . Hasson, B. Tekin, F. Bogo, I. Laptev, M. Pollefeys, and C. Schmid. Leveraging photometric consistency over time for sparsely supervised hand-object reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 571–580, 2020

    2020

  19. [27]

    Hasson, G

    Y . Hasson, G. Varol, D. Tzionas, I. Kalevatykh, M. J. Black, I. Laptev, and C. Schmid. Learning joint reconstruction of hands and manipulated objects. In CVPR, 2019

    2019

  20. [28]

    Hasson, G

    Y . Hasson, G. Varol, D. Tzionas, I. Kalevatykh, M. J. Black, I. Laptev, and C. Schmid. Learning joint reconstruction of hands and manipulated objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11807–11816, 2019

    2019

  21. [29]

    Hayoz, C

    M. Hayoz, C. Hahne, T. Kurmann, M. Allan, G. Beldi, D. Candinas, P. Márquez-Neila, and R. Sznitman. Online 3d reconstruction and dense tracking in endoscopic videos. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 444–454. Sprin...

    2024

  22. [30]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  23. [31]

    J. Hein, M. Seibold, F. Bogo, M. Farshad, M. Pollefeys, P. Fürnstahl, and N. Navab. Towards markerless surgical tool and hand pose estimation. International journal of computer assisted radiology and surgery, 16:799–808, 2021

    2021

  24. [32]

    M. Hu, L. Wang, S. Yan, D. Ma, Q. Ren, P. Xia, W. Feng, P. Duan, L. Ju, and Z. Ge. Nurvid: A large expert-level video database for nursing procedure activity understanding. Advances in Neural Information Processing Systems, 36:18146–18164, 2023

    2023

  25. [33]

    M. Hu, P. Xia, L. Wang, S. Yan, F. Tang, Z. Xu, Y . Luo, K. Song, J. Leitner, X. Cheng, et al. Ophnet: A large- scale video benchmark for ophthalmic surgical workflow understanding. arXiv preprint arXiv:2406.07471, 2024

    2024 arXiv

  26. [34]

    M. Hu, K. Yuan, Y . Shen, F. Tang, X. Xu, L. Zhou, W. Li, Y . Chen, Z. Xu, Z. Peng, et al. Ophclip: Hierarchical retrieval-augmented learning for ophthalmic surgical video-language pretraining. arXiv preprint arXiv:2411.15421, 2024

    2024 arXiv

  27. [35]

    Huang, X

    D. Huang, X. Ji, X. He, J. Sun, T. He, Q. Shuai, W. Ouyang, and X. Zhou. Reconstructing hand-held objects from monocular video. In SIGGRAPH Asia Conference Proceedings, 2022

    2022

  28. [36]

    Karunratanakul, J

    K. Karunratanakul, J. Yang, Y . Zhang, M. J. Black, K. Muandet, and S. Tang. Grasping field: Learning implicit representations for human grasps. In 2020 International Conference on 3D Vision (3DV), pages 333–344. IEEE, 2020

    2020

  29. [37]

    Khalid, M

    S. Khalid, M. Goldenberg, T. Grantcharov, B. Taati, and F. Rudzicz. Evaluation of deep learning models for identifying surgical actions and measuring performance. JAMA network open, 3(3):e201664–e201664, 2020

    2020

  30. [38]

    Kim, M.-G

    J. Kim, M.-G. Gwon, H. Park, H. Kwon, G.-M. Um, and W. Kim. Sampling is matter: Point-guided 3d human mesh reconstruction. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 12880–12889, 2023

    2023

  31. [39]

    see one, do one, teach one

    S. V . Kotsis and K. C. Chung. Application of the “see one, do one, teach one” concept in surgical training. Plastic and reconstructive surgery, 131(5):1194–1201, 2013. 11

    2013

  32. [40]

    T. Kwon, B. Tekin, J. Stühmer, F. Bogo, and M. Pollefeys. H2o: Two hands manipulating objects for first person interaction recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10138–10148, October 2021

    2021

  33. [41]

    T. Kwon, B. Tekin, J. Stühmer, F. Bogo, and M. Pollefeys. H2o: Two hands manipulating objects for first person interaction recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10138–10148, 2021

    2021

  34. [42]

    Lepetit, F

    V . Lepetit, F. Moreno-Noguer, and P. Fua. Ep n p: An accurate o (n) solution to the p n p problem. International journal of computer vision, 81:155–166, 2009

    2009

  35. [43]

    C. Li, Y . Tong, Y . Long, W. Si, D. C. M. Yeung, J. Y .-K. Chan, and Q. Dou. Extended reality with hmd-assisted guidance and console 3d overlay for robotic surgery remote mentoring. IEEE Robotics and Automation Letters, 2024

    2024

  36. [44]

    M. Li, L. An, H. Zhang, L. Wu, F. Chen, T. Yu, and Y . Liu. Interacting attention graph for single image two-hand reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2761–2770, 2022

    2022

  37. [45]

    M. Li, H. Zhang, Y . Zhang, R. Shao, T. Yu, and Y . Liu. Hhmr: Holistic hand mesh recovery by enhancing the multimodal controllability of graph diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 645–654, 2024

    2024

  38. [46]

    D. Lin, Y . Zhang, M. Li, Y . Liu, W. Jing, Q. Yan, Q. Wang, and H. Zhang. 4dhands: Reconstructing interactive hands in 4d with transformers. arXiv preprint arXiv:2405.20330, 2024

    2024 arXiv

  39. [47]

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. Focal loss for dense object detection. InProceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017

    2017

  40. [48]

    Z. Lin, C. Ding, H. Yao, Z. Kuang, and S. Huang. Harmonious feature learning for interactive hand-object pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12989–12998, 2023

    2023

  41. [49]

    D. Liu, Q. Li, T. Jiang, Y . Wang, R. Miao, F. Shan, and Z. Li. Towards unified surgical skill assessment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9522–9531, 2021

    2021

  42. [50]

    S. Liu, H. Jiang, J. Xu, S. Liu, and X. Wang. Semi-supervised 3d hand-object poses estimation with interactions in time. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14687–14697, 2021

    2021

  43. [51]

    X. Liu, A. Sinha, M. Ishii, G. D. Hager, A. Reiter, R. H. Taylor, and M. Unberath. Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Transactions on Medical Imaging, 39(5):1438–1447, 2020

    2020

  44. [52]

    Lugaresi, J

    C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee, et al. Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172, 2019

    1906 arXiv

  45. [53]

    Manni, C

    G. Manni, C. Lauretti, F. Prata, R. Papalia, L. Zollo, and P. Soda. Bodyslam: A generalized monocular visual slam framework for surgical applications. arXiv preprint arXiv:2408.03078, 2024

    2024 arXiv

  46. [54]

    R. Q. Mao, L. Lan, J. Kay, R. Lohre, O. R. Ayeni, D. P. Goel, et al. Immersive virtual reality for surgical training: a systematic review. Journal of Surgical Research, 268:40–58, 2021

    2021

  47. [55]

    G. Moon. Bringing inputs to shared domains for 3d interacting hands recovery in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17028–17037, 2023

    2023

  48. [56]

    Moon, S.-I

    G. Moon, S.-I. Yu, H. Wen, T. Shiratori, and K. M. Lee. Interhand2. 6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pag...

    2020

  49. [57]

    C. I. Nwoye, T. Yu, C. Gonzalez, B. Seeliger, P. Mascagni, D. Mutter, J. Marescaux, and N. Padoy. Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Medical Image Analysis, 78:102433, 2022

    2022

  50. [58]

    Özsoy, C

    E. Özsoy, C. Pellegrini, T. Czempiel, F. Tristram, K. Yuan, D. Bani-Harouni, U. Eck, B. Busam, M. Keicher, and N. Navab. Mm-or: A large multimodal operating room dataset for semantic understanding of high- intensity surgical environments. arXiv preprint arXiv:2503.02579, 2025

    2025 arXiv

  51. [59]

    Y . L. Pang, C. Oh, and A. Cavallaro. Sparse multi-view hand-object reconstruction for unseen environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 803–810, 2024

    2024

  52. [60]

    A. Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703, 2019. 12

    1912 arXiv

  53. [61]

    Pavlakos, D

    G. Pavlakos, D. Shan, I. Radosavovic, A. Kanazawa, D. Fouhey, and J. Malik. Reconstructing hands in 3d with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9826–9836, 2024

    2024

  54. [62]

    T.-H. Pham, N. Kyriazis, A. A. Argyros, and A. Kheddar. Hand-object contact force estimation from markerless visual tracking. IEEE transactions on pattern analysis and machine intelligence, 40(12):2883– 2896, 2017

    2017

  55. [63]

    J. D. Pitcher, J. T. Wilson, T.-C. Tsao, S. D. Schwartz, and J.-P. Hubschman. Robotic eye surgery: past, present, and future. J Comput Sci Syst Biol, 3(1):137, 2012

    2012

  56. [64]

    Prakash, M

    A. Prakash, M. Chang, M. Jin, R. Tu, and S. Gupta. 3d reconstruction of objects in hands without real world 3d supervision. In European Conference on Computer Vision, pages 126–145. Springer, 2024

    2024

  57. [65]

    H. Qi, C. Zhao, M. Salzmann, and A. Mathis. Hoisdf: Constraining 3d hand-object pose estimation with global signed distance fields. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10392–10402. IEEE, 2024

    2024

  58. [66]

    Qiu and H

    L. Qiu and H. Ren. Endoscope navigation with slam-based registration to computed tomography for transoral surgery. International Journal of Intelligent Robotics and Applications, 4(2):252–263, 2020

    2020

  59. [67]

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Dollár, and C. Feichtenhofer. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.0...

    2024 arXiv

  60. [68]

    P. Ren, C. Wen, X. Zheng, Z. Xue, H. Sun, Q. Qi, J. Wang, and J. Liao. Decoupled iterative refinement framework for interacting hands reconstruction from a single rgb image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8014–8025, 2023

    2023

  61. [69]

    Romero, D

    J. Romero, D. Tzionas, and M. J. Black. Embodied hands: Modeling and capturing hands and bodies together. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6), Nov. 2017

    2017

  62. [70]

    D. Shan, J. Geng, M. Shu, and D. F. Fouhey. Understanding human hands in contact at internet scale. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9869–9878, 2020

    2020

  63. [71]

    Spurr, U

    A. Spurr, U. Iqbal, P. Molchanov, O. Hilliges, and J. Kautz. Weakly supervised 3d hand pose estimation via biomechanical constraints. In European conference on computer vision, pages 211–228. Springer, 2020

    2020

  64. [72]

    Stevsic and O

    S. Stevsic and O. Hilliges. Spatial attention improves iterative 6d object pose estimation. In 2020 international conference on 3D vision (3DV), pages 1070–1078. IEEE, 2020

    2020

  65. [73]

    Sugiyama, S

    T. Sugiyama, S. Lama, L. S. Gan, Y . Maddahi, K. Zareinia, and G. R. Sutherland. Forces of tool-tissue interaction to assess surgical skill level. JAMA surgery, 153(3):234–242, 2018

    2018

  66. [74]

    Y . Sun, Q. Bao, W. Liu, Y . Fu, M. J. Black, and T. Mei. Monocular, one-stage, regression of multiple 3d people. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11179–11188, 2021

    2021

  67. [75]

    S. Tamai. History of microsurgery—from the beginning until the end of the 1970s. Microsurgery, 14(1):6–13, 1993

    1993

  68. [76]

    Tekin, F

    B. Tekin, F. Bogo, and M. Pollefeys. H+ o: Unified egocentric recognition of 3d hand-object poses and interactions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4511–4520, 2019

    2019

  69. [77]

    Q. Tian, Z. Chen, H. Liao, X. Huang, L. Li, S. Ourselin, and H. Liu. Endoomni: Zero-shot cross-dataset depth estimation in endoscopy by robust self-learning from noisy labels. arXiv preprint arXiv:2409.05442, 2024

    2024 arXiv

  70. [78]

    Tzionas, L

    D. Tzionas, L. Ballan, A. Srikantha, P. Aponte, M. Pollefeys, and J. Gall. Capturing hands in action using discriminative salient points and physics simulation. International Journal of Computer Vision, 118:172–193, 2016

    2016

  71. [79]

    P. Vaid, S. Yeung, and A. Rau. Robust semi-supervised detection of hands in diverse open surgery environments. In K. Deshpande, M. Fiterau, S. Joshi, Z. Lipton, R. Ranganath, I. Urteaga, and S. Yeung, editors, Proceedings of the 8th Machine Learning for Healthcare Conference, ...

    2023

  72. [80]

    H. Wang, Y . Long, Y . Chen, H.-C. Yip, M. Scheppach, P. W.-Y . Chiu, Y . Yam, H. M.-L. Meng, and Q. Dou. Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion. Medical Image Analysis, 103:103599, 2025

    2025

  73. [81]

    R. Wang, S. Ktistakis, S. Zhang, M. Meboldt, and Q. Lohmeyer. Pov-surgery: A dataset for egocentric hand and tool pose estimation during surgical activities. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 440–450. Springer, 2023. 13

    2023

  74. [82]

    J. Wu, G. Pavlakos, G. Gkioxari, and J. Malik. Reconstructing hand-held objects in 3d. arXiv preprint arXiv:2404.06507, 2024

    2024

  75. [83]

    Y . Xu, J. Zhang, Q. Zhang, and D. Tao. Vitpose: Simple vision transformer baselines for human pose estimation. Advances in Neural Information Processing Systems, 35:38571–38584, 2022

    2022

  76. [84]

    J. Yang, J. Li, G. Li, Z. Shen, H.-Y . Wu, Z. Fan, and H. Huang. Mlphand: Real time multi-view 3d hand mesh reconstruction via mlp modeling. arXiv preprint arXiv:2406.16137, 2024

    2024 arXiv

  77. [85]

    L. Yang, J. Xu, L. Zhong, X. Zhan, Z. Wang, K. Wu, and C. Lu. Poem: reconstructing hand in a point embedded multi-view stereo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21108–21117, 2023

    2023

  78. [86]

    Y . Yang, L. Guoliang, Q. Li, and R. Song. A slam framework based spinal endoscopic localization method. Procedia Computer Science, 250:81–87, 2024

    2024

  79. [87]

    Z. Yang, J. Pan, J. Dai, Z. Sun, and Y . Xiao. Self-supervised endoscopy depth estimation framework with clip-guidance segmentation. Biomedical Signal Processing and Control, 95:106410, 2024

    2024

  80. [88]

    Y . Ye, A. Gupta, and S. Tulsiani. What’s in your hands? 3d reconstruction of generic objects in hands. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3895–3905, 2022

    2022

  81. [89]

    Y . Ye, P. Hebbar, A. Gupta, and S. Tulsiani. Diffusion-guided reconstruction of everyday hand-object interaction clips. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19717–19728, 2023

    2023

  82. [90]

    Z. Yu, S. Huang, F. Chen, T. P. Breckon, and J. Wang. Acr: Attention collaboration-based regressor for arbitrary two-hand reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023

    2023

  83. [91]

    Z. Yu, S. Zafeiriou, and T. Birdal. Dyn-hamr: Recovering 4d interacting hand motion from a dynamic camera. arXiv preprint arXiv:2412.12861, 2024

    2024 arXiv

  84. [92]

    Zakharov, I

    S. Zakharov, I. Shugurov, and S. Ilic. Dpod: 6d pose object detector and refiner. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1941–1950, 2019

    1941

  85. [93]

    R. Zha, X. Cheng, H. Li, M. Harandi, and Z. Ge. Endosurf: Neural surface reconstruction of deformable tissues with stereo endoscope videos. In International conference on medical image computing and computer-assisted intervention, pages 13–23. Springer, 2023

    2023

  86. [94]

    Zhang, Y

    B. Zhang, Y . Wang, X. Deng, Y . Zhang, P. Tan, C. Ma, and H. Wang. Interacting two-hand 3d pose and shape reconstruction from single color image. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11354–11363, 2021

    2021

  87. [95]

    J. Y . Zhang, S. Pepose, H. Joo, D. Ramanan, J. Malik, and A. Kanazawa. Perceiving 3d human-object spatial arrangements from a single image in the wild. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16, pages 34–...

    2020

  88. [96]

    Start Recording

    C. Zimmermann, D. Ceylan, J. Yang, B. Russell, M. Argus, and T. Brox. Freihand: A dataset for markerless capture of hand pose and shape from single rgb images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 813–822, 2019. 14 A Related Work Su...

    2019

  89. [97]

    (15) 19 The shape prior Lβ similarly penalizes the shape coefficients βh t , encouraging plausible hand geometry: Lβ = X h∈{l,r} ∥βh∥2

  90. [98]

    Interaction Loss

    (16) These terms serve as soft constraints that prevent drift during optimization and help enforce physical realism. Interaction Loss. To model physical plausibility and guide the relative spatial arrangement of the hand and tool, we incorporate an interaction loss Linter comp...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.