Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Adding two learnable parameters to each 3D Gaussian sharpens hand-object interaction reconstruction from a single RGB video, without any object template.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 21:35 UTC pith:H4GX3SG5

load-bearing objection Plausible engineering advance for dynamic HOI reconstruction, but the 'interaction-aware' field is only weakly instantiated and the evaluation is too thin to support the SOTA claim. the 4 major comments →

arxiv 2511.14540 v2 pith:H4GX3SG5 submitted 2025-11-18 cs.CV

Interaction-Aware 4D Gaussian Splatting for Dynamic Hand-Object Interaction Reconstruction

Hao Tian , Chenyangguang Zhang , Rui Liu , Wen Shen , Xiaolin Qin This is my paper
classification cs.CV MSC 68T4568U05
keywords 4D Gaussian splattinghand-object interactiondynamic scene reconstructiondeformation fieldocclusion handlingedge sharpnessprogressive optimizationmonocular RGB video
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that a hand-object interaction scene can be reconstructed—both geometry and appearance—from a single egocentric RGB video without any object shape, category, or pose prior. Its central diagnosis is that existing dynamic Gaussian-splatting reconstructions fail at mutual occlusion and edge blur because they use a single deformation field for the whole scene. The fix is a set of interaction-aware hand-object Gaussians with two extra learnable parameters (weight and radius), separate deformation fields for hand, object, and background, and an object field that receives the key-frame hand position as extra input. The paper further claims that a progressive optimization schedule with explicit 3D losses—for hand position, object rotation, and hand-object contact/penetration—is needed for stable convergence. If correct, the method would let everyday hand-object manipulation be reconstructed from ordinary monocular video at higher fidelity than prior dynamic 3D-GS approaches.

Core claim

The authors claim that dynamic hand-object interaction is modeled more accurately by decomposing the scene into hand, object, and background Gaussian sets and augmenting each Gaussian with a learnable weight w and radius o. The weight encodes structural confidence and occlusion; the radius controls each Gaussian's spatial influence through a radial-basis kernel over its K nearest neighbors, implementing a piecewise-linear deformation prior that keeps contact edges crisp and suppresses non-physical merging of hand and object surfaces. The object field is made interaction-aware by concatenating hand and object positions at a key-frame just before contact with the timestamp, so grasping-induced

What carries the argument

The central object is the interaction-aware hand-object Gaussian, a 3D-GS ellipsoid augmented with two learnable scalars, weight w and radius o. In the HOI refinement step, each Gaussian's K nearest neighbors are weighted by an RBF kernel with radius o, normalized, and modulated by sigmoid(w); these weights drive a linear blend of local rigid transformations, inspired by linear blend skinning, that produces each Gaussian's deformed position and rotation at time t. This converts a global deformation field into locally sharp, occlusion-aware motion. The second load-bearing mechanism is the object deformation field, which takes as input the concatenation of hand and object canonical positions a

Load-bearing premise

The object deformation field receives hand and object positions only at a single fixed key-frame just before contact—not the hand pose at the current time—so any object deformation that depends on the instantaneous grasp pose cannot be represented by this field.

What would settle it

Render the same timestamp twice with the key-frame fixed but the current hand pose perturbed; if the object rendering is essentially unchanged, the object field carries no direct interaction signal. More decisively, train on a sequence with continuously changing grasp poses and compare held-out-frame PSNR with and without feeding the instantaneous hand pose into the object field—if the latter improves accuracy, the paper's interaction-aware claim is incomplete as stated.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Dynamic hand-object scenes can be reconstructed from monocular RGB video without any object template, category, or pose prior, with geometry and appearance modeled together.
  • The two added Gaussian parameters (weight and radius) reduce edge blur and mutual-occlusion artifacts that single-field dynamic 3D-GS methods exhibit at contact boundaries.
  • Conditioning the object deformation field on key-frame hand positions lets the model capture grasping-induced object deformation using only a lightweight hand tracker as 3D supervision.
  • The progressive optimization schedule with explicit 3D losses improves convergence stability and physical plausibility—smooth motion, realistic contact, and coherent lighting—over direct joint optimization.
  • The model extrapolates to held-out timestamps (novel interaction states) better than prior dynamic Gaussian-splatting baselines, as the paper demonstrates with alternate-frame evaluations on two benchmarks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Not tested in the paper: feeding the instantaneous hand pose at time t into the object field, instead of only the key-frame position, could further improve reconstruction during tight contact; this is a direct testable extension of the interaction-aware claim.
  • A diagnostic that would expose whether the object field is truly interaction-aware: fix the key-frame input and perturb the current hand pose; if object renderings at time t are unchanged, the object field does not actually use live interaction information.
  • Because the object Gaussians are initialized as random points in the bounding box of the hand tracker's mesh, the method's object reconstruction quality inherits the hand tracker's accuracy; replacing the tracker with a learned or self-calibrating hand representation could reduce error propagation.
  • The framework is naturally extensible to bimanual or tool-use scenes by adding further hand-conditioned fields, though the paper only evaluates single-hand grasping sequences.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a dynamic 3D Gaussian Splatting framework for monocular egocentric hand-object interaction (HOI) reconstruction without object shape priors. The method separates hand, object, and background Gaussians with three implicit deformation fields; introduces two learnable parameters, an importance weight w_i and a radius o_i, to model occlusion and edge sharpness; conditions the object deformation field on key-frame hand/object positions; and uses explicit 3D losses (hand Chamfer, hand-guided object rotation, and hand-object interaction losses) together with a progressive optimization schedule. Experiments on HOI4D and HO3D compare against 4DGS, Deform3DGS, SC-GS, HOLD, and BIGS, reporting improved PSNR/SSIM/LPIPS under alternate-frame and full-frame evaluation protocols.

Significance. If the reported results are reproducible, the paper makes a useful empirical contribution: it demonstrates that decoupling hand/object/background Gaussians, adding explicit 3D supervision, and using the w/o parameters can stabilize dynamic HOI reconstruction without relying on object meshes or templates. The proposed framework is plausible and the ablations are internally consistent. However, the headline claim of 'state-of-the-art' is currently supported by only eight sequences with no variance or significance estimates, and the central 'interaction-aware' mechanism is weaker than the text suggests because the object deformation field is conditioned on a static key-frame hand pose rather than the instantaneous hand pose. These issues are addressable, but they need to be resolved before the claim can be accepted at face value.

major comments (4)
  1. [§3.2, Eq. (5)] The object deformation field F^O is conditioned on hand and object positions at a fixed key-frame k—'the moment just before hand-object interaction'—concatenated with the canonical object position, plus the timestamp t. The hand coordinates (x_i^k, y_i^k, z_i^k) are therefore a per-sequence constant; the only time-varying input is t. This means the field can memorize or interpolate deformation as a function of time, but it cannot respond to the instantaneous hand pose during contact. The paper's central claim that 'incorporating hand information lets the object field capture dynamic changes caused by grasping' is thus only weakly instantiated. Please either (a) condition F^O on the current time-varying hand pose, e.g., MANO pose at time t, or (b) add an ablation that removes the key-frame hand input entirely and show that it matters. Without this, the interaction-aware mechanism is not e
  2. [§4.1, Tables 1-2] The state-of-the-art claim rests on four HOI4D sequences and four HO3D sequences, with no standard deviations, per-sequence breakdowns, or significance tests. The 'Ours* full-frame evaluation' rows are ambiguous: full-frame evaluation typically includes training frames, which would not measure extrapolation. Please report per-sequence results, clarify exactly which frames are used for training versus testing in both alternate-frame and full-frame protocols, and restrict the headline comparison to the held-out alternate-frame setting. With only eight sequences and no variance measures, 'surpasses state-of-the-art' is an overstatement; a qualified claim such as 'outperforms on the tested sequences' would be more appropriate.
  3. [§3.3, Eq. (8)] The object rotation loss is gated by ω_j(t) = σ(w_j^O), where w_j^O is a learnable per-Gaussian weight. Since w_j^O is optimized as part of the same objective, the model can minimize L_O^rot by driving w_j^O toward zero rather than by aligning object rotations with the hand's rotation trend. This makes the physical-plausibility claim of the loss circular unless w_j^O is independently controlled or the gate is derived from an external contact estimate (e.g., hand-object distance). Please provide evidence that w_j^O remains meaningful during contact (for example, visualize learned w_j^O against ground-truth contact or ablating the gate), or replace the gate with a non-learned contact signal.
  4. [§4.3, Table 3] The ablation study is reported on a single HOI4D scene, and the 'w/o Interaction-Aware Module' row removes both the learnable parameters (w, o) and 'their associated training scheme' jointly, confounding the contribution of the representation with the training procedure. To support the interaction-aware module as a standalone contribution, please run ablations on more than one sequence and separately ablate (i) w and o, (ii) the key-frame hand input to F^O, (iii) the HOI refinement LBS step, and (iv) each loss term. The current single-scene, grouped ablation is too weak to isolate the proposed mechanisms.
minor comments (5)
  1. [§3.1] Typo: 'wrap canonical Gaussians' should be 'warp'. Also, the phrase 'noise smooth' is used informally; define the noise schedule precisely in the implementation details.
  2. [§4.1] The alternate-frame testing protocol is not defined explicitly. State whether tested frames are excluded from training, how many frames are used for training, and how the timestamp t is encoded for held-out frames.
  3. [§3.3, after Eq. (9)] The text says 'we use a separate penetration loss' but no equation or explicit definition of this loss is provided. This omission makes the interaction loss incomplete as described.
  4. [§4, Implementation Details] The choice of key-frame k ('the timestamp just before hand-object contact') is not operationalized. How is contact detected? If it is manual or heuristic, this should be stated, because Eq. (5) depends on it.
  5. [Tables 1-3] The use of small red font for differences is hard to read and not journal-friendly. Use standard delta notation with explicit significance markers (e.g., bold/underline) and state any statistical tests, even if only paired per-sequence tests.

Circularity Check

0 steps flagged

No significant circularity: the central claims are tested against external held-out benchmarks and the stated mechanisms, while imperfect, are not reductions to their own inputs.

full rationale

The paper's main evaluation is external: held-out alternate-frame image metrics on HOI4D and HO3D against published baselines, so the SOTA claim is not built from the method's own fitted outputs. The object deformation field (Eq. 5) does use only static key-frame hand/object coordinates plus time, which is a representational weakness rather than a circular one: the conditioning input is not derived from the output, and no equation in the paper reduces the predicted offsets to the fitted parameters by construction. The rotation loss (Eq. 8) is gated by a learned sigmoid weight, which is self-referential as a regularizer and could in principle be gamed by driving the gate to zero, but the same weight participates in the Gaussian refinement and rendering loss, so the claimed improvement is not forced by the graph alone. There is no load-bearing self-citation, no author-imported uniqueness theorem, and no renaming of a known result as a derivation. The limitations section honestly concedes failure on extreme motion, which further indicates the claims are not tautological. Overall, the derivation chain does not reduce to its own inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The model introduces two new optimizable scalar fields (w and o) per Gaussian; these are fitted parameters rather than independently justified physical entities. The central load-bearing assumptions are the accuracy of the MANO hand tracker, the sufficiency of key-frame hand conditioning, and the sufficiency of random object initialization in a hand-centered box. No new physical entities are postulated.

free parameters (5)
  • w_i (per-Gaussian importance weight) = optimized, values not reported
    New optimizable parameter introduced in Eq. 3; used to gate the object rotation loss (Eq. 8) and to modulate KNN refinement weights (Eq. 12). No learned values or ranges are reported.
  • o_i (per-Gaussian radius) = optimized, values not reported
    New optimizable parameter used as RBF bandwidth in Eq. 10; controls local influence in the HOI refinement step. No learned values are reported.
  • K (number of nearest neighbors) = not specified
    Eqs. 10–13 define the HOI refinement over K-nearest neighbors, but the paper never reports the value of K.
  • loss weights and phase schedules = not specified
    L_H_trans, L_O_rot, L_interaction, penetration loss, and all progressive-phase iteration counts are not reported; these are chosen by hand and materially affect training.
  • key-frame k = timestamp just before hand-object contact, selected per sequence
    Eq. 5 conditions the object field on positions at key-frame k. The selection of k requires knowing contact timing, but the criterion is only described qualitatively.
axioms (6)
  • standard math 3D Gaussian Splatting is a differentiable and accurate scene representation with its standard optimization
    The method inherits the differentiable rasterizer and density-control pipeline from 3D-GS [13]; no modifications are made to the rendering equation.
  • domain assumption The off-the-shelf MANO hand tracker [46] provides sufficiently accurate coarse hand geometry and joint rotations
    Hand Gaussians are initialized from MANO vertices and supervised by Chamfer distance to MANO vertices; the object rotation prior is computed from MANO joint rotations. If the tracker is inaccurate, both hand and object supervision degrade. The authors acknowledge HO3D pose errors hurt all methods.
  • domain assumption Random uniform sampling of object Gaussians inside an AABB around MANO vertices provides a sufficient starting point for object reconstruction
    Section 3.4: 'GO is randomly initialized by uniformly sampling 3D points within an expanded axis-aligned bounding box (AABB) of the MANO vertices.' This assumes the object is near the hand and that photometric plus 3D losses can pull the random cloud onto the true surface without any object prior.
  • ad hoc to paper Static key-frame hand positions are a sufficient conditioning signal for object deformation
    Eq. 5 feeds only the key-frame k hand/object positions concatenated with canonical positions into F^O. The claim that this captures interaction-aware dynamic deformation is not backed by a formal argument or by an ablation replacing key-frame positions with time-varying hand positions.
  • standard math SVD-based rotation averaging of MANO joint rotations yields a valid global hand rotation prior
    Equation 8 uses R_target_hand(t) computed by SVD averaging [9]; this assumes the averaged SO(3) matrix is a meaningful dominant rotation for the whole hand.
  • domain assumption LBS-inspired linear blending of local rigid transformations with learned importance weights gives physically plausible hand-object motion
    Eq. 13 follows embedded deformation / LBS [35]; the paper assumes that KNN rigid blends with the learned w/o weights are sufficient to model complex HOI motion without explicit contact mechanics.

pith-pipeline@v1.3.0-alltime-deepseek · 12985 in / 14392 out tokens · 157205 ms · 2026-08-03T21:35:05.244104+00:00 · methodology

0 comments
read the original abstract

This paper focuses on a challenging setting of simultaneously modeling geometry and appearance of hand-object interaction scenes without any object priors. We follow the trend of dynamic 3D Gaussian Splatting based methods, and address several significant challenges. To model complex hand-object interaction with mutual occlusion and edge blur, we present interaction-aware hand-object Gaussians with newly introduced optimizable parameters aiming to adopt piecewise linear hypothesis for clearer structural representation. Moreover, considering the complementarity and tightness of hand shape and object shape during interaction dynamics, we incorporate hand information into object deformation field, constructing interaction-aware dynamic fields to model flexible motions. To further address difficulties in the optimization process, we propose a progressive strategy that handles dynamic regions and static background step by step. Correspondingly, explicit regularizations are designed to stabilize the hand-object representations for smooth motion transition, physical interaction reality, and coherent lighting. Experiments show that our approach surpasses existing dynamic 3D-GS-based methods and achieves state-of-the-art performance in reconstructing dynamic hand-object interaction.

Figures

Figures reproduced from arXiv: 2511.14540 by Chenyangguang Zhang, Hao Tian, Rui Liu, Wen Shen, Xiaolin Qin.

Figure 1
Figure 1. Figure 1: Differences between traditional 3D Gaussian-based hand-object reconstruction and our interaction-aware model￾ing. Conventional 3D Gaussian approaches model the entire HOI scene with a single, unified implicit field and rely primarily on 2D supervision. This design often leads to geometric ambiguities during close interactions—such as collapsed clearances, blurred contact boundaries, and non-physical mergin… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of interaction-aware hand-object Gaussians. We propose a novel framework for reconstructing dynamic HOI scenes from RGB videos without object shape priors. The framework consists of three components: (1) Specialized Implicit Fields: separate hand, object, and background fields disentangle dynamic interactions, with hand/object fields capturing high-frequency deformations and occlusions (leveraging… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison of our approach and the baseline methods. We present reconstructions from our model and SOTA baselines (4DGS [41], Deform3DGS [45], SC-GS [11]) on HOI4D and HO3D datasets. object pose estimates, inherent inaccuracies and pose er￾rors adversely affect the performance of all methods (Ta￾ble 2). 4DGS is highly sensitive to input noise; SC-GS’s sparse control points fail to model backgro… view at source ↗
Figure 5
Figure 5. Figure 5: Our method maintains consistently high rendering quality [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Disentangled rendering of hand, object, and background. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfer

    cs.CV 2026-07 conditional novelty 7.0

    A taxonomy of eight foundation-model priors organizes HOI reconstruction, generation, and embodied transfer, mapping what knowledge large models inject and where.

  2. Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfer

    cs.CV 2026-07 accept novelty 5.0

    Foundation-model HOI work is organized into eight geometric, semantic, and visual sub-priors that enter six reconstruction/generation tasks and three robot-transfer routes.

Reference graph

Works this paper leans on

50 extracted references · 4 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Push- ing the envelope for rgb-based dense 3d hand pose estimation via neural rendering

    Seungryul Baek, Kwang In Kim, and Tae-Kyun Kim. Push- ing the envelope for rgb-based dense 3d hand pose estimation via neural rendering. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1067–1076, 2019. 2

    2019

  2. [2]

    Bharat Lal Bhatnagar, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. Loopreg: Self-supervised learning of implicit surface correspondences, pose and shape for 3d human mesh registration.Advances in Neural Infor- mation Processing Systems, 33:12909–12922, 2020. 3

    2020

  3. [3]

    Camera-space hand mesh recovery via semantic aggregation and adaptive 2d-1d registration

    Xingyu Chen, Yufeng Liu, Chongyang Ma, Jianlong Chang, Huayan Wang, Tian Chen, Xiaoyan Guo, Pengfei Wan, and Wen Zheng. Camera-space hand mesh recovery via semantic aggregation and adaptive 2d-1d registration. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 13274–13283, 2021. 2

    2021

  4. [4]

    Arctic: A dataset for dexterous bimanual hand- object manipulation

    Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J Black, and Otmar Hilliges. Arctic: A dataset for dexterous bimanual hand- object manipulation. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 12943–12954, 2023. 2

    2023

  5. [5]

    Hold: Category-agnostic 3d reconstruction of in- teracting hands and objects from video

    Zicong Fan, Maria Parelli, Maria Eleni Kadoglou, Xu Chen, Muhammed Kocabas, Michael J Black, and Otmar Hilliges. Hold: Category-agnostic 3d reconstruction of in- teracting hands and objects from video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 494–504, 2024. 2, 6, 7

    2024

  6. [6]

    Forward flow for novel view synthesis of dynamic scenes

    Xiang Guo, Jiadai Sun, Yuchao Dai, Guanying Chen, Xiao- qing Ye, Xiao Tan, Errui Ding, Yumeng Zhang, and Jingdong Wang. Forward flow for novel view synthesis of dynamic scenes. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 16022–16033, 2023. 2

    2023

  7. [7]

    Honnotate: A method for 3d annotation of hand and object poses

    Shreyas Hampali, Mahdi Rad, Markus Oberweger, and Vin- cent Lepetit. Honnotate: A method for 3d annotation of hand and object poses. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 3196–3206, 2020. 2, 6

    2020

  8. [8]

    Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system

    Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan Ratliff, and Dieter Fox. Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system. In2020 IEEE Inter- national Conference on Robotics and Automation (ICRA), pages 9164–9170. IEEE, 2020. 1

    2020

  9. [9]

    Rotation averaging.International Journal of Computer Vision, 103(3):267–305, 2013

    Richard Hartley, Jochen Trumpf, Yuchao Dai, and Hongdong Li. Rotation averaging.International Journal of Computer Vision, 103(3):267–305, 2013. 5

    2013

  10. [10]

    Learning joint reconstruction of hands and manipulated ob- jects

    Yana Hasson, Gul Varol, Dimitrios Tzionas, Igor Kale- vatykh, Michael J Black, Ivan Laptev, and Cordelia Schmid. Learning joint reconstruction of hands and manipulated ob- jects. InProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 11807–11816,

  11. [11]

    Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes

    Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4220–4230, 2024. 1, 2, 3, 4, 6, 7, 8

    2024

  12. [12]

    A compact dynamic 3d gaussian representation for real-time dynamic view synthesis

    Kai Katsumata, Duc Minh V o, and Hideki Nakayama. A compact dynamic 3d gaussian representation for real-time dynamic view synthesis. InEuropean Conference on Com- puter Vision, pages 394–412. Springer, 2024. 2

    2024

  13. [13]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1,

  14. [14]

    Dynmf: Neural motion factorization for real-time dynamic view synthesis with 3d gaussian splatting

    Agelos Kratimenos, Jiahui Lei, and Kostas Daniilidis. Dynmf: Neural motion factorization for real-time dynamic view synthesis with 3d gaussian splatting. InEuropean Con- ference on Computer Vision, pages 252–269. Springer, 2024. 2

    2024

  15. [15]

    Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds

    Jiahui Lei, Yijia Weng, Adam W Harley, Leonidas Guibas, and Kostas Daniilidis. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 6165–6177, 2025. 2

    2025

  16. [16]

    Neural scene flow fields for space-time view synthesis of dy- namic scenes

    Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dy- namic scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6498– 6508, 2021. 2

    2021

  17. [17]

    Gaussian-flow: 4d reconstruction with dynamic 3d gaus- sian particle

    Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian-flow: 4d reconstruction with dynamic 3d gaus- sian particle. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21136– 21145, 2024. 2

    2024

  18. [18]

    Hosnerf: Dynamic human-object-scene neural ra- diance fields from a single video

    Jia-Wei Liu, Yan-Pei Cao, Tianyuan Yang, Zhongcong Xu, Jussi Keppo, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Hosnerf: Dynamic human-object-scene neural ra- diance fields from a single video. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 18483–18494, 2023. 1

    2023

  19. [19]

    Generalizable hand-object modeling from monocular RGB images via 3d gaussians

    Xingyu Liu, Pengfei Ren, Qi Qi, Haifeng Sun, Zirui Zhuang, Jing Wang, Jianxin Liao, and Jingyu Wang. Generalizable hand-object modeling from monocular RGB images via 3d gaussians. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. 2

    2025

  20. [20]

    Hoi4d: A 4d egocentric dataset for category-level human- object interaction

    Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human- object interaction. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 21013–21022, 2022. 6 9

    2022

  21. [21]

    Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis. In2024 International Con- ference on 3D Vision (3DV), pages 800–809. IEEE, 2024. 2

    2024

  22. [22]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021. 1, 2

    2021

  23. [23]

    Interhand2

    Gyeongsik Moon, Shoou-I Yu, He Wen, Takaaki Shiratori, and Kyoung Mu Lee. Interhand2. 6m: A dataset and base- line for 3d interacting hand pose estimation from a single rgb image. InComputer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Pro- ceedings, Part XX 16, pages 548–564. Springer, 2020. 2

    2020

  24. [24]

    Ganerated hands for real-time 3d hand tracking from monocular rgb

    Franziska Mueller, Florian Bernard, Oleksandr Sotny- chenko, Dushyant Mehta, Srinath Sridhar, Dan Casas, and Christian Theobalt. Ganerated hands for real-time 3d hand tracking from monocular rgb. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 49–59, 2018. 2

    2018

  25. [25]

    Real-time pose and shape reconstruction of two interacting hands with a single depth camera.ACM Transactions on Graphics (ToG), 38(4):1–13,

    Franziska Mueller, Micah Davis, Florian Bernard, Oleksandr Sotnychenko, Mickeal Verschoor, Miguel A Otaduy, Dan Casas, and Christian Theobalt. Real-time pose and shape reconstruction of two interacting hands with a single depth camera.ACM Transactions on Graphics (ToG), 38(4):1–13,

  26. [26]

    Bigs: Bimanual category-agnostic interaction reconstruction from monocular videos via 3d gaussian splatting

    Jeongwan On, Kyeonghwan Gwak, Gunyoung Kang, Junuk Cha, Soohyun Hwang, Hyein Hwang, and Seungryul Baek. Bigs: Bimanual category-agnostic interaction reconstruction from monocular videos via 3d gaussian splatting. InPro- ceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 17437–17447, 2025. 2, 6, 7

    2025

  27. [27]

    Nerfies: Deformable neural radiance fields

    Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865–5874, 2021. 2

    2021

  28. [28]

    Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields.arXiv preprint arXiv:2106.13228, 2021

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin- Brualla, and Steven M Seitz. Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields.arXiv preprint arXiv:2106.13228, 2021

    Pith/arXiv arXiv 2021

  29. [29]

    D-nerf: Neural radiance fields for dynamic scenes

    Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 10318–10327, 2021. 2

    2021

  30. [30]

    Hoisdf: Constraining 3d hand-object pose estima- tion with global signed distance fields

    Haozhe Qi, Chen Zhao, Mathieu Salzmann, and Alexander Mathis. Hoisdf: Constraining 3d hand-object pose estima- tion with global signed distance fields. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10392–10402. IEEE, 2024. 1

    2024

  31. [31]

    Em- bodied hands: Modeling and capturing hands and bodies to- gether.arXiv preprint arXiv:2201.02610, 2022

    Javier Romero, Dimitrios Tzionas, and Michael J Black. Em- bodied hands: Modeling and capturing hands and bodies to- gether.arXiv preprint arXiv:2201.02610, 2022. 2, 5

    Pith/arXiv arXiv 2022

  32. [32]

    Frankmo- cap: Fast monocular 3d hand and body motion capture by regression and integration.arXiv preprint arXiv:2008.08324,

    Yu Rong, Takaaki Shiratori, and Hanbyul Joo. Frankmo- cap: Fast monocular 3d hand and body motion capture by regression and integration.arXiv preprint arXiv:2008.08324,

    Pith/arXiv arXiv 2008

  33. [33]

    Coda- 4dgs: Dynamic gaussian splatting with context and defor- mation awareness for autonomous driving

    Rui Song, Chenwei Liang, Yan Xia, Walter Zimmer, Hu Cao, Holger Caesar, Andreas Festag, and Alois Knoll. Coda- 4dgs: Dynamic gaussian splatting with context and defor- mation awareness for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 28031–28041, 2025. 2

    2025

  34. [34]

    Dynamic gaussian marbles for novel view synthesis of casual monocular videos

    Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic gaussian marbles for novel view synthesis of casual monocular videos. InSIGGRAPH Asia 2024 Conference Pa- pers, pages 1–11, 2024. 2

    2024

  35. [35]

    Em- bedded deformation for shape manipulation

    Robert W Sumner, Johannes Schmid, and Mark Pauly. Em- bedded deformation for shape manipulation. InACM sig- graph 2007 papers, pages 80–es, 2007. 5

    2007

  36. [36]

    Grab: A dataset of whole-body human grasp- ing of objects

    Omid Taheri, Nima Ghorbani, Michael J Black, and Dim- itrios Tzionas. Grab: A dataset of whole-body human grasp- ing of objects. InComputer Vision–ECCV 2020: 16th Eu- ropean Conference, Glasgow, UK, August 23–28, 2020, Pro- ceedings, Part IV 16, pages 581–600. Springer, 2020. 1

    2020

  37. [37]

    Non- rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video

    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollh¨ofer, Christoph Lassner, and Christian Theobalt. Non- rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12959–12970, 2021. 2

    2021

  38. [38]

    Capturing hands in action using discriminative salient points and physics sim- ulation.International Journal of Computer Vision, 118:172– 193, 2016

    Dimitrios Tzionas, Luca Ballan, Abhilash Srikantha, Pablo Aponte, Marc Pollefeys, and Juergen Gall. Capturing hands in action using discriminative salient points and physics sim- ulation.International Journal of Computer Vision, 118:172– 193, 2016. 2

    2016

  39. [39]

    Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004. 6

    2004

  40. [40]

    Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects

    Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas M ¨uller, Alex Evans, Dieter Fox, Jan Kautz, and Stan Birchfield. Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 606–617, 2023. 1

    2023

  41. [41]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20310–20320, 2024. 1, 2, 3, 4, 6, 7

    2024

  42. [42]

    PhysGaussian: Physics- Integrated 3D Gaussians for Generative Dynamics

    Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. PhysGaussian: Physics- Integrated 3D Gaussians for Generative Dynamics . In 2024 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 4389–4398, Los Alamitos, CA, USA, 2024. IEEE Computer Society

    2024

  43. [43]

    Grid4d: 4d decomposed hash encoding for high-fidelity dynamic gaus- sian splatting.Advances in Neural Information Processing Systems, 37:123787–123811, 2024

    Jiawei Xu, Zexin Fan, Jian Yang, and Jin Xie. Grid4d: 4d decomposed hash encoding for high-fidelity dynamic gaus- sian splatting.Advances in Neural Information Processing Systems, 37:123787–123811, 2024. 2 10

    2024

  44. [44]

    Nerf-ds: Neural ra- diance fields for dynamic specular objects

    Zhiwen Yan, Chen Li, and Gim Hee Lee. Nerf-ds: Neural ra- diance fields for dynamic specular objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8285–8295, 2023. 2

    2023

  45. [45]

    Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20331–20341, 2024. 1, 2, 3, 4, 6, 7, 8

    2024

  46. [46]

    Predicting 4d hand trajectory from monocular videos.arXiv preprint arXiv:2501.08329,

    Yufei Ye, Yao Feng, Omid Taheri, Haiwen Feng, Shubham Tulsiani, and Michael J Black. Predicting 4d hand trajectory from monocular videos.arXiv preprint arXiv:2501.08329,

    Pith/arXiv arXiv

  47. [47]

    Egogaussian: Dynamic scene understanding from egocentric video with 3d gaussian splatting, 2024

    Daiwei Zhang, Gengyan Li, Jiajie Li, Micka¨el Bressieux, Ot- mar Hilliges, Marc Pollefeys, Luc Van Gool, and Xi Wang. Egogaussian: Dynamic scene understanding from egocentric video with 3d gaussian splatting, 2024. 2, 6

    2024

  48. [48]

    Neural- dome: A neural modeling pipeline on multi-view human- object interactions

    Juze Zhang, Haimin Luo, Hongdi Yang, Xinru Xu, Qianyang Wu, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Neural- dome: A neural modeling pipeline on multi-view human- object interactions. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 8834–8845, 2023. 1

    2023

  49. [49]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6

    2018

  50. [50]

    Dynamic 2d gaussians: Geo- metrically accurate radiance fields for dynamic objects

    Shuai Zhang, Guanjun Wu, Zhoufeng Xie, Xinggang Wang, Bin Feng, and Wenyu Liu. Dynamic 2d gaussians: Geo- metrically accurate radiance fields for dynamic objects. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 8144–8153, 2025. 2 11

    2025