Pith. sign in

REVIEW 2 major objections 5 minor 121 references

TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactions

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One diffusion network can model all seven 3D human-object interaction distributions.

desk verdict TriDi delivers on its central promise—one network for all seven HOI conditioning modes—with a fixable but real soft spot in the contact-label evaluation. read the letter →

arxiv 2412.06334 v3 pith:SI7SWDXR submitted 2024-12-09 cs.CV

Ilya A. Petrov , Riccardo Marin , Julian Chibane , Gerard Pons-Moll This is my paper · ORCID
classification cs.CV
keywords 3Dhuman-objectinteractiontrilateraldiffusionmultimodaljointdistributionmodelingcontactmapstext-conditionedgenerationSMPL+Hscenepopulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TriDi claims that a single diffusion model can represent the joint distribution of a 3D human body, an object, and their interaction, and therefore can generate any of the seven marginal or conditional distributions that arise from these three modalities. Previous methods train a separate network for each direction, such as human-from-object or object-from-human, whereas TriDi trains once and switches direction by setting the conditioning modalities' noise timestep to zero. The paper argues that this unification is not a trade-off: on GRAB and BEHAVE, one three-way model beats one-way specialized baselines on distribution-quality and geometric-consistency metrics, and it enables modes no prior work had, such as text- or contact-map-conditioned human-object generation and scene population. If true, this collapses a family of one-way HOI systems into a single architecture.

What carries the argument

The mechanism is the trilateral diffusion objective extended from UniDiffuser to three modalities, implemented with a token-level transformer and a shared Contact-Text interaction latent. In the objective, the denoiser predicts $(H_0, O_0, I_0)$ from $\mathrm{H}_{t_H}, \mathrm{O}_{t_O}, \mathrm{I}_{t_I}$, and the timestep rule $t=0$ versus $t=T$ selects conditioning versus marginalization. Token-wise self-attention between modalities lets the network discover relations rather than receiving them as fixed structure. Interaction $I$ is a 128-dimensional latent $z_I$ produced by two encoders, one for contact maps $\phi_I \in \{0,1\}^{690}$ and one for text, trained with BCE reconstruction and latent-similarity losses so that text and contact maps share one code. At inference, classifier guidance $F(\hat H, \hat O, \hat I)=\sum_j |\hat\phi_I^j \hat d_j|$ pushes predicted human vertices toward the predicted object where the contact map is active.

What would settle it

Take a sample of BEHAVE and GRAB test pairs, have annotators mark which body parts truly touch the object, and compare with the paper's thresholded distance maps. If the agreement is well below the reported contact accuracy, or if the same TriDi predictions evaluated against hand-verified labels score substantially lower than Acccont, the contact supervision is falsified.

Watch

Extended reading notes

Core claim

TriDi models the joint distribution $P(H,O,I)$ of human pose and shape $H = (\theta_H, \beta_H, g_H)$, object 6-DoF pose $O = g_O$, and interaction latent $I = z_I$, a 128-dimensional code shared by contact maps and text descriptions. The network is trained with a trilateral diffusion objective: one transformer receives independently noised tokens $\mathrm{H}_{t_H}, \mathrm{O}_{t_O}, \mathrm{I}_{t_I}$ with timestep tuple $(t_H, t_O, t_I)$ and an object class condition, and predicts the clean triplet. Setting a modality's timestep to $0$ conditions on it; setting it to $T$ marginalizes it out, so one network samples all $2^3 - 1 = 7$ distributions. The central demonstration is that despite covering every direction, TriDi surpasses the specialized one-way baselines GNet and Object Pop-up with a conditional VAE on Coverage, MMD, and 1-NNA, with contact accuracy close to ground-truth data.

Load-bearing premise

The load-bearing premise is that the automatically thresholded vertex-distance maps used as contact labels are faithful enough to teach the interaction modality and to score contact accuracy; the paper does not validate these labels against manual annotations.

Editorial extensions

If this is right

  • A single TriDi model replaces separate networks for human-from-object and object-from-human generation by covering all seven modes: $P(H,O,I)$, $P(H,O|I)$, $P(H,I|O)$, $P(O,I|H)$, $P(H|O,I)$, $P(O|H,I)$, and $P(I|H,O)$.;
  • On GRAB and BEHAVE, the unified model reports better Coverage, MMD, and 1-NNA than GNet and Object Pop-up plus a conditional VAE, with reported improvements up to 47% in distribution-matching metrics.;
  • Contact accuracy for TriDi is comparable to contact computed from predicted meshes, and penetration statistics are close to ground-truth values, indicating the interaction latent is doing real work rather than being a side channel.;
  • Beyond the seven sampling modes, TriDi generalizes to unseen object geometry, populates scenes from HPS scans, and generates objects for human-contact datasets, which are applications no one-way baseline supports.;

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same timestep rule should extend to other modality triples such as human motion, object trajectory, and action label, because the condition-by-$t=0$ and marginalize-by-$t=T$ mechanism is modality-agnostic.;
  • The Contact-Text latent is a general recipe for aligning a hard-to-control representation with an easy-to-control one, so similar shared latents could align keypoints, gaze, or object affordances with language in other generative models.;
  • A testable extension is to use TriDi as a data synthesizer: generate diverse $(H,O,I)$ samples to train one-way regressors such as Object Pop-up, which currently produce deterministic predictions, and measure whether the regressors improve.;
  • The left-right mirror augmentation suggests a cheap and general debiasing step for other human-object interaction datasets that are dominated by right-handed captures.;
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. TriDi proposes a unified three-way diffusion model over human pose/shape (H), object 6-DoF pose (O), and an interaction latent (I) that jointly embeds body contact maps and CLIP text embeddings. Built on the UniDiffuser paradigm and a transformer with token-level attention, the same network is trained with a single objective (Eq. 9) and can sample from the joint distribution P(H,O,I) and all six conditional/marginal modes, conditioned on an object class and canonical geometry. The paper reports extensive experiments on BEHAVE and GRAB, comparing against one-way baselines (GNet, ObjPOP, ObjPOP+cVAE, COINS) and single-mode variants of the same architecture (s-TriDi), with distribution metrics (1-NNA, COV, MMD), geometric consistency errors (MPJPE, Ev2v, Ec), contact accuracy, penetration analysis, ablations, a user study, and applications to scene population, interaction reconstruction from images, and generalization to unseen geometries.

Significance. If the results hold, TriDi is a significant step toward a unified, scalable model for 3D human-object interaction: one network trained once covers the seven conditioning configurations that previously required separate models. The paper is unusually thorough in its evaluation: it reports established distribution metrics with variances, ablates guidance and augmentation, includes an SDF-based penetration analysis, conducts a user study (40 responses, 89% preference over baselines), and demonstrates generalization to unseen object geometries and cross-dataset inference. The joint embedding of contact maps and text is a practically useful design that combines fine-grained spatial control with user-friendly semantic control. The main risk is the reliance on automatically thresholded contact labels for both supervision and evaluation, which may inflate the reported contact accuracy and bias the learned interaction representation; this concern is central to the interaction-quality claims and warrants additional validation.

major comments (2)
  1. [Sec. 4.3 and Eq. (29)] The contact maps phi_I are generated by thresholding human-object vertex distances 'following [4]' with no explicit threshold value. These same auto-generated maps are used (i) to train the contact encoder and the interaction latent via Eq. (6), (ii) as the training target for the diffused interaction modality in Eq. (9), and (iii) to compute the reported contact accuracy Acccont in Eq. (29). Because the evaluation compares the predicted contact map against the same thresholding rule that produced the training labels, Acccont measures agreement with the auto-annotation procedure rather than with physically validated contact. The paper provides no validation against manual contact annotations (e.g., DAMON labels used in Sec. 5.3) and no sensitivity analysis over the threshold. This directly affects the contact-accuracy numbers in Table 2 and the 'contact accuracy' statements in the abstract. Please validate the auto-labels on a manually annotated subset, report the exact threshold chosen, and show how Acccont varies with the threshold.
  2. [Sec. 5.1 and Tables 1-2] The GNet baseline [75] is originally a 4D whole-body motion generation model. The paper states 'we rely on GNet' for the H,I|O comparison but does not describe how GNet is adapted to static single-frame generation, nor how its sampling is performed for the test set. Likewise, the 'ObjPOP+cVAE' baseline is introduced in a single sentence with no architecture or training details. These missing details are load-bearing because the headline claim that TriDi 'surpasses one-way specialized baselines' depends on a fair and reproducible comparison. Without a description of the adaptation, it is difficult to rule out that GNet's poor performance (e.g., 1-NNA 89.64 on GRAB) is due to an unsuitable adaptation rather than to an inherent limitation of the one-way approach. Please provide full implementation and adaptation details for both baselines in the supplementary material.
minor comments (5)
  1. [Sec. 5.1] The text states that H,I|O and O,I|H are 'the only two that have existing methods working with static HOI,' but the supplementary material compares TriDi with COINS [109] on H|O,I (Table S3), which is a static HOI setting. Please revise this sentence to acknowledge the COINS comparison.
  2. [Sec. 1 and Sec. 4.2] The paper uses 'joint unconditional distribution' in the introduction, but Section 4.2 states that the only required conditioning is the object representation CO. Thus all seven modes are conditioned on the object class and canonical geometry. Please clarify this terminology so that 'unconditional' is not misread as fully unconditional over object class.
  3. [Supplementary Eq. (16)] The object loss is written as LO_n = ||gO - \hat{gH}||_1, which appears to be a typo; it should likely be ||gO - \hat{gO}||_1.
  4. [Supplementary Eq. (20)] The definition of the cone distance field contains an apparent typo: 'from the face℧' should likely be 'from the face'.
  5. [Sec. 4.3] Please specify the exact threshold used for contact-map binarization and the precise reference in [4], as the current text ('threshold them following [4]') is not reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the seven-mode capability follows mathematically from the three-way UniDiffuser-style objective, and the main distribution metrics are evaluated on held-out ground-truth samples; the contact-accuracy metric is self-referential to the auto-thresholded labels, but that is a label-validity caveat rather than a circular derivation.

full rationale

The central claim — that one network can sample from P(H,O,I) and all six conditionals — is a direct corollary of the three-way diffusion objective in Eq. 9. Setting any subset of the timesteps tH, tO, tI to T yields marginals, and setting them to 0 yields conditionals, exactly as in the cited UniDiffuser framework [3], which the paper explicitly extends from two to three modalities. This is a mathematical property of the training objective, not a fitted or self-imported result. The distribution-quality metrics in Table 1 (1-NNA, COV, MMD) are computed on held-out test sets against ground-truth samples using L2 distances defined in Sec. 13, so they do not reduce to the model's own fitted values. The only potentially self-referential quantity is the contact accuracy Acccont in Eq. 29: the ground-truth contact map phi_I is obtained by thresholding human-object vertex distances (Sec. 4.3, following [4]), and the same auto-thresholded maps supervise the contact encoder/decoder in Eq. 6 and the distance supervision in Eq. 16. Consequently, Acccont measures agreement with the paper's own auto-labeling rule rather than with independently validated physical contact, and the paper does not validate the threshold against manual annotations. This is a legitimate limitation of what the contact metric proves, but it is not circularity in the derivation chain: the model is trained on a training split and evaluated on a held-out test split, the interaction latent is a representation choice rather than a renamed known result, and the joint-modeling and conditional-sampling claims do not depend on the contact labels being physically perfect. No load-bearing argument reduces to a self-citation: reference [4] supplies the thresholding rule and dataset, reference [64] is used only as a baseline, and reference [3] supplies the multimodal-diffusion framework that the paper openly builds upon.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central claims rest mainly on the learned model and its hyperparameters. No new physical entities are introduced; the only new 'thing' is the learned interaction latent, which is internal. The main unstated assumptions are the fidelity of auto-generated contact labels and the validity of the symmetry augmentation.

free parameters (5)
  • Guidance scale lambda = 2.0
    Applied to the reconstruction guidance for the last 200 of 1000 denoising steps (Supp. Sec. 8). Chosen by hand; no sensitivity analysis reported.
  • Loss weights (lambda_Hn, lambda_On, lambda_In, lambda_Hv, lambda_Ov, lambda_Iv) = 2, 1, 1, 6, 2, 4
    Set by hand in Supp. Eq. 17; no ablation on these weights.
  • Contact map threshold = Not specified (follows BEHAVE [4])
    Binary contact maps are obtained by thresholding vertex distances following BEHAVE; the threshold value is not reported in the paper.
  • Interaction latent dimension = 128
    Dimension of the shared contact-text latent zI (Sec. 4.1).
  • Augmentation mirror probability = Not explicitly stated (randomly mirroring)
    Samples are mirrored through the ZY plane during training; the exact probability is not reported.
assumptions (5)
  • standard math The multi-modal diffusion objective (Eq. 9) trained with uniformly sampled timesteps yields a model that can sample from all seven distributions by setting timesteps to 0 or T.
    This is the UniDiffuser result [3] extended to three modalities; it is assumed valid for three modalities.
  • domain assumption SMPL+H with a decimated 690-vertex template is a sufficient representation of the human body for HOI generation.
    Used in Sec. 4.1 for H modality; if too coarse, fine-grained contact may be lost.
  • domain assumption A binary contact map on human body vertices, obtained by thresholding vertex-to-vertex distances, faithfully represents the interaction.
    Used for supervision, training of the interaction latent, and evaluation (Sec. 4.3, Eq. 29).
  • domain assumption Left-right mirroring through the ZY plane produces valid training samples; body asymmetry artifacts are negligible.
    Used for data augmentation in Sec. 4.3.
  • ad hoc to paper The reconstruction guidance function F (Eq. 11) that pulls predicted human and object toward the predicted contact map is a valid way to enforce contact.
    Introduced in Sec. 4.2; no theoretical guarantee that it preserves the data distribution.
invented entities (1)
  • Shared contact-text latent zI
    purpose: Represents the interaction I in a 128-dimensional latent space that can be encoded from either a binary contact map or a CLIP text embedding, and decoded back to a contact map.
    This is an internal learned representation; it has no external observable handle beyond the model's outputs, and its quality is only measured through the model's contact accuracy and user studies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactions." pith.science (2026). https://pith.science/paper/SI7SWDXR

@misc{pith2026241206334,
  author       = {Pith},
  title        = {Pith review of: TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SI7SWDXR}},
  note         = {Machine review of arXiv:2412.06334}
}
read the original abstract

Modeling 3D human-object interaction (HOI) is a problem of great interest for computer vision and a key enabler for virtual and mixed-reality applications. Existing methods work in a one-way direction: some recover plausible human interactions conditioned on a 3D object; others recover the object pose conditioned on a human pose. Instead, we provide the first unified model - TriDi which works in any direction. Concretely, we generate Human, Object, and Interaction modalities simultaneously with a new three-way diffusion process, allowing to model seven distributions with one network. We implement TriDi as a transformer attending to the various modalities' tokens, thereby discovering conditional relations between them. The user can control the interaction either as a text description of HOI or a contact map. We embed these two representations into a shared latent space, combining the practicality of text descriptions with the expressiveness of contact maps. Using a single network, TriDi unifies all the special cases of prior work and extends to new ones, modeling a family of seven distributions. Remarkably, despite using a single model, TriDi generated samples surpass one-way specialized baselines on GRAB and BEHAVE in terms of both qualitative and quantitative metrics, and demonstrating better diversity. We show the applicability of TriDi to scene population, generating objects for human-contact datasets, and generalization to unseen object geometry. The project page is available at: https://virtualhumans.mpi-inf.mpg.de/tridi.

Figures

Figures reproduced from arXiv: 2412.06334 by the authors.

Figure 1
Figure 1. TriDi. We present TriDi, the first joint probabilistic model of human pose (H), object (O) and human-object interaction (I). The joint model unifies these three modalities, capturing mutual dependencies between them, and allows for sampling in seven conditioning configurations, covering the use cases treated in isolation by previous works. The colors on the image encode prediction and condition. Abstract Modeling 3D… view at source ↗
Figure 2
Figure 2. TriDi Overview. TriDi is a Trilateral Diffusion for Human H (pose θH, identity βH, and 6-DoF global pose gH), Object O (6-DoF global pose gO) and Interaction I (Contact-Text latent zI ). In this figure the model is configured to sample p(H, I|O). One of the seven operating modes is chosen by adjusting the timestamp to be 0 for a given condition (t O above) and T for the desired prediction (t H and t I above), and su… view at source ↗
Figure 3
Figure 3. Architecture of Contact-Text Interactions model. We train a mapping from the contact map EϕI and CLIP embedding ETI to a joint latent space zI that is used to represent the interac￾tion I. Jointly, we train the decoder DϕI that maps the latent back to the contact map. Interactions. Representing interaction I is particularly challenging as we want to combine the intuitiveness of text descriptions with the expressiven… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison with baselines. In the two left-most columns, we show three samples for p(H, I|O) and p(O, I|H) from BEHAVE and GRAB test sets. TriDi’s generations are better aligned with the condition, causing less interpenetration (e.g., for basketball), respecting fine-g…
Figure 5
Figure 5. Figure 5: Text Interaction. TriDi supports text condition￾ing for I modality, providing user control on the contact. We perform an ablation to analyze the role of the augmentation, I modality modeling, and the guid￾ance (we report quantita￾tive results in Sup. Mat.). The full mo…
Figure 6
Figure 6. Figure 6: Scene populating. Using 3D scans from HPS [27], we validate the practicality of TriDi for scene population in various conditioning cases. On the left, we demonstrate conditional synthesis of human-object interactions. On the right, TriDi is used for the joint generatio…
Figure 7
Figure 7. Figure 7: Interaction reconstruction. DECO [78] annotates hu￾man H and contact I for the RGB image, while our TriDi recovers the object O, showing generalization on unseen data distributions. ful model trained on more datasets in the Sup. Mat., show￾ing scalability to a variety …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

121 extracted references · 64 canonical work pages

  1. [4]

    Petrov, Cris- tian Sminchisescu, Christian Theobalt, and Gerard Pons- Moll

    Bharat Lal Bhatnagar, Xianghui Xie, Ilya A. Petrov, Cris- tian Sminchisescu, Christian Theobalt, and Gerard Pons- Moll. BEHA VE: Dataset and Method for Tracking Human Object Interactions. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 15935–15946, 2022. 3, 5, 6, 7

    2022

  2. [75]

    Black, and Dimitrios Tzionas

    Omid Taheri, Vasileios Choutas, Michael J. Black, and Dimitrios Tzionas. GOAL: Generating 4D whole-body motion for hand-object grasping. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) , pages 13263–13273, 2022. 2, 6, 7

    work page 2022

  3. [1]

    Learning representations and generative models for 3d point clouds

    Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. In International conference on machine learning, pages 40–49. PMLR, 2018. 6, 7, 8

    2018

  4. [2]

    SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Image

    Dimitrije Anti ´c, Georgios Paschalidis, Shashank Tripathi, Theo Gevers, Sai Kumar Dwivedi, and Dimitrios Tzionas. Sdfit: 3d object pose and shape by fitting a morphable sdf to a single image. arXiv preprint arXiv:2409.16178, 2024. 8

    work page Pith review arXiv 2024

  5. [3]

    One transformer fits all distributions in multi-modal diffu- sion at scale

    Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, and Jun Zhu. One transformer fits all distributions in multi-modal diffu- sion at scale. In Proceedings of the 40th International Con- ference on Machine Learning. JMLR.org, 2023. 2, 3

    2023

  6. [5]

    Bedlam: A synthetic dataset of bodies exhibit- ing detailed lifelike animated motion

    Michael J Black, Priyanka Patel, Joachim Tesch, and Jin- long Yang. Bedlam: A synthetic dataset of bodies exhibit- ing detailed lifelike animated motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8726–8737, 2023. 2

    2023

  7. [6]

    Contactdb: Analyzing and predicting grasp contact via thermal imaging

    Samarth Brahmbhatt, Cusuh Ham, Charles C Kemp, and James Hays. Contactdb: Analyzing and predicting grasp contact via thermal imaging. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8709–8719, 2019. 3

    2019

  8. [7]

    Contactpose: A dataset of grasps with object contact and hand pose

    Samarth Brahmbhatt, Chengcheng Tang, Christopher D Twigg, Charles C Kemp, and James Hays. Contactpose: A dataset of grasps with object contact and hand pose. In European Conference on Computer Vision, pages 361–378. Springer, 2020. 3

    2020

Show all 121 references
  1. [8]

    Physically plausible full-body hand-object interaction synthesis

    Jona Braun, Sammy Christen, Muhammed Kocabas, Emre Aksan, and Otmar Hilliges. Physically plausible full-body hand-object interaction synthesis. In 2024 International Conference on 3D Vision (3DV) , pages 464–473. IEEE,

    2024

  2. [9]

    Learning to sit: Synthesizing human-chair interactions via hierarchical control

    Yu-Wei Chao, Jimei Yang, Weifeng Chen, and Jia Deng. Learning to sit: Synthesizing human-chair interactions via hierarchical control. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pages 5887–5895, 2021. 2

    2021

  3. [10]

    Detecting human-object contact in im- ages

    Yixin Chen, Sai Kumar Dwivedi, Michael J Black, and Dimitrios Tzionas. Detecting human-object contact in im- ages. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 17100–17110,

  4. [11]

    Comboverse: Compositional 3d assets creation using spatially-aware diffusion guidance

    Yongwei Chen, Tengfei Wang, Tong Wu, Xingang Pan, Kui Jia, and Ziwei Liu. Comboverse: Compositional 3d assets creation using spatially-aware diffusion guidance. In Euro- pean Conference on Computer Vision. Springer, 2024. 3

    2024

  5. [12]

    Progressive3d: Progressively local editing for text-to-3d content cre- ation with complex semantic prompts

    Xinhua Cheng, Tianyu Yang, Jianan Wang, Yu Li, Lei Zhang, Jian Zhang, and Li Yuan. Progressive3d: Progressively local editing for text-to-3d content cre- ation with complex semantic prompts. arXiv preprint arXiv:2310.11784, 2023. 3

    arXiv

  6. [13]

    D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand- Object Interactions

    Sammy Christen, Muhammed Kocabas, Emre Aksan, Jemin Hwangbo, Jie Song, and Otmar Hilliges. D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand- Object Interactions. arXiv:2112.03028 [cs], 2022. 2

    2022 arXiv

  7. [14]

    Diffh2o: Diffusion-based synthesis of hand-object interactions from textual descriptions

    Sammy Christen, Shreyas Hampali, Fadime Sener, Edoardo Remelli, Tomas Hodan, Eric Sauser, Shugao Ma, and Bugra Tekin. Diffh2o: Diffusion-based synthesis of hand-object interactions from textual descriptions. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 4

    2024

  8. [15]

    Lakshmipathy, Agniv Chatterjee, Michael J

    Alp ´ar Cseke, Shashank Tripathi, Sai Kumar Dwivedi, Ar- jun S. Lakshmipathy, Agniv Chatterjee, Michael J. Black, and Dimitrios Tzionas. Pico: Reconstructing 3d people in contact with objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2025

  9. [16]

    In- terfusion: Text-driven generation of 3d human-object in- teraction

    Sisi Dai, Wenhao Li, Haowen Sun, Haibin Huang, Chongyang Ma, Hui Huang, Kai Xu, and Ruizhen Hu. In- terfusion: Text-driven generation of 3d human-object in- teraction. In European Conference on Computer Vision . Springer, 2024. 3

    2024

  10. [17]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 5

    2021

  11. [18]

    Cg-hoi: Contact-guided 3d human-object interaction generation

    Christian Diller and Angela Dai. Cg-hoi: Contact-guided 3d human-object interaction generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19888–19901, 2024. 3, 4

    2024

  12. [19]

    Black, and Dim- itrios Tzionas

    Sai Kumar Dwivedi, Dimitrije Anti ´c, Shashank Tripathi, Omid Taheri, Cordelia Schmid, Michael J. Black, and Dim- itrios Tzionas. Interactvlm: 3d interaction reasoning from 2d foundational models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2025

  13. [20]

    Black, and Otmar 9 Hilliges

    Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J. Black, and Otmar 9 Hilliges. ARCTIC: A dataset for dexterous bimanual hand- object manipulation. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 3

    2023

  14. [21]

    Bullock, and Aaron M

    Thomas Feix, Ian M. Bullock, and Aaron M. Dollar. Anal- ysis of Human Grasping Behavior: Object Characteristics and Grasp Type. IEEE Transactions on Haptics, 7(3):311– 323, 2014. 2

    2014

  15. [22]

    Dollar, and Danica Kragic

    Thomas Feix, Javier Romero, Heinz-Bodo Schmiedmayer, Aaron M. Dollar, and Danica Kragic. The GRASP Tax- onomy of Human Grasp Types. IEEE Transactions on Human-Machine Systems, 46(1):66–77, 2016. 2

    2016

  16. [23]

    Imos: Intent- driven full-body motion synthesis for human-object inter- actions

    Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Christian Theobalt, and Philipp Slusallek. Imos: Intent- driven full-body motion synthesis for human-object inter- actions. In Computer Graphics Forum, pages 1–12. Wiley Online Library, 2023. 3

    2023

  17. [24]

    Generalized procrustes analysis

    John C Gower. Generalized procrustes analysis. Psychome- trika, 40(1):33–51, 1975. 6

    1975

  18. [25]

    Twigg, Minh V o, Samarth Brahmbhatt, and Charles C

    Patrick Grady, Chengcheng Tang, Christopher D. Twigg, Minh V o, Samarth Brahmbhatt, and Charles C. Kemp. Con- tactOpt: Optimizing Contact to Improve Grasps. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1471–1481, 2021. 3

    2021

  19. [26]

    Ac- tion2motion: Conditioned generation of 3d human motions

    Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Ac- tion2motion: Conditioned generation of 3d human motions. In Proceedings of the 28th ACM International Conference on Multimedia, pages 2021–2029, 2020. 3, 8

    2021

  20. [27]

    Human poseitioning system (hps): 3d human pose estimation and self-localization in large scenes from body-mounted sensors

    Vladimir Guzov, Aymen Mir, Torsten Sattler, and Gerard Pons-Moll. Human poseitioning system (hps): 3d human pose estimation and self-localization in large scenes from body-mounted sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , p...

    2021

  21. [28]

    Blendify–python rendering framework for blender

    Vladimir Guzov, Ilya A Petrov, and Gerard Pons-Moll. Blendify–python rendering framework for blender. arXiv preprint arXiv:2410.17858, 2024. 1

    2024 arXiv

  22. [29]

    Chorus: Learning canon- icalized 3d human-object spatial relations from unbounded synthesized images

    Sookwan Han and Hanbyul Joo. Chorus: Learning canon- icalized 3d human-object spatial relations from unbounded synthesized images. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 15835– 15846, 2023. 3

    2023

  23. [30]

    Resolving 3d human pose ambiguities with 3d scene constraints

    Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J Black. Resolving 3d human pose ambiguities with 3d scene constraints. In Proceedings of the IEEE/CVF international conference on computer vision , pages 2282– 2292, 2019. 2

    2019

  24. [31]

    Stochas- tic scene-aware motion prediction

    Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael J Black. Stochas- tic scene-aware motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11374–11384, 2021. 2

    2021

  25. [32]

    Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, and Michael J. Black. Populating 3D Scenes by Learning Human-Scene Interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14708–14718, 2021. 2, 3

    2021

  26. [33]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural informa- tion processing systems, 33:6840–6851, 2020. 3, 1

    2020

  27. [34]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 5

    2022

  28. [35]

    Capturing and inferring dense full-body human-scene contact

    Chun-Hao P Huang, Hongwei Yi, Markus H ¨oschle, Matvey Safroshkin, Tsvetelina Alexiadis, Senya Polikovsky, Daniel Scharstein, and Michael J Black. Capturing and inferring dense full-body human-scene contact. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2022

  29. [36]

    Intercap: Joint markerless 3d tracking of humans and objects in interaction

    Yinghao Huang, Omid Taheri, Michael J Black, and Dim- itrios Tzionas. Intercap: Joint markerless 3d tracking of humans and objects in interaction. In DAGM German Con- ference on Pattern Recognition , pages 281–299. Springer,

  30. [37]

    Hand-object contact consistency reasoning for hu- man grasps generation

    Hanwen Jiang, Shaowei Liu, Jiashun Wang, and Xiaolong Wang. Hand-object contact consistency reasoning for hu- man grasps generation. InProceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 11107– 11116, 2021. 2

    2021

  31. [38]

    Full-body articulated human-object interac- tion

    Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Zhiyuan Zhang, Yixin Chen, He Wang, Yixin Zhu, and Siyuan Huang. Full-body articulated human-object interac- tion. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 9365–9376, 2023. 3

    2023

  32. [39]

    Scaling up dynamic human-scene interaction mod- eling

    Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scene interaction mod- eling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1737– 1747, 2024. 2, 8

    2024

  33. [40]

    Neuralho- fusion: Neural volumetric rendering under human-object interactions

    Yuheng Jiang, Suyi Jiang, Guoxing Sun, Zhuo Su, Kai- wen Guo, Minye Wu, Jingyi Yu, and Lan Xu. Neuralho- fusion: Neural volumetric rendering under human-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6155– 6165, 2022. 3

    2022

  34. [41]

    Maximizing parallelism in the construc- tion of bvhs, octrees, and k-d trees

    Tero Karras. Maximizing parallelism in the construc- tion of bvhs, octrees, and k-d trees. In Proceedings of the Fourth ACM SIGGRAPH/Eurographics Conference on High-Performance Graphics, pages 33–37, 2012. 7

    2012

  35. [42]

    Black, Krikamol Muandet, and Siyu Tang

    Korrawe Karunratanakul, Jinlong Yang, Yan Zhang, Michael J. Black, Krikamol Muandet, and Siyu Tang. Grasping Field: Learning Implicit Representations for Hu- man Grasps. In 2020 International Conference on 3D Vi- sion (3DV), pages 333–344, 2020. 3

    2020

  36. [43]

    Beyond the contact: Discovering comprehen- sive affordance for 3d objects from pre-trained 2d diffu- sion models

    Hyeonwoo Kim, Sookwan Han, Patrick Kwon, and Han- byul Joo. Beyond the contact: Discovering comprehen- sive affordance for 3d objects from pre-trained 2d diffu- sion models. In European Conference on Computer Vision. Springer, 2024. 3

    2024

  37. [44]

    Parahome: Parameterizing everyday home activities towards 3d generative modeling of human-object interac- tions, 2024

    Jeonghwan Kim, Jisoo Kim, Jeonghyeon Na, and Hanbyul Joo. Parahome: Parameterizing everyday home activities towards 3d generative modeling of human-object interac- tions, 2024. 3, 8 10

    2024

  38. [45]

    Ncho: Unsupervised learning for neural 3d composition of hu- mans and objects

    Taeksoo Kim, Shunsuke Saito, and Hanbyul Joo. Ncho: Unsupervised learning for neural 3d composition of hu- mans and objects. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 14817– 14828, 2023. 3

    2023

  39. [46]

    Nifty: Neural object interaction fields for guided human motion synthesis

    Nilesh Kulkarni, Davis Rempe, Kyle Genova, Abhi- jit Kundu, Justin Johnson, David Fouhey, and Leonidas Guibas. Nifty: Neural object interaction fields for guided human motion synthesis. arXiv preprint arXiv:2307.07511,

    arXiv

  40. [47]

    Object motion guided human motion synthesis

    Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Transactions on Graphics (TOG), 42(6):1–11, 2023. 2, 4, 6

    2023

  41. [48]

    Controllable human-object interaction synthesis

    Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Controllable human-object interaction synthesis. In European Conference on Com- puter Vision. Springer, 2024. 2

    2024

  42. [49]

    Task-oriented human-object interactions generation with implicit neural representations

    Quanzhou Li, Jingbo Wang, Chen Change Loy, and Bo Dai. Task-oriented human-object interactions generation with implicit neural representations. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3035–3044, 2024. 2

    2024

  43. [50]

    Cliff: Carrying location information in full frames into human pose and shape estimation

    Zhihao Li, Jianzhuang Liu, Zhensong Zhang, Songcen Xu, and Youliang Yan. Cliff: Carrying location information in full frames into human pose and shape estimation. In ECCV, 2022. 8

    2022

  44. [51]

    Dextrous manipulation from a grasping pose

    C Karen Liu. Dextrous manipulation from a grasping pose. In ACM SIGGRAPH 2009 papers, pages 1–6. 2009. 2

    2009

  45. [52]

    Primitive-based 3d human-object interac- tion modelling and programming

    Siqi Liu, Yong-Lu Li, Zhou Fang, Xinpeng Liu, Yang You, and Cewu Lu. Primitive-based 3d human-object interac- tion modelling and programming. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 3711– 3719, 2024. 3

    2024

  46. [53]

    Synthesizing diverse and physically stable grasps with arbitrary hand structures using differentiable force closure estimator

    Tengyu Liu, Zeyu Liu, Ziyuan Jiao, Yixin Zhu, and Song- Chun Zhu. Synthesizing diverse and physically stable grasps with arbitrary hand structures using differentiable force closure estimator. IEEE Robotics and Automation Letters, 7(1):470–477, 2021. 2, 7

    2021

  47. [54]

    Hoi4d: A 4d egocentric dataset for category-level human-object interaction

    Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CV...

    2022

  48. [55]

    Smpl: A skinned multi-person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Ger- ard Pons-Moll, and Michael J Black. Smpl: A skinned multi-person linear model. ACM transactions on graphics (TOG), 34(6):1–16, 2015. 4

    2015

  49. [56]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 1

    2017 arXiv

  50. [57]

    Physics-aware hand-object interaction denoising

    Haowen Luo, Yunze Liu, and Li Yi. Physics-aware hand-object interaction denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2341–2350, 2024. 2

    2024

  51. [58]

    Nicp: Neural icp for 3d human registration at scale

    Riccardo Marin, Enric Corona, and Gerard Pons-Moll. Nicp: Neural icp for 3d human registration at scale. In European Conference on Computer Vision, pages 265–285. Springer, 2024. 8

    2024

  52. [59]

    Generating continual human motion in diverse 3d scenes

    Aymen Mir, Xavier Puig, Angjoo Kanazawa, and Gerard Pons-Moll. Generating continual human motion in diverse 3d scenes. arXiv preprint arXiv:2304.02061, 2023. 2

    2023 arXiv

  53. [60]

    Generative proxemics: A prior for 3d social interaction from images

    Lea M ¨uller, Vickie Ye, Georgios Pavlakos, Michael Black, and Angjoo Kanazawa. Generative proxemics: A prior for 3d social interaction from images. arXiv preprint arXiv:2306.09337, 2023. 3

    2023 arXiv

  54. [61]

    Joint reconstruction of 3d human and object via contact-based refinement transformer

    Hyeongjin Nam, Daniel Sungho Jung, Gyeongsik Moon, and Kyoung Mu Lee. Joint reconstruction of 3d human and object via contact-based refinement transformer. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE ,

  55. [62]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32,

  56. [63]

    Agora: Avatars in geography optimized for regres- sion analysis

    Priyanka Patel, Chun-Hao P Huang, Joachim Tesch, David T Hoffmann, Shashank Tripathi, and Michael J Black. Agora: Avatars in geography optimized for regres- sion analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13468– 13478, 2021. 2

    2021

  57. [64]

    Ilya A Petrov, Riccardo Marin, Julian Chibane, and Gerard Pons-Moll. Object pop-up: Can we infer 3d objects and their poses from human interactions alone? In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4726–4736, 2023. 2, 3, 6, 7

    2023

  58. [65]

    Hierarchical generation of human-object inter- actions with diffusion probabilistic models

    Huaijin Pi, Sida Peng, Minghui Yang, Xiaowei Zhou, and Hujun Bao. Hierarchical generation of human-object inter- actions with diffusion probabilistic models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15061–15073, 2023. 2

    2023

  59. [66]

    Pointnext: Revisiting pointnet++ with improved training and scaling strategies

    Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in Neural Infor- mation Processing Systems, 35:23192–23204, 2022. 4

    2022

  60. [67]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022. 3

    2022 arXiv

  61. [68]

    Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: Modeling and capturing hands and bod- ies together. ACM Transactions on Graphics, (Proc. SIG- GRAPH Asia), 36(6), 2017. 4

    2017

  62. [69]

    4deform: Neural surface deformation for robust shape interpolation

    Lu Sang, Zehranaz Canfes, Dongliang Cao, Riccardo Marin, Florian Bernard, and Daniel Cremers. 4deform: Neural surface deformation for robust shape interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6542–6551,

  63. [70]

    Twosquared: 4d generation from 2d image pairs, 2025

    Lu Sang, Zehranaz Canfes, Dongliang Cao, Riccardo Marin, Florian Bernard, and Daniel Cremers. Twosquared: 4d generation from 2d image pairs, 2025. 2

    2025

  64. [71]

    Replay: Multi- modal multi-view acted videos for casual holography

    Roman Shapovalov, Yanir Kleiman, Ignacio Rocco, David Novotny, Andrea Vedaldi, Changan Chen, Filippos Kokki- 11 nos, Ben Graham, and Natalia Neverova. Replay: Multi- modal multi-view acted videos for casual holography. In Proceedings of the IEEE/CVF International Conference on...

    2023

  65. [72]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 3

    2015

  66. [73]

    Hoianimator: Generating text-prompt human-object anima- tions using novel perceptive diffusion models

    Wenfeng Song, Xinyu Zhang, Shuai Li, Yang Gao, Aimin Hao, Xia Hou, Chenglizhao Chen, Ning Li, and Hong Qin. Hoianimator: Generating text-prompt human-object anima- tions using novel perceptive diffusion models. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and...

    2024

  67. [74]

    Black, and Dim- itrios Tzionas

    Omid Taheri, Nima Ghorbani, Michael J. Black, and Dim- itrios Tzionas. GRAB: A Dataset of Whole-Body Human Grasping of Objects. In Computer Vision – ECCV 2020 , pages 581–600. Springer International Publishing, Cham,

    2020

  68. [76]

    Grip: Generating interaction poses using spatial cues and latent consistency

    Omid Taheri, Yi Zhou, Dimitrios Tzionas, Yang Zhou, Duygu Ceylan, Soren Pirk, and Michael J Black. Grip: Generating interaction poses using spatial cues and latent consistency. In 2024 International Conference on 3D Vi- sion (3DV), pages 933–943. IEEE, 2024. 2

    2024

  69. [77]

    Flex: Full-body grasping without full-body grasps

    Purva Tendulkar, D ´ıdac Sur´ıs, and Carl V ondrick. Flex: Full-body grasping without full-body grasps. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21179–21189, 2023. 3

    2023

  70. [78]

    Deco: Dense estimation of 3d human-scene contact in the wild

    Shashank Tripathi, Agniv Chatterjee, Jean-Claude Passy, Hongwei Yi, Dimitrios Tzionas, and Michael J Black. Deco: Dense estimation of 3d human-scene contact in the wild. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 8001–8013, 2023. 2, 3, 8, 5

    2023

  71. [79]

    Capturing hands in action using discriminative salient points and physics simulation

    Dimitrios Tzionas, Luca Ballan, Abhilash Srikantha, Pablo Aponte, Marc Pollefeys, and Juergen Gall. Capturing hands in action using discriminative salient points and physics simulation. International Journal of Computer Vision, 118: 172–193, 2016. 7

    2016

  72. [80]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 5

    2017

  73. [81]

    Synthesizing long-term 3d human mo- tion and interaction in 3d scenes

    Jiashun Wang, Huazhe Xu, Jingwei Xu, Sifei Liu, and Xiaolong Wang. Synthesizing long-term 3d human mo- tion and interaction in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9401–9411, 2021. 2

    2021

  74. [82]

    Scene- aware generative network for human motion synthesis

    Jingbo Wang, Sijie Yan, Bo Dai, and Dahua Lin. Scene- aware generative network for human motion synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 12206–12215,

  75. [83]

    Dexgraspnet: A large-scale robotic dexterous grasp dataset for general ob- jects based on simulation

    Ruicheng Wang, Jialiang Zhang, Jiayi Chen, Yinzhen Xu, Puhao Li, Tengyu Liu, and He Wang. Dexgraspnet: A large-scale robotic dexterous grasp dataset for general ob- jects based on simulation. In 2023 IEEE International Con- ference on Robotics and Automation (ICRA), pages 1135...

    2023

  76. [84]

    Reconstructing action- conditioned human-object interactions using commonsense knowledge priors

    Xi Wang, Gen Li, Yen-Ling Kuo, Muhammed Kocabas, Emre Aksan, and Otmar Hilliges. Reconstructing action- conditioned human-object interactions using commonsense knowledge priors. In 2022 International Conference on 3D Vision (3DV), pages 353–362. IEEE, 2022. 3

    2022

  77. [85]

    Holistic 3d human and scene mesh estimation from single view images

    Zhenzhen Weng and Serena Yeung. Holistic 3d human and scene mesh estimation from single view images. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 334–343, 2021. 3

    2021

  78. [86]

    Saga: Stochastic whole- body grasping with contact

    Yan Wu, Jiahao Wang, Yan Zhang, Siwei Zhang, Otmar Hilliges, Fisher Yu, and Siyu Tang. Saga: Stochastic whole- body grasping with contact. InProceedings of the European Conference on Computer Vision (ECCV), 2022. 2, 3

    2022

  79. [87]

    Chore: Contact, human and object reconstruction from a single rgb image

    Xianghui Xie, Bharat Lal Bhatnagar, and Gerard Pons- Moll. Chore: Contact, human and object reconstruction from a single rgb image. In European Conference on Com- puter Vision (ECCV). Springer, 2022. 3

    2022

  80. [88]

    Visibility aware human-object interaction tracking from single rgb camera

    Xianghui Xie, Bharat Lal Bhatnagar, and Gerard Pons- Moll. Visibility aware human-object interaction tracking from single rgb camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4757–4768, 2023. 3, 6

    2023

  81. [89]

    Template free reconstruction of human- object interaction with procedural interaction generation

    Xianghui Xie, Bharat Lal Bhatnagar, Jan Eric Lenssen, and Gerard Pons-Moll. Template free reconstruction of human- object interaction with procedural interaction generation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 10003–10...

    2024

  82. [90]

    Interdiff: Generating 3d human-object interactions with physics-informed diffusion

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang-Yan Gui. Interdiff: Generating 3d human-object interactions with physics-informed diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 14928–14940, 2023. 3

    2023

  83. [91]

    Pointflow: 3d point cloud generation with continuous normalizing flows

    Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4541–4550, 2019. 6, 8

    2019

  84. [92]

    F-hoi: Toward fine-grained semantic- aligned 3d human-object interactions

    Jie Yang, Xuesong Niu, Nan Jiang, Ruimao Zhang, and Siyuan Huang. F-hoi: Toward fine-grained semantic- aligned 3d human-object interactions. In European Con- ference on Computer Vision. Springer, 2024. 3

    2024

  85. [93]

    Lemon: Learning 3d human-object in- teraction relation from 2d images

    Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, and Zheng-Jun Zha. Lemon: Learning 3d human-object in- teraction relation from 2d images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16284–16295, 2024. 3

    2024

  86. [94]

    Scene synthesis from hu- 12 man motion

    Sifan Ye, Yixing Wang, Jiaman Li, Dennis Park, C Karen Liu, Huazhe Xu, and Jiajun Wu. Scene synthesis from hu- 12 man motion. In SIGGRAPH Asia 2022 Conference Papers, pages 1–9, 2022. 3

    2022

  87. [95]

    Synthesis of detailed hand manipulations using contact sampling

    Yuting Ye and C Karen Liu. Synthesis of detailed hand manipulations using contact sampling. ACM Transactions on Graphics (ToG), 31(4):1–10, 2012. 2

    2012

  88. [96]

    Huang, Dimitrios Tzionas, Muhammed Kocabas, Mohamed Hassan, Siyu Tang, Justus Thies, and Michael J

    Hongwei Yi, Chun-Hao P. Huang, Dimitrios Tzionas, Muhammed Kocabas, Mohamed Hassan, Siyu Tang, Justus Thies, and Michael J. Black. Human-Aware Object Place- ment for Visual Environment Reconstruction. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2022

  89. [97]

    Mime: Human-aware 3d scene generation

    Hongwei Yi, Chun-Hao P Huang, Shashank Tripathi, Lea Hering, Justus Thies, and Michael J Black. Mime: Human-aware 3d scene generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12965–12976, 2023. 3

    2023

  90. [98]

    Generating human interaction motions in scenes with text control

    Hongwei Yi, Justus Thies, Michael J Black, Xue Bin Peng, and Davis Rempe. Generating human interaction motions in scenes with text control. In European Conference on Computer Vision. Springer, 2024. 2, 3

    2024

  91. [99]

    Hi4d: 4d instance seg- mentation of close human interaction

    Yifei Yin, Chen Guo, Manuel Kaufmann, Juan Jose Zarate, Jie Song, and Otmar Hilliges. Hi4d: 4d instance seg- mentation of close human interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17016–17027, 2023. 3

    2023

  92. [100]

    Manipnet: Neural manipulation synthesis with a hand-object spatial representation

    He Zhang, Yuting Ye, Takaaki Shiratori, and Taku Ko- mura. Manipnet: Neural manipulation synthesis with a hand-object spatial representation. ACM Transactions on Graphics (ToG), 40(4):1–14, 2021. 2

    2021

  93. [101]

    Neuraldome: A neural modeling pipeline on multi- view human-object interactions

    Juze Zhang, Haimin Luo, Hongdi Yang, Xinru Xu, Qianyang Wu, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Neuraldome: A neural modeling pipeline on multi- view human-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages ...

    2023

  94. [102]

    Hoi-mˆ 3: Capture multiple humans and objects in- teraction within contextual environment

    Juze Zhang, Jingyan Zhang, Zining Song, Zhanhe Shi, Chengfeng Zhao, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Hoi-mˆ 3: Capture multiple humans and objects in- teraction within contextual environment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2024

  95. [103]

    Perceiving 3d human-object spatial arrangements from a single image in the wild

    Jason Y Zhang, Sam Pepose, Hanbyul Joo, Deva Ramanan, Jitendra Malik, and Angjoo Kanazawa. Perceiving 3d human-object spatial arrangements from a single image in the wild. InEuropean conference on computer vision, pages 34–51. Springer, 2020. 3

    2020

  96. [104]

    Black, and Siyu Tang

    Siwei Zhang, Yan Zhang, Qianli Ma, Michael J. Black, and Siyu Tang. PLACE: Proximity Learning of Articulation and Contact in 3D Environments. In2020 International Confer- ence on 3D Vision (3DV), pages 642–651, 2020. 2, 3

    2020

  97. [105]

    Couch: Towards controllable human-chair interactions

    Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons-Moll. Couch: Towards controllable human-chair interactions. In European Con- ference on Computer Vision (ECCV). Springer, 2022. 2

    2022

  98. [106]

    Force: Dataset and method for intuitive physics guided human-object interac- tion

    Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Ilya Petrov, Vladimir Guzov, Helisa Dhamo, Eduardo P´erez-Pellitero, and Gerard Pons-Moll. Force: Dataset and method for intuitive physics guided human-object interac- tion. arXiv preprint arXiv:2403.11237, 2024. 2

    2024 arXiv

  99. [107]

    Generating 3d people in scenes with- out people

    Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J Black, and Siyu Tang. Generating 3d people in scenes with- out people. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6194–6204,

  100. [108]

    I’m hoi: Inertia-aware monocular capture of 3d human-object interactions

    Chengfeng Zhao, Juze Zhang, Jiashen Du, Ziwei Shan, Junye Wang, Jingyi Yu, Jingya Wang, and Lan Xu. I’m hoi: Inertia-aware monocular capture of 3d human-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 729– 741, 2024. 3

    2024

  101. [109]

    Compositional human-scene interaction synthesis with semantic control

    Kaifeng Zhao, Shaofei Wang, Yan Zhang, Thabo Beeler, , and Siyu Tang. Compositional human-scene interaction synthesis with semantic control. In European conference on computer vision (ECCV), 2022. 3, 4

    2022

  102. [110]

    Toch: Spatio-temporal object-to-hand correspondence for motion refinement

    Keyang Zhou, Bharat Lal Bhatnagar, Jan Eric Lenssen, and Gerard Pons-Moll. Toch: Spatio-temporal object-to-hand correspondence for motion refinement. In European Con- ference on Computer Vision , pages 1–19. Springer, 2022. 2

    2022

  103. [111]

    Gears: Local geometry-aware hand-object interaction synthesis

    Keyang Zhou, Bharat Lal Bhatnagar, Jan Eric Lenssen, and Gerard Pons-Moll. Gears: Local geometry-aware hand-object interaction synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20634–20643, 2024. 2

    2024

  104. [112]

    On the continuity of rotation representations in neu- ral networks

    Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neu- ral networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5745– 5753, 2019. 1 13 TriDi: Trilateral Diffusion of ...

    2019

  105. [113]

    We follow the formulation of Denoising Diffusion Probabilistic Model (DDPM) [33] to obtain a closed-form expression for zt given the original sample z0

    Background and Notation Background. We follow the formulation of Denoising Diffusion Probabilistic Model (DDPM) [33] to obtain a closed-form expression for zt given the original sample z0. Let αi = 1 − βi, ¯αt = Qt i=1 αi, and ϵ ∼ N(0, I): q(zt|z0) = N (zt; √ ¯αtz0, (1 − ¯αt)I...

  106. [114]

    <body parts> <is / are? in contact with <object class>

    Implementation details The denoising network has a total of15M parameters, and it is trained end-to-end. We use a batch size of 1024, a learn- ing rate of 1e − 4 with a cosine scheduler, and warm up the training during the first 50k steps. The parameters are optimized with Ada...

  107. [115]

    Generic templates: - <body parts> <is / are> in contact with <object class> - <object class> is in contact with <body parts> - <body parts> touch(-es) <object class> - <object class> <touches> <body parts>

  108. [116]

    Text labels

    Interaction specific templates: 2.1 Basketball template - a person is dribbling basketball 2.2 Sitting templates - <body parts> <is / are> on <object class> - a person <is / sits> on <object class> 2.3 Hands-only templates - <object class> is in <body parts> - <body parts> <ho...

  109. [117]

    This section introduces details on the user study that was used to evaluate TriDi

    Additional Evaluation User study. This section introduces details on the user study that was used to evaluate TriDi. We have designed and run a user study, asking participants to rate the quality of the generated interactions. We compared TriDi against one baseline method and ...

  110. [118]

    Since our method studies human interaction, analysis of subjects’ be- havior may be included in surveillance applications, leading to privacy issues

    Broader Impacts Our method provides an invaluable tool for general con- tent creation and supports analysis of different disciplines like behavioral sciences or ergonomic studies. Since our method studies human interaction, analysis of subjects’ be- havior may be included in s...

  111. [119]

    BEHA VE [4] captures 8 subjects interacting with 20 different objects, represented as SMPL+H meshes and global configuration, respectively

    Datasets BEHA VE. BEHA VE [4] captures 8 subjects interacting with 20 different objects, represented as SMPL+H meshes and global configuration, respectively. We downsample the 30f pstrain sequences to 10f psand consider the official 1f pstest subset. GRAB. We use the subset of...

  112. [120]

    In some cases, TriDi’s samples may miss perfect plausibility of fine grained details, especially for smaller objects

    Post-processing refinement Motivation. In some cases, TriDi’s samples may miss perfect plausibility of fine grained details, especially for smaller objects. Such behavior is naturally caused by a lack of detailed hand modeling in the majority of the train- ing data. To counter...

    2000

  113. [121]

    To evaluate our fit- ting to the target distribution, we use three metrics

    Error Metrics Quality of Generated Distribution. To evaluate our fit- ting to the target distribution, we use three metrics. The Coverage (COV)[1]: COV (Sg, Sr) = |{arg min r∈Sr D(g, r)|g ∈ Sg}| |Sr| , (22) 7 where D(g, r) is L2 distance between corresponding fea- ture vectors...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.