Pith. sign in

REVIEW 4 major objections 4 minor 134 references

InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read InterAct claims to be the largest 3D human-object interaction benchmark to date, and shows a unified model reaching state-of-the-art on six generation tasks.

desk verdict The consolidated benchmark is a real contribution, but the correction-quality gains are partly measured by its own objectives and the hand-contact indicator inherits the inaccuracies it is meant to fix. read the letter →

arxiv 2509.09555 v1 pith:NQWJHMY3 submitted 2025-09-11 cs.CV

Sirui Xu , Dongting Li , Yucheng Zhang , Xiyan Xu , Qi Long , Ziyin Wang , Yunzhi Lu , Shuchang Dong
show 4 more authors
Hezi Jiang Akshat Gupta Yu-Xiong Wang Liang-Yan Gui
This is my paper
classification cs.CV
keywords 3Dhuman-objectinteractionHOIbenchmarkmotiongenerationtext-to-interactioncontactcorrectiondataaugmentationmulti-taskdiffusionmarker-basedrepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the main bottleneck in 3D human-object interaction generation is data, not models, and presents InterAct as the largest such benchmark: 21.81 hours of standardized, text-annotated motion from seven existing datasets, expanded to 30.70 hours through a contact-preserving augmentation. It claims that a unified optimization pipeline removes common MoCap artifacts such as penetration, floating, and faulty hand poses, and that a single multi-task diffusion model trained on this data achieves state-of-the-art results across six tasks, including text-to-interaction and physics-based imitation. If true, the work gives the field a common data resource, a correction recipe, and a baseline, making interaction generation research more reproducible and more capable of handling new objects.

What carries the argument

The method rests on four pieces: a marker-based human representation chosen because contact happens on the body surface, not at joints; a contact indicator ci that gates where the optimizer enforces hand-object contact; contact-invariance augmentation, which displaces the object and re-optimizes the body so that close vertex pairs stay close (the Ealign distance-matrix loss); and a multi-task output eta giving each marker's vector to the object surface, which lets one diffusion model handle text-, action- and object-conditioned generation jointly.

What would settle it

Re-capture a subset of corrected sequences with a high-fidelity hand tracking system (e.g., magnetic or glove-based) and measure whether the corrected hand poses are actually closer to this ground truth than the raw poses; if not, or if regions the indicator labeled non-contact are revealed to be in genuine contact, the correction claim is falsified. A simpler check: train the same generation model on raw versus corrected data; if raw-trained FID or contact metrics match or beat corrected-trained ones, the correction step is not delivering its claimed benefit.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that heterogeneous, artifact-laden human-object interaction recordings can be turned into a large, clean, richly annotated resource by (1) unifying human representation with surface markers, (2) correcting contacts and hand poses through gradient-based optimization, and (3) applying the principle of contact invariance to synthesize new training sequences at no extra MoCap cost. The paper then claims that a multi-task transformer-diffusion model, given this resource, outperforms existing approaches on all six benchmark tasks, and that corrected data improve downstream imitation success from 84.4% to 90.7%.

Load-bearing premise

The correction stage decides where to enforce contact using a contact indicator computed from the hand-object distances in the very data it is correcting; the paper itself notes (Limitations, Sec. E) that large distances may be classified as no-contact and left uncorrected, so the pipeline's success depends on the indicator labeling contact correctly despite the source data being faulty.

Editorial extensions

If this is right

  • A single transformer-diffusion model, trained once on InterAct-X, covers five kinematic generation tasks plus imitation, replacing task-specific pipelines.
  • Physics-based imitation policies trained on corrected data succeed 90.7% of the time versus 84.4% on raw data, so higher-quality inputs translate into better control.
  • Training the interaction prediction model on three copies of the dataset lowers global per-marker error from 0.106 m to 0.091 m, suggesting the benchmark supports scaling-law-style improvements.
  • Marker representations reduce penetration artifacts compared with joint- or SMPL-based representations, giving other HOI methods a simple drop-in representation.
  • The benchmark's 217 objects and six tasks establish a common evaluation ground for future HOI generation work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The contact-invariance augmentation principle is stated for single human-object pairs, but the same distance-matrix alignment could be applied to multi-object scenes or human-human-object interactions, as long as the contact graph is preserved.
  • The geometric contact indicator could be replaced by a learned contact prior trained on a small, high-accuracy hand tracking dataset; if that prior corrected mislabeled contacts, the pipeline might fix the very failures it currently leaves untouched.
  • Because the interaction-aware text encoder is trained with contrastive learning on 30 hours of paired text-motion data, it may transfer to video-based HOI retrieval or captioning tasks that currently lack paired 3D data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces InterAct, a large-scale 3D human-object interaction (HOI) benchmark built by consolidating seven existing datasets (21.81 h of base data, expanded to 30.70 h via optimization-based augmentation), with unified marker-based human representations and detailed text/action annotations. It also proposes a unified multi-task transformer/diffusion framework covering six HOI generation tasks (text-to-interaction, action-to-interaction, object-to-human, human-to-object, interaction prediction, and interaction imitation). A three-stage optimization pipeline is used for full-body correction, hand correction, and interaction augmentation. Experiments report improved contact/penetration statistics, state-of-the-art results on language- and action-conditioned tasks, scaling benefits for interaction prediction, and an improved physics-based imitation success rate.

Significance. If the data-quality and augmentation claims hold, InterAct would be a substantial community resource: it standardizes heterogeneous HOI data, supplies text annotations at scale, defines six benchmarking tasks, and releases data and models. The marker-based representation and the multi-task formulation with contact prediction are useful methodological contributions. The strongest independent evidence is the physics-based imitation success gain (90.7% vs. 84.4% on corrected vs. raw data), which suggests the correction helps global control. However, the central data-quality claim rests partly on metrics that coincide with the optimization objectives, so the benchmark's foundational value needs additional independent validation.

major comments (4)
  1. [Sec. 5.1, Table 2] The two quantitative quality metrics reported in Table 2—penetration depth and contact ratio—are exactly the quantities minimized and maximized by the optimization losses E_pene and E_cont (Sec. 3.2, Supplementary Eqs. 1 and 4). Reporting improvements on these metrics after optimization is partly a check that the optimizer did what it was asked, not independent evidence of physical correctness. The user study uses only 30 tuples and only the BEHAVE and OMOMO subsets, so it is too small to establish per-contact localization across the seven datasets. I recommend independent validation: manual contact labels on a held-out subset, comparison against a separate capture modality, or a downstream metric not used in the correction objective.
  2. [Sec. 3.2, Supplementary Eq. (4)] The contact indicator c_i is a piecewise function of min_j d_j[i], the hand-object distance computed from the very poses being corrected. If the initial reconstruction places a truly touching hand more than epsilon_2 = 0.10 m away, c_i = 0 and no contact-promotion gradient is applied; conversely, a noisy near-but-not-touching hand can be pushed into the object. The paper's own limitation section (Sec. E) concedes that large distances may be identified as no contact and remain uncorrected. Since the motivation for correction is precisely that source hand poses are inaccurate (BEHAVE, OMOMO), this circularity weakens the central data-quality claim. Please validate the indicator against an independent source, e.g., manual contact annotation or a separate hand tracker.
  3. [Sec. 5.2, Tables 3 and 4] The FID and MM-Dist feature encoder used for evaluation is trained on InterAct corrected data and then used to score models trained on the same corrected data. This is acceptable for ranking design variants within the paper, but it cannot fully support the absolute 'state-of-the-art' claim or the data-quality claim, because the encoder may have learned the specific correction statistics and artifacts of InterAct. Please report results with a fixed feature extractor not trained on the corrected data, or at least demonstrate that the relative FID ranking is stable across different feature extractors.
  4. [Sec. 3.2, Interaction Augmentation] The 'contact invariance' principle is an assumed axiom: displaced object trajectories with re-optimized human motion are asserted to preserve interaction semantics. The filtering step removes some low-quality augmentations, but no metric directly demonstrates semantic equivalence for the retained augmented data. The physical imitation success gain is useful but is reported on only four sequences and does not isolate augmentation quality. Please provide an ablation that trains on corrected data with and without augmentation, or a per-sample semantic-preservation evaluation (automatic or human).
minor comments (4)
  1. [Table 2] The table formatting is very hard to read: the check marks and numeric columns are run together (e.g., '× ×0.017 0.048 22.3'). Please reformat with clear column separators so each row unambiguously shows the configuration and corresponding metric values.
  2. [Supplementary Sec. B.2] The sentence 'We only select sequences that contain rigid objects' appears twice verbatim in the Chairs processing paragraph. Please remove the duplicate.
  3. [Supplementary Sec. C.6] Typo: 'Addtional Implementation Details' should be 'Additional Implementation Details'.
  4. [Abstract and Sec. 1] The claim of being 'the most extensive 3D HOI benchmark' is based on 21.81 h of base data, with 30.70 h including synthetic augmentation. Please state the base versus augmented split clearly in the abstract or contribution list to avoid overstating the captured data volume.

Circularity Check

2 steps flagged · score 6.0 of 10

The benchmark's central data-quality claim is partly self-referential: the correction pipeline optimizes the same penetration and contact quantities used as quality metrics, and its contact indicator is computed from the very hand-object distances the correction is meant to fix.

  1. self definitional [Sec. 3.2 (Hand Correction) and Supplementary Eq. 4 (B.3); Sec. 5.1 metrics]
    "Econt = Σ_i c_i Σ_j d_j[i] ... The contact indicator ci, inferred from ground truth data, is a function based on hand-object distance minj dj[i], which we provide details in supplementary. ... ci = 1 if minj dj[i] ≤ ε; 0 if minj dj[i] > ε2 ... where minj dj[i] refers to hand-object chamfer distance"

    The contact indicator that decides where contact is promoted is computed from the same hand-object chamfer distances that the correction stage is supposed to repair. If a truly contacting hand is reconstructed more than ε2=10cm away, c_i=0 and the loss never pulls it into contact; the paper's own Sec. E concedes this. The Contact Ratio metric used in Table 2 counts vertices whose distance to the object is under a threshold, so minimizing Econt for the c_i>0 regions directly increases the reported metric. The claimed correction gain is therefore partly the optimization objective itself, not an independent measurement of physical fidelity.

  2. self definitional [Sec. 5.1 (metrics) vs Supplementary Eq. 1 and Eq. 5 (Full-Body Correction)]
    "Penetration refers to the intersection depth – maximum of negative sign distances from human vertices to the object's surface – average across the sequence. ... Epene = −Σ_i Σ_do min(sdf_i(v_o^i[k]),0). ... E = λpeneEpene + λsmoothEsmooth + λrecErec"

    The primary quality metric used to demonstrate correction success is the same function that the optimizer minimizes. Full-body correction takes E with a penetration term Epene, and Table 2 then reports 'Pene (m)' as evidence that the correction improves quality. Reducing the optimization objective reduces the reported penetration metric by construction (modulo reconstruction and smoothness tradeoffs). Thus the quantitative comparison does not independently validate the physical accuracy of the corrected data; the evaluation criterion is built into the correction objective.

full rationale

InterAct's scale claim and the multi-task generation results are not circular: they are measured against external baselines (OMOMO, HOI-Diff, InterDiff, PhysHOI) with standard metrics, and the physics-based imitation success (90.7% vs 84.4%) plus the user study provide partial independent support. However, the paper's core second contribution—that the optimization framework enhances data quality—rests on Table 2, whose two quantitative metrics (penetration and contact ratio) are, by construction, the same quantities minimized and maximized by the correction losses. The contact indicator in Eq. 4 also uses the erroneous hand-object distances as its own ground truth, and Sec. E admits that large-distance artifacts are therefore uncorrected. This is not a full 8-10 circularity because there are non-self-referential signals (imitation, user preference), but the foundational data-quality evidence is partly self-validating, so a score of 6 is appropriate.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-chosen thresholds and weights, on the assumption that marker correspondences transfer across body models within 1 cm, on contact invariance as a validity principle for augmentation, and on the reliability of the contact indicator computed from the very data being corrected. No new physical entities are introduced.

free parameters (5)
  • contact distance threshold epsilon = 0.02 m
    Used to define contact indicator c_i in hand correction; chosen following [4], but central to which contact is enforced.
  • non-contact threshold epsilon_2 = 0.10 m
    Used in contact indicator linear interpolation; hand-chosen.
  • beta (regularization weight in augmentation) = 5
    Weights the non-interaction mask in augmentation regularization.
  • guidance weights tau1, tau2 = 0.1, 0.2
    Control gradient update magnitude in classifier guidance; chosen by hand.
  • object motion loss weight = 0.9 relative to human loss
    Chosen for text-to-interaction training; any value changes balance between human and object.
assumptions (4)
  • domain assumption Marker correspondences between SMPL-H and SMPL-X are consistent within 1 cm across poses.
    The unified marker representation (Sec 3.1) depends on this; the paper states it was 'extensively evaluated' but provides no data.
  • ad hoc to paper Contact invariance: displaced object trajectories with re-optimized human motion preserve interaction semantics.
    Introduced in Sec 3.2 to justify augmentation; no independent evidence that semantics are preserved.
  • domain assumption The optimization framework can recover natural hand poses even when the source hand poses are inaccurate.
    Hand correction (Sec 3.2) relies on the contact indicator computed from the same inaccurate hand poses.
  • domain assumption GPT-4 generated text and action labels are correct and aligned with the motion.
    The annotation pipeline (Sec 3.1) uses GPT-4 for rewriting and labeling; no quantitative evaluation of label accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation." pith.science (2026). https://pith.science/paper/NQWJHMY3

@misc{pith2026250909555,
  author       = {Pith},
  title        = {Pith review of: InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NQWJHMY3}},
  note         = {Machine review of arXiv:2509.09555}
}
read the original abstract

While large-scale human motion capture datasets have advanced human motion generation, modeling and generating dynamic 3D human-object interactions (HOIs) remain challenging due to dataset limitations. Existing datasets often lack extensive, high-quality motion and annotation and exhibit artifacts such as contact penetration, floating, and incorrect hand motions. To address these issues, we introduce InterAct, a large-scale 3D HOI benchmark featuring dataset and methodological advancements. First, we consolidate and standardize 21.81 hours of HOI data from diverse sources, enriching it with detailed textual annotations. Second, we propose a unified optimization framework to enhance data quality by reducing artifacts and correcting hand motions. Leveraging the principle of contact invariance, we maintain human-object relationships while introducing motion variations, expanding the dataset to 30.70 hours. Third, we define six benchmarking tasks and develop a unified HOI generative modeling perspective, achieving state-of-the-art performance. Extensive experiments validate the utility of our dataset as a foundational resource for advancing 3D human-object interaction generation. To support continued research in this area, the dataset is publicly available at https://github.com/wzyabcas/InterAct, and will be actively maintained.

Figures

Figures reproduced from arXiv: 2509.09555 by the authors.

Figure 1
Figure 1. An overview of InterAct, our large-scale 3D human-object interaction (HOI) benchmark, covering six HOI generation tasks. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Our data processing pipeline consolidating data, [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Marker-based representation for human. icantly surpasses existing datasets, as summarized in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative evaluation of interaction correction (bottom) on the OMOMO [38] dataset shows hand recovery compared to the ground truth interaction (top). Zoom in to see details of the hand recovery. Dataset Correction Augmentation Pene (m)↓ Cont Ratio User Study (%) BEHA…
Figure 5
Figure 5. Figure 5: Qualitative evaluation of interaction augmentation (bottom) shows high-quality synthetic data varied from original (top). HOI-Aware Object Enc. HOI-Aware Text Enc. Contact Generation Contact Guidance R-Precision↑ FID↓ MM Dist↓ Multimodality↑ Diversity→ Top 1 Top 2 Top …
Figure 6
Figure 6. Figure 6: Qualitative results demonstrate the successful imitation [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

134 extracted references · 35 linked inside Pith

  1. [1]

    http:// mocap.cs.cmu.edu/

    CMU graphics lab motion capture database. http:// mocap.cs.cmu.edu/. 2

  2. [2]

    Easymocap - make human motion capture easier. Github,

  3. [3]

    Pmp: Learning to physically interact with environments using part-wise motion priors

    Jinseok Bae, Jungdam Won, Donggeun Lim, Cheol-Hui Min, and Young Min Kim. Pmp: Learning to physically interact with environments using part-wise motion priors. In SIGGRAPH, 2023. 3, 5

    2023

  4. [4]

    BEHA VE: Dataset and method for tracking human object interactions

    Bharat Lal Bhatnagar, Xianghui Xie, Ilya Petrov, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. BEHA VE: Dataset and method for tracking human object interactions. InCVPR, 2022. 2, 3, 4, 6, 7, 1

    2022

  5. [5]

    Bedlam: A synthetic dataset of bodies exhibiting detailed lifelike animated motion

    Michael J Black, Priyanka Patel, Joachim Tesch, and Jinlong Yang. Bedlam: A synthetic dataset of bodies exhibiting detailed lifelike animated motion. InCVPR, 2023. 5

    2023

  6. [6]

    Physically plausible full- body hand-object interaction synthesis.arXiv preprint arXiv:2309.07907, 2023

    Jona Braun, Sammy Christen, Muhammed Kocabas, Emre Aksan, and Otmar Hilliges. Physically plausible full- body hand-object interaction synthesis.arXiv preprint arXiv:2309.07907, 2023. 3

    arXiv 2023

  7. [7]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. InNeurIPS, 2020. 3

    2020

  8. [8]

    Long-term human motion prediction with scene context

    Zhe Cao, Hang Gao, Karttikeya Mangalam, Qi-Zhi Cai, Minh V o, and Jitendra Malik. Long-term human motion prediction with scene context. InECCV, 2020. 2

    2020

Show all 134 references
  1. [9]

    Text2hoi: Text-guided 3d motion generation for hand- object interaction

    Junuk Cha, Jihyeon Kim, Jae Shin Yoon, and Seungryul Baek. Text2hoi: Text-guided 3d motion generation for hand- object interaction. InCVPR, 2024. 3

    2024

  2. [10]

    Learning to sit: Synthesizing human-chair interactions via hierarchical control

    Yu-Wei Chao, Jimei Yang, Weifeng Chen, and Jia Deng. Learning to sit: Synthesizing human-chair interactions via hierarchical control. InAAAI, 2021. 3

    2021

  3. [11]

    Dexycb: A benchmark for capturing hand grasping of objects

    Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, et al. Dexycb: A benchmark for capturing hand grasping of objects. InCVPR,

  4. [12]

    Allava: Harnessing gpt4v-synthesized data for a lite vision-language model

    Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. Allava: Harnessing gpt4v-synthesized data for a lite vision-language model. arXiv preprint arXiv:2402.11684, 2024. 5

    2024 arXiv

  5. [13]

    Diffh2o: Diffusion-based synthesis of hand- object interactions from textual descriptions.arXiv preprint arXiv:2403.17827, 2024

    Sammy Christen, Shreyas Hampali, Fadime Sener, Edoardo Remelli, Tomas Hodan, Eric Sauser, Shugao Ma, and Bu- gra Tekin. Diffh2o: Diffusion-based synthesis of hand- object interactions from textual descriptions.arXiv preprint arXiv:2403.17827, 2024. 3

    2024 arXiv

  6. [14]

    Context-aware human motion prediction

    Enric Corona, Albert Pumarola, Guillem Alenya, and Francesc Moreno-Noguer. Context-aware human motion prediction. InCVPR, 2020. 3

    2020

  7. [15]

    AnySkill: Learning open- vocabulary physical skill for interactive agents

    Jieming Cui, Tengyu Liu, Nian Liu, Yaodong Yang, Yixin Zhu, and Siyuan Huang. AnySkill: Learning open- vocabulary physical skill for interactive agents. InCVPR,

  8. [16]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. InNeurIPS, 2021. 6, 7

    2021

  9. [17]

    Exploiting cyclic symmetry in convolutional neural net- works

    Sander Dieleman, Jeffrey De Fauw, and Koray Kavukcuoglu. Exploiting cyclic symmetry in convolutional neural net- works. InICML, 2016. 5

    2016

  10. [18]

    CG-HOI: Contact-guided 3d human-object interaction generation

    Christian Diller and Angela Dai. CG-HOI: Contact-guided 3d human-object interaction generation. InCVPR, 2024. 3, 5, 6

    2024

  11. [19]

    Scaling laws of synthetic images for model training

    Lijie Fan, Kaifeng Chen, Dilip Krishnan, Dina Katabi, Phillip Isola, and Yonglong Tian. Scaling laws of synthetic images for model training... for now. InCVPR, 2024. 5

    2024

  12. [20]

    Black, and Otmar Hilliges

    Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J. Black, and Otmar Hilliges. ARCTIC: A dataset for dexterous bimanual hand- object manipulation. InCVPR, 2023. 3

    2023

  13. [21]

    IMoS: Intent-driven full-body motion synthesis for human-object interactions

    Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Chris- tian Theobalt, and Philipp Slusallek. IMoS: Intent-driven full-body motion synthesis for human-object interactions. arXiv preprint arXiv:2212.07555, 2022. 3

    2022 arXiv

  14. [22]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. InCVPR, 2022. 1, 4, 6

    2022

  15. [23]

    Honnotate: A method for 3d annotation of hand and object poses

    Shreyas Hampali, Mahdi Rad, Markus Oberweger, and Vin- cent Lepetit. Honnotate: A method for 3d annotation of hand and object poses. InCVPR, 2020. 2

    2020

  16. [24]

    Resolving 3d human pose ambiguities with 3d scene constraints

    Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J Black. Resolving 3d human pose ambiguities with 3d scene constraints. InICCV, 2019. 2

    2019

  17. [25]

    Stochastic scene- aware motion prediction

    Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael Black. Stochastic scene- aware motion prediction. InICCV, 2021. 2

    2021

  18. [26]

    Synthesizing physi- cal character-scene interactions

    Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, and Xue Bin Peng. Synthesizing physi- cal character-scene interactions. InSIGGRAPH, 2023. 3

    2023

  19. [27]

    Composi- tional 3d human-object neural animation.arXiv preprint arXiv:2304.14070, 2023

    Zhi Hou, Baosheng Yu, and Dacheng Tao. Composi- tional 3d human-object neural animation.arXiv preprint arXiv:2304.14070, 2023. 3

    2023 arXiv

  20. [28]

    Black, and Dim- itrios Tzionas

    Yinghao Huang, Omid Taheri, Michael J. Black, and Dim- itrios Tzionas. InterCap: Joint markerless 3D tracking of humans and objects in interaction. InGCPR, 2022. 2, 3, 1, 4

    2022

  21. [29]

    CHAIRS: Towards full-body articulated human-object interaction

    Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Yixin Chen, He Wang, Yixin Zhu, and Siyuan Huang. CHAIRS: Towards full-body articulated human-object interaction. In ICCV, 2023. 2, 3, 1, 4

    2023

  22. [30]

    Scaling up dynamic human-scene interaction mod- eling

    Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scene interaction mod- eling. InCVPR, 2024. 3

    2024

  23. [31]

    From skin to skeleton: Towards biomechanically accurate 3d digital humans.ACM Transactions on Graphics (TOG), 42(6):1–12,

    Marilyn Keller, Keenon Werling, Soyong Shin, Scott Delp, Sergi Pujades, C Karen Liu, and Michael J Black. From skin to skeleton: Towards biomechanically accurate 3d digital humans.ACM Transactions on Graphics (TOG), 42(6):1–12,

  24. [32]

    ParaHome: Parameterizing everyday home activities to- wards 3d generative modeling of human-object interactions

    Jeonghwan Kim, Jisoo Kim, Jeonghyeon Na, and Hanbyul Joo. ParaHome: Parameterizing everyday home activities to- wards 3d generative modeling of human-object interactions. arXiv preprint arXiv:2401.10232, 2024. 3

    2024 arXiv

  25. [33]

    NCHO: Unsupervised learning for neural 3d composition of humans and objects

    Taeksoo Kim, Shunsuke Saito, and Hanbyul Joo. NCHO: Unsupervised learning for neural 3d composition of humans and objects. InICCV, 2023. 3

    2023

  26. [34]

    The kit bimanual manipulation dataset

    Franziska Krebs, Andre Meixner, Isabel Patzer, and Tamim Asfour. The kit bimanual manipulation dataset. InHu- manoids, 2021. 3

    2021

  27. [35]

    NIFTY: Neural object interaction fields for guided human motion synthesis.arXiv preprint arXiv:2307.07511, 2023

    Nilesh Kulkarni, Davis Rempe, Kyle Genova, Abhijit Kundu, Justin Johnson, David Fouhey, and Leonidas Guibas. NIFTY: Neural object interaction fields for guided human motion synthesis.arXiv preprint arXiv:2307.07511, 2023

    2023 arXiv

  28. [36]

    Locomotion-Action- Manipulation: Synthesizing human-scene interactions in complex 3d environments

    Jiye Lee and Hanbyul Joo. Locomotion-Action- Manipulation: Synthesizing human-scene interactions in complex 3d environments. InICCV, 2023. 3

    2023

  29. [37]

    Controllable human-object in- teraction synthesis.arXiv preprint arXiv:2312.03913, 2023

    Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Controllable human-object in- teraction synthesis.arXiv preprint arXiv:2312.03913, 2023. 3, 5, 6

    2023 arXiv

  30. [38]

    Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG), 42(6):1–11, 2023

    Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG), 42(6):1–11, 2023. 2, 3, 4, 5, 6, 7, 8

    2023

  31. [39]

    Task-oriented human-object interactions genera- tion with implicit neural representations.arXiv preprint arXiv:2303.13129, 2023

    Quanzhou Li, Jingbo Wang, Chen Change Loy, and Bo Dai. Task-oriented human-object interactions genera- tion with implicit neural representations.arXiv preprint arXiv:2303.13129, 2023. 3

    2023 arXiv

  32. [40]

    InterGen: Diffusion-based multi-human motion generation under complex interactions.arXiv preprint arXiv:2304.05684, 2023

    Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. InterGen: Diffusion-based multi-human motion generation under complex interactions.arXiv preprint arXiv:2304.05684, 2023. 2

    2023 arXiv

  33. [41]

    Motion-X: A large- scale 3d expressive whole-body human motion dataset

    Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-X: A large- scale 3d expressive whole-body human motion dataset. In NeurIPS, 2023. 1

    2023

  34. [42]

    Learning basketball drib- bling skills using trajectory optimization and deep reinforce- ment learning.ACM Transactions on Graphics (TOG), 37 (4):1–14, 2018

    Libin Liu and Jessica Hodgins. Learning basketball drib- bling skills using trajectory optimization and deep reinforce- ment learning.ACM Transactions on Graphics (TOG), 37 (4):1–14, 2018. 3

    2018

  35. [43]

    Contactgen: Generative contact modeling for grasp generation

    Shaowei Liu, Yang Zhou, Jimei Yang, Saurabh Gupta, and Shenlong Wang. Contactgen: Generative contact modeling for grasp generation. InICCV, 2023. 3

    2023

  36. [44]

    Hoi4d: A 4d egocentric dataset for category-level human- object interaction

    Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human- object interaction. InCVPR, 2022. 2

    2022

  37. [45]

    SMPL: A skinned multi- person linear model.ACM transactions on graphics, 2015

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. SMPL: A skinned multi- person linear model.ACM transactions on graphics, 2015. 3, 5

    2015

  38. [46]

    Hu- mantomato: Text-aligned whole-body motion generation

    Shunlin Lu, Ling-Hao Chen, Ailing Zeng, Jing Lin, Ruimao Zhang, Lei Zhang, and Heung-Yeung Shum. Hu- mantomato: Text-aligned whole-body motion generation. arxiv:2310.12978, 2023. 6

    2023 arXiv

  39. [47]

    Smplolympics: Sports environ- ments for physically simulated humanoids.arXiv preprint arXiv:2407.00187, 2024

    Zhengyi Luo, Jiashun Wang, Kangni Liu, Haotian Zhang, Chen Tessler, Jingbo Wang, Ye Yuan, Jinkun Cao, Zihui Lin, Fengyi Wang, et al. Smplolympics: Sports environ- ments for physically simulated humanoids.arXiv preprint arXiv:2407.00187, 2024. 3

    2024 arXiv

  40. [48]

    Himo: A new benchmark for full-body human interacting with multiple objects

    Xintao Lv, Liang Xu, Yichao Yan, Xin Jin, Congsheng Xu, Shuwen Wu, Yifan Liu, Lincheng Li, Mengxiao Bi, Wenjun Zeng, et al. Himo: A new benchmark for full-body human interacting with multiple objects. InECCV, 2025. 2

    2025

  41. [49]

    Diff-ip2d: Diffusion-based hand-object interaction predic- tion on egocentric videos.arXiv preprint arXiv:2405.04370,

    Junyi Ma, Jingyi Xu, Xieyuanli Chen, and Hesheng Wang. Diff-ip2d: Diffusion-based hand-object interaction predic- tion on egocentric videos.arXiv preprint arXiv:2405.04370,

  42. [50]

    AMASS: Archive of motion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. AMASS: Archive of motion capture as surface shapes. InICCV, 2019. 1

    2019

  43. [51]

    Isaac gym: High performance gpu-based physics simulation for robot learning.arXiv preprint arXiv:2108.10470, 2021

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning.arXiv preprint arXiv:2108.10470, 2021. 8

    2021 arXiv

  44. [52]

    The kit whole-body human motion database

    Christian Mandery, ¨Omer Terlemez, Martin Do, Nikolaus Vahrenkamp, and Tamim Asfour. The kit whole-body human motion database. InICAR, 2015. 3

    2015

  45. [53]

    Unifying representations and large-scale whole-body motion databases for studying human motion.IEEE Transactions on Robotics, 32(4):796– 809, 2016

    Christian Mandery, ¨Omer Terlemez, Martin Do, Nikolaus Vahrenkamp, and Tamim Asfour. Unifying representations and large-scale whole-body motion databases for studying human motion.IEEE Transactions on Robotics, 32(4):796– 809, 2016. 3

    2016

  46. [54]

    Single-shot multi-person 3D pose estimation from monocular RGB

    Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Srinath Sridhar, Gerard Pons-Moll, and Christian Theobalt. Single-shot multi-person 3D pose estimation from monocular RGB. In3DV, 2018. 2

    2018

  47. [55]

    Catch & carry: reusable neu- ral controllers for vision-guided whole-body tasks.ACM Transactions on Graphics (TOG), 39(4):39–1, 2020

    Josh Merel, Saran Tunyasuvunakool, Arun Ahuja, Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne, and Nicolas Heess. Catch & carry: reusable neu- ral controllers for vision-guided whole-body tasks.ACM Transactions on Graphics (TOG), 39(4):39–1, 2020. 3

    2020

  48. [56]

    Interhand2

    Gyeongsik Moon, Shoou-I Yu, He Wen, Takaaki Shiratori, and Kyoung Mu Lee. Interhand2. 6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image. InECCV, 2020. 2

    2020

  49. [57]

    Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic segmentation

    Quang Nguyen, Truong Vu, Anh Tran, and Khoi Nguyen. Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic segmentation. InNeurIPS, 2024. 5

    2024

  50. [58]

    Assemblyhands: Towards ego- centric activity understanding via 3d hand pose estimation

    Takehiko Ohkawa, Kun He, Fadime Sener, Tomas Hodan, Luan Tran, and Cem Keskin. Assemblyhands: Towards ego- centric activity understanding via 3d hand pose estimation. InCVPR, 2023. 2

    2023

  51. [59]

    Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018. 6

    2018 arXiv

  52. [60]

    OpenAI. ChatGPT. https://chat.openai.com/ ,

  53. [61]

    Synthesizing physically plausible human motions in 3d scenes.arXiv preprint arXiv:2308.09036, 2023

    Liang Pan, Jingbo Wang, Buzhen Huang, Junyu Zhang, Haofan Wang, Xu Tang, and Yangang Wang. Synthesizing physically plausible human motions in 3d scenes.arXiv preprint arXiv:2308.09036, 2023. 3

    2023 arXiv

  54. [62]

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. InCVPR, 2019. 3, 4, 1, 2

    2019

  55. [63]

    HOI-Diff: Text-driven syn- thesis of 3d human-object interactions using diffusion mod- els.arXiv preprint arXiv:2312.06553, 2023

    Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, and Huaizu Jiang. HOI-Diff: Text-driven syn- thesis of 3d human-object interactions using diffusion mod- els.arXiv preprint arXiv:2312.06553, 2023. 2, 3, 5, 6, 7

    2023 arXiv

  56. [64]

    Object pop-up: Can we infer 3d objects and their poses from human interactions alone? InCVPR, 2023

    Ilya A Petrov, Riccardo Marin, Julian Chibane, and Gerard Pons-Moll. Object pop-up: Can we infer 3d objects and their poses from human interactions alone? InCVPR, 2023. 3

    2023

  57. [65]

    Black, and G¨ul Varol

    Mathis Petrovich, Michael J. Black, and G¨ul Varol. TEMOS: Generating diverse human motions from textual descriptions. InECCV, 2022. 4

    2022

  58. [66]

    TMR: Text-to-motion retrieval using contrastive 3d human motion synthesis

    Mathis Petrovich, Michael J Black, and G ¨ul Varol. TMR: Text-to-motion retrieval using contrastive 3d human motion synthesis. InICCV, 2023. 6

    2023

  59. [67]

    The kit motion-language dataset.Big data, 4(4):236–252,

    Matthias Plappert, Christian Mandery, and Tamim Asfour. The kit motion-language dataset.Big data, 4(4):236–252,

  60. [68]

    Efficient learning on point clouds with basis point sets

    Sergey Prokudin, Christoph Lassner, and Javier Romero. Efficient learning on point clouds with basis point sets. In ICCV, 2019. 5, 7

    2019

  61. [69]

    Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, and Michael J

    Abhinanda R. Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, and Michael J. Black. BABEL: Bodies, action and behavior with english labels. InCVPR, 2021. 1

    2021

  62. [70]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. InNeurIPS, 2017. 7

    2017

  63. [71]

    Action-conditioned gen- eration of bimanual object manipulation sequences

    Haziq Razali and Yiannis Demiris. Action-conditioned gen- eration of bimanual object manipulation sequences. InAAAI,

  64. [72]

    Sentence-bert: Sentence embeddings using siamese bert-networks.arXiv preprint arXiv:1908.10084, 2019

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks.arXiv preprint arXiv:1908.10084, 2019. 6

    1908 arXiv

  65. [73]

    Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: Modeling and capturing hands and bodies together.ACM Transactions on Graphics, 36(6), 2017. 3, 4, 1, 2

    2017

  66. [74]

    Hoianimator: Generating text-prompt human-object anima- tions using novel perceptive diffusion models

    Wenfeng Song, Xinyu Zhang, Shuai Li, Yang Gao, Aimin Hao, Xia Hou, Chenglizhao Chen, Ning Li, and Hong Qin. Hoianimator: Generating text-prompt human-object anima- tions using novel perceptive diffusion models. InCVPR,

  67. [75]

    Neural state machine for character-scene interactions.ACM Trans

    Sebastian Starke, He Zhang, Taku Komura, and Jun Saito. Neural state machine for character-scene interactions.ACM Trans. Graph., 38(6):209–1, 2019. 3

    2019

  68. [76]

    Local motion phases for learning multi-contact char- acter movements.ACM Transactions on Graphics (TOG), 39(4):54–1, 2020

    Sebastian Starke, Yiwei Zhao, Taku Komura, and Kazi Za- man. Local motion phases for learning multi-contact char- acter movements.ACM Transactions on Graphics (TOG), 39(4):54–1, 2020. 3

    2020

  69. [77]

    GRAB: A dataset of whole-body human grasping of objects

    Omid Taheri, Nima Ghorbani, Michael J Black, and Dim- itrios Tzionas. GRAB: A dataset of whole-body human grasping of objects. InECCV, 2020. 2, 3, 1, 4

    2020

  70. [78]

    GOAL: Generating 4d whole-body motion for hand-object grasping

    Omid Taheri, Vasileios Choutas, Michael J Black, and Dim- itrios Tzionas. GOAL: Generating 4d whole-body motion for hand-object grasping. InCVPR, 2022. 3

    2022

  71. [79]

    Grip: Generating interaction poses using spatial cues and latent consistency

    Omid Taheri, Yi Zhou, Dimitrios Tzionas, Yang Zhou, Duygu Ceylan, Soren Pirk, and Michael J Black. Grip: Generating interaction poses using spatial cues and latent consistency. In3DV, 2024. 4

    2024

  72. [80]

    Flex: Full-body grasping without full-body grasps

    Purva Tendulkar, D´ıdac Sur´ıs, and Carl V ondrick. Flex: Full-body grasping without full-body grasps. InICCV, 2023. 2

    2023

  73. [81]

    Maskedmimic: Unified physics-based charac- ter control through masked motion inpainting.arXiv preprint arXiv:2409.14393, 2024

    Chen Tessler, Yunrong Guo, Ofir Nabati, Gal Chechik, and Xue Bin Peng. Maskedmimic: Unified physics-based charac- ter control through masked motion inpainting.arXiv preprint arXiv:2409.14393, 2024. 3

    2024 arXiv

  74. [82]

    Human motion diffusion model.arXiv preprint arXiv:2209.14916, 2022

    Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H Bermano. Human motion diffusion model.arXiv preprint arXiv:2209.14916, 2022. 4, 6

    2022 arXiv

  75. [83]

    Closd: Closing the loop between simulation and diffusion for multi-task character control.arXiv preprint arXiv:2410.03441, 2024

    Guy Tevet, Sigal Raab, Setareh Cohan, Daniele Reda, Zhengyi Luo, Xue Bin Peng, Amit H Bermano, and Michiel van de Panne. Closd: Closing the loop between simulation and diffusion for multi-task character control.arXiv preprint arXiv:2410.03441, 2024. 3

    2024 arXiv

  76. [84]

    Gaze-guided hand-object in- teraction synthesis: Benchmark and method.arXiv preprint arXiv:2403.16169, 2024

    Jie Tian, Lingxiao Yang, Ran Ji, Yuexin Ma, Lan Xu, Jingyi Yu, Ye Shi, and Jingya Wang. Gaze-guided hand-object in- teraction synthesis: Benchmark and method.arXiv preprint arXiv:2403.16169, 2024. 3

    2024

  77. [85]

    Learn to predict how hu- mans manipulate large-sized objects from interactive mo- tions.IEEE Robotics and Automation Letters, 2022

    Weilin Wan, Lei Yang, Lingjie Liu, Zhuoying Zhang, Ruix- ing Jia, Yi-King Choi, Jia Pan, Christian Theobalt, Taku Komura, and Wenping Wang. Learn to predict how hu- mans manipulate large-sized objects from interactive mo- tions.IEEE Robotics and Automation Letters, 2022. 3

    2022

  78. [86]

    Strategy and skill learning for physics-based table tennis animation

    Jiashun Wang, Jessica Hodgins, and Jungdam Won. Strategy and skill learning for physics-based table tennis animation. InSIGGRAPH, 2024. 3

    2024

  79. [87]

    Reconstructing action- conditioned human-object interactions using commonsense knowledge priors

    Xi Wang, Gen Li, Yen-Ling Kuo, Muhammed Kocabas, Emre Aksan, and Otmar Hilliges. Reconstructing action- conditioned human-object interactions using commonsense knowledge priors. In3DV, 2022. 3

    2022

  80. [88]

    PhysHOI: Physics-based imita- tion of dynamic human-object interaction.arXiv preprint arXiv:2312.04393, 2023

    Yinhuai Wang, Jing Lin, Ailing Zeng, Zhengyi Luo, Jian Zhang, and Lei Zhang. PhysHOI: Physics-based imita- tion of dynamic human-object interaction.arXiv preprint arXiv:2312.04393, 2023. 3, 5, 8

    2023 arXiv

  81. [89]

    Skillmimic: Learning reusable basketball skills from demonstrations.arXiv preprint arXiv:2408.15270,

    Yinhuai Wang, Qihan Zhao, Runyi Yu, Ailing Zeng, Jing Lin, Zhengyi Luo, Hok Wai Tsui, Jiwen Yu, Xiu Li, Qifeng Chen, et al. Skillmimic: Learning reusable basketball skills from demonstrations.arXiv preprint arXiv:2408.15270,

    arXiv

  82. [90]

    Inter- Control: Generate human motion interactions by controlling every joint.arXiv preprint arXiv:2311.15864, 2023

    Zhenzhi Wang, Jingbo Wang, Dahua Lin, and Bo Dai. Inter- Control: Generate human motion interactions by controlling every joint.arXiv preprint arXiv:2311.15864, 2023. 3

    2023 arXiv

  83. [91]

    Hoh: Markerless mul- timodal human-object-human handover dataset with large object count

    Noah Wiederhold, Ava Megyeri, DiMaggio Paris, Sean Banerjee, and Natasha Banerjee. Hoh: Markerless mul- timodal human-object-human handover dataset with large object count. InNeurIPS, 2024. 2

    2024

  84. [92]

    Daydreamer: World models for physical robot learning

    Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Daydreamer: World models for physical robot learning. InCoRL, 2023. 3

    2023

  85. [93]

    THOR: Text to human-object inter- action diffusion via relation intervention.arXiv preprint arXiv:2403.11208, 2024

    Qianyang Wu, Ye Shi, Xiaoshui Huang, Jingyi Yu, Lan Xu, and Jingya Wang. THOR: Text to human-object inter- action diffusion via relation intervention.arXiv preprint arXiv:2403.11208, 2024. 3, 5, 6

    2024 arXiv

  86. [94]

    SAGA: Stochastic whole- body grasping with contact

    Yan Wu, Jiahao Wang, Yan Zhang, Siwei Zhang, Otmar Hilliges, Fisher Yu, and Siyu Tang. SAGA: Stochastic whole- body grasping with contact. InECCV, 2022. 3, 4

    2022

  87. [95]

    Human-object interaction from human-level instructions.arXiv preprint arXiv:2406.17840, 2024

    Zhen Wu, Jiaman Li, and C Karen Liu. Human-object interaction from human-level instructions.arXiv preprint arXiv:2406.17840, 2024. 3

    2024 arXiv

  88. [96]

    Chore: Contact, human and object reconstruction from a single rgb image

    Xianghui Xie, Bharat Lal Bhatnagar, and Gerard Pons-Moll. Chore: Contact, human and object reconstruction from a single rgb image. InECCV, 2022. 3

    2022

  89. [97]

    In- terTrack: Tracking human object interaction without object templates.arXiv preprint arXiv:2408.13953, 2024

    Xianghui Xie, Jan Eric Lenssen, and Gerard Pons-Moll. In- terTrack: Tracking human object interaction without object templates.arXiv preprint arXiv:2408.13953, 2024. 3

    2024 arXiv

  90. [98]

    OmniControl: Control any joint at any time for human motion generation.arXiv preprint arXiv:2310.08580, 2023

    Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. OmniControl: Control any joint at any time for human motion generation.arXiv preprint arXiv:2310.08580, 2023. 3

    2023 arXiv

  91. [99]

    Learning soccer juggling skills with layer- wise mixture-of-experts

    Zhaoming Xie, Sebastian Starke, Hung Yu Ling, and Michiel van de Panne. Learning soccer juggling skills with layer- wise mixture-of-experts. InSIGGRAPH, 2022. 3

    2022

  92. [100]

    Hierarchical planning and control for box loco-manipulation.arXiv preprint arXiv:2306.09532, 2023

    Zhaoming Xie, Jonathan Tseng, Sebastian Starke, Michiel van de Panne, and C Karen Liu. Hierarchical planning and control for box loco-manipulation.arXiv preprint arXiv:2306.09532, 2023. 3

    2023 arXiv

  93. [101]

    Inter-x: Towards versatile human-human interaction analysis.arXiv preprint arXiv:2312.16051, 2023

    Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Con- gsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xing- dong Sheng, et al. Inter-x: Towards versatile human-human interaction analysis.arXiv preprint arXiv:2312.16051, 2023. 2

    2023 arXiv

  94. [102]

    InterDiff: Generating 3d human-object interactions with physics-informed diffusion

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang-Yan Gui. InterDiff: Generating 3d human-object interactions with physics-informed diffusion. InICCV, 2023. 1, 3, 4, 5, 8

    2023

  95. [103]

    Interdreamer: Zero-shot text to 3d dynamic human-object interaction.arXiv preprint arXiv:2403.19652, 2024

    Sirui Xu, Ziyin Wang, Yu-Xiong Wang, and Liang-Yan Gui. Interdreamer: Zero-shot text to 3d dynamic human-object interaction.arXiv preprint arXiv:2403.19652, 2024. 3

    2024

  96. [104]

    InterMimic: Towards universal whole-body control for physics-based human-object interactions

    Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, and Liang-Yan Gui. InterMimic: Towards universal whole-body control for physics-based human-object interactions. InCVPR, 2025. 1, 3, 4

    2025

  97. [105]

    D3D-HOI: Dynamic 3d human-object interactions from videos.arXiv preprint arXiv:2108.08420, 2021

    Xiang Xu, Hanbyul Joo, Greg Mori, and Manolis Savva. D3D-HOI: Dynamic 3d human-object interactions from videos.arXiv preprint arXiv:2108.08420, 2021. 3

    2021 arXiv

  98. [106]

    Person in place: Generating associa- tive skeleton-guidance maps for human-object interaction image editing

    ChangHee Yang, ChanHee Kang, Kyeongbo Kong, Hanni Oh, and Suk-Ju Kang. Person in place: Generating associa- tive skeleton-guidance maps for human-object interaction image editing. InCVPR, 2024. 3

    2024

  99. [107]

    F-HOI: Toward fine-grained semantic- aligned 3d human-object interactions

    Jie Yang, Xuesong Niu, Nan Jiang, Ruimao Zhang, and Siyuan Huang. F-HOI: Toward fine-grained semantic- aligned 3d human-object interactions. InECCV, 2024. 3

    2024

  100. [108]

    Lemon: Learning 3d human-object interac- tion relation from 2d images

    Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, and Zheng-Jun Zha. Lemon: Learning 3d human-object interac- tion relation from 2d images. InCVPR, 2024. 3

    2024

  101. [109]

    Learning to use chopsticks in diverse gripping styles.ACM Transactions on Graphics (TOG), 41(4):1–17, 2022

    Zeshi Yang, Kangkang Yin, and Libin Liu. Learning to use chopsticks in diverse gripping styles.ACM Transactions on Graphics (TOG), 41(4):1–17, 2022. 3

    2022

  102. [110]

    Moconvq: Unified physics- based motion control via scalable discrete representations

    Heyuan Yao, Zhenhua Song, Yuyang Zhou, Tenglong Ao, Baoquan Chen, and Libin Liu. Moconvq: Unified physics- based motion control via scalable discrete representations. ACM Transactions on Graphics (TOG), 43(4):1–21, 2024. 3

    2024

  103. [111]

    Affordance diffusion: Synthesizing hand-object inter- actions

    Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, and Sifei Liu. Affordance diffusion: Synthesizing hand-object inter- actions. InCVPR, 2023. 3

    2023

  104. [112]

    Oakink2: A dataset of bimanual hands-object manipulation in complex task completion.arXiv preprint arXiv:2403.19417, 2024

    Xinyu Zhan, Lixin Yang, Yifei Zhao, Kangrui Mao, Han- lin Xu, Zenan Lin, Kailin Li, and Cewu Lu. Oakink2: A dataset of bimanual hands-object manipulation in complex task completion.arXiv preprint arXiv:2403.19417, 2024. 2

    2024 arXiv

  105. [113]

    Core4d: A 4d human-object-human interac- tion dataset for collaborative object rearrangement.arXiv preprint arXiv:2406.19353, 2024

    Chengwen Zhang, Yun Liu, Ruofan Xing, Bingda Tang, and Li Yi. Core4d: A 4d human-object-human interac- tion dataset for collaborative object rearrangement.arXiv preprint arXiv:2406.19353, 2024. 2

    2024 arXiv

  106. [114]

    ArtiGrasp: Physically plausible synthesis of bi-manual dexterous grasp- ing and articulation.arXiv preprint arXiv:2309.03891, 2023

    Hui Zhang, Sammy Christen, Zicong Fan, Luocheng Zheng, Jemin Hwangbo, Jie Song, and Otmar Hilliges. ArtiGrasp: Physically plausible synthesis of bi-manual dexterous grasp- ing and articulation.arXiv preprint arXiv:2309.03891, 2023. 3

    2023 arXiv

  107. [115]

    Neural- Dome: A neural modeling pipeline on multi-view human- object interactions

    Juze Zhang, Haimin Luo, Hongdi Yang, Xinru Xu, Qianyang Wu, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Neural- Dome: A neural modeling pipeline on multi-view human- object interactions. InCVPR, 2023. 2, 3, 1, 4

    2023

  108. [116]

    T2M-GPT: Generating human motion from textual descriptions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2M-GPT: Generating human motion from textual descriptions with discrete representations. InCVPR, 2023. 4

    2023

  109. [117]

    Hoi-mˆ 3: Capture multiple humans and objects in- teraction within contextual environment

    Juze Zhang, Jingyan Zhang, Zining Song, Zhanhe Shi, Chengfeng Zhao, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Hoi-mˆ 3: Capture multiple humans and objects in- teraction within contextual environment. InCVPR, 2024. 3

    2024

  110. [118]

    Manidext: Hand-object manipulation synthesis via continuous corre- spondence embeddings and residual-guided diffusion.arXiv preprint arXiv:2409.09300, 2024

    Jiajun Zhang, Yuxiang Zhang, Liang An, Mengcheng Li, Hongwen Zhang, Zonghai Hu, and Yebin Liu. Manidext: Hand-object manipulation synthesis via continuous corre- spondence embeddings and residual-guided diffusion.arXiv preprint arXiv:2409.09300, 2024. 3

    2024 arXiv

  111. [119]

    Perceiving 3d human-object spatial arrangements from a single image in the wild

    Jason Y Zhang, Sam Pepose, Hanbyul Joo, Deva Ramanan, Jitendra Malik, and Angjoo Kanazawa. Perceiving 3d human-object spatial arrangements from a single image in the wild. InECCV, 2020. 3

    2020

  112. [120]

    COUCH: Towards controllable human-chair interactions

    Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons-Moll. COUCH: Towards controllable human-chair interactions. InECCV, 2022. 3

    2022

  113. [121]

    FORCE: Dataset and method for intuitive physics guided human-object interac- tion.arXiv preprint arXiv:2403.11237, 2024

    Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Ilya Petrov, Vladimir Guzov, Helisa Dhamo, Eduardo P´erez- Pellitero, and Gerard Pons-Moll. FORCE: Dataset and method for intuitive physics guided human-object interac- tion.arXiv preprint arXiv:2403.11237, 2024. 2

    2024 arXiv

  114. [122]

    Simulation and retargeting of complex multi-character interactions

    Yunbo Zhang, Deepak Gopinath, Yuting Ye, Jessica Hodgins, Greg Turk, and Jungdam Won. Simulation and retargeting of complex multi-character interactions. InSIGGRAPH, 2023. 5

    2023

  115. [123]

    I’M HOI: Inertia-aware monocular capture of 3d human-object interactions

    Chengfeng Zhao, Juze Zhang, Jiashen Du, Ziwei Shan, Junye Wang, Jingyi Yu, Jingya Wang, and Lan Xu. I’M HOI: Inertia-aware monocular capture of 3d human-object interactions. InCVPR, 2024. 2, 3, 4

    2024

  116. [124]

    Compositional human-scene interaction synthesis with semantic control

    Kaifeng Zhao, Shaofei Wang, Yan Zhang, Thabo Beeler, and Siyu Tang. Compositional human-scene interaction synthesis with semantic control. InECCV, 2022. 3

    2022

  117. [125]

    Synthesizing diverse human motions in 3d indoor scenes

    Kaifeng Zhao, Yan Zhang, Shaofei Wang, Thabo Beeler, and Siyu Tang. Synthesizing diverse human motions in 3d indoor scenes. InICCV, 2023. 3

    2023

  118. [126]

    CAMS: Canonicalized manipulation spaces for category-level functional hand-object manipulation synthe- sis

    Juntian Zheng, Qingyuan Zheng, Lixing Fang, Yun Liu, and Li Yi. CAMS: Canonicalized manipulation spaces for category-level functional hand-object manipulation synthe- sis. InCVPR, 2023. 3

    2023

  119. [127]

    A person [action] the [object name]

    Keyang Zhou, Bharat Lal Bhatnagar, Jan Eric Lenssen, and Gerard Pons-Moll. Toch: Spatio-temporal object-to-hand correspondence for motion refinement. InECCV, 2022. 3, 4 InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation Supplementary Material We w...

    2022

  120. [128]

    GRAB [77] uses Software Copyright License for non- commercial scientific research purposes

  121. [129]

    BEHA VE [4] uses Software Copyright License for non- commercial scientific research purposes

  122. [130]

    InterCap [28] uses Software Copyright License for non- commercial scientific research purposes

  123. [131]

    Chairs [29] does not indicate their license but requires to follow the license of each sub-modules they used

  124. [132]

    HODome [115] uses Apache License

  125. [133]

    OMOMO [38] does not indicate their license

  126. [134]

    IMHD [123] uses Dataset Copyright License for Non- commercial Scientific Research Purposes E. Discussion Limitations.Despite that our dataset significantly expands the number of objects to 217 – nearly ten times more than existing HOI datasets – we acknowledge that it still po...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.