REVIEW 3 major objections 6 minor 194 references
Surface Keypoint Representation for Multi-Object and Articulated Human-Object Interaction Generation
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Surface keypoint trajectories are a lossless, joint-type-agnostic representation of object motion that lets a single diffusion pipeline generate multi-object and articulated human-object interactions.
desk verdict A genuinely useful representational contribution to HOI generation with unusually honest kinematic auditing, but the joint-type-agnostic claim is only approximately true and the contact metric partially shares its objective with the training loss. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the surface keypoint trajectory. For each rigid component, $K$ canonical surface points are chosen by farthest point sampling ($K=3$ in the main experiments); at every time step their global 3D positions form the component's state, and the per-frame rigid pose is recovered in closed form by minimizing $\|R O_0 + t - O_t\|^2$ via the Kabsch algorithm, which is exact whenever the keypoints are non-collinear. Because the representation lives in Euclidean space, the generative model never has to predict on the $\mathrm{SE}(3)$ manifold, and because unused slots in a fixed-size component buffer are masked during training, a single Transformer-based flow-matching diffusion model can handle any number of components. The second mechanism is the contact distance field $D_t = \sigma((\tau - \mathrm{dist}(H_t^c, S_t))/\alpha)$, computed between a subset of 47 body markers and 384 dense object surface points per component; Stage II predicts this field, and Stage III uses it as a soft contact prior together with an object signed-distance-field penetration penalty. Stage III performs Diffusion Noise Optimization, updating the initial noise rather than the output motion, so the refined motion stays on the learned manifold while contact violations are reduced.
What would settle it
Prompt a trained model to open a prismatic drawer in a cabinet it has not seen, fit a line to each generated drawer trajectory, and measure the per-frame deviation of the drawer's recovered pose from that line and from the cabinet's track. If the median deviation exceeds the paper's own 2 cm assembly-connectivity threshold, or if the recovered joint value leaves the ground-truth range by more than 10% for a substantial fraction of frames, the claim that joint mechanisms are learned from keypoint trajectories alone is refuted; the paper's own appendix reports a 5.20% frame-level connectivity violation rate, so the test should show that rate stays near zero across novel articulated objects.
Extended reading notes
Core claim
The central claim, stated on the paper's own terms, is that surface keypoint trajectories are a lossless and joint-type-agnostic object motion representation: for each rigid component, a small set of non-collinear surface points tracked over time fully determines the component's pose via the Kabsch algorithm, and the trajectory patterns of revolute, prismatic, and screw joints are distinctive enough for a generative model to learn them directly from data. The paper further claims that a spatio-temporal contact distance field, computed as a sigmoid of the distance between 138 body surface markers and 384 surface points per component, captures when and where each body region contacts each object, and that this field is a more learnable intermediate signal than binary contact labels, raw Euclidean distances, or fixed contact pairs. The two representations are combined in a factorized three-stage pipeline—object motion diffusion, contact-field diffusion, and body motion diffusion refined by contact-guided noise optimization—and the authors report that this pipeline matches or outperforms existing methods on single-object, multi-object, and articulated interaction benchmarks.
Load-bearing premise
The whole approach rests on the assumption that a learned model can read the type of a joint—hinge, sliding, or screw—straight off the trajectories of a few surface points, so articulated parts stay on their axes and stay connected without any explicit joint constraint.
Editorial extensions
If this is right
- A single generation model can produce interactions with one object, several objects, or an articulated assembly without being told the object count or the joint type first.
- Object motion can be generated directly from text or sparse waypoints as point trajectories and converted to rigid poses afterward, so no separate per-object pose parameterization or per-joint handling is needed.
- The spatio-temporal contact distance field provides a learnable whole-body contact signal; the paper's ablations find it improves contact accuracy over binary labels, raw Euclidean distances, and fixed contact pairs.
- Articulated generation extends beyond revolute joints to prismatic and screw mechanisms; when exact joint constraints are required, the paper shows the generated trajectory can be projected onto the constrained motion without retraining.
- On unseen object compositions, the method keeps object motion smooth and contact accuracy high relative to a fixed-count baseline, suggesting the representation generalizes to object combinations not seen together in training.
Reading between the lines
- Editorial inference: because the representation is joint-type-agnostic, the same pipeline could plausibly extend to more complex mechanisms such as multi-axis linkages or gear trains without architectural changes; the paper does not test this.
- Editorial inference: the paper's robustness table shows over-determined keypoint sets reduce rotation error under noise, while its generation ablation favors three keypoints; a natural hybrid is to generate with three keypoints but recover pose from a denser set, a combination the paper does not evaluate.
- Editorial inference: the predicted contact distance field is a standalone, time-varying whole-body contact prior that could be reused for retargeting or physics-based control beyond the three-stage pipeline, which the paper leaves unexamined.
- Editorial inference: the approximately four-minute per-sequence cost is dominated by noise optimization; a testable extension suggested by the paper's own discussion is to distill the optimized noise into a feed-forward correction network, which would make real-time deployment plausible.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces surface keypoint trajectories as an alternative to SE(3) object representations for human-object interaction generation. Each rigid component is represented by K non-collinear surface keypoints, and the authors use the Kabsch algorithm to recover its pose. Contact is modeled by a sigmoid contact distance field between 138 SMPL-X body markers and 384 surface points per component. The proposed method factorizes generation into three diffusion stages: object keypoint trajectory generation, contact distance field prediction, and human marker motion synthesis with diffusion noise optimization for contact refinement. Experiments are reported on ParaHome, HIMO, ARCTIC, and OMOMO, with comparisons against HIMO-Gen, CoDA, HOI-Diff, CHOIS, ROG, and HOIDiNi, together with ablations, a perceptual user study, and kinematic-validity audits.
Significance. If the central claims hold, the keypoint-trajectory representation would be a useful unified object representation that avoids SE(3) manifold issues and explicit joint-type modeling, and the spatio-temporal contact distance field is a richer supervision signal than binary contact labels. The paper is strong in scope: it evaluates on four benchmarks, reports ablations for the object representation and the contact formulation, conducts a perceptual user study, and provides reconstruction and kinematic-validity audits. The strengths include the explicit robustness analysis for Kabsch recovery, the permutation study for slot ordering, and the optional projection step that makes residual articulation errors transparent. However, the load-bearing articulation and contact-accuracy claims need qualification and more stringent evaluation before the paper's central claims can be taken at face value.
major comments (3)
- [Sec. 3.1, Appendix Table 6, Appendix Table 3] The paper's central claim that articulated objects are generated 'without explicit joint-type specification' is only approximately supported by the reported evidence. Since Eq. (1) recovers component pose from K non-collinear keypoints and the Stage I diffusion model (Eq. (2)) imposes no joint constraints, generated components are not guaranteed to stay on their joint axes or remain assembled. The paper's own audit (Appendix Table 6) reports 5.20% frame-level assembly-connectivity violations at a 2 cm threshold and 1.46% frames with axis deviation above 10 degrees, and the optional 'Ours + Correction' (Appendix Table 3) requires a predefined pivot/axis and joint type. Please state explicitly whether any main-table result uses the correction, report how these residuals behave beyond the T=120 training horizon, and clarify when the correction is necessary; without that, the 'joint-type-agnostic' contribution is considerably weaker than claimed.
- [Eq. (3), Eq. (6), Appendix Eq. (9)] The contact accuracy metric is structurally aligned with both the contact field definition and the Stage III optimization target, which can inflate the reported contact improvements. Eq. (3) defines the field as sigmoid((tau - dist)/alpha), Appendix Eq. (9) scores contact with the same thresholded-distance notion at tau=2 cm, and Eq. (6) optimizes L_contact against the predicted field by thresholding D_hat. Thus, part of the improvement in C_tem/body acc may reflect agreement of human markers with the model's own predicted field rather than with genuinely observed contact. I recommend reporting contact accuracy under alternative thresholds (e.g., 1 cm and 3 cm) or a surface-area/contact-patch metric that is not derived from Eq. (3), and reporting the no-DNO ablation in Table 1 with the same metrics to quantify how much of the gain comes from optimizing the measured quantity.
- [Tables 1, 3, 4, 8, 9] Most head-to-head numbers are reported for a single run without error bars, and several differences are small. For example, Table 5 shows C_body acc of 0.902 vs 0.896 for the Euclidean-distance baseline and Table 1 shows 0.896 vs 0.902 across object counts; Table 2 is the only table with error bars. This makes it difficult to judge whether the claimed advantage over baselines and the ranking of ablations are statistically robust. Please provide at least three seeds (or a paired evaluation over the test set) and report mean plus/minus standard deviation or confidence intervals for the main tables, and state the significance test used.
minor comments (6)
- [Fig. 3 vs Eq. (4)] The Fig. 3 caption refers to the Stage II model as f_c, but Eq. (4) uses f_d; unify the notation.
- [Table 2 caption] The Table 2 caption cites 'CoDA [Lv et al. 2024]', but CoDA is [Pi et al. 2025] in the text and references; fix the citation.
- [Eq. (6)] L_contact in Eq. (6) is not defined in the main text, and the Appendix sentence after Eq. (6) does not contain its explicit formula; define it in the main text or point to the exact supplement equation.
- [Table 4] Table 4 reports T_obj and O_obj without defining them in the main text; define translation error and orientation error and state how they are computed from the recovered poses.
- [Eq. (3) and Sec. 4.1] The hyperparameters tau and alpha in Eq. (3), along with K, N, M_c, and Q, are only given in the appendix; add a single hyperparameter table in the main paper because the representation claims depend on these choices.
- [Appendix numbering] The main-text and appendix table numbering collide (both contain a Table 3); renumber the appendix tables to avoid confusion.
Circularity Check
Partial circularity in the contact evidence: Stage III explicitly optimizes the same thresholded distance that defines both the predicted contact field and the contact-accuracy metric, while the central surface-keypoint representation claim remains independently grounded.
-
fitted input called prediction
[Sec. 3.2 Stage II Eq. (3), Stage III DNO paragraph; Appendix B Eq. (9)]
"D_t = sigma((tau - dist(H^c_t, S_t))/alpha) ... L_contact encourages predicted contact pairs (identified by thresholding D_hat_t) to be spatially close ... and a binary contact label using a distance threshold tau=2cm, g^o_t = I(d^o_t < tau)."
The contact distance field (Eq. 3) is a sigmoid of (tau minus marker-surface distance), i.e., a soft threshold of that distance; the Appendix contact metric (Eq. 9) is a hard threshold of the same distance at the same tau. Stage III then uses DNO to explicitly minimize marker-surface distance for pairs whose predicted field exceeds a threshold. Consequently, the reported C_tem/C_body improvements partly measure how well the test-time optimizer satisfies a field that is itself a smoothed version of the evaluation quantity. Because the field is independently shown to be accurate (F1 = 0.927), optimizing it is nearly equivalent to optimizing the metric, so the contact-accuracy evidence is partially self-confirming.
full rationale
The central claimed derivation chain, surface keypoint trajectories as a lossless joint-type-agnostic representation, is self-contained: the losslessness follows from the external Kabsch result and is empirically tested against SE(3) parameterizations with matched architectures, so it does not reduce to a fitted input. Self-citations (e.g., HOI-Diff and Meng et al.) appear only as baselines or related work and are not load-bearing. The genuine circularity is in the contact-modeling evidence: Eq. (3), the test-time DNO loss, and Appendix Eq. (9) all operate on the same thresholded marker-surface distance, so Stage III is effectively optimizing a soft version of the metric being reported. This is a partial reduction of the contact-accuracy claim to its own optimization objective, although the metric is computed against ground-truth labels and therefore depends on the learned field being accurate. The joint-type-agnostic claim is also only approximately supported by the paper's own kinematic audit (Appendix Table 6, 5.20% assembly-connectivity violations), but that is a correctness and robustness limitation, not a circularity. Overall, the main representation contribution has independent content; the circularity is confined to the contact evaluation pathway.
Assumptions & free parameters
free parameters (6)
- K (object keypoints per component) =
3
- tau (contact threshold in Eq. 3) =
2 cm in evaluation; training value not reported
- alpha (sigmoid sharpness) =
not reported
- N (maximum component slots) =
not reported
- M_c and Q (contact markers and surface points) =
M_c=47, Q=384
- lambda_pene =
0.001
assumptions (5)
- standard math Three non-collinear points determine a rigid transformation, so Kabsch recovery is lossless for clean rigid motion.
- domain assumption Scenes are collections of rigid components; articulated objects decompose into such components.
- ad hoc to paper Joint mechanisms (revolute, prismatic, screw) can be learned implicitly from keypoint trajectory patterns without explicit joint-type specification.
- domain assumption Contact is adequately modeled by a smooth function of Euclidean distance between body surface markers and object surface points.
- domain assumption 138 SMPL-X surface markers sufficiently represent the body for whole-body contact and motion synthesis.
Cite this review
Pith. "Pith review of Surface Keypoint Representation for Multi-Object and Articulated Human-Object Interaction Generation." pith.science (2026). https://pith.science/paper/2WVCCX2G
@misc{pith2026260803158,
author = {Pith},
title = {Pith review of: Surface Keypoint Representation for Multi-Object and Articulated Human-Object Interaction Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/2WVCCX2G}},
note = {Machine review of arXiv:2608.03158}
}
read the original abstract
Daily activities require humans to coordinate whole-body motion with the motion of surrounding objects. Despite recent progress in human-object interaction (HOI) generation, most existing methods assume interactions with a single rigid object and do not extend well to scenarios involving a variable number of objects or articulated objects with diverse joint mechanisms. We propose surface keypoint trajectories as an object motion representation: for each rigid component, whether a standalone object or one part of an articulated assembly, we track a small set of non-collinear surface points over time. This representation handles multi-object coordination and diverse articulation mechanisms directly from point dynamics without requiring explicit joint-type specification. To model when and where each body region contacts each object, we introduce a spatio-temporal contact distance field that extends distance-based contact modeling to whole-body, multi-object, and articulated settings. We factorize HOI generation into three stages: generating object motions from text or waypoints, predicting the contact distance field, and synthesizing whole-body motion with contact-guided optimization. Experiments on ParaHome, HIMO, ARCTIC, and OMOMO demonstrate better or comparable performance to existing methods across single-object, multi-object, and articulated interaction settings.
Figures
Figures from the paper (12 more)
Reference graph
Works this paper leans on
-
[1]
ICLR , year=
Human Motion Diffusion Model , author=. ICLR , year=
-
[2]
ICCV , year =
Karunratanakul, Korrawe and Preechakul, Konpat and Suwajanakorn, Supasorn and Tang, Siyu , title =. ICCV , year =
-
[3]
2023 , journal=
Human Motion Diffusion as a Generative Prior , author=. 2023 , journal=
2023
-
[4]
Advances in neural information processing systems , volume=
Openshape: Scaling up 3d shape representation towards open-world understanding , author=. Advances in neural information processing systems , volume=
-
[5]
NeurIPS , year=
Diffusion models beat gans on image synthesis , author=. NeurIPS , year=
-
[6]
TOG , year=
Deepmimic: Example-guided deep reinforcement learning of physics-based character skills , author=. TOG , year=
-
[7]
TOG , year=
SuperTrack: motion tracking for physically simulated characters using supervised learning , author=. TOG , year=
-
[8]
TOG , year=
Amp: Adversarial motion priors for stylized physics-based character control , author=. TOG , year=
Show all 194 references
-
[9]
TOG , year=
Generalizing locomotion style to new animals with inverse optimal regression , author=. TOG , year=
-
[10]
ACM SIGGRAPH Asia 2009 papers , year=
Robust task-based control policies for physics-based characters , author=. ACM SIGGRAPH Asia 2009 papers , year=
2009
-
[11]
SIGGRAPH , year=
Construction and optimal search of interpolated motion graphs , author=. SIGGRAPH , year=
-
[12]
ACM SIGGRAPH 2008 classes , year=
Motion graphs , author=. ACM SIGGRAPH 2008 classes , year=
2008
-
[13]
arXiv preprint arXiv:2209.03003 , year=
Flow straight and fast: Learning to generate and transfer data with rectified flow , author=. arXiv preprint arXiv:2209.03003 , year=
-
[14]
CVPR , year=
Executing your Commands via Motion Diffusion in Latent Space , author=. CVPR , year=
-
[15]
arXiv , year=
Motiondiffuse: Text-driven human motion generation with diffusion model , author=. arXiv , year=
-
[16]
CVPR , year=
MoDi: UnconditionHal Motion Synthesis from Diverse Data , author=. CVPR , year=
-
[17]
SIGGRAPH Asia 2015 technical briefs , year=
Learning motion manifolds with convolutional autoencoders , author=. SIGGRAPH Asia 2015 technical briefs , year=
2015
-
[18]
CVPR , year=
Humor: 3d human motion model for robust pose estimation , author=. CVPR , year=
-
[19]
CVPR , year=
Gfpose: Learning 3d human pose prior with gradient fields , author=. CVPR , year=
-
[20]
3DV , year=
Task-generic hierarchical human motion prior using vaes , author=. 3DV , year=
-
[21]
CVPR , year=
Mofusion: A framework for denoising-diffusion-based motion synthesis , author=. CVPR , year=
-
[22]
3DV , year=
Language2pose: Natural language grounded pose forecasting , author=. 3DV , year=
-
[23]
ECCV , year=
Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts , author=. ECCV , year=
-
[24]
ECCV , year=
TEMOS: Generating diverse human motions from textual descriptions , author=. ECCV , year=
-
[25]
arXiv , year=
TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis , author=. arXiv , year=
-
[26]
NeurIPS , year=
Dancing to music , author=. NeurIPS , year=
-
[27]
AAAI , year=
Danceformer: Music conditioned 3d dance generation with parametric motion transformer , author=. AAAI , year=
-
[28]
CVPR , year=
Edge: Editable dance generation from music , author=. CVPR , year=
-
[29]
arXiv , year=
Learning Variational Motion Prior for Video-based Motion Capture , author=. arXiv , year=
-
[30]
3DV , year=
Convolutional autoencoders for human motion infilling , author=. 3DV , year=
-
[31]
CGF , year=
IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions , author=. CGF , year=
-
[32]
arXiv , year=
NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis , author=. arXiv , year=
-
[33]
arXiv , year=
Synthesizing Diverse Human Motions in 3D Indoor Scenes , author=. arXiv , year=
-
[34]
ECCV , year=
Compositional human-scene interaction synthesis with semantic control , author=. ECCV , year=
-
[35]
TOG , year=
Moglow: Probabilistic and controllable motion synthesis using normalising flows , author=. TOG , year=
-
[36]
NeurIPS , year=
Improving diffusion models for inverse problems using manifold constraints , author=. NeurIPS , year=
-
[37]
arXiv , year=
Planning with diffusion for flexible behavior synthesis , author=. arXiv , year=
-
[38]
NeurIPS , year=
Denoising diffusion probabilistic models , author=. NeurIPS , year=
-
[39]
ICML , year=
Improved denoising diffusion probabilistic models , author=. ICML , year=
-
[40]
arXiv , year=
Imagen video: High definition video generation with diffusion models , author=. arXiv , year=
-
[41]
CVPR , year=
Align your latents: High-resolution video synthesis with latent diffusion models , author=. CVPR , year=
-
[42]
arXiv , year=
Ilvr: Conditioning method for denoising diffusion probabilistic models , author=. arXiv , year=
-
[43]
CVPR , year =
Guo, Chuan and Zou, Shihao and Zuo, Xinxin and Wang, Sen and Ji, Wei and Li, Xingyu and Cheng, Li , title =. CVPR , year =
-
[44]
and Pons-Moll, Gerard and Black, Michael J
Mahmood, Naureen and Ghorbani, Nima and Troje, Nikolaus F. and Pons-Moll, Gerard and Black, Michael J. , booktitle =
-
[45]
ACM MM , year=
Action2motion: Conditioned generation of 3d human motions , author=. ACM MM , year=
-
[46]
NeurIPS Workshop , year=
Classifier-Free Diffusion Guidance , author=. NeurIPS Workshop , year=
-
[47]
ICML , year=
Learning transferable visual models from natural language supervision , author=. ICML , year=
-
[48]
arXiv , year=
Controllable Motion Diffusion Model , author=. arXiv , year=
-
[49]
TOG , year=
Character controllers using motion vaes , author=. TOG , year=
-
[50]
arXiv , year=
An image is worth one word: Personalizing text-to-image generation using textual inversion , author=. arXiv , year=
-
[51]
CVPR , year=
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation , author=. CVPR , year=
-
[52]
arXiv , year=
Instantbooth: Personalized text-to-image generation without test-time finetuning , author=. arXiv , year=
-
[53]
arXiv , year=
HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models , author=. arXiv , year=
-
[54]
CVPR , year=
Multi-concept customization of text-to-image diffusion , author=. CVPR , year=
-
[55]
arXiv , year=
Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation , author=. arXiv , year=
-
[56]
arXiv , year=
Taming encoder for zero fine-tuning image customization with text-to-image diffusion models , author=. arXiv , year=
-
[57]
, author=
Neural state machine for character-scene interactions. , author=. TOG , year=
-
[58]
TOG , year=
Deepphase: Periodic autoencoders for learning motion phase manifolds , author=. TOG , year=
-
[59]
CVPR , year=
Convolutional sequence generation for skeleton-based action synthesis , author=. CVPR , year=
-
[60]
CVPR , year=
Action-conditioned 3D human motion synthesis with transformer VAE , author=. CVPR , year=
-
[61]
ECCV , year=
Posegpt: Quantization-based 3d human motion generation and forecasting , author=. ECCV , year=
-
[62]
The KIT Motion-Language Dataset , journal=
Plappert, Matthias and Mandery, Christian and Asfour, Tamim , year=. The KIT Motion-Language Dataset , journal=
-
[63]
CVPR , year=
Rempe, Davis and Luo, Zhengyi and Peng, Xue Bin and Yuan, Ye and Kitani, Kris and Kreis, Karsten and Fidler, Sanja and Litany, Or , title=. CVPR , year=
-
[64]
Guided Conditional Diffusion for Controllable Traffic Simulation , year=
Zhong, Ziyuan and Rempe, Davis and Xu, Danfei and Chen, Yuxiao and Veer, Sushant and Che, Tong and Ray, Baishakhi and Pavone, Marco , booktitle=. Guided Conditional Diffusion for Controllable Traffic Simulation , year=
-
[65]
CVPR , year=
High-resolution image synthesis with latent diffusion models , author=. CVPR , year=
-
[66]
ICLR , year=
Decoupled weight decay regularization , author=. ICLR , year=
-
[67]
ICCV , year=
PhysDiff: Physics-Guided Human Motion Diffusion Model , author=. ICCV , year=
-
[68]
CVPR , year=
Diffusion-based Generation, Optimization, and Planning in 3D Scenes , author=. CVPR , year=
-
[69]
arXiv , year=
CHAIRS: Towards Full-Body Articulated Human-Object Interaction , author=. arXiv , year=
-
[70]
ICCV , year=
InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion , author=. ICCV , year=
-
[71]
CVPR , year=
Towards diverse and natural scene-aware 3d human motion synthesis , author=. CVPR , year=
-
[72]
NeurIPS , year=
Humanise: Language-conditioned human motion generation in 3d scenes , author=. NeurIPS , year=
-
[73]
ICCV , year =
Stochastic Scene-Aware Motion Prediction , author =. ICCV , year =
-
[74]
ECCV , year =
COUCH: Towards Controllable Human-Chair Interactions , author =. ECCV , year =
-
[75]
arXiv , year=
InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions , author=. arXiv , year=
-
[76]
NeurIPS , year=
MotionGPT: Human Motion as a Foreign Language , author=. NeurIPS , year=
-
[77]
arXiv , year=
MotionGPT: Finetuned LLMs are General-Purpose Motion Generators , author=. arXiv , year=
-
[78]
CVPR , year=
T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations , author=. CVPR , year=
-
[79]
ECCV , year=
Motionclip: Exposing human motion generation to clip space , author=. ECCV , year=
-
[80]
AAAI , year=
Flame: Free-form language-based motion synthesis & editing , author=. AAAI , year=
-
[81]
2021 , booktitle=
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ , author=. 2021 , booktitle=
2021
-
[82]
BMVC , year=
A recurrent variational autoencoder for human motion synthesis , author=. BMVC , year=
-
[83]
ICCV , year=
Adding conditional control to text-to-image diffusion models , author=. ICCV , year=
-
[84]
SIGGRAPH Asia , year=
PADL: Language-Directed Physics-Based Character Control , author=. SIGGRAPH Asia , year=
-
[85]
CVPR , year=
GLIGEN: Open-Set Grounded Text-to-Image Generation , author=. CVPR , year=
-
[86]
International conference on medical image computing and computer-assisted intervention , pages=
Contrastive diffusion model with auxiliary guidance for coarse-to-fine PET reconstruction , author=. International conference on medical image computing and computer-assisted intervention , pages=. 2023 , organization=
2023
-
[87]
NeurIPS , year=
Photorealistic text-to-image diffusion models with deep language understanding , author=. NeurIPS , year=
-
[88]
ICML , year=
Zero-shot text-to-image generation , author=. ICML , year=
-
[89]
ICCV , year =
Pi, Huaijin and Peng, Sida and Yang, Minghui and Zhou, Xiaowei and Bao, Hujun , title =. ICCV , year =
-
[90]
TOG , year=
Object motion guided human motion synthesis , author=. TOG , year=
-
[91]
ECCV , year =
SAGA: Stochastic Whole-Body Grasping with Contact , author =. ECCV , year =
-
[92]
and Tzionas, Dimitrios , booktitle =
Taheri, Omid and Choutas, Vasileios and Black, Michael J. and Tzionas, Dimitrios , booktitle =
-
[93]
AAAI , year=
Action-conditioned generation of bimanual object manipulation sequences , author=. AAAI , year=
-
[94]
arXiv , year=
Can We Use Diffusion Probabilistic Models for 3D Motion Prediction? , author=. arXiv , year=
-
[95]
ICCV , year=
Belfusion: Latent diffusion for behavior-driven human motion prediction , author=. ICCV , year=
-
[96]
arXiv , year=
HumanMAC: Masked Motion Completion for Human Motion Prediction , author=. arXiv , year=
-
[97]
arXiv , year=
Towards Globally Consistent Stochastic Human Motion Prediction via Motion Diffusion , author=. arXiv , year=
-
[98]
arXiv , year=
TransFusion: A practical and effective transformer-based diffusion model for 3d human motion prediction , author=. arXiv , year=
-
[99]
arXiv , year=
Understanding Text-driven Motion Synthesis with Keyframe Collaboration via Diffusion Models , author=. arXiv , year=
-
[100]
arXiv , year=
ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model , author=. arXiv , year=
-
[101]
arXiv , year=
TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis , author=. arXiv , year=
-
[102]
CVPR , year =
BEHAVE: Dataset and Method for Tracking Human Object Interactions , author=. CVPR , year =
-
[103]
OmniControl: Control Any Joint at Any Time for Human Motion Generation , author=
-
[104]
ICCV , year=
Guided Motion Diffusion for Controllable Human Motion Synthesis , author=. ICCV , year=
-
[105]
NeurIPS , year=
Attention is all you need , author=. NeurIPS , year=
-
[106]
NeurIPS , year=
Pytorch: An imperative style, high-performance deep learning library , author=. NeurIPS , year=
-
[107]
NeurIPS , year=
Pointnet++: Deep hierarchical feature learning on point sets in a metric space , author=. NeurIPS , year=
-
[108]
arXiv , year=
Open-vocabulary affordance detection in 3d point clouds , author=. arXiv , year=
-
[109]
CVPR , year=
3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding , author=. CVPR , year=
-
[110]
International Conference on Humanoid Robotics (Humanoids) , year=
Affordance detection for task-specific grasping using deep learning , author=. International Conference on Humanoid Robotics (Humanoids) , year=
-
[111]
Sensors , year=
Affordance-based grasping point detection using graph convolutional networks for industrial bin-picking applications , author=. Sensors , year=
-
[112]
CoRL , year=
O2O-Afford: Annotation-free large-scale object-object affordance learning , author=. CoRL , year=
-
[113]
ICRA , year=
Semantic labeling of 3d point clouds with object affordance for robot manipulation , author=. ICRA , year=
-
[114]
IROS , year=
Interactive affordance map building for a robotic task , author=. IROS , year=
-
[115]
Denoising Diffusion Probabilistic Models
Ho, Jonathan and Jain, Ajay and Abbeel, Pieter. Denoising Diffusion Probabilistic Models
-
[116]
, title =
Loper, Matthew and Mahmood, Naureen and Romero, Javier and Pons-Moll, Gerard and Black, Michael J. , title =. ACM Trans. Graphics (Proc. SIGGRAPH Asia) , year =
-
[117]
arXiv , year=
Gaussian error linear units (gelus) , author=. arXiv , year=
-
[118]
CG-HOI: Contact-Guided 3D Human-Object Interaction Generation , author=. Proc. Computer Vision and Pattern Recognition (CVPR), IEEE , year=
-
[119]
2023 , eprint=
Controllable Human-Object Interaction Synthesis , author=. 2023 , eprint=
2023
-
[120]
arXiv preprint arXiv:2312.04393 , year =
Wang, Yinhuai and Lin, Jing and Zeng, Ailing and Luo, Zhengyi and Zhang, Jian and Zhang, Lei , title =. arXiv preprint arXiv:2312.04393 , year =
-
[121]
https://chat.openai.com , year =
OpenAI , title =. https://chat.openai.com , year =
-
[122]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Make-an-animation: Large-scale text-conditional 3D human motion generation , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[123]
arXiv preprint arXiv:2403.05530 , year=
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context , author=. arXiv preprint arXiv:2403.05530 , year=
-
[124]
arXiv preprint arXiv:2307.09288 , year=
Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=
-
[125]
arXiv preprint arXiv:2408.00118 , year=
Gemma 2: Improving open language models at a practical size , author=. arXiv preprint arXiv:2408.00118 , year=
-
[126]
International Journal of Computer Vision , pages=
Intergen: Diffusion-based multi-human motion generation under complex interactions , author=. International Journal of Computer Vision , pages=. 2024 , publisher=
2024
-
[127]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
HOIAnimator: Generating Text-prompt Human-object Animations using Novel Perceptive Diffusion Models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[128]
arXiv preprint arXiv:2403.19652 , year=
InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction , author=. arXiv preprint arXiv:2403.19652 , year=
-
[129]
arXiv preprint arXiv:2403.15612 , year=
InterFusion: Text-Driven Generation of 3D Human-Object Interaction , author=. arXiv preprint arXiv:2403.15612 , year=
-
[130]
arXiv preprint arXiv:2407.12435 , year=
F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions , author=. arXiv preprint arXiv:2407.12435 , year=
-
[131]
arXiv preprint arXiv:2407.11385 , year=
Grasping diverse objects with simulated humanoids , author=. arXiv preprint arXiv:2407.11385 , year=
-
[132]
European Conference on Computer Vision , pages=
HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects , author=. European Conference on Computer Vision , pages=. 2024 , organization=
2024
-
[133]
arXiv preprint arXiv:2312.06553 , year=
Hoi-diff: Text-driven synthesis of 3d human-object interactions using diffusion models , author=. arXiv preprint arXiv:2312.06553 , year=
-
[134]
arXiv preprint arXiv:2412.20657 , year=
Diffgrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model , author=. arXiv preprint arXiv:2412.20657 , year=
-
[135]
arXiv preprint arXiv:2412.15664 , year=
SCENIC: Scene-aware Semantic Navigation with Instruction-guided Control , author=. arXiv preprint arXiv:2412.15664 , year=
-
[136]
arXiv preprint arXiv:2412.18600 , year=
ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generation , author=. arXiv preprint arXiv:2412.18600 , year=
-
[137]
International Journal of Advanced Robotic Systems , volume=
Perceptual perspective taking and action recognition , author=. International Journal of Advanced Robotic Systems , volume=. 2005 , publisher=
2005
-
[138]
2014 , publisher=
The ecological approach to visual perception: classic edition , author=. 2014 , publisher=
2014
-
[139]
arXiv preprint arXiv:2412.06334 , year=
TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactions , author=. arXiv preprint arXiv:2412.06334 , year=
-
[140]
arXiv preprint arXiv:2411.18660 , year=
OOD-HOI: Text-Driven 3D Whole-Body Human-Object Interactions Generation Beyond Training Domains , author=. arXiv preprint arXiv:2411.18660 , year=
-
[141]
arXiv preprint arXiv:2410.13911 , year=
Graspdiffusion: Synthesizing realistic whole-body hand-object interaction , author=. arXiv preprint arXiv:2410.13911 , year=
-
[142]
SIGGRAPH Asia 2024 Conference Papers , pages=
Autonomous character-scene interaction synthesis from text instruction , author=. SIGGRAPH Asia 2024 Conference Papers , pages=
2024
-
[143]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Scaling up dynamic human-scene interaction modeling , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[144]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
I'm hoi: Inertia-aware monocular capture of 3d human-object interactions , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[145]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Neuraldome: A neural modeling pipeline on multi-view human-object interactions , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[146]
Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part IV 16 , pages=
GRAB: A dataset of whole-body human grasping of objects , author=. Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part IV 16 , pages=. 2020 , organization=
2020
-
[147]
Advances in Neural Information Processing Systems , volume=
Autoregressive image generation without vector quantization , author=. Advances in Neural Information Processing Systems , volume=
-
[148]
2018 , publisher=
Improving language understanding by generative pre-training , author=. 2018 , publisher=
2018
-
[149]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Mmm: Generative masked motion model , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[150]
arXiv preprint arXiv:2411.16575 , year=
Rethinking Diffusion for Text-Driven Human Motion Generation , author=. arXiv preprint arXiv:2411.16575 , year=
-
[151]
arXiv preprint arXiv:2410.05260 , year=
Dart: A diffusion-based autoregressive motion model for real-time text-driven motion control , author=. arXiv preprint arXiv:2410.05260 , year=
-
[152]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Diffusionposer: Real-time human motion reconstruction from arbitrary sparse sensors using autoregressive diffusion , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[153]
ACM Transactions on Graphics (TOG) , volume=
Interactive character control with auto-regressive motion diffusion models , author=. ACM Transactions on Graphics (TOG) , volume=. 2024 , publisher=
2024
-
[154]
ACM SIGGRAPH 2024 Conference Papers , pages=
Taming diffusion probabilistic models for character control , author=. ACM SIGGRAPH 2024 Conference Papers , pages=
2024
-
[155]
Advances in Neural Information Processing Systems , volume=
Omnigrasp: Grasping diverse objects with simulated humanoids , author=. Advances in Neural Information Processing Systems , volume=
-
[156]
arXiv preprint arXiv:2503.13130 , year=
ChainHOI: Joint-based Kinematic Chain Modeling for Human-Object Interaction Generation , author=. arXiv preprint arXiv:2503.13130 , year=
-
[157]
arXiv preprint arXiv:2503.20172 , year=
Guiding Human-Object Interactions with Rich Geometry and Relations , author=. arXiv preprint arXiv:2503.20172 , year=
-
[158]
A solution for the best rotation to relate two sets of vectors
Kabsch, W. A solution for the best rotation to relate two sets of vectors. Acta Crystallographica Section A. 1976. doi:10.1107/S0567739476001873 , url =
1976 doi
-
[159]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
On the continuity of rotation representations in neural networks , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[160]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Projective manifold gradient layer for deep rotation regression , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[161]
University of Malaga, Tech
A tutorial on se (3) transformation parameterizations and on-manifold optimization , author=. University of Malaga, Tech. Rep , volume=. 2010 , publisher=
2010
-
[162]
1994 , publisher=
Minimization on the Lie group SO (3) and related manifolds , author=. 1994 , publisher=
1994
-
[163]
arXiv preprint arXiv:2410.03441 , year=
CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control , author=. arXiv preprint arXiv:2410.03441 , year=
-
[164]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Optimizing diffusion noise can serve as universal motion priors , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[165]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Scalable diffusion models with transformers , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[166]
2023 , eprint=
NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis , author=. 2023 , eprint=
2023
-
[167]
arXiv preprint arXiv:2210.02747 , year=
Flow matching for generative modeling , author=. arXiv preprint arXiv:2210.02747 , year=
-
[168]
Advances in neural information processing systems , volume=
Denoising diffusion probabilistic models , author=. Advances in neural information processing systems , volume=
-
[169]
arXiv preprint arXiv:2505 , year=
Absolute Coordinates Make Motion Generation Easy , author=. arXiv preprint arXiv:2505 , year=
-
[170]
Neurocomputing , volume=
Roformer: Enhanced transformer with rotary position embedding , author=. Neurocomputing , volume=. 2024 , publisher=
2024
-
[171]
arXiv preprint arXiv:2010.04245 , year=
Query-key normalization for transformers , author=. arXiv preprint arXiv:2010.04245 , year=
2010 arXiv
-
[172]
arXiv preprint arXiv:2002.05202 , year=
Glu variants improve transformer , author=. arXiv preprint arXiv:2002.05202 , year=
2002 arXiv
-
[173]
arXiv preprint arXiv:2503.15451 , year=
MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space , author=. arXiv preprint arXiv:2503.15451 , year=
-
[174]
Pavlakos, Georgios and Choutas, Vasileios and Ghorbani, Nima and Bolkart, Timo and Osman, Ahmed A. A. and Tzionas, Dimitrios and Black, Michael J. , booktitle =. Expressive Body Capture:
-
[175]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Efficient learning on point clouds with basis point sets , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[176]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objects , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
-
[177]
International Conference on 3D Vision (3DV) , year=
ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generation , author=. International Conference on 3D Vision (3DV) , year=
-
[178]
CVPR , year =
Jeonghwan Kim and Jisoo Kim and Jeonghyeon Na and Hanbyul Joo , title =. CVPR , year =
-
[179]
Advances in Neural Information Processing Systems , year=
CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects , author=. Advances in Neural Information Processing Systems , year=
-
[180]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Text2hoi: Text-guided 3d motion generation for hand-object interaction , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[181]
arXiv preprint arXiv:2506.15625 , year=
HOIDiNi: Human-Object Interaction through Diffusion Noise Optimization , author=. arXiv preprint arXiv:2506.15625 , year=
-
[182]
Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
HOIGPT: Learning Long-Sequence Hand-Object Interaction with Language Models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
-
[183]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Affordpose: A large-scale dataset of hand-object interactions with affordance-driven hand pose , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[184]
arXiv preprint arXiv:2502.20041 , year=
3d-affordancellm: Harnessing large language models for open-vocabulary affordance detection in 3d worlds , author=. arXiv preprint arXiv:2502.20041 , year=
-
[185]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Lemon: Learning 3d human-object interaction relation from 2d images , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[186]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Deco: Dense estimation of 3d human-scene contact in the wild , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[187]
Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
PICO: Reconstructing 3D people in contact with objects , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
-
[188]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Contactgen: Generative contact modeling for grasp generation , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[189]
G lo V e: Global Vectors for Word Representation
Pennington, Jeffrey and Socher, Richard and Manning, Christopher. G lo V e: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ). 2014. doi:10.3115/v1/D14-1162
2014 doi
-
[190]
arXiv preprint arXiv:1412.3555 , year=
Empirical evaluation of gated recurrent neural networks on sequence modeling , author=. arXiv preprint arXiv:1412.3555 , year=
-
[191]
Xu, Sirui and Li, Dongting and Zhang, Yucheng and Xu, Xiyan and Long, Qi and Wang, Ziyin and Lu, Yunzhi and Dong, Shuchang and Jiang, Hezi and Gupta, Akshat and Wang, Yu-Xiong and Gui, Liang-Yan , booktitle =
-
[192]
2025 , booktitle =
Liu, Sheng and Liang, Yuanzhi and Wang, Jiepeng and Du, Sidan and Zhang, Chi and Li, Xuelong , title =. 2025 , booktitle =
2025
-
[193]
2021 , volume =
Zhang, He and Ye, Yuting and Shiratori, Takaaki and Komura, Taku , title =. 2021 , volume =
2021
-
[194]
and Hilliges, Otmar , booktitle =
Fan, Zicong and Taheri, Omid and Tzionas, Dimitrios and Kocabas, Muhammed and Kaufmann, Manuel and Black, Michael J. and Hilliges, Otmar , booktitle =
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.