Pith. sign in

REVIEW 5 major objections 5 minor 60 references

Move-in-2D: 2D-Conditioned Human Motion Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Given a scene image and a text prompt, Move-in-2D generates a motion sequence that matches the text and projects naturally onto the scene.

desk verdict New task and large dataset are solid, but the main evaluation discards the scene-placement coordinates, so the central claim is only weakly supported. read the letter →

arxiv 2412.13185 v1 pith:752Y6NLB submitted 2024-12-17 cs.CV

Hsin-Ping Huang , Yang Zhou , Jui-Hsien Wang , Difan Liu , Feng Liu , Ming-Hsuan Yang , Zhan Xu This is my paper · ORCID
classification cs.CV
keywords humanmotiongeneration2D-conditionedtext-to-motionscene-conditioneddiffusiontransformerSMPLbodymodelmotion-guidedvideoHiC-Motiondataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Move-in-2D introduces a new task: generating a human motion sequence from a scene image and a text prompt, so that the motion matches the text and lands naturally on the image when projected. The paper argues that a single 2D photo carries enough spatial and semantic layout information to condition motion, making 3D scene reconstruction unnecessary for many applications. To support this, the authors assemble HiC-Motion, a 300k-video corpus of single-person activities with auto-extracted SMPL body poses and background images. A diffusion transformer trained on this corpus with in-context conditioning on text and image tokens reports the best FID (44.639) and prompt accuracy (0.661) among the compared text- and scene-conditioned models. If this holds, it gives video generation systems a practical way to obtain scene-compatible motion without motion capture or 3D scanning.

What carries the argument

The load-bearing machinery is a multi-conditional diffusion transformer. The motion representation is a sequence of SMPL body poses (SMPL is a standard skinned human body model): pose parameters $\theta_b \in \mathbb{R}^{23\times 6}$ for 23 body joints, a global orientation $\theta_g \in \mathbb{R}^6$, and a camera translation $\pi \in \mathbb{R}^3$, so the generated pose can be projected onto the scene image with a fixed-focal-length perspective camera. The text prompt is encoded into a single token and the scene image into patch tokens by a vision encoder that preserves spatial layout; all condition tokens are then concatenated with the noisy motion tokens (in-context conditioning) inside transformer blocks that use adaptive layer normalization for the diffusion timestep. This shared token space lets the model align motion with both modalities, while the two-stage training first learns diverse scene-aware motion from 300k videos and then fine-tunes on a subset emphasizing large motion and static backgrounds to separate body motion from camera-induced movement.

What would settle it

Collect 100 held-out videos with verified static cameras, manually mark the 2D positions of the person's feet and pelvis every 16 frames, generate a motion from the scene image and text, and compare the generated joints projected onto the image against those manual marks; a median projection error above roughly 20 pixels on trained actions would contradict the claim that the motion projects naturally onto the scene.

Watch

Extended reading notes

Core claim

The central claim is that 2D-conditioned human motion generation is feasible and useful: a diffusion model accepting a language-encoded text token and patch-level scene-image tokens, concatenated in-context with the noisy motion tokens, can output a 256-frame SMPL body-pose sequence plus a camera translation $\pi \in \mathbb{R}^3$ that lets the motion be projected onto the background image with correct placement. Trained in two stages on HiC-Motion (the full corpus, then a 150k subset biased toward large motion and static backgrounds), the model achieves FID 44.639, accuracy 0.661, and diversity 26.027, beating text-only models, point-cloud-conditioned models, and a same-corpus text-only variant. The authors further show that feeding the generated motion to motion-guided video animators produces smoother, geometrically cleaner human video than unguided video diffusion, and that the in-context conditioning design outperforms cross-attention in ablations.

Load-bearing premise

The load-bearing premise is that the automatically extracted body poses from hundreds of thousands of internet videos are accurate enough to serve as both training targets and evaluation ground truth; if these pseudo-labels drift or conflate camera motion with body motion, the model learns a distorted motion distribution and the reported scores inherit that distortion.

Editorial extensions

If this is right

  • A single 2D photo plus a text prompt is enough to synthesize a motion sequence that can be re-projected onto the photo, so motion generation no longer requires 3D reconstruction or an existing driving motion.
  • The generated motions can serve as control signals for motion-guided video animators, producing videos with sharper human geometry and more consistent motion than unguided video diffusion.
  • Training on a large corpus of real-world videos with auto-extracted body poses substantially improves text-to-motion quality, since a same-corpus text-only model far outperforms its counterpart trained on smaller motion-capture datasets.
  • In-context conditioning, which concatenates text and scene tokens with motion tokens in a shared space, outperforms cross-attention for aligning generated motion with both modalities, and AdaLN timestep conditioning improves temporal smoothness.
  • The method covers large-dynamics activities and human-object interactions such as tennis, trampoline, climbing, horse riding, and petting a dog, because 2D scene images are far more abundant than annotated 3D scenes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the held-out test set is labeled by the same automatic reconstruction pipeline used for training, part of the reported advantage could come from the model learning the reconstruction model's biases rather than from true scene understanding; an independent motion-capture test set would separate these.
  • The paper does not model camera movement and admits the two-pass video pipeline is not jointly optimized; a natural extension is to predict a full camera trajectory and to fine-tune the motion generator and video animator together, which may further improve temporal consistency.
  • The automated VLM evaluation uses only the middle frame of 20 generated videos, so it is a weak test of temporal scene alignment; a per-frame projection metric or a user study with more samples would give a stricter check.
  • A controlled comparison in which 3D-scene baselines are retrained on the HiC-Motion corpus, rather than only evaluated with back-projected depth, would isolate how much of the improvement is due to 2D conditioning versus dataset scale.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Move-in-2D, a conditional diffusion model that generates SMPL motion sequences from a 2D scene image and a text prompt, with the stated goal of producing motion that 'can be projected naturally onto the scene image.' To support training, the authors introduce HiC-Motion, a dataset of 300k videos with pseudo-ground-truth SMPL motions extracted via 4D-Humans, inpainted background frames, and text captions. The model uses a transformer with in-context conditioning for text and scene tokens, AdaLN for the timestep, and classifier-free guidance. Experiments include FID, accuracy, diversity, multimodality, a VLM-based automated evaluation, and a downstream video-generation application with Champ and Gen-3. The central claim is that Move-in-2D is the first method to produce scene-compatible human motion from a 2D image plus text.

Significance. If the result holds, this is a valuable new task formulation and a large-scale dataset contribution. The HiC-Motion dataset at 300k sequences significantly exceeds existing scene-aware motion datasets, and the ablation study in Table 4 provides evidence that the in-context conditioning design contributes to the reported gains. The application to motion-guided video generation is practical and well-motivated. However, the evaluation does not currently substantiate the headline claim of scene compatibility: the main quantitative metrics discard the very global parameters needed for 2D projection, and the only direct scene-alignment evidence is a small VLM study. The reliance on pseudo-labels for both training and evaluation further weakens the empirical grounding. These issues are fixable, but they are load-bearing for the paper's central contribution.

major comments (5)
  1. [Sec. 5, 'Evaluation metrics' and Table 2] The paper's central claim is that generated motion 'can be projected naturally onto the scene image,' but the main quantitative metrics are computed after explicitly ignoring global orientation and translation ('To standardize outputs across models, we ignore global orientation and translation'). FID, Accuracy, Diversity, and Multimodality therefore measure body-pose distribution quality and action recognition, not whether the person is placed at a plausible scale, location, or orientation in the scene. A model that ignores the image entirely could in principle score well on these metrics. The paper should add metrics that directly use the predicted global translation π and orientation θg, such as 2D reprojection error against the ground-truth projection, foot-contact or floor-penetration checks, or human evaluation of the rendered full sequence.
  2. [Table 2 and Table 4] All metrics are reported as point estimates without variance, confidence intervals, or significance tests. The reported advantage of Ours over MDM+ (FID 44.639 vs. 46.035; Accuracy 0.661 vs. 0.620) may be within sampling noise, especially since both models are trained on the same data and the differences in Table 4 are similar in magnitude. Please report results over multiple random seeds or provide confidence intervals and, where appropriate, paired statistical tests.
  3. [Sec. 5.2, 'Automated evaluation' and Table 3] The only direct scene-alignment evaluation is the VLM study, which averages over just 20 test videos, scores only the middle frame of each sequence, and uses ChatGPT-4o without any reported inter-rater agreement, repeat-run variance, or confidence intervals. Twenty samples is far too small to support the claim that 'our method consistently outperforms' all baselines on scene alignment. The paper should evaluate on a substantially larger sample, score multiple frames per sequence, report the prompt template, and provide variance or agreement statistics.
  4. [Sec. 3, 'Data preprocessing', and Sec. 5, 'Evaluation data'] The ground-truth motions are pseudo-labels extracted by 4D-Humans, and the held-out test set is drawn from the same pseudo-labeled distribution. If these pseudo-labels contain errors—such as camera-motion contamination, drift, or failures under occlusion—those errors directly bias both training and evaluation. The paper should quantify the reliability of the pseudo-labels (e.g., by manually verifying a random subset or by comparing against a small motion-capture dataset), and should report how the optical-flow filtering in Sec. 4.4 changes the distribution of the pseudo-labels (e.g., the distribution of global translation magnitudes before and after filtering).
  5. [Sec. 5, 'Evaluation metrics'] The STGCN classifier used for FID and accuracy is trained by the authors on their own pseudo-labeled motion data. This makes the metrics sensitive to the classifier's idiosyncrasies and to the pseudo-label distribution, and it is not clear that the classifier generalizes to open-domain motions. Please report the classifier's accuracy on a held-out set, compare against a classifier trained on an independent motion dataset if possible, or provide an alternative human-evaluation metric for action alignment.
minor comments (5)
  1. [Sec. 4.4, 'Selection of fine-tuning set'] The term 'fixed-background videos' is not defined; please specify the optical-flow threshold used to select this subset, and state the median-flow criterion precisely so that the filtering step is reproducible.
  2. [Sec. 5.1, Figure 5 caption] The caption reads 'MDM and SceneDiff produces implausible poses'; the verb should agree with the plural subject ('produce').
  3. [Sec. 2, paragraph beginning 'Given a 3D indoor scene'] The sentence 'prior works [19, 20, 46, 47, 57, 57]' cites reference [57] twice; one duplicate should be removed.
  4. [Sec. 5.2, 'Automated evaluation'] The paper states that 'there is currently no established metric to assess the compatibility between generated motion sequences and 2D background images,' but the authors themselves use a VLM as a new metric; it would be helpful to state the exact prompt and scoring rubric used for ChatGPT-4o, and to note whether the evaluation was repeated.
  5. [Sec. 6, Conclusions] The limitation 'our framework does not control camera movement in generated motions' is only mentioned in the conclusion; it should be stated earlier in the method section, since camera movement in the source videos is a key confound for the pseudo-label extraction.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity: the center is an empirical generative model evaluated on a held-out split; the minor self-citation to ActAnywhere is contextual, not load-bearing.

full rationale

No significant circularity. The paper's central claim—that a diffusion transformer conditioned on a scene image and text generates motion sequences projectable onto the image—is an empirical, trained system rather than a derivation from first principles. The motion representation includes θb, θg, and π; the training loss directly supervises these from 4D-Humans pseudo-labels, and the test set is a held-out portion of HiC-Motion. No fitted constant is relabeled as a prediction: the STGCN classifier is trained separately for evaluation, and the VLM study uses an external model. The reuse of the same pseudo-label source for both training and evaluation is a validity concern (noisy 4D-Humans reconstructions, metrics ignoring global orientation and translation), but it is not circular by construction: generated outputs are not derived from the test labels, and the paper explicitly notes limitations (no camera control, no joint optimization). The only self-citation, ActAnywhere [38], appears in a contextual list of human-centric video datasets and in 'Inspired by HiC'; it is not load-bearing for the method or the evaluation. Therefore no equation-to-equation reduction or fitted-parameter-as-prediction exists.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests mainly on the accuracy of pseudo-GT motion labels and on the assumption that a single background frame represents the scene. All other inputs are standard diffusion and transformer machinery. The reported gains are therefore only as reliable as the 4D-Humans reconstructions and the self-trained evaluation classifier.

free parameters (4)
  • Classifier-free guidance scale g = not reported
    Used in Eq. (1) to balance condition alignment; authors do not state the value used for reported samples.
  • Condition drop probability q = 0.1
    Sec. 4.2; hand-chosen probability for randomly dropping text and scene conditions during training.
  • Large-motion filtering threshold = 200 pixels
    Sec. 4.4; videos with less than 200 pixels of human movement are excluded from the fine-tuning set.
  • Fine-tuning dataset mix = 60% large-motion / 40% fixed-background
    Sec. 4.4; authors chose this ratio for the second training stage.
assumptions (6)
  • domain assumption Pseudo-ground-truth SMPL motions from 4D-Humans are sufficiently accurate to serve as training targets and evaluation ground truth.
    Sec. 3 data preprocessing; the entire dataset and test set are built from these automatic reconstructions.
  • domain assumption A single inpainted frame can represent the scene for the whole motion sequence.
    Sec. 3: 'we randomly select an inpainted frame from each video to serve as the background image'; this assumes the scene is static.
  • domain assumption Optical-flow median filtering removes enough camera motion to make the remaining motion interpretable as human motion.
    Sec. 4.4; camera motion is not modeled or removed explicitly, but the training objective treats the pseudo-GT SMPL sequence as pure human motion.
  • domain assumption The STGCN classifier trained by the authors measures motion quality, prompt accuracy, and diversity.
    Sec. 5 evaluation metrics; FID, Accuracy, Diversity, and Multimodality all use this self-trained classifier.
  • domain assumption ChatGPT-4o scores on a single middle frame capture scene alignment, text alignment, and pose quality.
    Sec. 5.2 automated evaluation; only 20 test videos are scored.
  • standard math Standard DDPM diffusion and classifier-free guidance are valid generative models for motion.
    Sec. 4.1; assumed background from [22,45].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Move-in-2D: 2D-Conditioned Human Motion Generation." pith.science (2026). https://pith.science/paper/752Y6NLB

@misc{pith2026241213185,
  author       = {Pith},
  title        = {Pith review of: Move-in-2D: 2D-Conditioned Human Motion Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/752Y6NLB}},
  note         = {Machine review of arXiv:2412.13185}
}
read the original abstract

Generating realistic human videos remains a challenging task, with the most effective methods currently relying on a human motion sequence as a control signal. Existing approaches often use existing motion extracted from other videos, which restricts applications to specific motion types and global scene matching. We propose Move-in-2D, a novel approach to generate human motion sequences conditioned on a scene image, allowing for diverse motion that adapts to different scenes. Our approach utilizes a diffusion model that accepts both a scene image and text prompt as inputs, producing a motion sequence tailored to the scene. To train this model, we collect a large-scale video dataset featuring single-human activities, annotating each video with the corresponding human motion as the target output. Experiments demonstrate that our method effectively predicts human motion that aligns with the scene image after projection. Furthermore, we show that the generated motion sequence improves human motion quality in video synthesis tasks.

Figures

Figures reproduced from arXiv: 2412.13185 by the authors.

Figure 1
Figure 1. 2D-conditioned human motion generation. Given an image representing the target scene and a text prompt describing the desired motion, we generate a motion sequence that aligns with the text description and projects naturally onto the scene image. This generated motion then serves as the control signal for the subsequent video generation tasks. spective, enabling the generation of affordant human mo￾tion without the … view at source ↗
Figure 2
Figure 2. Overview. The text prompt and background scene image are encoded by the CLIP and DINO encoders, and incorporated into the model via in-context conditioning. The AdaLN layer receives the diffusion timestep as input. Our multi-conditional transformer model then generates a human motion sequence through a diffusion denoising process, aligning the generated motion with both input conditions. images [29] as well as rando… view at source ↗
Figure 3
Figure 3. Affordance-aware human generation. Our model generates human poses consistent with both text prompts and scene context, such as standing on a cliff. It also supports complex human-scene interactions, including activities like petting a dog. A person playing tennis. A person jumping rope. A person jumping on a trampoline. A person swinging on a swing. A person playing basketball. A person doing lunge. A person riding… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Motion generation with large dynamics. Our results show motion sequences that are accurately placed and move within scenes, such as playing tennis, enabling the generation of complex human activities that are challenging for video generation models. 147, using an archi…
Figure 5
Figure 5. Figure 5: Comparison to state-of-the-art. MDM and SceneDiff produces implausible poses, MLD generates mismatched motion with the scene, and HUMANISE generates static poses. Our method generates coherent motion aligned with both the scene and text prompts [PITH_FULL_IMAGE:figure…
Figure 6
Figure 6. Figure 6: Motion-guided human video generation. Our approach generates scene-compatible motion sequences from a scene image and text prompt, which are then used to animate a reference human using Champ [60] or Gen-3 [11]. The generated motion ensures accurate human shapes and sm…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 44 canonical work pages

  1. [1]

    Text2action: Generative adversarial synthesis from language to action

    Hyemin Ahn, Timothy Ha, Yunho Choi, Hwiyeon Yoo, and Songhwai Oh. Text2action: Generative adversarial synthesis from language to action. In ICRA, 2018. 1, 3

    work page 2018

  2. [2]

    How do in- context examples affect compositional generalization? In ACL, 2023

    Shengnan An, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Jian-Guang Lou, and Dongmei Zhang. How do in- context examples affect compositional generalization? In ACL, 2023. 2

    work page 2023

  3. [3]

    Circle: Capture in rich contextual environ- ments

    Joao Pedro Ara ´ujo, Jiaman Li, Karthik Vetrivel, Rishi Agar- wal, Jiajun Wu, Deepak Gopinath, Alexander William Clegg, and Karen Liu. Circle: Capture in rich contextual environ- ments. In CVPR, 2023. 3

    work page 2023

  4. [4]

    Teach: Temporal action composition for 3d hu- mans

    Nikos Athanasiou, Mathis Petrovich, Michael J Black, and G¨ul Varol. Teach: Temporal action composition for 3d hu- mans. In 3DV, 2022. 3

    work page 2022

  5. [5]

    Stable video diffusion: Scaling la- tent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling la- tent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 8

    arXiv 2023

  6. [6]

    Hallucinating pose- compatible scenes

    Tim Brooks and Alexei A Efros. Hallucinating pose- compatible scenes. In ECCV, 2022. 3

    work page 2022

  7. [7]

    Z. Cao, G. Hidalgo Martinez, T. Simon, S. Wei, and Y . A. Sheikh. Openpose: Realtime multi-person 2d pose estima- tion using part affinity fields. IEEE TPAMI, 2019. 3

    work page 2019

  8. [8]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In CVPR, 2023. 1, 3, 4, 5, 7

    work page 2023

Show all 60 references
  1. [9]

    Cmu graphics lab motion capture database

    CMU. Cmu graphics lab motion capture database. http: //mocap.cs.cmu.edu/. 2

  2. [10]

    Laserhuman: Language-guided scene-aware hu- man motion generation in free environment

    Peishan Cong, Ziyi Wang, Zhiyang Dou, Yiming Ren, Wei Yin, Kai Cheng, Yujing Sun, Xiaoxiao Long, Xinge Zhu, and Yuexin Ma. Laserhuman: Language-guided scene-aware hu- man motion generation in free environment. arXiv preprint arXiv:2403.13307, 2024. 3

    2024 arXiv

  3. [11]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In ICCV, 2023. 8

    2023

  4. [12]

    Movi: A large multi-purpose human motion and video dataset

    Saeed Ghorbani, Kimia Mahdaviani, Anne Thaler, Konrad Kording, Douglas James Cook, Gunnar Blohm, and Niko- laus F Troje. Movi: A large multi-purpose human motion and video dataset. Plos one, 2021. 2

    2021

  5. [13]

    Detectron

    Ross Girshick, Ilija Radosavovic, Georgia Gkioxari, Piotr Doll´ar, and Kaiming He. Detectron. https://github. com/facebookresearch/detectron, 2018. 3

    2018

  6. [14]

    Humans in 4D: Reconstructing and tracking humans with transformers

    Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, and Jitendra Malik. Humans in 4D: Reconstructing and tracking humans with transformers. In ICCV, 2023. 2, 3

    2023

  7. [15]

    Ac- tion2motion: Conditioned generation of 3d human motions

    Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Ac- tion2motion: Conditioned generation of 3d human motions. In ACM MM, 2020. 2, 5

    2020

  8. [16]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In CVPR, 2022. 2, 3

    2022

  9. [17]

    Tm2t: Stochastic and tokenized modeling for the reciprocal gener- ation of 3d human motions and texts

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal gener- ation of 3d human motions and texts. In ECCV, 2022. 3

    2022

  10. [18]

    Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J. Black. Resolving 3D human pose ambigu- ities with 3D scene constraints. In ICCV, 2019. 3

    2019

  11. [19]

    Stochastic scene-aware motion prediction

    Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael J Black. Stochastic scene-aware motion prediction. In ICCV, 2021. 3

    2021

  12. [20]

    Populating 3d scenes by learning human-scene interaction

    Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, and Michael J Black. Populating 3d scenes by learning human-scene interaction. In CVPR, 2021. 3

    2021

  13. [21]

    Classifier-free diffusion guidance

    Jonathan Ho. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 4

    2022 arXiv

  14. [22]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 4

    2020

  15. [23]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. arXiv preprint arXiv:2311.17117, 2023. 1, 2

    2023 arXiv

  16. [24]

    Diffusion- based generation, optimization, and planning in 3d scenes

    Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song-Chun Zhu. Diffusion- based generation, optimization, and planning in 3d scenes. In CVPR, 2023. 1, 5, 7

    2023

  17. [25]

    Human3.6m: Large scale datasets and predic- tive methods for 3d human sensing in natural environments

    Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3.6m: Large scale datasets and predic- tive methods for 3d human sensing in natural environments. IEEE TPAMI, 2014. 2

    2014

  18. [26]

    Open source computer vision library

    Itseez. Open source computer vision library. https:// github.com/itseez/opencv, 2015. 3

    2015

  19. [27]

    Motiongpt: Human motion as a foreign language

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign language. In NeurIPS, 2024. 3

    2024

  20. [28]

    Dreampose: Fashion image-to-video synthesis via stable diffusion

    Johanna Karras, Aleksander Holynski, Ting-Chun Wang, and Ira Kemelmacher-Shlizerman. Dreampose: Fashion image-to-video synthesis via stable diffusion. In ICCV,

  21. [29]

    Training generative adver- sarial networks with limited data

    Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adver- sarial networks with limited data. In NeurIPS, 2020. 4

    2020

  22. [30]

    The kinetics human action video dataset

    Will Kay, Jo ˜ao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Apostol Natsev, Mustafa Suley- man, and Andrew Zisserman. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017. 3

    2017 arXiv

  23. [31]

    Efros, and Krishna Kumar Singh

    Sumith Kulal, Tim Brooks, Alex Aiken, Jiajun Wu, Jimei Yang, Jingwan Lu, Alexei A. Efros, and Krishna Kumar Singh. Putting people in their place: Affordance-aware hu- man insertion into scenes. In CVPR, 2023. 3

    2023

  24. [32]

    Motion-x: A large- scale 3d expressive whole-body human motion dataset

    Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x: A large- scale 3d expressive whole-body human motion dataset. In NeurIPS, 2023. 2, 3

    2023

  25. [33]

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi- person linear model. ACM TOG, 2015. 4 9

    2015

  26. [34]

    Troje, Ger- ard Pons-Moll, and Michael J

    Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Ger- ard Pons-Moll, and Michael J. Black. AMASS: Archive of motion capture as surface shapes. In ICCV, 2019. 2

    2019

  27. [35]

    Moments in time dataset: one million videos for event understanding

    Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ra- makrishnan, Sarah Adel Bargal, Tom Yan, Lisa Brown, Quanfu Fan, Dan Gutfruend, Carl V ondrick, et al. Moments in time dataset: one million videos for event understanding. IEEE TPAMI, pages 1–8, 2019. 3

    2019

  28. [36]

    Chatgpt, 2024

    OpenAI. Chatgpt, 2024. 7

    2024

  29. [37]

    Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Rus- sell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Ni...

    2023 arXiv

  30. [38]

    Guibas, and Jimei Yang

    Boxiao Pan, Zhan Xu, Chun-Hao Paul Huang, Krishna Ku- mar Singh, Yang Zhou, Leonidas J. Guibas, and Jimei Yang. Actanywhere: Subject-aware video background generation. In NeurIPS, 2024. 3

    2024

  31. [39]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 2, 4

    2023

  32. [40]

    Courville

    Ethan Perez, Florian Strub, Harm de Vries, Vincent Du- moulin, and Aaron C. Courville. Film: Visual reasoning with a general conditioning layer. In AAAI, 2018. 5

    2018

  33. [41]

    Black, and G ¨ul Varol

    Mathis Petrovich, Michael J. Black, and G ¨ul Varol. Action- conditioned 3D human motion synthesis with transformer V AE. InICCV, 2021. 5

    2021

  34. [42]

    The KIT motion-language dataset

    Matthias Plappert, Christian Mandery, and Tamim Asfour. The KIT motion-language dataset. Big Data, 4(4):236–252,

  35. [43]

    Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever

    Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from nat- ural language supervision. In ICML, 2021. 4, 6

    2021

  36. [44]

    Lidar-aid inertial poser: Large-scale human motion capture by sparse inertial and lidar sensors

    Yiming Ren, Chengfeng Zhao, Yannan He, Peishan Cong, Han Liang, Jingyi Yu, Lan Xu, and Yuexin Ma. Lidar-aid inertial poser: Large-scale human motion capture by sparse inertial and lidar sensors. IEEE TVCG, 2023. 3

    2023

  37. [45]

    Human motion diffu- sion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffu- sion model. In ICLR, 2023. 1, 3, 4, 5, 7

    2023

  38. [46]

    Synthesizing long-term 3d human motion and interaction in 3d scenes

    Jiashun Wang, Huazhe Xu, Jingwei Xu, Sifei Liu, and Xiao- long Wang. Synthesizing long-term 3d human motion and interaction in 3d scenes. In CVPR, 2021. 3

    2021

  39. [47]

    Scene- aware generative network for human motion synthesis

    Jingbo Wang, Sijie Yan, Bo Dai, and Dahua Lin. Scene- aware generative network for human motion synthesis. In CVPR, 2021. 3

    2021

  40. [48]

    Disco: Disentangled control for realistic human dance generation

    Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung- Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In CVPR, 2024. 1, 2

    2024

  41. [49]

    Humanise: Language-conditioned hu- man motion generation in 3d scenes

    Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Humanise: Language-conditioned hu- man motion generation in 3d scenes. In NeurIPS, 2022. 1, 3, 5, 7

    2022

  42. [50]

    Chi, Tat- sunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Bar- ret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tat- sunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language model...

    2022

  43. [51]

    Magicanimate: Temporally consistent human image animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. In CVPR, 2024. 1, 2

    2024

  44. [52]

    Cimi4d: A large multimodal climbing motion dataset under human-scene in- teractions

    Ming Yan, Xin Wang, Yudi Dai, Siqi Shen, Chenglu Wen, Lan Xu, Yuexin Ma, and Cheng Wang. Cimi4d: A large multimodal climbing motion dataset under human-scene in- teractions. In CVPR, 2023. 3

    2023

  45. [53]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, 2024. 5

    2024

  46. [54]

    Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting

    Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In AAAI, 2018. 5

    2018

  47. [55]

    Motiondif- fuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022. 3

    2022 arXiv

  48. [56]

    Re- modiffuse: Retrieval-augmented motion diffusion model

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Re- modiffuse: Retrieval-augmented motion diffusion model. In ICCV, 2023. 3

    2023

  49. [57]

    Generating 3d people in scenes with- out people

    Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J Black, and Siyu Tang. Generating 3d people in scenes with- out people. In CVPR, 2020. 3

    2020

  50. [58]

    Compositional human-scene interaction synthe- sis with semantic control

    Kaifeng Zhao, Shaofei Wang, Yan Zhang, Thabo Beeler, and Siyu Tang. Compositional human-scene interaction synthe- sis with semantic control. In ECCV, 2022. 3

    2022

  51. [59]

    Emdm: Efficient motion diffusion model for fast and high-quality motion generation

    Wenyang Zhou, Zhiyang Dou, Zeyu Cao, Zhouyingcheng Liao, Jingbo Wang, Wenjia Wang, Yuan Liu, Taku Komura, Wenping Wang, and Lingjie Liu. Emdm: Efficient motion diffusion model for fast and high-quality motion generation. In ECCV, 2024. 3

    2024

  52. [60]

    Champ: Controllable and consistent human image animation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In ECCV, 2024. 1, 2, 8 10

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.