REVIEW 4 major objections 5 minor 94 references
DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A monocular 3D Gaussian avatar can reach multi-view fidelity if a video generative model supplies a synthetic back-view video as pseudo-supervision, with fine-tuning and super-resolution making that video faithful.
desk verdict Plausible idea, weak evidence: back-view pseudo-supervision from an undisclosed pretrained model whose training data may overlap the test set undercuts the main claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the back-view synthetic video produced by Human4DiT, a diffusion-transformer video model conditioned on identity and pose, which the paper treats as additional pseudo-ground-truth frames for a 3D Gaussian avatar. The avatar is represented as two pose-conditioned Gaussian maps anchored to the SMPL template, following Animatable Gaussians. The two enabling mechanisms are: (1) Physical Identity Inversion, a subject-driven fine-tuning technique that injects a learnable identity token into the diffusion model so the generated rear view matches the input subject's appearance and motion; and (2) patch-based super-resolution generation, which splits the latent image into nine o
What would settle it
Run the method on a newly captured subject that provably does not appear in any public human video dataset, retrain or restrict the generative model so its training set excludes the test subject, and compare novel-view PSNR/LPIPS against the same avatar trained without the generated back view; if the improvement vanishes, the reported gains are memorization artifacts. A second check: replace the generated back view with the true multi-view back view on THuman4.0; if the avatar trained on the true back view does not clearly beat the one trained on the generated back view, the generator is repro
Extended reading notes
Core claim
The central claim is that generative priors from a 360-degree human video model can serve as pseudo-supervision for dynamic avatar reconstruction. The authors argue that the failure of prior work is a data problem, not just a representation problem: monocular input provides no observation of rear-view appearance, so pose-conditioned networks either smooth over unseen regions or hallucinate artifacts. DevilSight's contribution is to fill that missing observation by generating a rear-view video of the same person performing the same motion, then treating that video as ground truth. To make generation faithful, they fine-tune the generator with a learnable identity embedding initialized from th
Load-bearing premise
The pipeline rests on the assumption that the video generative model, once fine-tuned on the input clip, produces a rear-view video of the same person performing the same motion that is physically consistent — and that the model has not already memorized the test subject from its own training data.
Editorial extensions
If this is right
- Monocular avatar pipelines that currently restrict network complexity to avoid novel-view artifacts can instead keep full capacity and rely on generative pseudo-views for regularization.
- Reconstructed avatars support novel-view synthesis of the back of the body with quality comparable to input views, directly from a single video.
- The generated back view also improves dynamic details such as fluttering clothing that template-based methods miss, because the pseudo-supervision preserves physical identity rather than just appearance.
- The same back-view generation strategy extends to novel pose synthesis: generating front and back views for a new pose sequence and adding them as training data animates the avatar under arbitrary poses with consistent detail.
- Using generated videos as explicit pseudo-ground-truth, rather than as score-distillation targets, avoids the oversaturation and smoothing typical of SDS-based distillation while achieving better novel-view FID.
Reading between the lines
- The framework generates only one back view for efficiency, so a natural extension is to generate left, right, and top views as well; whether the view-consistency priors of the generator hold across more vantage points is a testable scaling question the paper leaves open.
- A risk not addressed in the paper is training-set overlap: if the video generative model was trained on the same multi-view datasets used for evaluation, its 'novel' back views could be memorized ground truth rather than genuine hallucination, which would make part of the reported gain circular.
- Physical identity inversion costs about 20 hours of per-subject fine-tuning, so an important practical follow-up is a lightweight adapter that transfers identity without per-subject gradient updates; the paper's ablations suggest the full fine-tuning is what captures dynamic clothing detail.
- The patch-based super-resolution result depends on overlap weighting, but the paper does not study sensitivity to patch overlap or stride; small controlled experiments varying those parameters would clarify how robust the resolution claim is.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DevilSight, a framework for reconstructing animatable 3D Gaussian avatars from monocular video. The key idea is to use Human4DiT, a video diffusion transformer, to generate a back-view video of the same subject and motion, and to use that generated video as pseudo-supervision for the avatar representation. Two supporting components are introduced: Physical Identity Inversion, which fine-tunes Human4DiT on the input video with a learnable identity embedding to improve motion and appearance consistency, and a patch-based super-resolution strategy that doubles the resolution of generated frames. The method is evaluated on THuman4.0 and a new Mono2K dataset against HumanNeRF, GaussianAvatar, AnimatableGaussians, and two video-to-4D baselines. The paper claims that the generated back view enriches unseen regions and regularizes the avatar, leading to superior reconstruction quality and novel-view synthesis.
Significance. If the claims were established, the idea of using a video generative model as a virtual viewpoint for dynamic human avatar reconstruction would be a timely and valuable contribution. The pipeline is nontrivial and the two proposed strategies—identity fine-tuning and patch-based super-resolution—are sensible engineering choices. The paper also provides a useful comparison against strong baselines. However, the central empirical claim of outperforming state-of-the-art methods is not supported by the reported numbers, and a training-data leakage risk in Human4DiT remains unresolved. As presented, the evidence does not demonstrate that the generative pseudo-supervision provides independent information about unseen views, which is the load-bearing contribution of the work.
major comments (4)
- [§4.2, Table 1] The sentence "our method significantly outperforms other state-of-the-art approaches" is not supported by the reported numbers. On input-view reconstruction, the method is worse than AnimatableGaussians (AG) on both datasets in PSNR and SSIM (THuman4.0: 32.97 vs 34.06 dB, 0.9769 vs 0.9798; Mono2K: 32.19 vs 33.15 dB, 0.9768 vs 0.9807). Novel-view gains are small (≤0.5 dB) and inconsistent: on Mono2K Novel View 1 the PSNR is lower than AG (26.26 vs 26.29), and per-subject Table B shows ties within noise. With only five evaluation sequences and no variance or significance testing, the central claim of SOTA superiority is not established. The claims in §4.2 and the abstract should be tempered or supported with a more extensive evaluation.
- [§3.2–3.3; Supplementary Table A] The core assumption is that Human4DiT provides independent pseudo-supervision for unseen back views. This requires that Human4DiT has not memorized the evaluation subjects. The paper states only that Human4DiT was trained on "large-scale multi-dimensional datasets" and does not disclose whether THuman4.0 or Mono2K subjects, or their multi-view captures, were used in its training. Since Human4DiT is from the same group and THuman4.0 is a multi-view dataset used here for evaluation, the risk is concrete: if any of the five test subjects appeared in training, the generated back view is a recalled ground-truth image and the pseudo-supervision is circular. The low FID of the generated video alone (59.25 vs AG's 100.34 on novel views, Supplementary Table A) is consistent with strong generalization but also with memorization. The authors must disclose the training split of Human4DiT and evaluat
- [§3.4, Supplement A] The generated video is used as hard pseudo-ground truth without accounting for generator error. Supplementary Table A shows that the generated video itself has substantially lower fidelity than the 3DGS baselines on input views (PSNR 32.38 vs AG 34.06; FID 38.55 vs 17.91). Table 1 shows a corresponding input-view degradation for the full method relative to AG on PSNR/SSIM, suggesting that hard supervision with generated content hurts the observed view. The paper should either use a confidence-weighted or uncertainty-aware supervision, or provide evidence that the input-view regression is acceptable; otherwise the trade-off undermines the claimed advantage.
- [§4.3, Supplement A] The two named contributions (Physical Identity Inversion and super-resolution generation) are only ablated qualitatively (Figs. 7 and 8). Without quantitative ablation, it is unclear whether the reported (modest) novel-view gains are due to the pseudo-supervision or to other training choices. Also, several hyperparameters are listed in Supplement A without numerical values: λ_LPIPS, λ_offset, λ_scale, the number/overlap of patches, fine-tuning steps, and the back-view camera rotation. These should be specified to make the method reproducible and to allow readers to assess the sensitivity of the claimed regularization.
minor comments (5)
- [§4] The evaluation uses ground-truth SMPL-X poses fitted from multiple views, which is a favorable condition for a 'monocular' reconstruction task. This should be stated prominently and discussed as a limitation.
- [§3.3] The term 'Physical Identity Inversion' is not formally defined. Please clarify what 'physical identity' means beyond appearance and specify exactly which parameters are fine-tuned (e.g., which attention layers).
- [Table 1] Tables 1 and A use unusual spacing in decimal numbers; harmonize formatting for readability.
- [§4.2 / Fig. 6] The comparison with video-to-4D methods L4GM and GVFDiffusion is only qualitative. If quantitative evaluation is possible, adding numbers would strengthen the comparison.
- [Eq. (3)] The conditions cref and cΘ are not defined with respect to actual inputs. Specify how the identity reference and pose conditions are computed, especially during the fine-tuning stage.
Circularity Check
No circular derivation found; Human4DiT is used as an external generative prior with empirical validation, and the claimed improvements are not by-construction identities.
full rationale
The paper's pipeline is: (1) estimate SMPL poses from the monocular video; (2) fine-tune Human4DiT on that video to inject identity ('Physical Identity Inversion'); (3) generate a back-view video with the same poses; (4) use that generated video as pseudo-supervision to train a 3DGS avatar. At no point is a target quantity defined in terms of itself. The generated back-view is not produced by the avatar being trained, nor is it a re-rendering of the input; it is an output of a fixed generative model conditioned on identity and pose. The avatar's loss on those frames is a standard reconstruction loss against this pseudo ground truth. The only self-referential element is the citation to Human4DiT [59], whose authors overlap with the present paper. This is a load-bearing tool, but it is not invoked as a 'uniqueness theorem' and the paper provides its own ablations (Fig. 7, Table A) testing the generated video's fidelity. The concern that Human4DiT may have memorized THuman4.0/Mono2K evaluation subjects is a data-leakage validity threat, not a circularity: the paper does not state that those subjects are in Human4DiT's training set, and no equation or construction in the manuscript reduces the novel-view predictions to the input video. Without evidence that the pseudo-supervision equals ground truth by construction, this remains a correctness risk rather than a demonstrated circular step. Therefore no circularity is established.
Assumptions & free parameters
free parameters (5)
- λ_LPIPS, λ_offset, λ_scale =
not specified
- Number of patches and overlap =
9 patches, overlap unspecified
- Fine-tuning steps =
~20k steps
- c_id learnable embedding =
learned
- Back-view camera rotation =
180 degrees
assumptions (5)
- domain assumption Human4DiT can generate high-quality, view-consistent human videos from a single image with pose control.
- standard math SMPL-X and linear blend skinning accurately model human body shape and dynamics.
- domain assumption Fine-tuning only identity-related attention layers preserves multi-view consistency while adding physical identity.
- domain assumption Patch-based denoising with overlapping windows maintains spatial consistency across patches.
- domain assumption Ground-truth SMPL-X poses fitted from multi-view are representative of monocular pose estimates.
Cite this review
Pith. "Pith review of DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective." pith.science (2026). https://pith.science/paper/5BBIVHCH
@misc{pith2026250900403,
author = {Pith},
title = {Pith review of: DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective},
year = {2026},
howpublished = {\url{https://pith.science/paper/5BBIVHCH}},
note = {Machine review of arXiv:2509.00403}
}
read the original abstract
We present a novel framework to reconstruct human avatars from monocular videos. Recent approaches have struggled either to capture the fine-grained dynamic details from the input or to generate plausible details at novel viewpoints, which mainly stem from the limited representational capacity of the avatar model and insufficient observational data. To overcome these challenges, we propose to leverage the advanced video generative model, Human4DiT, to generate the human motions from alternative perspective as an additional supervision signal. This approach not only enriches the details in previously unseen regions but also effectively regularizes the avatar representation to mitigate artifacts. Furthermore, we introduce two complementary strategies to enhance video generation: To ensure consistent reproduction of human motion, we inject the physical identity into the model through video fine-tuning. For higher-resolution outputs with finer details, a patch-based denoising algorithm is employed. Experimental results demonstrate that our method outperforms recent state-of-the-art approaches and validate the effectiveness of our proposed strategies.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Single-image 3d human digitization with shape-guided diffusion
Badour AlBahar, Shunsuke Saito, Hung-Yu Tseng, Changil Kim, Johannes Kopf, and Jia-Bin Huang. Single-image 3d human digitization with shape-guided diffusion. In SIG- GRAPH Asia 2023 Conference Papers , pages 1–11, 2023. 3
2023
-
[2]
Guide3D: Create 3D Avatars from Text and Image Guidance
Yukang Cao, Yan-Pei Cao, Kai Han, Ying Shan, and Kwan- Yee K Wong. Guide3d: Create 3d avatars from text and im- age guidance. arXiv preprint arXiv:2308.09705 , 2023. 2, 3
work page Pith review arXiv 2023
-
[3]
Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models
Yukang Cao, Yan-Pei Cao, Kai Han, Ying Shan, and Kwan- Yee K Wong. Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 958–968, 2024. 3
2024
-
[4]
Videocrafter2: Overcoming data limitations for high-quality video diffusion models, 2024
Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models, 2024. 3
2024
-
[5]
Gener- alizable human gaussians from single-view image
Jinnan Chen, Chen Li, Jianfeng Zhang, Lingting Zhu, Buzhen Huang, Hanlin Chen, and Gim Hee Lee. Gener- alizable human gaussians from single-view image. arXiv preprint arXiv:2406.06050, 2024. 3
arXiv 2024
-
[6]
Meshavatar: Learning high-quality triangular human avatars from multi-view videos
Yushuo Chen, Zerong Zheng, Zhe Li, Chao Xu, and Yebin Liu. Meshavatar: Learning high-quality triangular human avatars from multi-view videos. In European Conference on Computer Vision, pages 250–269. Springer, 2024. 1, 2, 4
2024
-
[7]
Scaling recti- fied flow transformers for high-resolution image synthesis
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning,
-
[8]
Learning neural volumetric representations of dy- namic humans in minutes
Chen Geng, Sida Peng, Zhen Xu, Hujun Bao, and Xiaowei Zhou. Learning neural volumetric representations of dy- namic humans in minutes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8759–8770, 2023. 2
2023
Show all 94 references
-
[9]
Humans in 4d: Re- constructing and tracking humans with transformers
Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, and Jitendra Malik. Humans in 4d: Re- constructing and tracking humans with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14783–14794, 2023. 3
2023
-
[10]
Vid2avatar: 3d avatar reconstruction from videos in the wild via self-supervised scene decomposition
Chen Guo, Tianjian Jiang, Xu Chen, Jie Song, and Otmar Hilliges. Vid2avatar: 3d avatar reconstruction from videos in the wild via self-supervised scene decomposition. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12858–12868, 2023. 1, 3
2023
-
[11]
Sparsectrl: Adding sparse controls to text-to-video diffusion models
Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In European Conference on Computer Vision, pages 330–348. Springer, 2024. 3
2024
-
[12]
Gans trained by a two time-scale update rule converge to a local nash equilib- rium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 13
2017
-
[13]
Sith: Single-view tex- tured human reconstruction with image-conditioned diffu- sion
I Ho, Jie Song, Otmar Hilliges, et al. Sith: Single-view tex- tured human reconstruction with image-conditioned diffu- sion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 538–549, 2024. 2, 3
2024
-
[14]
Expres- sive gaussian human avatars from monocular rgb video
Hezhen Hu, Zhiwen Fan, Tianhao Wu, Yihan Xi, Seoyoung Lee, Georgios Pavlakos, Zhangyang Wang, et al. Expres- sive gaussian human avatars from monocular rgb video. Ad- vances in Neural Information Processing Systems, 37:5646– 5660, 2025. 1, 3
2025
-
[15]
Animate anyone: Consistent and controllable image- to-video synthesis for character animation
Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 3
2024
-
[16]
Gaussianavatar: Towards realistic human avatar modeling from a single video via animatable 3d gaussians
Liangxiao Hu, Hongwen Zhang, Yuxiang Zhang, Boyao Zhou, Boning Liu, Shengping Zhang, and Liqiang Nie. Gaussianavatar: Towards realistic human avatar modeling from a single video via animatable 3d gaussians. In Pro- ceedings of the IEEE/CVF conference on computer vision and pat...
2024
-
[17]
Gauhuman: Articu- lated gaussian splatting from monocular human videos
Shoukang Hu, Tao Hu, and Ziwei Liu. Gauhuman: Articu- lated gaussian splatting from monocular human videos. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 20418–20431, 2024. 1, 3
2024
-
[18]
Humannorm: Learning normal diffusion model for high-quality and realistic 3d hu- man generation
Xin Huang, Ruizhi Shao, Qi Zhang, Hongwen Zhang, Ying Feng, Yebin Liu, and Qing Wang. Humannorm: Learning normal diffusion model for high-quality and realistic 3d hu- man generation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages...
2024
-
[19]
Dreamwaltz: Make a scene with complex 3d animatable avatars.Advances in Neural Information Processing Systems , 36:4566–4584,
Yukun Huang, Jianan Wang, Ailing Zeng, He Cao, Xianbiao Qi, Yukai Shi, Zheng-Jun Zha, and Lei Zhang. Dreamwaltz: Make a scene with complex 3d animatable avatars.Advances in Neural Information Processing Systems , 36:4566–4584,
-
[20]
Tech: Text-guided reconstruction of lifelike clothed humans
Yangyi Huang, Hongwei Yi, Yuliang Xiu, Tingting Liao, Ji- axiang Tang, Deng Cai, and Justus Thies. Tech: Text-guided reconstruction of lifelike clothed humans. In 2024 Interna- tional Conference on 3D Vision (3DV) , pages 1531–1542. IEEE, 2024. 3
2024
-
[21]
Sel- frecon: Self reconstruction your digital avatar from monoc- ular video
Boyi Jiang, Yang Hong, Hujun Bao, and Juyong Zhang. Sel- frecon: Self reconstruction your digital avatar from monoc- ular video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5605– 5615, 2022. 2
2022
-
[22]
Avatar- craft: Transforming text into neural human avatars with pa- rameterized shape and pose control
Ruixiang Jiang, Can Wang, Jingbo Zhang, Menglei Chai, Mingming He, Dongdong Chen, and Jing Liao. Avatar- craft: Transforming text into neural human avatars with pa- rameterized shape and pose control. In Proceedings of the IEEE/CVF International Conference on Computer Vision ,...
2023
-
[23]
Mvhuman: tailoring 2d diffusion with multi-view sampling for realistic 3d human generation
Suyi Jiang, Haimin Luo, Haoran Jiang, Ziyu Wang, Jingyi Yu, and Lan Xu. Mvhuman: tailoring 2d diffusion with multi-view sampling for realistic 3d human generation. arXiv preprint arXiv:2312.10120, 2023. 2, 3 9
2023 arXiv
-
[24]
In- stantavatar: Learning avatars from monocular video in 60 seconds
Tianjian Jiang, Xu Chen, Jie Song, and Otmar Hilliges. In- stantavatar: Learning avatars from monocular video in 60 seconds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16922– 16932, 2023. 3
2023
-
[25]
Neuman: Neural human radiance field from a single video
Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, and Anurag Ranjan. Neuman: Neural human radiance field from a single video. In European Conference on Computer Vision, pages 402–418. Springer, 2022. 1, 3
2022
-
[26]
3d gaussian splatting for real-time radiance field rendering
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4), 2023. 4
2023
-
[27]
Hugs: Human gaussian splats
Muhammed Kocabas, Jen-Hao Rick Chang, James Gabriel, Oncel Tuzel, and Anurag Ranjan. Hugs: Human gaussian splats. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 505–515, 2024. 3
2024
-
[28]
Dreamhuman: Animatable 3d avatars from text
Nikos Kolotouros, Thiemo Alldieck, Andrei Zanfir, Ed- uard Bazavan, Mihai Fieraru, and Cristian Sminchisescu. Dreamhuman: Animatable 3d avatars from text. Advances in Neural Information Processing Systems, 36:10516–10529,
-
[29]
Guess the unseen: Dynamic 3d scene reconstruction from partial 2d glimpses
Inhee Lee, Byungjun Kim, and Hanbyul Joo. Guess the unseen: Dynamic 3d scene reconstruction from partial 2d glimpses. 2024. 3
2024
-
[30]
Gart: Gaussian articulated template mod- els
Jiahui Lei, Yufu Wang, Georgios Pavlakos, Lingjie Liu, and Kostas Daniilidis. Gart: Gaussian articulated template mod- els. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 19876–19887,
-
[31]
Tava: Template-free animatable volumetric actors
Ruilong Li, Julian Tanke, Minh V o, Michael Zollh ¨ofer, J¨urgen Gall, Angjoo Kanazawa, and Christoph Lassner. Tava: Template-free animatable volumetric actors. In Eu- ropean Conference on Computer Vision , pages 419–436. Springer, 2022. 2
2022
-
[32]
Posevocab: Learning joint-structured pose embeddings for human avatar modeling
Zhe Li, Zerong Zheng, Yuxiao Liu, Boyao Zhou, and Yebin Liu. Posevocab: Learning joint-structured pose embeddings for human avatar modeling. In ACM SIGGRAPH 2023 con- ference proceedings, pages 1–11, 2023. 1, 2
2023
-
[33]
Ani- matable gaussians: Learning pose-dependent gaussian maps for high-fidelity human avatar modeling
Zhe Li, Zerong Zheng, Lizhen Wang, and Yebin Liu. Ani- matable gaussians: Learning pose-dependent gaussian maps for high-fidelity human avatar modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19711–19722, 2024. 1, 2, 4, 6, 13, 14
2024
-
[34]
Diffusion4d: Fast spatial-temporal consis- tent 4d generation via video diffusion models.arXiv preprint arXiv:2405.16645, 2024
Hanwen Liang, Yuyang Yin, Dejia Xu, Hanxue Liang, Zhangyang Wang, Konstantinos N Plataniotis, Yao Zhao, and Yunchao Wei. Diffusion4d: Fast spatial-temporal consis- tent 4d generation via video diffusion models.arXiv preprint arXiv:2405.16645, 2024. 3
2024 arXiv
-
[35]
Tada! text to animatable digital avatars
Tingting Liao, Hongwei Yi, Yuliang Xiu, Jiaxiang Tang, Yangyi Huang, Justus Thies, and Michael J Black. Tada! text to animatable digital avatars. In2024 International Con- ference on 3D Vision (3DV), pages 1508–1519. IEEE, 2024. 3
2024
-
[36]
Neural actor: Neural free-view synthesis of human actors with pose con- trol
Lingjie Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, and Christian Theobalt. Neural actor: Neural free-view synthesis of human actors with pose con- trol. ACM transactions on graphics (TOG) , 40(6):1–16,
-
[37]
Zero-1-to- 3: Zero-shot one image to 3d object
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 2
2023
-
[38]
Humangaus- sian: Text-driven 3d human generation with gaussian splat- ting
Xian Liu, Xiaohang Zhan, Jiaxiang Tang, Ying Shan, Gang Zeng, Dahua Lin, Xihui Liu, and Ziwei Liu. Humangaus- sian: Text-driven 3d human generation with gaussian splat- ting. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 6646–6657,
-
[39]
Wonder3d: Sin- gle image to 3d using cross-domain diffusion
Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Sin- gle image to 3d using cross-domain diffusion. In Proceed- ings of the IEEE/CVF conference on computer vision and pa...
2024
-
[40]
Matthew Loper, Naureen Mahmood, Javier Romero, Ger- ard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1–248:16, 2015. 1, 2
2015
-
[41]
Avatarstudio: Text-driven editing of 3d dynamic human head avatars
Mohit Mendiratta, Xingang Pan, Mohamed Elgharib, Kar- tik Teotia, Ayush Tewari, Vladislav Golyanik, Adam Ko- rtylewski, and Christian Theobalt. Avatarstudio: Text-driven editing of 3d dynamic human head avatars. ACM Transac- tions on Graphics (ToG), 42(6):1–18, 2023. 2, 3
2023
-
[42]
Expressive whole-body 3d gaussian avatar
Gyeongsik Moon, Takaaki Shiratori, and Shunsuke Saito. Expressive whole-body 3d gaussian avatar. In European Conference on Computer Vision , pages 19–35. Springer,
-
[43]
Humansplat: Generalizable single-image human gaus- sian splatting with structure priors
Panwang Pan, Zhuo Su, Chenguo Lin, Zhen Fan, Yongjie Zhang, Zeming Li, Tingting Shen, Yadong Mu, and Yebin Liu. Humansplat: Generalizable single-image human gaus- sian splatting with structure priors. Advances in Neural In- formation Processing Systems, 37:74383–74410, 2025. 3
2025
-
[44]
Effi- cient4d: Fast dynamic 3d object generation from a single- view video
Zijie Pan, Zeyu Yang, Xiatian Zhu, and Li Zhang. Effi- cient4d: Fast dynamic 3d object generation from a single- view video. arXiv preprint arXiv:2401.08742, 2024. 3
2024
-
[45]
Ash: Animatable gaussian splats for efficient and photoreal human rendering
Haokai Pang, Heming Zhu, Adam Kortylewski, Christian Theobalt, and Marc Habermann. Ash: Animatable gaussian splats for efficient and photoreal human rendering. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1165–1175, 2024. 3
2024
-
[46]
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pa...
2019
-
[47]
Scalable diffusion models with transformers
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,
-
[48]
Selfnerf: 10 Fast training nerf for human from monocular self-rotating video
Bo Peng, Jun Hu, Jingtao Zhou, and Juyong Zhang. Selfnerf: 10 Fast training nerf for human from monocular self-rotating video. arXiv preprint arXiv:2210.01651, 2022. 2
2022 arXiv
-
[49]
Ani- matable neural radiance fields for modeling dynamic human bodies
Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, and Hujun Bao. Ani- matable neural radiance fields for modeling dynamic human bodies. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 14314–14323, 2021. 1, 2
2021
-
[50]
Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans
Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. InProceedings of the IEEE/CVF conference on computer vision and ...
2021
-
[51]
Rmavatar: Photo- realistic human avatar reconstruction from monocular video based on rectified mesh-embedded gaussians
Sen Peng, Weixing Xie, Zilong Wang, Xiaohu Guo, Zhong- gui Chen, Baorong Yang, and Xiao Dong. Rmavatar: Photo- realistic human avatar reconstruction from monocular video based on rectified mesh-embedded gaussians. arXiv preprint arXiv:2501.07104, 2025. 3
2025 arXiv
-
[52]
Dreamfusion: Text-to-3d using 2d diffusion
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 2, 3
2022 arXiv
-
[53]
3dgs-avatar: Animatable avatars via deformable 3d gaussian splatting
Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, and Siyu Tang. 3dgs-avatar: Animatable avatars via deformable 3d gaussian splatting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5020–5030, 2024. 3
2024
-
[54]
Learning transferable visual models from natural language supervi- sion
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...
2021
-
[55]
Sam 2: Segment anything in images and videos
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R”adle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junt- ing Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Doll´ar, and Christoph Feic...
2024 arXiv
-
[56]
L4gm: Large 4d gaussian reconstruction model
Jiawei Ren, Cheng Xie, Ashkan Mirzaei, Karsten Kreis, Zi- wei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, Huan Ling, et al. L4gm: Large 4d gaussian reconstruction model. Advances in Neural Information Processing Systems, 37:56828–56858, 2024. 3, 7
2024
-
[57]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 3
2022
-
[58]
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...
2023
-
[59]
Human4dit: 360-degree human video generation with 4d diffusion transformer
Ruizhi Shao, Youxin Pang, Zerong Zheng, Jingxiang Sun, and Yebin Liu. Human4dit: 360-degree human video generation with 4d diffusion transformer. arXiv preprint arXiv:2405.17405, 2024. 2, 4
2024 arXiv
-
[60]
Splattingavatar: Realistic real-time human avatars with mesh-embedded gaussian splatting
Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, and Zeyu Wang. Splattingavatar: Realistic real-time human avatars with mesh-embedded gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...
2024
-
[61]
A-nerf: Articulated neural radiance fields for learn- ing human shape, appearance, and pose
Shih-Yang Su, Frank Yu, Michael Zollh ¨ofer, and Helge Rhodin. A-nerf: Articulated neural radiance fields for learn- ing human shape, appearance, and pose. Advances in neural information processing systems, 34:12278–12291, 2021. 2
2021
-
[62]
Danbo: Disentangled articulated neural body representations via graph neural networks
Shih-Yang Su, Timur Bagautdinov, and Helge Rhodin. Danbo: Disentangled articulated neural body representations via graph neural networks. InEuropean Conference on Com- puter Vision, pages 107–124. Springer, 2022. 2
2022
-
[63]
Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion
Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion. In European Conference on Computer...
2024
-
[64]
Disentangled clothed avatar generation from text descriptions
Jionghao Wang, Yuan Liu, Zhiyang Dou, Zhengming Yu, Yongqing Liang, Cheng Lin, Rong Xie, Li Song, Xin Li, and Wenping Wang. Disentangled clothed avatar generation from text descriptions. In European Conference on Com- puter Vision, pages 381–401. Springer, 2024. 3
2024
-
[65]
Arah: Animatable volume rendering of articulated hu- man sdfs
Shaofei Wang, Katja Schwarz, Andreas Geiger, and Siyu Tang. Arah: Animatable volume rendering of articulated hu- man sdfs. In European conference on computer vision, pages 1–19. Springer, 2022. 1, 2
2022
-
[66]
Disco: Disentangled control for realistic human dance generation
Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung- Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,...
2024
-
[67]
Videocomposer: Compositional video synthesis with motion controllability
Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis with motion controllability. Advances in Neural Information Processing Systems, 36:7594–7611, 2023. 3
2023
-
[68]
Emu3: Next-token prediction is all you need
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 3
2024 arXiv
-
[69]
Humancoser: Layered 3d human generation via semantic-aware diffusion model
Yi Wang, Jian Ma, Ruizhi Shao, Qiao Feng, Yu-Kun Lai, and Kun Li. Humancoser: Layered 3d human generation via semantic-aware diffusion model. In 2024 IEEE Inter- national Symposium on Mixed and Augmented Reality (IS- MAR), pages 436–445. IEEE, 2024. 3
2024
-
[70]
Tram: Global trajectory and motion of 3d humans from in- the-wild videos
Yufu Wang, Ziyun Wang, Lingjie Liu, and Kostas Daniilidis. Tram: Global trajectory and motion of 3d humans from in- the-wild videos. In European Conference on Computer Vi- sion, pages 467–487. Springer, 2024. 3
2024
-
[71]
Image quality assessment: from error visibility to 11 structural similarity
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to 11 structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 7
2004
-
[72]
Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems , 36:8406–8441, 2023. 3
2023
-
[73]
Wonderhuman: Hallucinating unseen parts in dynamic 3d human reconstruction
Zilong Wang, Zhiyang Dou, Yuan Liu, Cheng Lin, Xiao Dong, Yunhui Guo, Chenxu Zhang, Xin Li, Wenping Wang, and Xiaohu Guo. Wonderhuman: Hallucinating unseen parts in dynamic 3d human reconstruction. arXiv preprint arXiv:2502.01045, 2025. 3
2025
-
[74]
Gomavatar: Efficient an- imatable human modeling from monocular video using gaussians-on-mesh
Jing Wen, Xiaoming Zhao, Zhongzheng Ren, Alexander G Schwing, and Shenlong Wang. Gomavatar: Efficient an- imatable human modeling from monocular video using gaussians-on-mesh. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 2059–20...
-
[75]
Hu- mannerf: Free-viewpoint rendering of moving people from monocular video
Chung-Yi Weng, Brian Curless, Pratul P Srinivasan, Jonathan T Barron, and Ira Kemelmacher-Shlizerman. Hu- mannerf: Free-viewpoint rendering of moving people from monocular video. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern Recognition , pages 162...
2022
-
[76]
Show-o: One single transformer to unify multimodal understanding and generation
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024. 2, 3
2024 arXiv
-
[77]
Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency
Yiming Xie, Chun-Han Yao, Vikram V oleti, Huaizu Jiang, and Varun Jampani. Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency. arXiv preprint arXiv:2407.17470, 2024. 3
2024 arXiv
-
[78]
Seeavatar: Pho- torealistic text-to-3d avatar generation with constrained ge- ometry and appearance
Yuanyou Xu, Zongxin Yang, and Yi Yang. Seeavatar: Pho- torealistic text-to-3d avatar generation with constrained ge- ometry and appearance. arXiv preprint arXiv:2312.08889 ,
-
[79]
Magicanimate: Temporally consistent human im- age animation using diffusion model
Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...
2024
-
[80]
Diffusion 2: Dynamic 3d content generation via score composition of video and multi-view diffusion models
Zeyu Yang, Zijie Pan, Chun Gu, and Li Zhang. Diffusion 2: Dynamic 3d content generation via score composition of video and multi-view diffusion models. arXiv preprint arXiv:2404.02148, 2024. 3
2024 arXiv
-
[81]
Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,
-
[82]
Monohuman: Animatable human neu- ral field from monocular video
Zhengming Yu, Wei Cheng, Xian Liu, Wayne Wu, and Kwan-Yee Lin. Monohuman: Animatable human neu- ral field from monocular video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16943–16953, 2023. 2
2023
-
[83]
Mip-splatting: Alias-free 3d gaussian splat- ting
Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splat- ting. Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2024. 2
2024
-
[84]
Avatarbooth: High-quality and customizable 3d human avatar generation
Yifei Zeng, Yuanxun Lu, Xinya Ji, Yao Yao, Hao Zhu, and Xun Cao. Avatarbooth: High-quality and customizable 3d human avatar generation. arXiv preprint arXiv:2306.09864,
-
[85]
Gaussian vari- ation field diffusion for high-fidelity video-to-4d synthesis
Bowen Zhang, Sicheng Xu, Chuxin Wang, Jiaolong Yang, Feng Zhao, Dong Chen, and Baining Guo. Gaussian vari- ation field diffusion for high-fidelity video-to-4d synthesis. arXiv preprint arXiv:2507.23785, 2025. 3, 7
2025 arXiv
-
[86]
Avatarverse: High-quality & stable 3d avatar creation from text and pose
Huichao Zhang, Bowen Chen, Hao Yang, Liao Qu, Xu Wang, Li Chen, Chao Long, Feida Zhu, Daniel Du, and Min Zheng. Avatarverse: High-quality & stable 3d avatar creation from text and pose. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7124–7132, 2024. 3
2024
-
[87]
4diffusion: Multi-view video dif- fusion model for 4d generation
Haiyu Zhang, Xinyuan Chen, Yaohui Wang, Xihui Liu, Yun- hong Wang, and Yu Qiao. 4diffusion: Multi-view video dif- fusion model for 4d generation. Advances in Neural Infor- mation Processing Systems, 37:15272–15295, 2024. 3
2024
-
[88]
Adding conditional control to text-to-image diffusion models
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. 3
2023
-
[89]
The unreasonable effectiveness of deep features as a perceptual metric
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 7, 13
2018
-
[90]
Structured local radiance fields for human avatar modeling
Zerong Zheng, Han Huang, Tao Yu, Hongwen Zhang, Yan- dong Guo, and Yebin Liu. Structured local radiance fields for human avatar modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15893–15903, 2022. 1, 2, 6
2022
-
[91]
Avatarrex: Real-time expressive full- body avatars
Zerong Zheng, Xiaochen Zhao, Hongwen Zhang, Boning Liu, and Yebin Liu. Avatarrex: Real-time expressive full- body avatars. ACM Transactions on Graphics (TOG), 42(4): 1–19, 2023. 1, 2
2023
-
[92]
Champ: Controllable and consistent human image animation with 3d parametric guidance
Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision (ECCV), 2024. 3
2024
-
[93]
Driv- able 3d gaussian avatars
Wojciech Zielonka, Timur Bagautdinov, Shunsuke Saito, Michael Zollh ¨ofer, Justus Thies, and Javier Romero. Driv- able 3d gaussian avatars. arXiv preprint arXiv:2311.08581,
-
[2023]
back-view
3 12 DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective Supplementary Material A. Implementation Details The avatar is finally optimized by: L = Lrgb + λLPIPSLLPIPS + λoffsetLoffset + λscaleLscale, (4) where Lrgb is the L1 photometric lo...
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.