Pith. sign in

REVIEW 4 major objections 6 minor 73 references

One Shot, One Talk: Whole-body Talking Avatar from a Single Image

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read From one photograph, the paper constructs a photorealistic, precisely animatable whole-body talking avatar using diffusion-generated pseudo-videos and a coupled 3DGS-mesh representation.

desk verdict Solid one-shot avatar engineering with thin evidence; deserves review with major revision. read the letter →

arxiv 2412.01106 v1 pith:XV4JGICN submitted 2024-12-02 cs.CV cs.GR

Jun Xiang , Yudong Guo , Leipeng Hu , Boyang Guo , Yancheng Yuan , Juyong Zhang This is my paper · ORCID
classification cs.CVcs.GR
keywords one-shotavatarwhole-bodytalking3DGaussiansplattingSMPL-Xdiffusionpseudo-labelsperceptuallossimage-to-videoanimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a photorealistic, precisely animatable, and expressive whole-body talking avatar can be built from a single photograph. The strategy is to synthesize many pseudo-video frames of the person performing diverse gestures and expressions using pre-trained image-to-video diffusion models, then train a tightly coupled 3D Gaussian-and-mesh avatar using perceptual supervision on those imperfect frames and per-pixel supervision on the original photo. If the claim holds, it removes the need for multi-view or self-rotating video capture, making expressive avatar creation a one-click consumer operation.

What carries the argument

The load-bearing object is a coupled 3DGS-mesh hybrid avatar: 3D Gaussians are initialized on the surface of a whole-body parametric mesh (SMPL-X) via UV parameterization, and two deformation fields are optimized jointly—one deforming the Gaussians, one deforming the mesh—kept aligned by a mesh-Gaussian consistency loss. The mesh supplies geometric priors and smoothness through normal-consistency, mask, and Laplacian losses, while the Gaussians provide expressive rendering; re-tracking of the pseudo-frames corrects pose misalignment before perceptual supervision.

What would settle it

A concrete test is to train the identical pipeline with real captured video frames of the same subject as labels instead of diffusion pseudo-frames; if the real-label avatar is clearly better, residual pseudo-label noise is the bottleneck. A complementary check is to measure identity preservation between pseudo-frames and the source image (for example, face-verification similarity) and test whether avatar quality collapses below a detectable similarity threshold.

Watch

Extended reading notes

Core claim

The central claim is that noisy, diffusion-generated pseudo-videos, combined with a coupled 3DGS-mesh avatar representation and carefully chosen regularizations, are sufficient to convert a single image into a whole-body talking avatar that preserves identity and can be driven by novel gestures and expressions. The paper argues that per-pixel losses on pseudo-labels produce blur, flicker, and identity drift, whereas a perceptual loss (LPIPS) captures high-level human structure while the source image supplies exact detail; mesh constraints keep the Gaussian field geometrically coherent. In self-driven reenactment experiments the resulting one-shot avatar is reported to outperform methods that require short video input on MSE, L1, PSNR, SSIM, and LPIPS.

Load-bearing premise

The load-bearing premise is that the noise, identity drift, and pose misalignment in diffusion-generated pseudo-videos are small enough that, after re-tracking, perceptual-loss supervision fills occluded regions with the person's true appearance; if the diffusion models hallucinate identity-altering details or systematic pose errors, the avatar inherits them.

Editorial extensions

If this is right

  • One-shot avatar construction no longer requires multi-view rigs, light stages, or minutes of self-rotating monocular video.
  • Any SMPL-X pose or expression sequence can drive the avatar, enabling fine-grained control of hand gestures and facial expressions.
  • The perceptual-loss-on-pseudo-frames recipe demonstrates that imperfect diffusion outputs can serve as training labels for 3D avatar learning, provided the representation adds geometric regularity.
  • In the reported self-driven evaluation, the one-shot avatar beats video-input baselines across all five image-quality metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would be to measure pseudo-label fidelity (for example, identity similarity between pseudo-frames and the source image) and map how avatar quality degrades as that fidelity drops; the paper does not report such a map.
  • The same pipeline is likely transferable to other articulated categories that have parametric models and pose-guided video diffusion, such as animals or stylized characters.
  • If the approach generalizes, it suggests a broader recipe for 3D reconstruction from generated data: use diffusion to create coverage, re-track to fix alignment, and supervise with perceptual losses rather than pixel losses.
  • A natural upper-bound test is to train the identical pipeline with real multi-view video labels from the same subject; the gap between that result and the pseudo-label result would quantify the cost of trusting diffusion outputs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a pipeline that constructs a whole-body, animatable talking avatar from a single input image. The method generates imperfect pseudo-videos of the target person using pose-guided diffusion models (MimicMotion for body, Portrait4D-v2 for face), driven by motion sequences from the TED Gesture Dataset, then re-tracks these videos to obtain SMPL-X pose parameters. A coupled 3DGS-mesh avatar representation is optimized with per-pixel losses on the source image, LPIPS perceptual supervision on pseudo-frames, and several mesh/Gaussian regularizers. The authors report qualitative comparisons against ExAvatar, ELICIT, MimicMotion, and Make-Your-Anchor, plus a quantitative self-driven reenactment table, and conclude that the one-shot avatar surpasses video-input methods.

Significance. If the reported results are reliable, the paper would be a meaningful step toward practical one-shot avatar creation, enabling whole-body animation with gestures and expressions from a single photo. The hybrid 3DGS-mesh representation and the deliberate use of perceptual loss to tolerate noisy diffusion labels are sensible design choices, and the idea of distilling a pose-controllable avatar from off-the-shelf video diffusion models is timely and potentially influential. However, the current evidence is not sufficient to substantiate the strong advertised claims of 'photorealistic, precisely animatable' performance: the quantitative evaluation is thin and partly circular, the train/test pose-distribution gap is unexamined, and the ablation study is qualitative. The paper is an interesting engineering contribution, but it needs substantially stronger validation before the central claims can be accepted.

major comments (4)
  1. [4.2, Table 1] The quantitative comparison reports only a single value per metric per method, with no error bars, no subject count, and no statistical significance tests. To support 'Our method outperforms others across these metrics,' the authors should report per-subject mean and standard deviation, the number of test sequences, and paired significance tests (e.g., Wilcoxon). As written, the magnitude of the differences (e.g., LPIPS 0.030 vs 0.039) could be within inter-subject variability. Also, Make-Your-Anchor is compared qualitatively but omitted from Table 1; either include its numbers or explain its exclusion.
  2. [3.2 and 5 (quantitative comparison)] The body pseudo-labels are generated by MimicMotion (Eq. 6), and MimicMotion is then used as a baseline in Table 1 and Figs. 3-4. This is a teacher-student circularity: the proposed method is trained on MimicMotion's outputs, so a favorable comparison against MimicMotion is partially by construction and does not demonstrate superiority over the knowledge already contained in that teacher. The paper should explicitly discuss this dependency, and ideally compare against at least one pose-guided video diffusion model that was not used as a teacher, or evaluate on a setting (e.g., pose editing or out-of-distribution gestures) where the teacher itself is weak.
  3. [3.2, Fig. 9] Training uses re-tracked pseudo poses (⌢θ̂_N, φ̂_N) obtained from the generated frames, but inference feeds user-provided SMPL-X poses directly into LBS and Gaussian splatting without re-tracking. If re-tracking systematically biases the pose estimates (e.g., toward less extreme hand articulations or compensating for diffusion artifacts), the avatar learns a pose-to-appearance mapping on a distribution that differs from real-fitted test poses. The paper neither quantifies this gap nor provides pose-accuracy metrics. To substantiate 'precisely animatable,' the authors should measure the deviation between original driving poses and re-tracked poses, and report pose error or joint re-projection error on the self-driven reenactment task, or otherwise demonstrate that re-tracking is distribution-preserving.
  4. [3.3, Eq. (9), Fig. 6] Only the LPIPS loss supervises the pseudo frames; no per-pixel or geometric loss is applied to them. The authors motivate this by pseudo-label misalignment, but LPIPS alone can be satisfied by appearance averaging or by ignoring precise pose-conditioned texture changes, potentially causing blurring or identity drift in occluded regions (a risk acknowledged only qualitatively in the supplementary limitations). The ablation in Fig. 6 comparing perceptual guidance with L1/SSIM guidance is purely visual. Provide quantitative evidence on held-out real frames (e.g., LPIPS/L1 with respect to ground truth, or a user study) that perceptual guidance yields better fidelity, and ideally a sensitivity analysis for λ_LPIPS = 0.2.
minor comments (6)
  1. [Title and throughout] The title and several section headings contain 'A vatar' with a spurious space; correct these to 'Avatar'.
  2. [3.2, root pose augmentation] The rationale for the specific root pose perturbation ranges (pitch ∈ (−30°, 30°), yaw ∈ (−10°, 10°)) is not given, and it is unclear whether inference-time poses ever exceed these ranges; consider reporting the pose distribution of the test set.
  3. [4.3, Fig. 7] The abbreviation 'w/o mesh SC' is undefined; spell out 'mesh soft constraints' in the caption.
  4. [4.2, Table 1 caption] The caption states 'ExAvatar here denotes ExAvatar-40shot' after the table, but the main text does not consistently clarify that this setting uses 40 input images; make this explicit when first referencing the table.
  5. [4.2, qualitative comparison] The text claims Make-Your-Anchor 'struggles with short fine-tuning videos' but provides no quantitative support; consider adding numbers or a supplementary figure.
  6. [3.3, Eq. (7)] The mask loss L_M is used in Eq. (7) but never formally defined; specify how the rendered mesh mask is obtained and compared (e.g., L1 on alpha maps against the BiRefNet mask).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the one-shot avatar training uses external diffusion pseudo-labels and is validated against ground-truth video, so the central claim is not equivalent to its inputs.

full rationale

The paper's derivation chain is empirical rather than formal: a source image and SMPL-X registration are combined with pseudo-video frames generated by external diffusion models (MimicMotion and Portrait4D-v2), re-tracked poses, and a coupled 3DGS-mesh representation. No predicted quantity is defined as a fitted input. In Eq. (9), Ldiff = λLPIPS LPIPS(IN, G(θ̂N, φ̂N, P)) supervises a rendered avatar image against a pseudo frame; the rendered image is not equal to the pseudo frame by construction, and the pseudo frame comes from an independent generative model rather than from the avatar being optimized. The re-tracking step maps pseudo frames to pose parameters, but the loss still compares two different image generation processes (diffusion vs. 3DGS-mesh rendering), so the supervision is not circular. Evaluation is performed on self-driven ground-truth video from ActorsHQ and Casual Conversations, not on the pseudo-label training set, so the comparison against the teacher model MimicMotion is not the sole evidence. The only self-citation, [54] FlashAvatar, appears in Sec. 3.1 as "Inspired by [1, 39, 54]" for UV-based Gaussian initialization and is not load-bearing; the same initialization is supported by independent prior work [1, 39]. The supplementary limitations (Sec. B) acknowledge tracking inaccuracies, finger self-intersection, and reduced performance at large viewing angles; these are correctness and generalization limitations, not evidence of circularity. The concern that re-tracked training poses may differ from inference-time fitted poses is a real generalization risk, but the central claim remains empirically testable on held-out driving poses and is not forced by definition.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities; its contribution is a trained hybrid representation. The main uncharged inputs are hand-tuned loss weights, a warm-up schedule, pose augmentation ranges, and assumptions about SMPL-X, diffusion pseudo-label fidelity, LPIPS as a correctness proxy, and TED data diversity.

free parameters (4)
  • Loss weights for all objective terms = lambda_normal=1e-2, lambda_M=1e-1, lambda_MGC=1e1, lambda_lap=1e2, lambda_LPIPS=2e-1, lambda_RGB=8e-1…
    Chosen by hand in Sec. 3.4 to balance per-pixel, perceptual, mask, normal, Laplacian, and consistency losses. No sensitivity analysis is reported.
  • Perceptual loss warm-up step = 2000
    Empirically set in Sec. 3.4; before this step, only the other losses train the avatar. No ablation of this schedule is shown.
  • Root pose augmentation ranges = pitch in (-30,30) degrees, yaw in (-10,10) degrees
    Randomly set per segment to increase viewpoint generalization; these ranges are manually chosen in Sec. 3.2 and are not motivated by data statistics.
  • Gaussian count and UV resolution = approximately 150,000 Gaussians; 512x512 UV map
    Implementation choices in Sec. 3.4 that affect detail and memory; no ablation is reported for these values.
assumptions (5)
  • domain assumption SMPL-X provides an adequate parametric space for whole-body pose, hand, and expression control.
    Used throughout Sec. 3.1; if SMPL-X cannot represent fine finger or face motions, the avatar cannot either.
  • ad hoc to paper MimicMotion and Portrait4D-v2 generate pseudo-frames from the input image whose appearance and identity are reliable enough after re-tracking to serve as training labels.
    Secs. 3.2 and 3.3; this is the only source of supervision for regions unseen in the single image. The paper does not independently verify label correctness.
  • ad hoc to paper LPIPS perceptual similarity is an appropriate loss for noisy, misaligned pseudo-labels and correlates with correct human structure and identity.
    Eq. (9); the paper motivates this on empirical grounds but provides no validation that LPIPS minima correspond to identity-preserving geometry.
  • domain assumption The TED Gesture Dataset provides enough diversity of talking gestures and expressions to generalize to novel poses.
    Sec. 4.1 uses 100 TED videos to build the motion space; no analysis of hand or face pose coverage is given.
  • domain assumption 3D Gaussian Splatting with isotropic Gaussians and fixed opacity and rotation can represent the target's clothed body appearance.
    Sec. 3.4 follows prior work [16]; this is a standard assumption in the field but still a representational capacity assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of One Shot, One Talk: Whole-body Talking Avatar from a Single Image." pith.science (2026). https://pith.science/paper/XV4JGICN

@misc{pith2026241201106,
  author       = {Pith},
  title        = {Pith review of: One Shot, One Talk: Whole-body Talking Avatar from a Single Image},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XV4JGICN}},
  note         = {Machine review of arXiv:2412.01106}
}
read the original abstract

Building realistic and animatable avatars still requires minutes of multi-view or monocular self-rotating videos, and most methods lack precise control over gestures and expressions. To push this boundary, we address the challenge of constructing a whole-body talking avatar from a single image. We propose a novel pipeline that tackles two critical issues: 1) complex dynamic modeling and 2) generalization to novel gestures and expressions. To achieve seamless generalization, we leverage recent pose-guided image-to-video diffusion models to generate imperfect video frames as pseudo-labels. To overcome the dynamic modeling challenge posed by inconsistent and noisy pseudo-videos, we introduce a tightly coupled 3DGS-mesh hybrid avatar representation and apply several key regularizations to mitigate inconsistencies caused by imperfect labels. Extensive experiments on diverse subjects demonstrate that our method enables the creation of a photorealistic, precisely animatable, and expressive whole-body talking avatar from just a single image.

Figures

Figures reproduced from arXiv: 2412.01106 by the authors.

Figure 1
Figure 1. Given a one-shot image (e.g., your favorite photo) as input, our method reconstructs a fully expressive whole-body talking avatar [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview. Our method constructs an expressive whole-body talking avatar from a single image. We begin by generating pseudo [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparisons with representative methods [ [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons with representative methods [ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: More examples of cross-identity pose reenactment. Dif [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Perceptual diffusion guidance is of great importance to inpainting unseen regions and modeling natural and realistic textures. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Soft mesh constraints together with Gaussian Laplacian [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Re-Tracking step preserves better texture structures and [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: A detailed illustration of our pipeline. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Inaccurate tracking and finger self-intersection during [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Novel view results across diverse angles. [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 49 canonical work pages

  1. [1]

    Gaussian shell maps for efficient 3d human generation

    Rameen Abdal, Wang Yifan, Zifan Shi, Yinghao Xu, Ryan Po, Zhengfei Kuang, Qifeng Chen, Dit-Yan Yeung, and Gor- don Wetzstein. Gaussian shell maps for efficient 3d human generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9441– 9451, 2024. 3

    2024

  2. [2]

    Single-image 3d human digitization with shape-guided diffusion

    Badour AlBahar, Shunsuke Saito, Hung-Yu Tseng, Changil Kim, Johannes Kopf, and Jia-Bin Huang. Single-image 3d human digitization with shape-guided diffusion. In SIG- GRAPH Asia 2023 Conference Papers , pages 1–11, 2023. 3

    work page 2023

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3

    arXiv 2023

  4. [4]

    Structured 3d features for reconstructing control- lable avatars

    Enric Corona, Mihai Zanfir, Thiemo Alldieck, Ed- uard Gabriel Bazavan, Andrei Zanfir, and Cristian Smin- chisescu. Structured 3d features for reconstructing control- lable avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16954– 16964, 2023. 3

    work page 2023

  5. [5]

    Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer

    Yu Deng, Duomin Wang, and Baoyuan Wang. Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer. In Proceedings of the European conference on computer vision (ECCV), 2024. 2, 4, 5

    work page 2024

  6. [6]

    Ag3d: Learning to gener- ate 3d avatars from 2d image collections

    Zijian Dong, Xu Chen, Jinlong Yang, Michael J Black, Ot- mar Hilliges, and Andreas Geiger. Ag3d: Learning to gener- ate 3d avatars from 2d image collections. In Proceedings of the IEEE/CVF international conference on computer vision, pages 14916–14927, 2023. 3

    work page 2023

  7. [7]

    Dreamoving: A human video generation framework based on diffusion models

    Mengyang Feng, Jinlin Liu, Kai Yu, Yuan Yao, Zheng Hui, Xiefan Guo, Xianhui Lin, Haolan Xue, Chen Shi, Xiaowen Li, et al. Dreamoving: A human video generation framework based on diffusion models. arXiv e-prints , pages arXiv– 2312, 2023. 3

    work page 2023

  8. [8]

    Livepor- trait: Efficient portrait animation with stitching and retarget- ing control

    Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang. Livepor- trait: Efficient portrait animation with stitching and retarget- ing control. arXiv preprint arXiv:2407.03168, 2024. 4

    arXiv 2024

Show all 73 references
  1. [9]

    The re- lightables: V olumetric performance capture of humans with realistic relighting

    Kaiwen Guo, Peter Lincoln, Philip Davidson, Jay Busch, Xueming Yu, Matt Whalen, Geoff Harvey, Sergio Orts- Escolano, Rohit Pandey, Jason Dourgarian, et al. The re- lightables: V olumetric performance capture of humans with realistic relighting. ACM Transactions on Graphics (To...

    2019

  2. [10]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning, 2023. 3

    2023

  3. [11]

    Real-time deep dynamic characters

    Marc Habermann, Lingjie Liu, Weipeng Xu, Michael Zoll- hoefer, Gerard Pons-Moll, and Christian Theobalt. Real-time deep dynamic characters. ACM Transactions on Graphics (ToG), 40(4):1–16, 2021. 2

    2021

  4. [12]

    Towards measuring fairness in ai: the casual conversations dataset

    Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, and Cristian Canton Ferrer. Towards measuring fairness in ai: the casual conversations dataset. IEEE Transactions on Biometrics, Behavior, and Identity Science, 4(3):324–332, 2021. 6

    2021

  5. [13]

    Sith: Single-view tex- tured human reconstruction with image-conditioned diffu- sion

    I Ho, Jie Song, Otmar Hilliges, et al. Sith: Single-view tex- tured human reconstruction with image-conditioned diffu- sion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 538–549, 2024. 3

    2024

  6. [14]

    Expres- sive gaussian human avatars from monocular rgb video

    Hezhen Hu, Zhiwen Fan, Tianhao Wu, Yihan Xi, Seoyoung Lee, Georgios Pavlakos, and Zhangyang Wang. Expres- sive gaussian human avatars from monocular rgb video. In NeurIPS, 2024. 3, 4

    2024

  7. [15]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 2, 3, 4

    2024

  8. [16]

    Gaussianavatar: Towards realistic human avatar model- ing from a single video via animatable 3d gaussians

    Liangxiao Hu, Hongwen Zhang, Yuxiang Zhang, Boyao Zhou, Boning Liu, Shengping Zhang, and Liqiang Nie. Gaussianavatar: Towards realistic human avatar model- ing from a single video via animatable 3d gaussians. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (C...

    2024

  9. [17]

    Gauhuman: Articu- lated gaussian splatting from monocular human videos

    Shoukang Hu, Tao Hu, and Ziwei Liu. Gauhuman: Articu- lated gaussian splatting from monocular human videos. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20418–20431, 2024. 2, 3

    2024

  10. [18]

    One-shot implicit animatable avatars with model- based priors

    Yangyi Huang, Hongwei Yi, Weiyang Liu, Haofan Wang, Boxi Wu, Wenxiao Wang, Binbin Lin, Debing Zhang, and Deng Cai. One-shot implicit animatable avatars with model- based priors. In IEEE Conference on Computer Vision (ICCV), 2023. 3, 5, 6, 7

    2023

  11. [19]

    Tech: Text-guided reconstruction of lifelike clothed humans

    Yangyi Huang, Hongwei Yi, Yuliang Xiu, Tingting Liao, Ji- axiang Tang, Deng Cai, and Justus Thies. Tech: Text-guided reconstruction of lifelike clothed humans. In 2024 Interna- tional Conference on 3D Vision (3DV) , pages 1531–1542. IEEE, 2024. 3, 4

    2024

  12. [20]

    Make-your-anchor: A diffusion-based 2d avatar generation framework

    Ziyao Huang, Fan Tang, Yong Zhang, Xiaodong Cun, Juan Cao, Jintao Li, and Tong-Yee Lee. Make-your-anchor: A diffusion-based 2d avatar generation framework. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6997–7006, 2024. 3, 6, 7

    2024

  13. [21]

    Humanrf: High-fidelity neural radiance fields for humans in motion

    Mustafa Is ¸ık, Martin R ¨unz, Markos Georgopoulos, Taras Khakhulin, Jonathan Starck, Lourdes Agapito, and Matthias Nießner. Humanrf: High-fidelity neural radiance fields for humans in motion. ACM Transactions on Graphics (TOG), 42(4):1–12, 2023. 6

    2023

  14. [22]

    Humangen: Generating hu- man radiance fields with explicit priors

    Suyi Jiang, Haoran Jiang, Ziyu Wang, Haimin Luo, Wen- zheng Chen, and Lan Xu. Humangen: Generating hu- man radiance fields with explicit priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12543–12554, 2023. 3

    2023

  15. [23]

    Neuman: Neural human radiance field from a single video

    Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, and Anurag Ranjan. Neuman: Neural human radiance field from a single video. In Proceedings of the European confer- ence on computer vision (ECCV), 2022. 2

    2022

  16. [24]

    Hifi4g: 9 High-fidelity human performance rendering via compact gaussian splatting

    Yuheng Jiang, Zhehao Shen, Penghao Wang, Zhuo Su, Yu Hong, Yingliang Zhang, Jingyi Yu, and Lan Xu. Hifi4g: 9 High-fidelity human performance rendering via compact gaussian splatting. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages ...

    2024

  17. [25]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  18. [26]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 6

    2014 arXiv

  19. [27]

    HUGS: Human gaussian splatting

    Muhammed Kocabas, Jen-Hao Rick Chang, James Gabriel, Oncel Tuzel, and Anurag Ranjan. HUGS: Human gaussian splatting. In 2024 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2024. 2

    2024

  20. [28]

    Modular primitives for high-performance differentiable rendering

    Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primitives for high-performance differentiable rendering. ACM Transac- tions on Graphics, 39(6), 2020. 5

    2020

  21. [29]

    Gart: Gaussian articulated template mod- els

    Jiahui Lei, Yufu Wang, Georgios Pavlakos, Lingjie Liu, and Kostas Daniilidis. Gart: Gaussian articulated template mod- els. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 19876–19887,

  22. [30]

    Pshuman: Photorealistic single-view human reconstruction using cross-scale diffusion

    Peng Li, Wangguandong Zheng, Yuan Liu, Tao Yu, Yang- guang Li, Xingqun Qi, Mengfei Li, Xiaowei Chi, Siyu Xia, Wei Xue, et al. Pshuman: Photorealistic single-view human reconstruction using cross-scale diffusion. arXiv preprint arXiv:2409.10141, 2024. 3

    2024 arXiv

  23. [31]

    Learning a model of facial shape and expression from 4d scans

    Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Transactions on Graphics (ToG) , 36 (6):194–1, 2017. 2

    2017

  24. [32]

    Neural actor: Neural free-view synthesis of human actors with pose con- trol

    Lingjie Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, and Christian Theobalt. Neural actor: Neural free-view synthesis of human actors with pose con- trol. ACM Transactions on Graphics (TOG) , 40(6):1–16,

  25. [33]

    Human-vdm: Learning single-image 3d human gaussian splatting from video diffusion models

    Zhibin Liu, Haoye Dong, Aviral Chharia, and Hefeng Wu. Human-vdm: Learning single-image 3d human gaussian splatting from video diffusion models. arXiv preprint arXiv:2409.02851, 2024. 3

    2024

  26. [34]

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. Smpl: a skinned multi- person linear model. ACM Transactions on Graphics (ToG), 34(6), 2015. 2

    2015

  27. [35]

    The contextual loss for image transformation with non-aligned data

    Roey Mechrez, Itamar Talmi, and Lihi Zelnik-Manor. The contextual loss for image transformation with non-aligned data. In Proceedings of the European conference on com- puter vision (ECCV), pages 768–783, 2018. 6

    2018

  28. [36]

    Expressive whole-body 3D gaussian avatar

    Gyeongsik Moon, Takaaki Shiratori, and Shunsuke Saito. Expressive whole-body 3D gaussian avatar. In ECCV, 2024. 3, 4, 5, 6, 7

    2024

  29. [37]

    Human gaussian splatting: Real-time rendering of animatable avatars

    Arthur Moreau, Jifei Song, Helisa Dhamo, Richard Shaw, Yiren Zhou, and Eduardo P ´erez-Pellitero. Human gaussian splatting: Real-time rendering of animatable avatars. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 788–798, 2024. 3

    2024

  30. [38]

    Humansplat: Generalizable single-image human gaussian splatting with structure priors

    Panwang Pan, Zhuo Su, Chenguo Lin, Zhen Fan, Yongjie Zhang, Zeming Li, Tingting Shen, Yadong Mu, and Yebin Liu. Humansplat: Generalizable single-image human gaussian splatting with structure priors. arXiv preprint arXiv:2406.12459, 2024. 3

    2024 arXiv

  31. [39]

    Ash: Animatable gaussian splats for efficient and photoreal human rendering

    Haokai Pang, Heming Zhu, Adam Kortylewski, Christian Theobalt, and Marc Habermann. Ash: Animatable gaussian splats for efficient and photoreal human rendering. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1165–1175, 2024. 3

    2024

  32. [40]

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pa...

    2019

  33. [41]

    Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans

    Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In Proceed- ings of the IEEE/CVF Conference on Computer Vision a...

    2021

  34. [42]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 3, 4

    2022 arXiv

  35. [43]

    3dgs-avatar: Animatable avatars via deformable 3d gaussian splatting

    Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, and Siyu Tang. 3dgs-avatar: Animatable avatars via deformable 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2, 3

    2024

  36. [44]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  37. [45]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  38. [46]

    Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: modeling and capturing hands and bod- ies together. ACM Transactions on Graphics (ToG), 36(6),

  39. [47]

    Pifu: Pixel-aligned implicit function for high-resolution clothed human digitiza- tion

    Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Mor- ishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitiza- tion. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 2304–2314, 2019. 3

    2019

  40. [48]

    Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization

    Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 84–93, 2020. 3

    2020

  41. [49]

    10 SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting

    Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, and Zeyu Wang. 10 SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2024

  42. [50]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 6

    2014 arXiv

  43. [51]

    Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation. arXiv preprint arXiv:2309.16653,

    arXiv

  44. [52]

    Hu- mannerf: Free-viewpoint rendering of moving people from monocular video

    Chung-Yi Weng, Brian Curless, Pratul P Srinivasan, Jonathan T Barron, and Ira Kemelmacher-Shlizerman. Hu- mannerf: Free-viewpoint rendering of moving people from monocular video. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern Recognition , pages 162...

    2022

  45. [53]

    Driv- able avatar clothing: Faithful full-body telepresence with dy- namic clothing driven by sparse rgb-d input

    Donglai Xiang, Fabian Prada, Zhe Cao, Kaiwen Guo, Chen- glei Wu, Jessica Hodgins, and Timur Bagautdinov. Driv- able avatar clothing: Faithful full-body telepresence with dy- namic clothing driven by sparse rgb-d input. In SIGGRAPH Asia 2023 Conference Papers, pages 1–11, 2023. 1

    2023

  46. [54]

    Flashavatar: High-fidelity head avatar with efficient gaussian embedding

    Jun Xiang, Xuan Gao, Yudong Guo, and Juyong Zhang. Flashavatar: High-fidelity head avatar with efficient gaussian embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1802– 1812, 2024. 3

    2024

  47. [55]

    Get3dhuman: Lifting stylegan-human into a 3d generative model using pixel-aligned reconstruction priors

    Zhangyang Xiong, Di Kang, Derong Jin, Weikai Chen, Lin- chao Bao, Shuguang Cui, and Xiaoguang Han. Get3dhuman: Lifting stylegan-human into a 3d generative model using pixel-aligned reconstruction priors. In Proceedings of the IEEE/CVF International Conference on Computer Vision ,

  48. [56]

    Icon: Implicit clothed humans obtained from nor- mals

    Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J Black. Icon: Implicit clothed humans obtained from nor- mals. In 2022 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 13286–13296. IEEE, 2022. 3

    2022

  49. [57]

    Econ: Explicit clothed humans optimized via normal integration

    Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, and Michael J Black. Econ: Explicit clothed humans optimized via normal integration. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 512–523, 2023. 3

    2023

  50. [58]

    Puzzleavatar: Assembling 3d avatars from personal albums

    Yuliang Xiu, Yufei Ye, Zhen Liu, Dimitrios Tzionas, and Michael J Black. Puzzleavatar: Assembling 3d avatars from personal albums. ACM Transactions on Graphics (TOG) ,

  51. [59]

    Magicanimate: Temporally consistent human im- age animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  52. [60]

    Have-fun: Human avatar reconstruction from few-shot unconstrained images

    Xihe Yang, Xingyu Chen, Daiheng Gao, Shaohui Wang, Xi- aoguang Han, and Baoyuan Wang. Have-fun: Human avatar reconstruction from few-shot unconstrained images. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 742–752, 2024. 3, 4

    2024

  53. [61]

    Effec- tive whole-body pose estimation with two-stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effec- tive whole-body pose estimation with two-stages distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4210–4220, 2023. 2, 5

    2023

  54. [62]

    Robots learn social skills: End-to-end learning of co-speech gesture generation for humanoid robots

    Youngwoo Yoon, Woo-Ri Ko, Minsu Jang, Jaeyeon Lee, Jae- hong Kim, and Geehyuk Lee. Robots learn social skills: End-to-end learning of co-speech gesture generation for humanoid robots. In 2019 International Conference on Robotics and Automation (ICRA) , pages 4303–4309. IEEE,

    2019

  55. [63]

    Humanref: Single image to 3d human gen- eration via reference-guided diffusion

    Jingbo Zhang, Xiaoyu Li, Qi Zhang, Yanpei Cao, Ying Shan, and Jing Liao. Humanref: Single image to 3d human gen- eration via reference-guided diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1844–1854, 2024. 3, 4

    2024

  56. [64]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 2, 6

    2018

  57. [65]

    Mim- icmotion: High-quality human motion video generation with confidence-aware pose guidance

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mim- icmotion: High-quality human motion video generation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680, 2024. 2, 3, 4, 5, 6, 7

    2024 arXiv

  58. [66]

    Sifu: Side-view conditioned implicit function for real-world us- able clothed human reconstruction

    Zechuan Zhang, Zongxin Yang, and Yi Yang. Sifu: Side-view conditioned implicit function for real-world us- able clothed human reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9936–9947, 2024. 3

    2024

  59. [67]

    Humannerf: Efficiently gen- erated human radiance field from sparse inputs

    Fuqiang Zhao, Wei Yang, Jiakai Zhang, Pei Lin, Yingliang Zhang, Jingyi Yu, and Lan Xu. Humannerf: Efficiently gen- erated human radiance field from sparse inputs. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7743–7753, 2022. 2

    2022

  60. [68]

    Bilateral refer- ence for high-resolution dichotomous image segmentation

    Peng Zheng, Dehong Gao, Deng-Ping Fan, Li Liu, Jorma Laaksonen, Wanli Ouyang, and Nicu Sebe. Bilateral refer- ence for high-resolution dichotomous image segmentation. CAAI Artificial Intelligence Research, 3:9150038, 2024. 6

    2024

  61. [69]

    Gps- gaussian: Generalizable pixel-wise 3d gaussian splatting for real-time human novel view synthesis

    Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, and Yebin Liu. Gps- gaussian: Generalizable pixel-wise 3d gaussian splatting for real-time human novel view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2024

  62. [70]

    Pamir: Parametric model-conditioned implicit representa- tion for image-based human reconstruction

    Zerong Zheng, Tao Yu, Yebin Liu, and Qionghai Dai. Pamir: Parametric model-conditioned implicit representa- tion for image-based human reconstruction. IEEE transac- tions on pattern analysis and machine intelligence, 2021. 3

    2021

  63. [71]

    Champ: Controllable and consistent human image animation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Qingkun Su, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. arXiv preprint arXiv:2403.14781, 2024. 2, 3

    2024 arXiv

  64. [72]

    Driv- able 3d gaussian avatars

    Wojciech Zielonka, Timur Bagautdinov, Shunsuke Saito, Michael Zollh ¨ofer, Justus Thies, and Javier Romero. Driv- able 3d gaussian avatars. arXiv preprint arXiv:2311.08581,

    arXiv

  65. [2023]

    A detailed illustration of our pipeline

    3 11 One Shot, One Talk: Whole-body Talking A vatar from a Single Image Supplementary Material Perceptual Supervision Generative Models Training Poses  (, ) Re-Track LBS & 3DGS Per-pixel Supervision Gaussian & Mesh Deformation  (෠, ෠) UV Init Inference Poses LBS & 3DGS O...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.