Text-based Animatable 3D Avatars with Morphable Model Alignment

Malte Prinzler; Siyu Tang; Xiaogang Jin; Yiqian Wu

REVIEW 4 major objections 8 minor 18 references

Text-based Animatable 3D Avatars with Morphable Model Alignment

T0 review · 4 major / 8 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A text prompt alone can now produce an animatable 3D head avatar with roughly half the alignment error of prior text-based methods.

desk verdict The alignment gains are real and externally measured, but the quality claim is not supported by the paper's own numbers; still a solid, citable contribution worth serious peer review. read the letter →

arxiv 2504.15835 v1 pith:F54TYY2U submitted 2025-04-22 cs.CV

Yiqian Wu , Malte Prinzler , Xiaogang Jin , Siyu Tang This is my paper · ORCID

classification cs.CV

keywords text-basedavatargenerationanimatable3DheadavatarsGaussiansplattingmorphablemodelalignmentdiffusionmodelsControlNetscoredistillationsamplingSMPL-Xrigging

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the two persistent failures of text-driven animatable head avatars — blurry, unrealistic appearance and visible misalignment with the driving parametric model during animation — both stem from ambiguities in the 2D diffusion guidance used during 3D distillation, and that both can be fixed in one pipeline. The diagnosis is that a text prompt underconstrains appearance and geometry, and raw diffusion predictions carry no semantics of the parametric head model. Its method, AnimPortrait3D, first initializes a 3D Gaussian avatar from a static text-to-3D model so appearance, geometry, and rigging are well defined, then optimizes the avatar for dynamic expressions under a ControlNet that injects dense normal and segmentation maps rendered from the morphable model. In the paper's quantitative comparison the method posts the lowest landmark deviation (0.0148 versus 0.0263 for the landmark-conditioned HeadStudio) and the lowest average expression distance (0.1265 versus 0.3136). If the paper is right, text alone can now produce animatable head avatars whose motion tracks the underlying model closely enough for content-production use.

What carries the argument

The load-bearing object is a ControlNet conditioned jointly on dense normal maps and segmentation maps rendered from the SMPL-X morphable model; this four-channel condition (three channels of normals, one of semantics) tells the diffusion model where the eyes, teeth, and iris are, which sparse facial landmarks cannot convey. A second carrier is the initialization: 3D Gaussian splatting whose Gaussians are sampled from a fitted SMPL-X body plus segmented hair and clothing meshes and rigged to SMPL-X faces, so appearance and animation share one skeleton from the start. The optimization stack — Interval Score Matching for the mouth and full avatar, SDEdit-based regeneration for the eyes and final cleanup — depends on this ControlNet guidance; the authors' ablation shows that removing the ControlNet degrades the eyes, lips, and rigging noticeably even with careful initialization.

What would settle it

Generate avatars from text prompts whose appearance lies far from the ControlNet's training distribution — heavy prosthetics, face paint, extreme aging, or bulky headgear that SMPL-X cannot render — and measure landmark deviation and average expression distance against the driving model. The paper's claim predicts the alignment advantage over HeadStudio persists because the conditions are geometric; if deviations climb back to HeadStudio's level, the domain gap is the breaking point.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that score-distillation-based text-to-avatar generation fails for two separable reasons — underconstrained appearance and geometry, and insufficient semantic alignment between diffusion predictions and the parametric head model — and that a strong static prior plus dense geometric-semantic conditioning removes both. The avatar is a set of 3D Gaussians rigged to the SMPL-X morphable model: the initialization stage fits SMPL-X to a static text-to-3D portrait, carves hair and clothing meshes from it, segments them, and trains the Gaussians on multi-view renders so every splat is bound to a SMPL-X face that moves it. The dynamic stage trains a ControlNet on normal and segmentation maps derived from real portraits, then conditions it at inference on maps rendered from the deformed SMPL-X model, using Interval Score Matching with dedicated pre-training for the eyes and mouth interior. The paper claims this achieves state-of-the-art synthesis quality and animation fidelity, and its headline evidence is the nearly twofold reduction in landmark deviation and the roughly 60 percent reduction in average expression distance relative to HeadStudio.

Load-bearing premise

The alignment gains rest on the ControlNet transferring from training conditions that are normal maps estimated from real photographs to inference conditions that are maps rendered from the synthetic SMPL-X model; if that domain gap is large, the reported alignment improvements would not survive on avatars outside the training distribution.

Editorial extensions

If this is right

Text prompts can now produce head avatars whose motions track a driving SMPL-X model about twice as accurately as the landmark-conditioned HeadStudio baseline (landmark deviation 0.0148 versus 0.0263).
Dense normal-plus-semantic conditioning from the parametric model is a stronger control signal than sparse landmarks for diffusion-guided avatar optimization.
Initializing from a static text-to-3D avatar removes the appearance and geometry ambiguity of raw score distillation, reducing blur and Janus artifacts without extra prompt engineering.
Mouth interiors and eyelids, which are invisible or artifact-prone under a neutral expression, can be recovered through region-specific pre-training starting from generic proxy geometry.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Retraining the ControlNet on normal and segmentation maps rendered directly from SMPL-X with realistic textures would eliminate the estimated-versus-rendered domain gap and could improve robustness on prompts far outside the FFHQ and LPFF portrait distribution.
The same four-channel geometric-semantic conditioning likely transfers to full-body animatable avatar generation, where limb, hand, and clothing alignment is a harder instance of the same ambiguity problem.
If landmark deviation and average expression distance become standard evaluation quantities, the field gains a driveability metric separate from visual quality — a method that looks good in a neutral pose but breaks under animation would be exposed.
At roughly 3.5 hours per avatar on a single GPU with real-time 3DGS rendering, the pipeline is near practical use for games, film previsualization, and embodied assistants.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read the paper. The one thing to know: the contribution is alignment, not quality. The landmark deviation and AED gains over HeadStudio are large and measured with external tools, so I believe the alignment story. But the abstract's "synthesis quality" sentence is not supported by Table 1, where TADA has a higher HyperIQA score, or by Table A.2, where removing ControlNet yields higher HyperIQA and DSL-FIQA than the full model.

What is new and good: the dense normal-plus-segmentation ControlNet, the region-specific eye and mouth pre-training, and the use of a Portrait3D static avatar as a rigged initialization are a real combination I don't recall seeing in prior work. The pipeline is documented carefully, and the appendices include ablation baselines (refine-only, super-resolution) that many papers skip. The evaluation is externally measured via landmark detection, expression estimation, and CLIP, so the headline metrics are not self-metrics. The limitations section is honest about static Gaussian features, segmentation dependency, and blend-shape constraints.

The soft spots, in proportion. The quality claim is the biggest. The method is best on alignment and comparable or sometimes worse on perceptual quality. The authors need to weaken that claim or justify why the IQA metrics are wrong. There are no error bars; 20 prompts is a small base. There is also a hyperparameter inconsistency: the position regularization weight appears as 1e5 in A1.3 and 1e-2 in A1.6. Probably a typo, but it should be fixed.

The stress-test concern about the ControlNet's train/inference domain gap is real, though not fatal. The ControlNet is trained on normal maps from a single-image face reconstruction method; those maps don't represent open-mouth interiors or dynamically posed eyes. At inference, the conditionals are rendered from SMPL-X, so the eye and mouth regions are out-of-distribution precisely where the method claims improvements. The authors do add NeRSemble mouth images and closed-eye augmentations, so the gap is partially addressed, but they never measure control-map adherence on rendered maps. I'd trust the Table 1 gains on this evaluation distribution, but I wouldn't extrapolate them to all prompts and expressions. Section 5 discusses segmentation and blend-shape issues but not this train/inference gap.

Who this is for: anyone building text-to-3D avatar pipelines. The alignment mechanism is worth studying and citing. It deserves a serious referee, not a desk reject. I'd ask the authors to add error bars, soften the quality claim, fix the weight inconsistency, and add at least one experiment or analysis addressing the rendered-map domain gap before acceptance.

Referee Report

4 major / 8 minor

Summary. AnimPortrait3D is a two-stage pipeline for generating animatable 3D head avatars from text. In the first stage, it uses Portrait3D to create a static NeRF avatar, fits SMPL-X, extracts and segments hair/clothing meshes, and initializes a rigged 3DGS representation with color initialization from multi-view renders. In the second stage, it optimizes the avatar for dynamic poses and expressions using a ControlNet conditioned on SMPL-X normal and segmentation maps, with dedicated eye and mouth pretraining, ISM-based full optimization, and a final SDEdit refinement. The paper evaluates against HeadStudio, TADA, HumanGaussian, PortraitGen, GPAvatar, and GAGAvatar, reporting landmark deviation, AED, CLIP, HyperIQA, and DSL-FIQA, and claims state-of-the-art synthesis quality, alignment, and animation fidelity.

Significance. The alignment results are the paper's strongest contribution. Table 1 shows large reductions in landmark deviation (0.0148 vs 0.0263 for HeadStudio) and AED (0.1265 vs 0.3136), and these are measured with external tools rather than self-reported outputs. The ablation study in Table A.2 is unusually complete, and the ControlNet training set of 453,385 pairs is a substantial resource. If the domain-gap concern for rendered SMPL-X conditionals is addressed, the method is a credible step forward for text-driven animatable avatars. The synthesis-quality part of the claim, however, is not supported by the paper's own numbers, and the lack of a control-map generalization analysis leaves the scope of the alignment improvement uncertain.

major comments (4)

[§4.2.2 / Table A.2] The abstract's claim that the method 'outperforms existing approaches in terms of synthesis quality' is contradicted by the paper's own quantitative results. In Table 1, TADA has a higher HyperIQA score (60.1467 vs 59.6879), and in Table A.2, two ablations, namely removing ControlNet and removing geometry initialization, have both higher HyperIQA (61.5838 and 62.9362) and higher DSL-FIQA (0.6587 and 0.6486) than the full model (59.6879 and 0.6426). The statement in Appendix A2.2 that 'the difference is minimal' does not reconcile a roughly 3-point HyperIQA gap, and the abstract does not mention that the quality claim is limited to face-specific metrics. Please either revise the claim, add a human perceptual study, or report a metric that directly supports the quality advantage.
[§3.2.1 / Appendix A1.5] The central alignment mechanism is a ControlNet trained on normal maps estimated from real RGB portraits by Deng et al. (2019) and on segmentation maps from MediaPipe/EasyPortrait, but at inference it is conditioned on normal and segmentation maps rendered from SMPL-X with open mouths, dynamic eyelids, and gaze changes (Sections 3.2.2 and 3.2.3). These rendered conditionals are out of the training distribution of the ControlNet in exactly the eye and mouth regions that the method most needs to fix. The paper does not report any quantitative check of how well the ControlNet follows rendered SMPL-X conditionals, for example by comparing generated mouth and eye crops against the conditional maps or by measuring landmark deviation as a function of expression openness. Without such evidence, the headline alignment gains in Table 1 cannot be confidently attributed to the ControlNet or expected to transfer outside the evaluated prompts and datasets. Please add this analysis or explicitly limit the claim.
[§4.2.2 / Table A.2] The semantic-alignment metric CLIP shows only a small advantage over HeadStudio (0.2749 vs 0.2687), and the table reports averages over 20 prompts and 100 random images per prompt without variance or significance tests. Table A.2 further shows that the full model's CLIP (0.2749) is below the no-ControlNet ablation (0.2775) and below one progressive-ablation row, so the CLIP-based superiority is not established. Please report standard deviations and, where feasible, per-prompt breakdowns or significance tests.
[§4.2 / Figure 8] The image-based methods GPAvatar and GAGAvatar are evaluated by feeding them a frontal image from Portrait3D, even though they are reconstruction methods rather than text-to-3D generators. Their lower quantitative scores may partly reflect this input mismatch rather than a genuine deficiency in their intended setting. Please add a sentence clarifying what conclusion can and cannot be drawn from those two comparison rows.

minor comments (8)

[Abstract / Section 6] The phrase 'synthesis quality' in the abstract and conclusion should be made consistent with the quantitative evidence, which supports alignment and DSL-FIQA advantages but not a general HyperIQA advantage.
[Appendix A1.5] The sentence 'The resolution of training images is fixed at 5122' should read '512 × 512'.
[Eq. (5)] The LPIPS term L_lpips is never defined; please state that it is the standard learned perceptual similarity and give the backbone used.
[Section 4.2.2] For the AED metric, please clarify whether the expression coefficients from the reference video and from the generated avatars are compared in the same parameter space and whether any canonicalization or re-targeting is applied.
[Section 3.2.4 / Appendix A4] The claim that 'we use low noise levels' to preserve identity is vague; please specify the actual noise ranges or editing strengths used in the full-avatar refinement.
[Figure 4] Figure 4 shows only training-distribution conditionals; adding one example with SMPL-X-rendered normal and segmentation maps would help readers judge the domain gap discussed in the major comments.
[Section 3.2.2] The eye-region pretraining uses a fixed SDEdit strength of 0.9; please either provide an ablation for this value or state why it was fixed.
[Code availability] The text says 'Code and model for this paper are at AnimPortrait3D' but no URL is given; please provide a working repository link in the final version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline comparisons are measured with external tools, and the self-cited Portrait3D initialization is a pipeline component rather than the validation criterion.

full rationale

No load-bearing step in the paper reduces, by its own equations or by self-citation, to its inputs. The method's two claimed contributions are (i) initialization from the authors' previous Portrait3D model and (ii) ControlNet-guided dynamic optimization (Sections 3.1 and 3.2). Portrait3D is used as a building block, not as the validation criterion; it is independently published and externally falsifiable, so citing it is normal provenance rather than circular evidence. The quantitative claims in Table 1 are computed with external tools: landmark deviation uses the off-the-shelf STAR Loss detector (Zhou et al. 2023), AED uses the face-capture estimator of Retsinas et al. (2024), and semantic/quality metrics use CLIP, HyperIQA, and DSL-FIQA. None of these metrics is computed from the method's own ControlNet, diffusion model, or Portrait3D output, so the headline improvements are not forced by construction. The AED protocol animates all methods with the same reference video and measures the deviation between estimated expressions; although the avatars are driven by SMPL-X parameters, the measured quantity is the residual error after rendering and rigging, which differs across methods and is not definitionally equal to the input. The ControlNet conditions (normal and segmentation maps) are rendered from the SMPL-X model, which is external to the text prompt and to the fitted appearance; the ISM and SDEdit losses use diffusion-model predictions as training signal, not as evaluation outcome. The main identified risk, that a ControlNet trained on normal maps estimated from real portraits may not transfer to SMPL-X-rendered maps in the eye and mouth regions, is a generalization/domain-gap concern, not circularity. No equation equates a predicted quantity with a fitted quantity, and no uniqueness or forced-choice argument is imported from the authors' prior work. Therefore the derivation chain is self-contained with respect to the external benchmarks, and the correct circularity finding is 0.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The central claim rests on several hand-tuned hyperparameters (noise schedules, editing strengths, ISM step size, regularization weights, proxy mouth colors) and on strong generalization assumptions about the ControlNet, the SMPL-X fit, and the evaluation tools. The paper introduces no new physical entities; the only novel artifacts are trained network components and a purchased teeth mesh, neither of which is an invented entity in the sense of a postulated force or particle.

free parameters (6)

ISM inversion step size delta_T = 50
Chosen for dynamic optimization; the ISM gradient (Eq. 2) depends on it and it affects alignment and stability.
Noise schedule endpoints for mouth pre-training = t from 750 to 15
Hand-tuned in Appendix A1.6 to balance detail and artifacts in the mouth interior.
Noise schedule endpoints for full optimization = t from 300 to 15
Hand-tuned in Appendix A1.6; directly controls the trade-off between added detail and high-frequency artifacts.
SDEdit editing strengths = 0.9 for eye pre-training, 0.3 for final refinement
Used in Eqs. 1 and 5; controls how much of the rendered image structure is preserved versus replaced by the diffusion model, and is central to the claimed artifact removal.
Position regularization weight = 1e5 (A1.3) vs 1e-2 (A1.6)
Reported inconsistently across sections; this value controls Gaussian drift and thus rigging integrity.
Generic mouth and teeth colors = teeth RGB (141.6, 133.8, 122.4); inner mouth (64.0, 30.5, 29.5)
Substitutes for the unobserved mouth interior during initialization (Appendix A1.3); the final mouth realism depends on later diffusion refinement correcting this proxy.

assumptions (7)

standard math The interval score matching gradient (Eq. 2) with deterministic inversion gives a valid optimization signal for 3DGS, as established in LucidDreamer.
The paper adopts ISM as a loss without re-deriving its validity; the strongest claim's optimization relies on it.
domain assumption The normal and segmentation maps rendered from SMPL-X adequately represent the avatar's true geometry and semantics during optimization, and the ControlNet trained on estimated maps transfers to rendered maps.
Section 3.2.1 and Appendix A1.5; if this train-inference domain gap is large, the alignment guidance fails.
domain assumption The multi-view head tracking (Qian 2024) yields a SMPL-X fit that is aligned well enough to the Portrait3D avatar to define correct rigging.
Section 3.1.1; all subsequent rigging and alignment inherit errors from this fit.
domain assumption The off-the-shelf landmark detector (Zhou et al. 2023) and expression estimator (Retsinas et al. 2024) provide unbiased measurements for generated avatars.
Used in Section 4.2.2 to quantify geometric alignment and AED; if these tools are biased toward real faces or SMPL-X morphology, the reported improvement could be inflated.
domain assumption The diffusion prior with ControlNet refines the generic teeth and mouth proxy without losing identity or creating artifacts.
Appendix A1.3-A1.5; the mouth interior is invisible during initialization, so its realism depends on this assumption.
domain assumption The ControlNet training datasets (FFHQ, LPFF, NeRSemble) and the Realistic Vision V5.1 base model are representative of the text prompt distribution encountered at inference.
Appendix A1.5; out-of-distribution prompts with stylized appearance or unusual hair and garments may degrade ControlNet guidance.
ad hoc to paper Using abstract region-specific text prompts (for example, 'right eye region, a boy') instead of the detailed full-avatar prompt improves eye and mouth guidance.
Appendix A1.5 and Figure A.3; an empirical choice specific to this paper's ControlNet training, with no principled justification.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Text-based Animatable 3D Avatars with Morphable Model Alignment." pith.science (2026). https://pith.science/paper/F54TYY2U

@misc{pith2026250415835,
  author       = {Pith},
  title        = {Pith review of: Text-based Animatable 3D Avatars with Morphable Model Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F54TYY2U}},
  note         = {Machine review of arXiv:2504.15835}
}

read the original abstract

The generation of high-quality, animatable 3D head avatars from text has enormous potential in content creation applications such as games, movies, and embodied virtual assistants. Current text-to-3D generation methods typically combine parametric head models with 2D diffusion models using score distillation sampling to produce 3D-consistent results. However, they struggle to synthesize realistic details and suffer from misalignments between the appearance and the driving parametric model, resulting in unnatural animation results. We discovered that these limitations stem from ambiguities in the 2D diffusion predictions during 3D avatar distillation, specifically: i) the avatar's appearance and geometry is underconstrained by the text input, and ii) the semantic alignment between the predictions and the parametric head model is insufficient because the diffusion model alone cannot incorporate information from the parametric model. In this work, we propose a novel framework, AnimPortrait3D, for text-based realistic animatable 3DGS avatar generation with morphable model alignment, and introduce two key strategies to address these challenges. First, we tackle appearance and geometry ambiguities by utilizing prior information from a pretrained text-to-3D model to initialize a 3D avatar with robust appearance, geometry, and rigging relationships to the morphable model. Second, we refine the initial 3D avatar for dynamic expressions using a ControlNet that is conditioned on semantic and normal maps of the morphable model to ensure accurate alignment. As a result, our method outperforms existing approaches in terms of synthesis quality, alignment, and animation fidelity. Our experiments show that the proposed method advances the state of the art in text-based, animatable 3D head avatar generation.

Figures

Figures reproduced from arXiv: 2504.15835 by the authors.

**Figure 1.** Our method generates high-quality, realistic, and animatable 3D avatars from text descriptions which can be driven with morphable model parameters. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** Overview of AnimPortrait3D. Given an input text, the 3D Avatar Initialization stage (Section 3.1) generates a well-defined initial avatar that provides [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** The visualization of (a) the static 3D avatar [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: (a) The ground truth image from ControlNet’s training dataset (orig [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 6.** Figure 6: Generated results of our method. For each 3D avatar, we present rendered images with varying expressions and poses across different camera views, [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Ablation Study. We conduct two types of ablation studies: a progressive ablation study and a subtractive ablation study. The mesh renderings and the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

**Figure 8.** Figure 8: Comparison with HeadStudio [Zhou et al. 2024], TADA [Liao et al. 2024], HumanGaussian [Liu et al. 2024], PortraitGen [Gao et al. 2024], GPAvatar [Chu et al. 2024], and GAGAvatar [Chu and Harada 2024]. While other methods take a text prompt as input (shown at the top), …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 12 canonical work pages

[1]

In the subtractive ab- lation study, we individually remove 1) appearance initialization,

the results after the 3D Initialization stage, 2) the avatar after pre-training the mouth- and eye region, and 3) the avatar after full optimization but without refinement. In the subtractive ab- lation study, we individually remove 1) appearance initialization,

work page
[2]

5 5 y . o m a n , t r a v e l e r c l o t h e s , s t a n d i n g i n t h e f o r e s t

geometry initialization, 3) eye and mouth pre-training, and 4) ControlNet from our full model to evaluate their contributions. In Appendix A2.1, we additionally conduct an ablation study on replacing the dynamic avatar optimization stage . We replace the entire dynamic avatar optimization stage with 1) the final re- finement and 2) a super-resolution meth...

work page 2024
[4]

ACM Trans

Reconstructing Personalized Semantic Facial NeRF Models from Monocular Video. ACM Trans. Graph. 41, 6 (2022), 200:1–200:12. Simon Giebenhain, Tobias Kirschstein, Markos Georgopoulos, Martin Rünz, Lourdes Agapito, and Matthias Nießner. 2023. Learning Neural Parametric Head Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023...

2022
[5]

Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang

IEEE, 10456–10465. Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang. 2024. LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control. CoRR abs/2407.03168 (2024). Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. In Advances in Neural Information...

arXiv 2024
[6]

NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads.ACM Trans. Graph. 42, 4 (2023), 161:1–161:14. Nikos Kolotouros, Thiemo Alldieck, Andrei Zanfir, Eduard Gabriel Bazavan, Mihai Fieraru, and Cristian Sminchisescu. 2023. DreamHuman: Animatable 3D Avatars from Text. CoRR abs/2306.09329 (2023). Junxuan Li, Chen Cao, Gabriel Schwartz, Rawal Khi...

arXiv 2023
[7]

MagicMirror: Fast and High-Quality Avatar Generation with a Constrained Search Space

Luciddreamer: Towards high-fidelity text-to-3d generation via interval score matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6517–6526. Tingting Liao, Hongwei Yi, Yuliang Xiu, Jiaxiang Tang, Yangyi Huang, Justus Thies, and Michael J. Black. 2024. TADA! Text to Animatable Digital Avatars. In International Con...

work page Pith review arXiv 2024
[8]

In Computer Vision - ECCV 2022 - 17th European Conference (Lecture Notes in Computer Science, Vol

KeypointNeRF: Generalizing Image-Based Volumetric Avatars Using Relative Spatial Encoding of Keypoints. In Computer Vision - ECCV 2022 - 17th European Conference (Lecture Notes in Computer Science, Vol. 13675) . Springer, 179–197. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ra- mamoorthi, and Ren Ng. 2020. NeRF: Represen...

work page 2022
[10]

Shenhan Qian

Joker: Conditional 3D Head Synthesis with Extreme Facial Expressions. Shenhan Qian. 2024. VHAP. https://github.com/ShenhanQian/VHAP. Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Gieben- hain, and Matthias Nießner. 2024. GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians. In IEEE/CVF Conference on Computer Visi...

work page 2024

Show all 18 references

[11]

InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

High-Resolution Image Synthesis With Latent Diffusion Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 10684–10695. Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, and Zeyu Wang. 2024...

2024
[13]

Yuelang Xu, Bengwang Chen, Zhe Li, Hongwen Zhang, Lizhen Wang, Zerong Zheng, and Yebin Liu

IEEE, 656–665. Yuelang Xu, Bengwang Chen, Zhe Li, Hongwen Zhang, Lizhen Wang, Zerong Zheng, and Yebin Liu. 2024. Gaussian Head Avatar: Ultra High-Fidelity Head Avatar via Dynamic Gaussians. In IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, CVPR 2024. IEEE, 1...

2023 arXiv
[14]

CoRR abs/2408.06019 (2024)

HeadGAP: Few-shot 3D Head Avatar via Generalizable Gaussian Priors. CoRR abs/2408.06019 (2024). Ziming Zhong, Yanxu Xu, Jing Li, Jiale Xu, Zhengxin Li, Chaohui Yu, and Shenghua Gao. 2024. MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis. CoRR abs/2407....

2024 arXiv
[15]

A T e e n b o y , p e n s i v e l o o k , d a r k h a i r . P r e p p y s w e a t e r , c o l l a r e d s h i r t , m o o d y r o o m , 8 0 s m e m o r a b i l i a

STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 . IEEE, 15475–15484. Zhenglin Zhou, Fan Ma, Hehe Fan, and Yi Yang. 2024. HeadStudio: Text to Animatable Head Avatars with 3D Gaussi...

2023
[2009]

In Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, A VSS 2009, Stefano Tubaro and Jean-Luc Dugelay (Eds.)

A 3D Face Model for Pose and Illumination Invariant Face Recognition. In Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, A VSS 2009, Stefano Tubaro and Jean-Luc Dugelay (Eds.). IEEE Computer Society, 296–301. Ben Poole, Ajay Jain, Jonathan ...

2009
[2020]

In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020

Blindly Assess Image Quality in the Wild Guided by a Self-Adaptive Hyper Network. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020. Computer Vision Foundation / IEEE, 3664–3673. Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. 2021. Real-ES...

2024 arXiv
[2022]

In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022

RigNeRF: Fully Controllable Neural 3D Portraits. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 . IEEE, 20332–20341. Wei-Ting Chen, Gurunandan Krishnan, Qiang Gao, Sy-Yen Kuo, Sizhuo Ma, and Jian Wang. 2024. DSL-FIQA: Assessing Facial Image Qualit...

2022
[2023]

In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023

DiffusionRig: Learning Personalized Priors for Facial Appearance Editing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 . IEEE, 12736–12746. Guy Gafni, Justus Thies, Michael Zollhöfer, and Matthias Nießner. 2021. Dynamic Neural Radiance Fields fo...

2023
[2024]

In SIGGRAPH Asia 2024 Conference Papers, SA 2024 , Takeo Igarashi, Ariel Shamir, and Hao (Richard) Zhang (Eds.)

Portrait Video Editing Empowered by Multimodal Generative Priors. In SIGGRAPH Asia 2024 Conference Papers, SA 2024 , Takeo Igarashi, Ariel Shamir, and Hao (Richard) Zhang (Eds.). ACM, 104:1–104:11. Xuan Gao, Chenglai Zhong, Jun Xiang, Yang Hong, Yudong Guo, and Juyong Zhang

2024
[5122]

a teen boy, pensive look, dark hair, preppy sweater, collared shirt, moody room, 80s memorabilia

To ensure approximately balanced quantities of face, mouth, and eye data, we duplicate relevant samples. For data augmentation, we employ random resized cropping during training. For ControlNet guidance on the face region, we utilize the com- plete text prompt describing the f...

2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

In the subtractive ab- lation study, we individually remove 1) appearance initialization,

the results after the 3D Initialization stage, 2) the avatar after pre-training the mouth- and eye region, and 3) the avatar after full optimization but without refinement. In the subtractive ab- lation study, we individually remove 1) appearance initialization,

work page

[2] [2]

5 5 y . o m a n , t r a v e l e r c l o t h e s , s t a n d i n g i n t h e f o r e s t

geometry initialization, 3) eye and mouth pre-training, and 4) ControlNet from our full model to evaluate their contributions. In Appendix A2.1, we additionally conduct an ablation study on replacing the dynamic avatar optimization stage . We replace the entire dynamic avatar optimization stage with 1) the final re- finement and 2) a super-resolution meth...

work page 2024

[3] [4]

ACM Trans

Reconstructing Personalized Semantic Facial NeRF Models from Monocular Video. ACM Trans. Graph. 41, 6 (2022), 200:1–200:12. Simon Giebenhain, Tobias Kirschstein, Markos Georgopoulos, Martin Rünz, Lourdes Agapito, and Matthias Nießner. 2023. Learning Neural Parametric Head Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023...

2022

[4] [5]

Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang

IEEE, 10456–10465. Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang. 2024. LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control. CoRR abs/2407.03168 (2024). Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. In Advances in Neural Information...

arXiv 2024

[5] [6]

NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads.ACM Trans. Graph. 42, 4 (2023), 161:1–161:14. Nikos Kolotouros, Thiemo Alldieck, Andrei Zanfir, Eduard Gabriel Bazavan, Mihai Fieraru, and Cristian Sminchisescu. 2023. DreamHuman: Animatable 3D Avatars from Text. CoRR abs/2306.09329 (2023). Junxuan Li, Chen Cao, Gabriel Schwartz, Rawal Khi...

arXiv 2023

[6] [7]

MagicMirror: Fast and High-Quality Avatar Generation with a Constrained Search Space

Luciddreamer: Towards high-fidelity text-to-3d generation via interval score matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6517–6526. Tingting Liao, Hongwei Yi, Yuliang Xiu, Jiaxiang Tang, Yangyi Huang, Justus Thies, and Michael J. Black. 2024. TADA! Text to Animatable Digital Avatars. In International Con...

work page Pith review arXiv 2024

[7] [8]

In Computer Vision - ECCV 2022 - 17th European Conference (Lecture Notes in Computer Science, Vol

KeypointNeRF: Generalizing Image-Based Volumetric Avatars Using Relative Spatial Encoding of Keypoints. In Computer Vision - ECCV 2022 - 17th European Conference (Lecture Notes in Computer Science, Vol. 13675) . Springer, 179–197. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ra- mamoorthi, and Ren Ng. 2020. NeRF: Represen...

work page 2022

[8] [10]

Shenhan Qian

Joker: Conditional 3D Head Synthesis with Extreme Facial Expressions. Shenhan Qian. 2024. VHAP. https://github.com/ShenhanQian/VHAP. Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Gieben- hain, and Matthias Nießner. 2024. GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians. In IEEE/CVF Conference on Computer Visi...

work page 2024