Pith. sign in

REVIEW 4 major objections 4 minor 61 references

GRMM augments a classical 3DMM with learned residual geometry and appearance to render photorealistic full-head avatars at 75 fps.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

GRMM combines a classic 3D face template with learned fine detail residuals to render controllable full-head avatars in real time.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A solid engineering contribution with a valuable new dataset, but the paper oversells its disentangled appearance residuals and the SOTA claims outrun the evidence: missing closest baselines, no error bars, and an architecture text that contradicts the actual decoder inputs. the 4 major comments →

arxiv 2509.02141 v1 pith:4BY5AIC7 submitted 2025-09-02 cs.GR cs.CV

GRMM: Real-Time High-Fidelity Gaussian Morphable Head Model with Learned Residuals

Mohit Mendiratta , Mayur Deshmukh , Kartik Teotia , Vladislav Golyanik , Adam Kortylewski , Christian Theobalt This is my paper
classification cs.GR cs.CV
keywords gaussian splatting3D morphable modelhead avatarexpression disentanglementresidual learningnovel-view synthesisexpression transfermulti-view head dataset
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish GRMM, an open-source full-head morphable model that layers learned residual geometry and appearance on top of a classic mesh-based 3DMM. If correct, it gives one parametric model fine-grained control over identity, expression, and pose while rendering detailed full-head portraits at 1K resolution in real time at 75 FPS. The central design claim is that separating interpretable coarse 3DMM parameters from low-dimensional residual codes keeps the model editable and expressive. To make those residuals transferable, the authors introduce EXPRESS-50, a multiview dataset with 50 identities performing 60 manually aligned expressions, and argue that this cross-identity alignment is what allows identity and expression to be disentangled.

Core claim

GRMM's central claim is that a full-head Gaussian morphable model can be built by additive residual refinement of a base FLAME 3DMM: a shared identity code z_id and a shared expression code z_exp drive lightweight decoders that predict vertex-level displacements and per-Gaussian transforms, while a screen-space CNN cleans up the rasterized image. The deformed mesh v_d = v_rec + v_delta,id + M_face v_delta,exp and the final image I = Psi_ref([I_rgb, I_feature, I_norm_depth]) capture the two stages. Against state-of-the-art morphable head models, the paper reports higher reconstruction fidelity and novel-view quality, and its user study shows expression transfer that preserves identity across

What carries the argument

UV-anchored Gaussian primitives on an enhanced FLAME mesh. Gaussians are initialized by uniform sampling in the mesh's UV space, so they deform coherently with predicted mesh displacements; a transformation decoder refines their position, rotation, and scale, an opacity decoder conditions only on identity, and an appearance decoder conditions on identity and view direction. The residual codes z_id and z_exp are the disentanglement mechanism: expression-specific detail is carried by a shared global code, while identity-specific detail stays in the per-identity code.

Load-bearing premise

The model's claim that expressions transfer cleanly across people depends on its hand-picked 'peak expression' frames actually matching semantically for every subject; if those alignments are off, the expression code silently carries identity information and the edit fails.

What would settle it

Take two subjects whose mouth shapes differ strongly, fit GRMM to the same labeled expression from each, then render each subject with the other subject's expression code. If the resulting mouth reveals the source identity's tooth or tongue geometry rather than the intended expression, identity has leaked into the shared expression residual. A quantitative check is to train a simple identity classifier on mouth-crop renderings generated with z_exp held constant; above-chance classification would indicate identity leakage.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A single fitted portrait can be re-rendered from arbitrary viewpoints with consistent identity and expression, because the same 3D Gaussian representation is shared across views.
  • Expression transfer reduces to copying the shared z_exp code: the paper shows that this preserves the target expression and the source identity, including subtle mouth articulation.
  • Fine details like wrinkles, hairline variation, and inner-mouth geometry are recovered by the learned residuals rather than by raising the 3DMM's PCA dimensionality, so they stay locally editable.
  • Because the full head, including teeth and inner mouth, is modeled, GRMM can be used for tasks face-only models cannot handle, such as avatars with talking or expressive mouths.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to automate peak-expression selection: if the manual alignment in EXPRESS-50 is replaced by a learned expression embedding, the same residual decomposition could scale to thousands of identities.
  • The architecture implies a direct control knob for hairstyle and face shape through z_id while z_exp controls expression; one could test this by interpolating two identities' z_id codes and checking whether rendered hair and face shape interpolate independently of expression.
  • Because opacity is conditioned only on identity, occluding features like hair are decoupled from expression; a testable corollary is that editing z_id alone should change occlusion boundaries without disturbing the expression readout.
  • The reported 75 FPS for 1K rendering suggests the same coarse-to-fine residual design could be carried into relighting or gaze editing with additional latent codes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. GRMM proposes a full-head Gaussian morphable head model that combines a FLAME-based mesh decoder with learned residual identity and expression latent codes, UV-anchored 3D Gaussian primitives, CNN decoders, and a screen-space refinement network. The method is trained on a newly introduced EXPRESS-50 dataset (50 identities, 60 aligned expressions) together with RenderMe-360, and is evaluated on monocular reconstruction, novel-view synthesis, and expression transfer against HeadNeRF, MoFaNeRF, and Morphable Diffusion. The paper claims state-of-the-art fidelity, disentangled control, and real-time 1K rendering at 75 FPS.

Significance. The work has several strengths: it is explicitly positioned as open-source, introduces a novel expression-aligned dataset (EXPRESS-50) that addresses a real gap in the literature, and provides ablations supporting the value of the residual parameterization, the mesh decoder, the refinement network, and joint dataset training. If the claims are substantiated, GRMM would be a practically useful controllable head model with photorealistic real-time rendering. However, the quantitative evidence is incomplete: the closest Gaussian-based head models (GPHM, HeadGAP) are not compared, and the reported numbers lack error bars and are based on small test sets. In addition, there is an internal architectural mismatch between the stated inputs to the decoders and their actual conditioning, which affects the central claim of expression-dependent appearance residuals.

major comments (4)
  1. [Sec. 3.3, Eqs. (4), (13)-(14)] Equation (4) states that all decoders take zid, zexp, αexp, θneck, θjaw, and d. However, the detailed specification shows that the appearance decoder Φapp is conditioned only on zid and view direction d, and the opacity decoder Φα is conditioned only on zid. Consequently, the per-Gaussian RGB and 32-D feature vectors are identical across expressions for a fixed identity, up to view direction. Expression-dependent appearance—wrinkles, creases, teeth/tongue visibility, skin-stretch shading—must therefore be carried by geometric warping of a fixed identity texture or by the screen-space CNN Ψref, not by a 3D appearance residual. This contradicts the abstract's claim of 'residual geometry and appearance components' that capture 'subject- and expression-specific detail', and the overview sentence in Sec. 3.3 stating that the decoders 'map identity and expression codes to geometric and appeara
  2. [Sec. 4.2, Tables 2 and 3] The quantitative comparisons are against HeadNeRF and MoFaNeRF, both NeRF-based methods from 2022, plus a qualitative comparison to Morphable Diffusion. The closest Gaussian head models—GPHM and HeadGAP, which are discussed in Related Work—are not included in any quantitative comparison. Since the paper's central claim is to outperform prior morphable face models, including Gaussian-based ones, the evidence is incomplete. In addition, Table 3 reports RMSE and FID on only 10 randomly selected RAVDESS identities without variance; Table 2 does not state test-set size or error bars. Please add the missing Gaussian baselines and report mean and standard deviation over multiple seeds or splits.
  3. [Sec. 3.1; Supp. Sec. 7] The disentanglement of identity and expression is partly built into the architecture: z_exp is shared per expression across identities, z_id is per identity, and L_z regularizes both. The semantic validity of this separation depends entirely on the quality of the manual expression alignment. For EXPRESS-50, peak frames are selected by subjective visual matching against a reference identity; for RenderMe-360, the last frame of each video is used, with the assertion that it 'typically captures the peak'. No quantitative alignment accuracy or inter-annotator agreement is reported. The supplementary material itself shows the risk: Sec. 7 states that before adding mouth-interior geometry, zeroing z_exp severely distorts the mouth, indicating that intra-oral appearance was entangled with the expression channel. This does not invalidate the method, but it means the disentanglement and transfer
  4. [Sec. 3.1 and 3.4; Supp. Sec. 6] Depth supervision uses ProbeSDF, a learned surface reconstruction method, as 'ground-truth' depth. No validation of ProbeSDF's accuracy on the specific datasets, nor an analysis of potential systematic bias, is provided. Since the depth loss directly supervises the mesh and Gaussian geometry, geometric errors in ProbeSDF could propagate into the learned residuals and affect the claimed fidelity. A quantitative comparison of ProbeSDF against known geometry (for example, on a dataset with 3D scans) or a sensitivity analysis would strengthen the geometric claims.
minor comments (4)
  1. [Abstract; Sec. 1] The 75 FPS figure is reported without hardware specification. Please state the GPU and rendering resolution used.
  2. [Fig. 17 caption] Typo: 'Distangled' should be 'Disentangled'.
  3. [References] References [28] and [37] appear to have malformed page markers ('4, 1' and '4, 1'); please clean up the bibliography.
  4. [Supp. Sec. 8, Table 4] The user study pooling is reported as percentages, but it is unclear how ties are handled across the 'Expression' and 'Identity' rows. Please clarify the denominator and whether the 600 judgments are 30 per participant or another allocation.

Circularity Check

0 steps flagged

No significant circularity; the central residual formulation and evaluations are self-contained.

full rationale

GRMM's derivation is self-contained: the base FLAME prior, ProbeSDF depth supervision, and multi-view RGB losses are external inputs, and the learned residual latents zid and zexp are optimized against held-out identities and views, so the novel-view synthesis and expression-transfer evaluations are genuine generalizations rather than re-statements of training targets. The shared zexp / per-identity zid design enforces a particular code structure but does not by itself guarantee identity-invariant expression rendering, since the FiLM modulation in Eqs. 7-9 couples zexp with zid; thus the disentanglement claim remains empirically testable and is in fact evaluated on unseen subjects. The main weaknesses noted in the manuscript — the appearance decoder Φapp(zid,d) not receiving zexp, and the mouth-interior shortcut discussed in Supplementary Sec. 7 — are internal architectural/quality gaps, not circular derivations: they do not make Eq. 11 or Eq. 14 equivalent to their inputs. Self-citations to GaussianHeads, ASH, and related works are used for standard architectural components and are not load-bearing for the central residual-formulation or generalization claims. No circular step of the enumerated kinds is present.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The central model is a large learned system: its predictive power comes from per-subject and per-expression latents plus decoder weights fitted to the training data. The dataset alignment and external tooling (VHAP, ProbeSDF, FLAME) supply assumptions that are not validated inside the paper. No new physical entities are introduced.

free parameters (4)
  • residual identity latent codes z_id = R^512 per training identity (330 identities)
    Learned per subject; identity control and residual geometry depend on them. Fitted via reconstruction loss during training.
  • residual expression latent codes z_exp = R^256 per expression (60+12 expressions)
    Learned per expression and shared across identities; the disentanglement claim depends on these codes.
  • decoder network weights = Not itemized
    MLP and CNN weights fit end-to-end via reconstruction loss; they parameterize all predictions.
  • loss weights, latent dimensions, Gaussian grid resolution = Ng=512, dims 512/256, lambda values in Sec. 3.4
    Chosen by hand; affect fidelity, disentanglement, and training stability.
axioms (6)
  • domain assumption FLAME provides an adequate fixed topology and parametric space for coarse head geometry.
    The method builds on FLAME (Li et al. 2017), adopting its vertex semantics and parameterization; if FLAME's model is insufficient for a subject, residuals must compensate.
  • domain assumption VHAP-based tracking yields accurate FLAME parameters for all training frames.
    Coarse geometry v_rec comes from VHAP fits; errors in tracking propagate to the deformed mesh (Eq. 11) and cannot be fully corrected by residuals.
  • domain assumption ProbeSDF depth maps are valid ground truth geometry for supervision.
    Depth loss (Eq. 15) supervises geometry with ProbeSDF reconstructions; any systematic bias in ProbeSDF is inherited by GRMM.
  • domain assumption Manual peak-expression annotation produces semantically consistent alignment across identities.
    Shared z_exp assumes the same expression index has the same meaning across subjects; alignment is by subjective frame selection (Sec. 3.1).
  • domain assumption A single UV-anchored Gaussian grid with barycentric interpolation maintains point correspondence through deformation.
    Gaussian positions are derived from mesh vertices via barycentric interpolation (Sec. 3.3); violation of correspondence would break cross-subject consistency.
  • standard math 3D Gaussian Splatting is a sufficient rendering model for photorealistic heads with the described attribute set.
    Uses 3DGS and its differentiable rasterizer; this is a published rendering technique.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of GRMM: Real-Time High-Fidelity Gaussian Morphable Head Model with Learned Residuals." pith.science (2026). https://pith.science/paper/4BY5AIC7

@misc{pith2026250902141,
  author       = {Pith},
  title        = {Pith review of: GRMM: Real-Time High-Fidelity Gaussian Morphable Head Model with Learned Residuals},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4BY5AIC7}},
  note         = {Machine review of arXiv:2509.02141}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

3D Morphable Models (3DMMs) enable controllable facial geometry and expression editing for reconstruction, animation, and AR/VR, but traditional PCA-based mesh models are limited in resolution, detail, and photorealism. Neural volumetric methods improve realism but remain too slow for interactive use. Recent Gaussian Splatting (3DGS) based facial models achieve fast, high-quality rendering but still depend solely on a mesh-based 3DMM prior for expression control, limiting their ability to capture fine-grained geometry, expressions, and full-head coverage. We introduce GRMM, the first full-head Gaussian 3D morphable model that augments a base 3DMM with residual geometry and appearance components, additive refinements that recover high-frequency details such as wrinkles, fine skin texture, and hairline variations. GRMM provides disentangled control through low-dimensional, interpretable parameters (e.g., identity shape, facial expressions) while separately modelling residuals that capture subject- and expression-specific detail beyond the base model's capacity. Coarse decoders produce vertex-level mesh deformations, fine decoders represent per-Gaussian appearance, and a lightweight CNN refines rasterised images for enhanced realism, all while maintaining 75 FPS real-time rendering. To learn consistent, high-fidelity residuals, we present EXPRESS-50, the first dataset with 60 aligned expressions across 50 identities, enabling robust disentanglement of identity and expression in Gaussian-based 3DMMs. Across monocular 3D face reconstruction, novel-view synthesis, and expression transfer, GRMM surpasses state-of-the-art methods in fidelity and expression accuracy while delivering interactive real-time performance.

Figures

Figures reproduced from arXiv: 2509.02141 by Adam Kortylewski, Christian Theobalt, Kartik Teotia, Mayur Deshmukh, Mohit Mendiratta, Vladislav Golyanik.

Figure 1
Figure 1. Figure 1: GRMM provides disentangled control over a base 3DMM and learned residuals, fitting unseen identities from input images [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Method pipeline. Identity and expression latents zid ∈ R 512 and zexp ∈ R 256, together with FLAME pose/expression param￾eters (θneck, θjaw, αexp), drive the coarse mesh decoder Φmesh to predict per-vertex displacements vδ. Adding these to the tracked mesh vrec yields the deformed mesh Md = (vd, F). UV-anchored 3D Gaussians with initial (pin, rin, sin) are placed on Md. The transfor￾mation decoder ΦT(zid, … view at source ↗
Figure 3
Figure 3. Figure 3: Residual parameterization improves fidelity. Qualita￾tive ablation comparing W/O residuals vs. With residuals. Resid￾uals yield finer hair detail and better mouth articulation (e.g., for ID1, EXP2 the mouth cannot roll in without residuals), with higher PSNR (dB): W/O residuals 28.91 vs. With residuals 30.54 (+1.63). Please zoom in for details. Combining Datasets. We conduct an ablation study to assess the… view at source ↗
Figure 4
Figure 4. Figure 4: Combining Datasets. (left to right) Ground Truth, Without EXPRESS-50, without RenderMe and Ours. Our model trained with the combined datasets leads to overall better identity and expression fidelity [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Novel view synthesis (left) and monocular reconstruction (right). Left: given one or few posed views, GRMM synthesizes unseen viewpoints while preserving identity and expression. Right: from a single RGB frame, GRMM reconstructs the subject and renders both the input and novel views. We compare against Morphable Diffusion [6], MoFaNeRF [52], and HeadNeRF [14] in both settings. FID is reported only in the m… view at source ↗
Figure 6
Figure 6. Figure 6: Expression transfer. From a single frontal image, we invert (Sec. 3.5), swap expression parameters, and render novel views on EXPRESS-50 and NeRSemble. GRMM preserves iden￾tity and subtle expressions with multi-view consistency, whereas Morphable Diffusion (MD) generates inconsistent expressions across views. Please zoom in for details. Expression Transfer. We compare GRMM to Morphable Diffusion for expres… view at source ↗
Figure 8
Figure 8. Figure 8: Examples of 25 distinct identities from the EXPRESS-50 [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Diverse facial expressions from the EXPRESS-50 dataset are aligned consistently across all identities. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Examples of aligned expressions from the Ren [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 13
Figure 13. Figure 13: Zeroing the expression residual zexp exposes a short￾cut: intra-oral appearance is entangled with expression, severely distorting the mouth. Type Ours Morphable Diffusion Tie ++ + ++ + Expression 86.7% 6.0% 3.7% 1.0% 2.7% Identity 89.3% 6.3% 2.7% 1.7% 0.0% Overall 88.0% 6.2% 3.2% 1.3% 1.3% [PITH_FULL_IMAGE:figures/full_fig_p013_13.png] view at source ↗
Figure 11
Figure 11. Figure 11: Examples of reconstructed depth from ProbeSDF. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: UV enhancement. The mouth interior enhancement is highlighted in yellow 9. Locality regularisation for inverse rendering Locality regularization. During the second-stage re￾finement, we regularize the model to remain close to the pretrained solution for interpolations between the fine-tuned subject and dataset identities. Specifically, Ground Truth With Mouth Interior W/O Mouth Interior [PITH_FULL_IMAGE:… view at source ↗
Figure 14
Figure 14. Figure 14: Locality loss preserves the prior and improves novel views. Qualitative ablation comparing With locality loss vs. W/O locality loss. The locality loss preserves the prior and yields sharper detail and fewer artifacts in novel views. PSNR (dB): With 28.43 vs. W/O 27.75 (+0.68). Please zoom in for details. 10. Additional ablations. No Mesh Decoder. Disabling Φmesh and learning only fine residual offsets for… view at source ↗
Figure 16
Figure 16. Figure 16: Screen-space refinement improves fidelity. Quali￾tative ablation comparing With vs. W/O Ψref. Eliminating Ψref degrades hair texture, removes mouth details, and reduces overall sharpness and fidelity—even after 3D-aware rasterization. PSNR (dB): W/O 30.83 vs. With 31.60 (+0.77). Please zoom in for de￾tails. No Refinement Network. Eliminating Ψref leads to de￾graded hair texture, loss of mouth details, and… view at source ↗
Figure 15
Figure 15. Figure 15: No mesh decoder harms expressivity and realism. Qualitative ablation comparing With mesh decoder vs. W/O mesh decoder. Disabling Φmesh and learning only residual Gaussian offsets weakens mouth and cheek articulation and degrades pho￾torealism. PSNR (dB): W/O mesh decoder 32.34 vs. With mesh decoder 34.52 (+2.18). Please zoom in for details. Ground Truth Ground Truth With Refinement W/O Refinement With Ref… view at source ↗
Figure 17
Figure 17. Figure 17: Distangled control of GRMM parameters [PITH_FULL_IMAGE:figures/full_fig_p015_17.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 56 canonical work pages

  1. [1]

    Principal component analysis

    Herv ´e Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433–459, 2010. 2

    work page 2010

  2. [2]

    A morphable model for the synthesis of 3d faces

    V Blanz and T Vetter. A morphable model for the synthesis of 3d faces. In 26th Annual Conference on Computer Graph- ics and Interactive Techniques (SIGGRAPH 1999) , pages 187–194. ACM Press, 1999. 1, 2

    work page 1999

  3. [3]

    A 3d morphable model learnt from 10,000 faces

    James Booth, Anastasios Roussos, Stefanos Zafeiriou, Allan Ponniah, and David Dunaway. A 3d morphable model learnt from 10,000 faces. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 5543–5552,

    work page

  4. [4]

    How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks)

    Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). In International Conference on Computer Vision, 2017. 4

    work page 2017

  5. [5]

    Authentic volumetric avatars from a phone scan

    Chen Cao, Tomas Simon, Jin Kyu Kim, Gabe Schwartz, Michael Zollhoefer, Shun-Suke Saito, Stephen Lombardi, Shih-En Wei, Danielle Belko, Shoou-I Yu, Yaser Sheikh, and Jason Saragih. Authentic volumetric avatars from a phone scan. ACM Trans. Graph., 41(4), 2022. 2

    work page 2022

  6. [6]

    Morphable diffusion: 3d- consistent diffusion for single-image avatar creation

    Xiyi Chen, Marko Mihajlovic, Shaofei Wang, Sergey Prokudin, and Siyu Tang. Morphable diffusion: 3d- consistent diffusion for single-image avatar creation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10359–10370, 2024. 2, 7, 8

    work page 2024

  7. [7]

    Generalizable and ani- matable gaussian head avatar

    Xuangeng Chu and Tatsuya Harada. Generalizable and ani- matable gaussian head avatar. Advances in Neural Informa- tion Processing Systems, 37:57642–57670, 2024. 3

    work page 2024

  8. [8]

    Emoca: Emotion driven monocular face capture and animation

    Radek Dan ˇeˇcek, Michael J Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20311–20322, 2022. 2

    work page 2022

  9. [9]

    Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer

    Yu Deng, Duomin Wang, and Baoyuan Wang. Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer. In European Conference on Computer Vision, pages 316–333. Springer, 2024. 3

    work page 2024

  10. [10]

    3d morphable face models—past, present, and future

    Bernhard Egger, William AP Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, et al. 3d morphable face models—past, present, and future. ACM Transactions on Graphics (ToG), 39(5):1–38, 2020. 1

    2020

  11. [11]

    Learning an animatable detailed 3d face model from in-the- wild images

    Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in-the- wild images. ACM Transactions on Graphics (ToG), 40(4): 1–13, 2021. 2

    work page 2021

  12. [12]

    Learning neural parametric head models

    Simon Giebenhain, Tobias Kirschstein, Markos Georgopou- los, Martin R ¨unz, Lourdes Agapito, and Matthias Nießner. Learning neural parametric head models. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR),

    work page

  13. [13]

    Vrmm: A volumetric re- lightable morphable head model

    Yang Haotian, Zheng Mingwu, Ma ChongYang, Lai Yu-Kun, Wan Pengfei, and Huang Haibin. Vrmm: A volumetric re- lightable morphable head model. In SIGGRAPH 2024 Con- ference Proceedings, 2024. 2

    work page 2024

  14. [14]

    Headnerf: A real-time nerf-based parametric head model

    Yang Hong, Bo Peng, Haiyao Xiao, Ligang Liu, and Juy- ong Zhang. Headnerf: A real-time nerf-based parametric head model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20374– 20384, 2022. 2, 7, 8

    work page 2022

  15. [15]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4), 2023. 2, 3, 4

    work page 2023

  16. [16]

    Sapiens: Foundation for human vision mod- els

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. arXiv preprint arXiv:2408.12569, 2024. 4

    Pith/arXiv arXiv 2024

  17. [17]

    Deep video portraits

    Hyeongwoo Kim, Pablo Garrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Niessner, Patrick P´erez, Christian Richardt, Michael Zollh ¨ofer, and Christian Theobalt. Deep video portraits. ACM Trans. Graph., 37(4), 2018. 2

    work page 2018

  18. [18]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. 4

    work page 2017

  19. [19]

    Nersemble: Multi-view ra- diance field reconstruction of human heads

    Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter, and Matthias Nießner. Nersemble: Multi-view ra- diance field reconstruction of human heads. ACM Trans. Graph., 42(4), 2023. 2, 3, 6

    work page 2023

  20. [20]

    Learning a model of facial shape and expression from 4d scans

    Tianye Li, Timo Bolkart, Michael J Black, and Javier Romero. Learning a model of facial shape and expression from 4d scans. 2017. 1, 2, 4

    work page 2017

  21. [21]

    Real-time high-resolution background matting

    Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sen- gupta, Brian Curless, Steve Seitz, and Ira Kemelmacher- Shlizerman. Real-time high-resolution background matting. arXiv, pages arXiv–2012, 2020. 4

    work page 2012

  22. [22]

    The ryer- son audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english

    Steven R Livingstone and Frank A Russo. The ryer- son audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english. PloS one , 13(5): e0196391, 2018. 6

    work page 2018

  23. [23]

    Mix- ture of volumetric primitives for efficient neural rendering

    Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, and Jason Saragih. Mix- ture of volumetric primitives for efficient neural rendering. ACM Trans. Graph., 40(4), 2021. 2

    work page 2021

  24. [24]

    Jewett, Simon Ven- shtain, Christopher Heilman, Yueh-Tung Chen, Sidi Fu, Mo- hamed Ezzeldin A

    Julieta Martinez, Emily Kim, Javier Romero, Timur Bagaut- dinov, Shunsuke Saito, Shoou-I Yu, Stuart Anderson, Michael Zollh ¨ofer, Te-Li Wang, Shaojie Bai, Chenghui Li, Shih-En Wei, Rohan Joshi, Wyatt Borsos, Tomas Simon, Jason Saragih, Paul Theodosis, Alexander Greene, Anjani Josyula, Silvio Mano Maeta, Andrew I. Jewett, Simon Ven- shtain, Christopher He...

    work page 2024

  25. [25]

    Renderme-360: A large dig- ital asset library and benchmarks towards high-fidelity head avatars

    Dongwei Pan, Long Zhuo, Jingtan Piao, Huiwen Luo, Wei Cheng, Yuxin Wang, Siming Fan, Shengqi Liu, Lei Yang, Bo Dai, Ziwei Liu, Chen Change Loy, Chen Qian, Wayne Wu, Dahua Lin, and Kwan-Yee Lin. Renderme-360: A large dig- ital asset library and benchmarks towards high-fidelity head avatars. Advances in Neural Information Processing Sys- tems, 36, 2024. 2, 3

    work page 2024

  26. [26]

    Ash: Animatable gaussian splats for efficient and photoreal human rendering, 2024

    Haokai Pang, Heming Zhu, Adam Kortylewski, Christian Theobalt, and Marc Habermann. Ash: Animatable gaussian splats for efficient and photoreal human rendering, 2024. 5

    work page 2024

  27. [27]

    Joker: Conditional 3d head syn- thesis with extreme facial expressions, 2024

    Malte Prinzler, Egor Zakharov, Vanessa Sklyarova, Berna Kabadayi, and Justus Thies. Joker: Conditional 3d head syn- thesis with extreme facial expressions, 2024. 3

    work page 2024

  28. [28]

    Vhap: Versatile head alignment with adaptive appearance priors, 2024

    Shenhan Qian. Vhap: Versatile head alignment with adaptive appearance priors, 2024. 4, 1

    work page 2024

  29. [29]

    Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J. Black. Generating 3d faces using convolutional mesh autoencoders, 2018. 2

    work page 2018

  30. [30]

    Bermano, and Daniel Cohen-Or

    Daniel Roich, Ron Mokady, Amit H. Bermano, and Daniel Cohen-Or. Pivotal tuning for latent-based editing of real im- ages, 2021. 6

    work page 2021

  31. [31]

    Relightable gaussian codec avatars

    Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars

    work page

  32. [32]

    Relightable gaussian codec avatars,

    Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars,

    work page

  33. [33]

    Very deep convo- lutional networks for large-scale image recognition, 2015

    Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition, 2015. 6

    2015

  34. [34]

    Gaus- sianheads: End-to-end learning of drivable gaussian head avatars from coarse-to-fine representations

    Kartik Teotia, Hyeongwoo Kim, Pablo Garrido, Marc Haber- mann, Mohamed Elgharib, and Christian Theobalt. Gaus- sianheads: End-to-end learning of drivable gaussian head avatars from coarse-to-fine representations. ACM Trans. Graph., 43(6), 2024. 4

    work page 2024

  35. [35]

    Mofa: Model-based deep convolutional face au- toencoder for unsupervised monocular reconstruction

    Ayush Tewari, Michael Zollhofer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Christian Theobalt. Mofa: Model-based deep convolutional face au- toencoder for unsupervised monocular reconstruction. In Proceedings of the IEEE International Conference on Com- puter Vision (ICCV) Workshops, 2017. 2

    work page 2017

  36. [36]

    Face2face: Real-time face capture and reenactment of rgb videos

    Justus Thies, Michael Zollhofer, Marc Stamminger, Chris- tian Theobalt, and Matthias Niessner. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 2387–2395, 2016. 2

    work page 2016

  37. [37]

    Probesdf: Light field probes for neural surface reconstruc- tion

    Briac Toussaint, Diego Thomas, and Jean-S ´ebastien Franco. Probesdf: Light field probes for neural surface reconstruc- tion. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 11026–11035, 2025. 4, 1

    work page 2025

  38. [38]

    Nonlinear 3d face morphable model, 2018

    Luan Tran and Xiaoming Liu. Nonlinear 3d face morphable model, 2018. 2

    work page 2018

  39. [39]

    Towards high- fidelity nonlinear 3d face morphable model, 2019

    Luan Tran, Feng Liu, and Xiaoming Liu. Towards high- fidelity nonlinear 3d face morphable model, 2019. 2

    work page 2019

  40. [40]

    Face transfer with multilinear models

    Daniel Vlasic, Matthew Brand, Hanspeter Pfister, and Jovan Popovi´c. Face transfer with multilinear models. ACM Trans. Graph., 24(3):426–433, 2005. 2

    work page 2005

  41. [41]

    V oge: A differentiable volume renderer using gaussian ellipsoids for analysis-by-synthesis

    Angtian Wang, Peng Wang, Jian Sun, Adam Kortylewski, and Alan Yuille. V oge: A differentiable volume renderer using gaussian ellipsoids for analysis-by-synthesis. In The Eleventh International Conference on Learning Representa- tions, 2022. 3

    work page 2022

  42. [42]

    Morf: Morphable radiance fields for multiview neural head modeling

    Daoye Wang, Prashanth Chandran, Gaspard Zoss, Derek Bradley, and Paulo Gotardo. Morf: Morphable radiance fields for multiview neural head modeling. In ACM SIG- GRAPH 2022 Conference Proceedings, pages 1–9, 2022. 2

    work page 2022

  43. [43]

    Faceverse: a fine-grained and detail- controllable 3d face morphable model from a hybrid dataset

    Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang Ma, Liang Li, and Yebin Liu. Faceverse: a fine-grained and detail- controllable 3d face morphable model from a hybrid dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 3

    work page 2022

  44. [44]

    Latentavatar: Learning latent expression code for expressive neural head avatar

    Yuelang Xu, Hongwen Zhang, Lizhen Wang, Xiaochen Zhao, Huang Han, Qi Guojun, and Yebin Liu. Latentavatar: Learning latent expression code for expressive neural head avatar. In ACM SIGGRAPH 2023 Conference Proceedings,

    work page 2023

  45. [45]

    3d gaussian parametric head model

    Yuelang Xu, Lizhen Wang, Zerong Zheng, Zhaoqi Su, and Yebin Liu. 3d gaussian parametric head model. In Pro- ceedings of the European Conference on Computer Vision (ECCV), 2024. 2, 3

    work page 2024

  46. [46]

    Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face pre- diction

    Haotian Yang, Hao Zhu, Yanru Wang, Mingkai Huang, Qiu Shen, Ruigang Yang, and Xun Cao. Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face pre- diction. In Proceedings of the ieee/cvf conference on com- puter vision and pattern recognition , pages 601–610, 2020. 1, 2, 3

    work page 2020

  47. [47]

    i3dmm: Deep implicit 3d morphable model of human heads

    T Yenamandra, A Tewari, F Bernard, HP Seidel, M El- gharib, D Cremers, and C Theobalt. i3dmm: Deep implicit 3d morphable model of human heads. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 1, 2

    work page 2021

  48. [48]

    Rade-gs: Rasterizing depth in gaussian splatting

    Baowen Zhang, Chuan Fang, Rakesh Shrestha, Yixun Liang, Xiaoxiao Long, and Ping Tan. Rade-gs: Rasterizing depth in gaussian splatting. arXiv preprint arXiv:2406.01467, 2024. 4

    Pith/arXiv arXiv 2024

  49. [49]

    Bilateral refer- ence for high-resolution dichotomous image segmentation

    Peng Zheng, Dehong Gao, Deng-Ping Fan, Li Liu, Jorma Laaksonen, Wanli Ouyang, and Nicu Sebe. Bilateral refer- ence for high-resolution dichotomous image segmentation. CAAI Artificial Intelligence Research, 2024. 4

    work page 2024

  50. [50]

    Headgap: Few-shot 3d head avatar via generalizable gaus- sian priors

    Xiaozheng Zheng, Chao Wen, Zhaohu Li, Weiyi Zhang, Zhuo Su, Xu Chang, Yang Zhao, Zheng Lv, Xiaoyuan Zhang, Yongjie Zhang, Guidong Wang, and Xu Lan. Headgap: Few-shot 3d head avatar via generalizable gaus- sian priors. arXiv preprint arXiv:2408.06019, 2024. 2, 3

    Pith/arXiv arXiv 2024

  51. [51]

    Headgap: Few-shot 3d head avatar via generalizable gaus- sian priors, 2025

    Xiaozheng Zheng, Chao Wen, Zhaohu Li, Weiyi Zhang, Zhuo Su, Xu Chang, Yang Zhao, Zheng Lv, Xiaoyuan Zhang, Yongjie Zhang, Guidong Wang, and Lan Xu. Headgap: Few-shot 3d head avatar via generalizable gaus- sian priors, 2025. 6

    work page 2025

  52. [52]

    Mofanerf: Morphable facial neural radiance field

    Yiyu Zhuang, Hao Zhu, Xusen Sun, and Xun Cao. Mofanerf: Morphable facial neural radiance field. In European confer- ence on computer vision, pages 268–285. Springer, 2022. 2, 7, 8 GRMM: Real-Time High-Fidelity Gaussian Morphable Head Model with Learned Residuals Supplementary Material

    work page 2022

  53. [53]

    Their combination enables robust learning of geometry, appearance, and expression disentanglement

    Dataset Details We utilise two datasets for training: EXPRESS-50 and RenderMe-360, each offering distinct advantages in terms of expression coverage, identity diversity, and multi-view supervision. Their combination enables robust learning of geometry, appearance, and expression disentanglement. EXPRESS-50 is a multi-view dataset containing 50 subjects (2...

    work page

  54. [54]

    Depth Supervision with ProbeSDF To supervise geometry, we leverage ground-truth depth maps Igt depth generated with ProbeSDF [37]. These depth images are spatially aligned with the corresponding input RGB views, enabling us to directly measure consistency be- tween the reconstructed geometry and the reference depth. As shown in Figure 11, ProbeSDF provide...

    work page

  55. [55]

    Mesh and UV Enhancement Without explicit mouth–interior geometry, the model ex- ploits a shortcut: Mouth interior geometry and appear- ance is implicitly encoded in the expression residual zexp, entangling expression and intra-oral appearance (see Fig- ure 13). Qualitatively, when we zero out the expression residual code zexp the mouth interior becomes se...

    work page

  56. [56]

    Each trial pre- sented three images: a frontal reference (ground truth) on the left and two novel-view renderings from the two meth- ods, labeled A and B

    User study We conducted a user study to qualitatively assess how well GRMM and Morphable Diffusion preserve (i) the reference person’s facial expression and (ii) identity. Each trial pre- sented three images: a frontal reference (ground truth) on the left and two novel-view renderings from the two meth- ods, labeled A and B. For every example, participant...

    work page

  57. [57]

    ++” denotes strong pref- erence and “+

    Locality regularisation for inverse rendering Locality regularization. During the second-stage re- finement, we regularize the model to remain close to the pretrained solution for interpolations between the fine-tuned subject and dataset identities. Specifically, Ground Truth With Mouth Interior W/O Mouth Interior Figure 13. Zeroing the expression residua...

    work page

  58. [58]

    No Mesh Decoder

    Additional ablations. No Mesh Decoder. Disabling Φmesh and learning only fine residual offsets for the Gaussian primitives reduces mouth and cheek articulation, which yields poorer facial ex- pressivity and reduced photorealism, as shown in Figure 15. Ground Truth Ground Truth With Mesh Decoder W/O Mesh Decoder With Mesh Decoder W/O Mesh Decoder Figure 15...

    work page

  59. [59]

    The model is trained for 250,000 iterations with a batch size of 1 on four NVIDIA A100 GPUs

    Implementation details. The model is trained for 250,000 iterations with a batch size of 1 on four NVIDIA A100 GPUs. We use the Adam [18] optimizer with a learning rate of 1 × 10−4 for all learnable parameters

    work page

  60. [60]

    Disentangled control

    Additional Results. Disentangled control. We demonstrate disentangled con- trol over 3DMM and residual parameters; see Figure 17

    work page

  61. [61]

    Limitations and Future Work While significantly advancing the state of the art, our model is not without limitations. One notable limitation of the method lies in its difficulty in handling out-of-distribution subjects, such as individuals with long hair or unconven- tional facial features, which may deviate significantly from the training data. Additiona...

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.