Pith. sign in

REVIEW 3 major objections 5 minor 3 references

TeGA: Texture Space Gaussian Avatars for High-Resolution Dynamic Head Modeling

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper presents TeGA, a hybrid head-avatar model that embeds 3D Gaussians in the continuous UV-displacement tangent space of a tracked face mesh, warps them with an expression-dependent field, and renders high-fidelity 4K reenactments…

desk verdict Genuinely new Gaussian avatar method with strong qualitative results, but the quantitative tables contradict themselves—needs major revision before the SOTA claim is credible. read the letter →

arxiv 2505.05672 v1 pith:D65WFQ3C submitted 2025-05-08 cs.CV

Gengyan Li , Paulo Gotardo , Timo Bolkart , Stephan Garbin , Kripasindhu Sarkar , Abhimitra Meka , Alexandros Lattas , Thabo Beeler This is my paper · ORCID
classification cs.CV
keywords 3DGaussiansplattingheadavatarsUVDtexturespaceexpression-dependentdeformationdynamicshading4Krenderingadaptivedensificationfacialreenactment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to close the gap between animatable head avatars and per-expression reconstructions by building a 3D Gaussian representation in the continuous UVD (UV plus surface-normal displacement) tangent space of a tracked face mesh. This lets Gaussians move freely across mesh triangles during optimization, so adaptive densification can concentrate detail where it is needed instead of being limited by a fixed UV texture resolution. A U-Net reads rasterized vertex displacements and produces expression-dependent codes that drive a learned residual deformation field and a monochromatic shading term, capturing transient features such as wrinkles and skin darkening. The authors report that, on multiview video of eight in-studio subjects plus the Multiface dataset, TeGA improves landmark alignment, perceptual similarity, and close-up fidelity over GaussianAvatars, GaussianHeadAvatars, RGCA, and MVP, while rendering at 4K with up to four million Gaussians. The value of the claim, if correct, is that controllable avatars no longer trade animation flexibility for high-resolution detail.

What carries the argument

The load-bearing object is the continuous UVD tangent space: each canonical Gaussian lives at a UV texture coordinate plus a scalar displacement $D$ along the locally interpolated surface normal, so the whole set acts as a sparse volumetric texture indexed by the mesh. The mapping $F(\boldsymbol{\mu}_{uvd})$ to world space, together with the Jacobian-based covariance transform $\Sigma_{xyz} = J_F \Sigma_{uvd} J_F^T$, converts triangle deformation into Gaussian rotation and stretching, including non-rigid stretching. A shared U-Net encodes rasterized neutral-relative vertex displacements into a $256 \times 256 \times 64$ feature texture; bilinear interpolation gives each Gaussian a local expression code $f_{uv}$, which conditions a shallow MLP residual deformation field and a shallow MLP shading head. Because the feature texture dimension is decoupled from the Gaussian count, the model keeps 3D Gaussian splatting's clone-and-split densification, bounded at 4 million Gaussians, and thus allocates primitives where detail demands them.

What would settle it

Take a multiview capture with known ground-truth geometry (for example, synchronized 4D scans) and compare the full TeGA model trained on the provided tracked meshes versus the same model trained on the ground-truth meshes, measuring close-up fidelity in the mouth interior; if both versions show the same artifacts there, the tracked-mesh assumption is not the limiting factor, whereas if the ground-truth version improves, the central detail claim depends on mesh accuracy as the paper acknowledges.

Watch

Extended reading notes

Core claim

The central claim is that a tracked 3DMM mesh can serve as a coarse deformation layer while photorealistic detail lives in a sparse volumetric texture of 3D Gaussians embedded in the mesh's continuous UVD space. Canonical Gaussians are parameterized by $\boldsymbol{\mu}_{uvd}$ with a displacement $D$ along the surface normal; the mapping $F$ from UVD to world coordinates and its Jacobian $J_F$ propagate mesh deformation to each Gaussian's position and covariance. A residual translation field $D(\cdot)$ conditioned on U-Net features $f_{uv}$ adds fine, expression-dependent motion, and the combined Jacobian $J_{(F+D)}$ reshapes Gaussians accordingly. An expression-dependent shading factor $S(\boldsymbol{\mu}_{uvd}; f_{uv})$ darkens wrinkles without changing chromaticity. The paper argues that this combination, non-greedy cross-triangle movement, adaptive densification up to millions of Gaussians, and network-based deformation and shading, is what allows 4K close-ups with sharp wrinkles, stubble, and brows while remaining controllable through expression and pose parameters.

Load-bearing premise

The load-bearing premise is that the tracked 3D meshes give dense, accurate correspondence across expressions, so the UV tangent space and its Jacobian faithfully describe how the skin deforms; the paper itself notes this is unreliable for teeth, tongue, and the mouth interior.

Editorial extensions

If this is right

  • High-resolution close-ups of animated heads can be rendered directly from 3D Gaussians, without a separate super-resolution network on the image plane; the paper's 4K results are produced by the representation itself.
  • The same UVD rigging works on any tracked mesh sequence in correspondence, not only the paper's 3DMM, since it is demonstrated on Multiface meshes; the method is therefore portable across tracking systems.
  • Because densification is retained, per-region detail such as stubble, brow hairs, and wrinkles is allocated adaptively during optimization rather than being fixed by a UV-map texel budget.
  • The residual deformation field plus Jacobian propagation extracts rotation and stretching from a simple learned translation field, making expression-dependent motion easier to learn than directly predicting full pose and scale per Gaussian.
  • Expression-dependent monochromatic shading captures occlusion-like darkening in wrinkles; the paper's limitation is that it cannot change Gaussian chromaticity, so dynamic skin-color effects such as blood flow remain outside the model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • [Editorial inference] The UVD-plus-Jacobian rigging is a general template for animating any deformable surface with a UV atlas, so the same design could transfer to hands, bodies, or clothing if a tracked template mesh is available; the paper does not test these cases.
  • [Editorial inference] A likely testable extension is replacing the monochromatic shading with a small chromatic spherical-harmonics or albedo correction conditioned on the same expression codes; this would directly address the paper's stated blood-flow limitation.
  • [Editorial inference] The paper's appendix finding that a scaled-rigid per-triangle transform matches the Jacobian when no residual field is used suggests the larger wins come from cross-triangle Gaussian movement and densification; validating the full model with the residual field active against scaled-rigid rotation would isolate each contribution.
  • [Editorial inference] If mesh tracking accuracy improves for the mouth interior and teeth, the same pipeline should inherit those gains, since the paper identifies tracked-mesh quality as the limiting factor for those regions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces TeGA, an animatable 3D head avatar model built on 3D Gaussian splatting. The key idea is to define canonical Gaussians in the continuous UVD tangent space of a tracked 3DMM mesh, propagate mesh deformation to Gaussian covariance through the Jacobian of the UVD-to-world mapping, and add a U-Net/MLP-based residual deformation field and an expression-dependent dynamic shading term. Adaptive densification is retained from 3DGS, allowing millions of Gaussians, and the method is evaluated on a new 8-subject studio dataset and on 4 Multiface subjects against GaussianAvatars, GaussianHeadAvatars, RGCA, and MVP, with reenactment and novel-view metrics. The paper claims state-of-the-art quality at high resolution, with extensive ablations of the deformation field, shading, densification, triangle updates, and loss terms.

Significance. If the reported results are reproducible, TeGA is a meaningful advance for high-resolution animatable head avatars: it decouples the latent texture resolution from the number of Gaussians, uses a principled Jacobian-based covariance transformation, and demonstrates qualitative gains in close-up detail in the included figures. The experimental design is strong in breadth: four baselines, two datasets, multiple ablations, and explicit discussion of limitations such as mouth interior artifacts and the dependence on tracked meshes. The main obstacle is that the quantitative core of the paper is not internally consistent as printed: the same named configurations receive different numbers in the main and appendix tables, and the Gaussian-count cap is stated differently in two places. The headline SOTA claim therefore cannot yet be verified from the manuscript as written.

major comments (3)
  1. [§5.1, Table 1 vs Appendix G, Tables 4–5] The main quantitative table and the appendix tables report conflicting values for identically labeled configurations. For example, the 'Ours' row on 'our dataset' gives Landmark 31.7 ± 28.6, LPIPS 0.150, SSIM 0.779, PSNR 24.4 and novel-view LPIPS 0.123 in Table 1, while Appendix Table 4 gives Landmark 25.7 ± 10.5, LPIPS 0.120, SSIM 0.781, PSNR 24.9 and novel-view LPIPS 0.099. The 'Ours (200K GS.)' rows differ similarly (Landmark 49.0 ± 26.1 vs 37.2 ± 19.4), and every ablation row in Table 1 differs from the corresponding row in Appendix Table 4. Because the central claim is a quantitative improvement over the state of the art, the paper must present one consistent set of per-dataset numbers, state precisely how the aggregates in each table are computed, and preferably release per-subject metric logs so the results can be verified. This issue is load-bearing and must be resolved before the paper can be accepted.
  2. [§3.8 vs §D.2, Table 2] The Gaussian-count upper bound is stated inconsistently. Section 3.8 says 'we set an upper bound of 4 million Gaussians to avoid running out of memory,' while Appendix D.2 says 'we set an upper limit of at most 6M Gaussians' and Table 2 reports results for a 6M row. The appendix explanation that most subjects do not exceed 4M is plausible, but as printed the two statements conflict. Please state the exact cap used in each experiment reported in Tables 1, 2, 4, and 5, and distinguish the implementation cap from the observed Gaussian counts.
  3. [§5.2.3, Table 4 vs Table 1] The claim that adaptive densification is 'critical' is not uniformly supported by the appendix table. In Table 1, removing densification causes a large drop (Landmark 122.7 ± 147, LPIPS 0.274, SSIM 0.738, PSNR 21.3 with full-model values of 31.7, 0.150, 0.779, 24.4). In Appendix Table 4, however, the same ablation on the same named dataset gives Landmark 68.6 ± 19.3, LPIPS 0.223, SSIM 0.780, PSNR 24.4, which is only marginally worse than the full model in SSIM and PSNR. This discrepancy affects the paper's explanation of why densification matters and should be reconciled with a clear statement of which table corresponds to the reported protocol.
minor comments (5)
  1. [Abstract, §1, §4] The abstract and introduction describe the method as rendering at '4K resolution,' but the studio data are trained at 3072×2048 after downsampling and Multiface at 2048×1334. Please clarify the relationship between the native camera resolution, the training resolution, and the '4K' claim.
  2. [§3.6, Eq. (6)-(8)] Equation (6) introduces λ_JD for the Jacobian smoothness term, but the following paragraph refers to λ_smooth with values 1.0 decaying to 0.1. Please use consistent notation and specify where λ_smooth enters Eq. (6).
  3. [§5, Landmark metric] The landmark evaluation is described as the 'mean-squared difference' between detected keypoints, but the reported values are not given units. Please state whether the metric is squared pixel distance, mean Euclidean pixel distance, or normalized coordinates, and whether the average is taken over landmarks or frames.
  4. [Tables 1, 4, 5] The baseline name appears as 'GaussianHeadAvatar' in Appendix Tables 4–5 and 'GaussianHeadAvatars' in the main text and Table 1. Please use one name consistently.
  5. [§E.1] The U-Net description says each block has 'two convolutional layers, the first layer downsampling or upsampling using striding or transposing respectively,' but Figure 13 appears to show a skip connection with a separate strided/transposed convolution. Please make the block description match the figure precisely.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation found; the SOTA claim rests on held-out comparisons, with self-citations limited to related work and implementation details.

full rationale

The paper's central claim—higher-fidelity animatable head avatars than GaussianAvatars, GaussianHeadAvatars, RGCA, and MVP—is supported by held-out evaluation, not by a quantity that is fed back into the model. Section 5 states 'For each subject, we evaluate on test frames from three training views, using three cameras distributed horizontally in front of the face' plus holdout cameras, and Tables 1, 4, and 5 compare against external baselines run with the same geometry. Equations (1)-(5) define the UVD-to-world map, Jacobian covariance propagation, residual deformation, and dynamic shading; they are modeling definitions, not predictions derived from their own outputs. The deformation field is learned end-to-end, but it is an internal function optimized on the training split, not a fitted parameter renamed as a reported result. Self-citations (e.g., ShellNeRF, LitNeRF, Gaussian Eigen Models) appear in related work or as minor implementation choices (e.g., conditioning deformation on local codes, 'As noted in Bai et al. [2023]'), and none carries the SOTA claim. Two non-circular issues should be noted: the main text (Sec 3.8) sets a 4M Gaussian upper bound while Appendix D states 6M, and the 'Ours' row differs between Table 1 and Table 4; these are reproducibility and consistency problems, not circular derivation. The Sec 6 limitation about teeth, tongue, and mouth interior likewise concerns mesh-tracking accuracy, not self-reference. No load-bearing circular step is identifiable.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method is an empirical system; the central claim depends on a network trained with hand-chosen hyperparameters, a high-quality tracked mesh input, and the assumption that local surface deformation is captured by the Jacobian of the UV-to-world map. No new physical entities are introduced. The free parameters listed are the main manual choices; the Gaussian positions and attributes are optimized rather than fitted in the sense of a closed-form derivation.

free parameters (6)
  • Training loss weights = lambda_1=0.8, lambda_D-SSIM=0.2, lambda_VGG=0 to 0.1 with a ramp after 50 percent of training, lambda_smooth=1.0 to 0.1
    Chosen by hand and not derived; the central training objective in Eq. 6 depends on them. The VGG and smoothness schedules are motivated by observed training stability, not by a first-principles argument.
  • Densification upper bound = 4,000,000 Gaussians in Sec 3.8; 6,000,000 in Appendix D.2
    Set to avoid out-of-memory on an 80 GB H100. The bound directly controls the claimed detail level and is inconsistent between the main text and the appendix.
  • Gaussian standard deviation range = between 0.02 mm and 5 mm
    Applied as a hard clamp on Gaussian sizes during optimization, following Li et al. 2024a; chosen by hand.
  • U-Net latent texture resolution N = 256
    The U-Net output resolution for deformation and shading codes; it decouples representation capacity from Gaussian count but is still a manual choice.
  • Densification gradient thresholds = defaults from 3D Gaussian Splatting (Kerbl et al. 2023)
    Used unmodified for cloning and splitting Gaussians; chosen by the original 3DGS authors, not by this paper.
  • Smoothness loss sampling count = 100,000 UVD points uniformly sampled
    Number of sample points for L_JD in Eq. 8; chosen to balance coverage and compute.
assumptions (4)
  • domain assumption The tracked 3DMM mesh sequence provides dense correspondence and a consistent UV parameterization across all expressions and subjects.
    Used to define the canonical UVD space in Sec 3.3 and to generate the vertex-displacement texture that is the U-Net input in Sec 3.4.
  • domain assumption The local deformation of the face surface is well approximated by the Jacobian of the UVD-to-world mapping F.
    Eq. 2 propagates Gaussian covariance through J_F. Appendix F shows a scaled-rigid approximation yields similar quality, suggesting the assumption is reasonable on the Gaussian scale.
  • domain assumption The learned deformation field D is continuous and smooth enough that its Jacobian, added to J_F in Eq. 4, is a valid local linearization.
    The smoothness regularizer in Eq. 8 is imposed precisely to keep J_D small and smooth; the paper relies on this for stable training.
  • domain assumption Multiview training images are photometrically consistent and accurately matted.
    The rendering loss in Eq. 6 compares rendered and captured images; errors in calibration or matting would be absorbed by the model as geometry or shading artifacts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TeGA: Texture Space Gaussian Avatars for High-Resolution Dynamic Head Modeling." pith.science (2026). https://pith.science/paper/D65WFQ3C

@misc{pith2026250505672,
  author       = {Pith},
  title        = {Pith review of: TeGA: Texture Space Gaussian Avatars for High-Resolution Dynamic Head Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D65WFQ3C}},
  note         = {Machine review of arXiv:2505.05672}
}
read the original abstract

Sparse volumetric reconstruction and rendering via 3D Gaussian splatting have recently enabled animatable 3D head avatars that are rendered under arbitrary viewpoints with impressive photorealism. Today, such photoreal avatars are seen as a key component in emerging applications in telepresence, extended reality, and entertainment. Building a photoreal avatar requires estimating the complex non-rigid motion of different facial components as seen in input video images; due to inaccurate motion estimation, animatable models typically present a loss of fidelity and detail when compared to their non-animatable counterparts, built from an individual facial expression. Also, recent state-of-the-art models are often affected by memory limitations that reduce the number of 3D Gaussians used for modeling, leading to lower detail and quality. To address these problems, we present a new high-detail 3D head avatar model that improves upon the state of the art, largely increasing the number of 3D Gaussians and modeling quality for rendering at 4K resolution. Our high-quality model is reconstructed from multiview input video and builds on top of a mesh-based 3D morphable model, which provides a coarse deformation layer for the head. Photoreal appearance is modelled by 3D Gaussians embedded within the continuous UVD tangent space of this mesh, allowing for more effective densification where most needed. Additionally, these Gaussians are warped by a novel UVD deformation field to capture subtle, localized motion. Our key contribution is the novel deformable Gaussian encoding and overall fitting procedure that allows our head model to preserve appearance detail, while capturing facial motion and other transient high-frequency features such as skin wrinkling.

Figures

Figures reproduced from arXiv: 2505.05672 by the authors.

Figure 1
Figure 1. We propose a novel method that builds photorealistic 3D head avatars for rendering at high resolution with controllable expressions and viewpoints. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our method learns a dense set of canonical 3D Gaussians within the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The UVD deformation field and dynamic shading components are [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Our method manages to extrapolate better when compared to RGCA, GA, MVP and GHA. Thanks to the warpfield and densification we can more [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: We compare zoomed in patches of the red highlighted area. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Close-up view of the highlighted patch, showing that our dynamic [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 8
Figure 8. Figure 8: Our method manages to extrapolate better when compared to RGCA, GA, MVP and GHA. Thanks to the warpfield and densification we can more [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: In comparison to RGCA, GA, MVP and GHA our approach has significantly higher fidelity on details, such as stubble (top, middle, subjects from [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 13
Figure 13. Figure 13: All layer weights are uniform initialized using the default [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 10
Figure 10. Figure 10: Comparison between maximum number of Gaussians for densification. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 12
Figure 12. Figure 12: A schematic of our U-Net architecture [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 11
Figure 11. Figure 11: Our UV layout: red areas indicate invalid UV locations that are not [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 14
Figure 14. Figure 14: A schematic of our MLP heads The positions of each Gaussian in UVD space is used as a sec￾ondary input, and is positionally encoded using 8 frequencies. While the output of the deformation network is used directly, the output of the shading network has the following f…
Figure 15
Figure 15. Figure 15: A comparison of our Jacobian transformation with a scaled rigid [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

3 extracted references · 3 linked inside Pith

  1. [3]

    single-shot

    High-resolution image synthesis with latent diffusion models. In Conference on Computer Vision and Pattern Recognition (CVPR) . CVF / IEEE, 10684–10695. Shunsuke Saito, Stanislav Pidhorskyi, Igor Santesteban, Forrest Iandola, Divam Gupta, Anuj Pahuja, Nemanja Bartolovic, Frank Yu, Emanuel Garbin, and Tomas Simon. 2024a. SqueezeMe: Efficient Gaussian Avata...

    arXiv 2024

  2. [2009]

    In International conference on advanced video and signal based surveillance

    A 3D face model for pose and illumination invariant face recognition. In International conference on advanced video and signal based surveillance . IEEE, 296– 301. Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. 2024. GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians. In Confere...

    arXiv 2024

  3. [2022]

    in-the-wild

    Efficient geometry-aware 3D generative adversarial networks. In Conference on Computer Vision and Pattern Recognition (CVPR) . CVF / IEEE, 16123–16133. Yufan Chen, Lizhen Wang, Qijing Li, Hongjiang Xiao, Shengping Zhang, Hongxun Yao, and Yebin Liu. 2024. MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar. In SIGGRAPH Conference Papers (SA) . A...

    arXiv doi:10.48550/arxiv.1412.6980 2012

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.