Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatars

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read SqueezeMe distills a Gaussian avatar's neural pose decoder into a 0.45 ms linear layer, enabling three full-body avatars to be animated and rendered at 72 FPS on a standalone Quest 3 headset.

desk verdict A solid systems paper that delivers a real mobile VR demo; the distillation is simple and honestly evaluated, though the 'minimal degradation' claim needs per-frame error statistics. read the letter →

arxiv 2412.15171 v4 pith:PTWHZV4R submitted 2024-12-19 cs.CV

classification cs.CV
keywords 3DGaussianSplattingfull-bodyavatarpose-dependentcorrectiveslineardistillationcorrectivesharingmobileVRrenderingVulkanUV-spacemaps
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the expensive neural decoder that predicts how a photorealistic Gaussian avatar deforms with pose can be replaced by a much cheaper linear model without losing much fidelity. The authors train a CNN decoder that emits pose-dependent correctives on a UV map, then distill that decoder by solving a least-squares regression from a 64-dimensional PCA pose basis to the corrective values. They further share one corrective among each 4x4 block of nearby Gaussians, cutting the number of correctives from roughly 60,000 to 4,000. The result is a decoder that runs in 0.45 ms on a Meta Quest 3's NPU, which is fast enough to animate and render three such avatars simultaneously at 72 FPS in VR. If true, photorealistic, drivable full-body avatars move from desktop GPUs to standalone headsets, which is what real-time social VR and telepresence would need.

What carries the argument

The carrying mechanism is the PCA-linearized pose corrective: a linear map from a 64-dimensional PCA-compressed pose vector to per-Gaussian displacements in position, rotation, scale, and spherical harmonics, fitted by the normal equations so that the pseudo-inverse $(C^T C)^{-1} C^T$ maps decoder outputs to the linear layer's weights. Corrective sharing is the second half of the machinery: the decoder is trained to emit one corrective per 4x4 neighborhood of the UV map (4096 correctives instead of 65536), and nearest upsampling broadcasts each corrective to its block. The linear layer's dominant cost drops from 5.0 ms to 0.45 ms on the Quest 3's NPU, and the paper attributes the feasibility of this compression to pose-dependent correctives being lower-frequency than the static Gaussian fields.

What would settle it

Take the distilled 0.45 ms model and the CNN teacher, feed both a held-out set of extreme poses that the 512 distillation key frames do not cover, and measure per-pixel LPIPS and per-Gaussian corrective error separately at joint neighborhoods (shoulders, elbows, hips, knees) versus torso and limbs; if the joint-region error gap between teacher and distilled model grows well beyond the Table 1 average gap, the low-frequency/linear-subspace assumption is falsified rather than merely occasionally stressed.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that pose-dependent Gaussian correctives are low-frequency, and that this property supports a two-stage distillation that preserves quality while making mobile inference practical. A teacher avatar built from a CNN decoder (60k Gaussians, 60k correctives) already matches the quality of Animatable Gaussians with five times fewer Gaussians. The teacher's decoder is then replaced by a two-stage linear model: a least-squares fit from a 64-dim PCA-compressed pose vector, including a ones column, to the corrective map, followed by an expansion of six spherical-harmonic channels into the full SH coefficients. Sharing correctives among 4x4 UV blocks shrinks the linear matrix roughly 16x, reducing decoder latency from 5.0 ms to 0.45 ms. With a custom Vulkan splatting pipeline, this makes it possible to animate and render three full-body Gaussian avatars simultaneously at 72 FPS on a Meta Quest 3, with quantitative quality close to the CNN teacher (L1 0.036 vs 0.039, LPIPS 0.146 vs 0.151 on the combined GCS+linearized model).

Load-bearing premise

The load-bearing premise is that pose-dependent correctives are low-frequency and lie in a low-dimensional linear subspace, so a 64-dim PCA pose basis plus one corrective per 4x4 UV block can reproduce what the CNN decoder learned.

Editorial extensions

If this is right

  • A decoder cost of 0.45 ms per avatar is small enough that three full-body avatars can be animated and rendered simultaneously at 72 FPS on a standalone VR headset, meeting the refresh-rate budget VR requires.
  • The combined GCS+linearized model stays within 0.003 L1 and 0.005 LPIPS of the full CNN teacher on held-out poses and novel views.
  • Distillation is more reliable than training a linear decoder from scratch, which diverged for one of four identities and scored clearly worse when it did converge.
  • Because the avatar is driven by on-the-fly pose inputs rather than pre-recorded video, the method supports novel animation, unlike mobile volumetric-video streaming approaches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The low-frequency assumption suggests an immediate extension: use adaptive UV block sizes, small near joints and sleeve/armpit boundaries and large on the torso, to target exactly the artifacts Figure 6 shows in the 4k and linearized models.
  • The same two-stage recipe (CNN teacher, PCA linearization, spatially shared correctives) should transfer to other animatable Gaussian representations whose bottleneck is pose decoding, such as head avatars or 4D scene models, whenever correctives are smooth in space.
  • The small gap between the no-decoder and linearized models (0.040 vs 0.039 L1) suggests that after linearization, remaining quality loss comes more from the static template and shared corrective resolution than from the pose model's nonlinearity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper presents SqueezeMe, a system for converting high-fidelity Gaussian full-body avatars into a mobile-ready representation. The authors train a CNN decoder that predicts pose-dependent Gaussian correctives on a UV map, then distill it into a linear layer via PCA on poses and least-squares regression on correctives. They further reduce memory by sharing correctives among 4x4 UV neighborhoods (GCS). They implement a Vulkan-based splatting pipeline and report real-time animation and rendering of three avatars at 72 FPS on a Meta Quest 3. Quantitative evaluation on an internal 4-identity dataset and on AvatarRex compares favorably to Animatable Gaussians, with a GCS+Linearized model reducing decoder latency to 0.45 ms.

Significance. The paper is a solid systems contribution. Its strengths include evaluation on held-out poses and cameras, comparison to Animatable Gaussians on two datasets, a clear ablation of distillation and sharing, and an unusually honest discussion of failure cases. If the quality claims survive closer scrutiny, the result is practically significant for VR. The main weakness is that the 'minimal degradation' claim is supported only by aggregate means over four identities, while the paper's own failure figures show localized artifacts at joints and garment boundaries.

major comments (3)
  1. [Section 6, Table 1] The headline claim that 'quality degradation by the proposed distillation is minimal' (Introduction) is supported only by mean metrics over 4 identities, with no per-identity or per-frame breakdown. Given that Fig. 6(b,d,e) shows visible artifacts in exactly the GCS and linearized models, the averages could hide a long tail of high-error frames. Please report per-identity results, standard deviations, and/or quantiles of per-frame errors, and state how often the failure modes in Fig. 6 occur.
  2. [Section 4.2, Eq. (8)-(9) and Section 6 (Limitations)] The load-bearing assumption of Gaussian corrective sharing is that pose-dependent correctives are low-frequency in UV space. The manuscript's own Fig. 6(b,d,e) demonstrates that this assumption fails at armpits, sleeve edges, and the pants seat with legs apart. Since the memory reduction from 60k to 4k correctives depends on this assumption, the claim that the loss in visual fidelity is 'minimal' requires quantitative evidence about how often and how severely these failure modes occur. Please report, for example, the per-frame error distribution separated by pose difficulty, or a UV-space error map for representative frames.
  3. [Table 1, footnote] The 'Linear from scratch' baseline is averaged over only 3 identities because one training run diverged. This makes the comparison against the distilled models (averaged over 4 identities) inconsistent and weakens the claim that distillation is necessary for stable training. Please report per-identity results for all rows, including the divergent run (e.g., as 'did not converge'), or use the same set of identities for all rows.
minor comments (4)
  1. [Section 6, Quantization] The word 'quantative' should be 'quantitative'.
  2. [Appendix C] The sentence 'SqueezeMe (GCS) outperforms AG on L1 and SSIM, and SqueezeMe (GCS) matches AG on L1' is internally contradictory; based on Table 3, GCS matches AG on L1 and outperforms on SSIM.
  3. [Conclusion] The stated baseline of 50 ms for the CNN decoder is not reported in Table 1 or Section 5; please specify the hardware and measurement conditions for this number.
  4. [Abstract and Introduction] The phrase 'for the first time' should be softened to 'to the best of our knowledge' unless a comprehensive literature search is provided.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the linearized decoder is a supervised fit, but the reported evaluation is on held-out poses and cameras against ground truth and an external baseline.

full rationale

The paper's derivation chain is a standard supervised distillation. The CNN pose-corrective decoder is compressed by PCA on poses, least-squares regression from 64-d pose codes to the decoder's corrective outputs on 512 training frames, and UV-space corrective sharing with nearest-neighbor upsampling. None of these steps defines the reported evaluation metric in terms of the fitted quantities. The held-out protocol is stated explicitly: 'The evaluation set is held-out from the dataset that is used for training, linear distillation, and quantization.' Quality is measured with rendered-image L1, LPIPS, PSNR, and SSIM against ground-truth images, and the main comparison is against Animatable Gaussians, an external baseline, on novel camera views and poses. The linear model's match to the teacher on the distillation frames is guaranteed by least squares, but the paper does not present that as the evidence; the evidence is held-out rendered-image quality. The low-frequency-corrective assumption is an explicit design hypothesis, and the Limitations section honestly documents where it fails, e.g., 'the 4k and linearized models struggle with the edge of a t-shirt sleeve' and 'the arm and leg joints Gaussians may move more independently, causing artifacts to sometimes appear in those regions.' That is the opposite of hiding a circular fit. Self-citations such as Pixel Codec Avatars, Drivable 3D Gaussian Avatars, and Sapiens appear only as background, related work, or tooling, and none is load-bearing for the central claim. Therefore no circular step is exhibited; the correct finding is no significant circularity.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities. Its central empirical result rests on free design choices, including PCA dimension, sharing grid, keyframe count, and loss weights, and on the untested assumption that pose-dependent correctives are low-frequency and well approximated by a 64-dim linear subspace. The paper's own failure cases show the assumption only partially holds near joints.

free parameters (7)
  • PCA pose dimension = 64
    Chosen by hand as the compressed input dimension in Algorithm 1; controls the capacity of the linear model.
  • Corrective sharing grid = 64x64 = 4096 correctives
    Output resolution of D_GCS; each corrective is shared by a 4x4 neighborhood of Gaussians after nearest upsampling (Section 4.2).
  • Distillation keyframe count = 512 frames
    512 training frames used to fit PCA and the least-squares corrective layer (Section 5); a larger or smaller set changes the fit.
  • Teacher loss weights = lambda_photo=1, lambda_lpips=0.1, lambda_opacity=0.01, lambda_scale=1, lambda_offset=1, lambda_alpha=0.1, lambda_kpt=0.1
    Hand-set weights in Eq. 6; they shape the teacher CNN and therefore the distilled target.
  • Latent code dimensions = face 32, body 32
    e_f and e_b dimensions set in Section 4.1; not derived from data statistics.
  • Upsampling method = nearest interpolation
    Appendix A shows bilinear is slightly better (PSNR 25.075 vs 25.024) but nearest was chosen for implementation ease; a modeling choice with measurable quality cost.
  • Visible Gaussian count after masking = 60,381 per identity
    Depends on the UV mask M applied to the 256x256 map; differs per identity and controls the linear layer width.
assumptions (6)
  • domain assumption 3D Gaussian splatting and LBS pose Gaussians faithfully
    Eqs. 1 to 5 are inherited from 3DGS and Animatable Gaussians; no proof is given that this representation captures hair and clothing detail.
  • domain assumption Pose-dependent correctives are low-frequency and locally shared
    Section 4.2 motivates PCA and 4x4 sharing; failure cases in Fig. 6b,d,e show the assumption partially breaks at joints.
  • domain assumption The CNN teacher's correctives are a valid distillation target
    The linear layer is fitted to M(D(pi)) in Algorithm 1; teacher errors propagate and no ground-truth corrective is measured.
  • domain assumption Sapiens segmentation masks correctly identify avatar vs background
    L_alpha uses these masks to enforce opacity; mask errors would bias the avatar's alpha.
  • domain assumption Held-out poses and cameras are representative of deployment motion
    Evaluation uses 2000 held-out frames and 5 cameras per identity (Section 5); out-of-distribution poses may exceed the 512-frame PCA coverage.
  • standard math PCA and normal equations are standard linear algebra
    Least-squares fitting in Algorithm 1 is treated as background math.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatars." pith.science (2026). https://pith.science/paper/PTWHZV4R

@misc{pith2026241215171,
  author       = {Pith},
  title        = {Pith review of: SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatars},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PTWHZV4R}},
  note         = {Machine review of arXiv:2412.15171}
}
read the original abstract

Gaussian-based human avatars have achieved an unprecedented level of visual fidelity. However, existing approaches based on high-capacity neural networks typically require a desktop GPU to achieve real-time performance for a single avatar, and it remains non-trivial to animate and render such avatars on mobile devices including a standalone VR headset due to substantially limited memory and computational bandwidth. In this paper, we present SqueezeMe, a simple and highly effective framework to convert high-fidelity 3D Gaussian full-body avatars into a lightweight representation that supports both animation and rendering with mobile-grade compute. Our key observation is that the decoding of pose-dependent Gaussian attributes from a neural network creates non-negligible memory and computational overhead. Inspired by blendshapes and linear pose correctives widely used in Computer Graphics, we address this by distilling the pose correctives learned with neural networks into linear layers. Moreover, we further reduce the parameters by sharing the correctives among nearby Gaussians. Combining them with a custom splatting pipeline based on Vulkan, we achieve, for the first time, simultaneous animation and rendering of 3 Gaussian avatars in real-time (72 FPS) on a Meta Quest 3 VR headset. Demo videos are available at https://forresti.github.io/squeezeme.

Figures

Figures reproduced from arXiv: 2412.15171 by the authors.

Figure 1
Figure 1. With SqueezeMe, we simultaneously run 3 Gaussian avatars locally on a Meta Quest 3 VR headset. Upper: Stereo view in VR. Lower: View into the VR headset lenses. Gaussian-based human avatars have achieved an unprecedented level of visual fidelity. However, existing approaches based on high-capacity neural networks typically require a desktop GPU to achieve real-time performance for a single avatar, and it remains non… view at source ↗
Figure 2
Figure 2. System Diagram during training. This is the configuration we use for training the model in Section 4.1. 4.1 Compact 3D Gaussian Avatar Our high-fidelity 3D avatar consists of a convolutional decoder that predicts a 2D gaussian map G ∈ R 𝐾×256×256 where each pixel represents a Gaussian and 𝐾 = 37 are the degrees of freedom of each Gaussian. In contrast to Animatable Gaussians [Li et al. 2024], which uses front/back o… view at source ↗
Figure 3
Figure 3. System Diagram with optimizations. We show the end-to-end optimized system, including the techniques from Sections 4.2 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of the mobile-ready avatars. Left: ground [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results. Our 0.45 ms model produces results that are competitive with far more expensive models [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Failure cases. (a) In all models and identities, hands are sometimes blurry. (b) The 4k and linearized models struggle with the edge of a t-shirt sleeve. (c) All models have unwanted transparency under the arm for this identity’s avatar, but it is worse in 4k and linea…
Figure 7
Figure 7. Figure 7: Comparing Bilinear and Nearest interpolation in SqueezeMe (GCS) [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Qualitative ablation study. This figure evaluates the effect of the losses used to train the convolutional decoder. Using the losses of Animatable Gaussians [Li et al. 2024] as our baseline, we incorporate an opacity and scale loss to encourage the model to use less an…
Figure 9
Figure 9. Figure 9: Qualitative results on AvatarRex [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Privacy-preserving Photorealistic Self-avatars in Mixed Reality

    cs.HC 2025-07 conditional novelty 6.0 of 10

    AvatarLDP and AvatarRotation distort identity embeddings to create de-identified but photorealistic 2D and 3D avatars while aiming to preserve age, race, and gender.

Reference graph

Works this paper leans on

18 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [2]

    Further, we provide a qualitative evaluation in Figure 7 and find that the two methods produce very similar results

    The numerical results show that Bilinear has a small advantage in quality over Nearest upsampling. Further, we provide a qualitative evaluation in Figure 7 and find that the two methods produce very similar results. For ease of imple- mentation, and to reduce the load on the mobile GPU, we selected nearest interpolation for our VR demo videos and for the ...

  2. [3]

    Results are averaged over two identities

    Results on AvatarRex. Results are averaged over two identities. Model # Gaussians # Correctives L1 ↓ LPIPS↓ PSNR↑ SSIM↑ Animatable Gaussians [Li et al. 2024] 300k 300k 0.059 0.151 19.542 0.844 SqueezeMe 60k 60k 0.057 0.156 20.178 0.851 SqueezeMe (GCS) 60k 4k 0.059 0.158 20.051 0.849 14 • Iandola et al. Ground Truth Baseline𝓛photo +𝓛lpips +𝓛offset 57516 vi...

  3. [4]

    High-fidelity and real-time novel view synthesis for dynamic scenes

    Haotong Lin, Sida Peng, Zhen Xu, Tao Xie, Xingyi He, Hujun Bao, and Xiaowei Zhou. High-fidelity and real-time novel view synthesis for dynamic scenes. In SIGGRAPH Asia 2023 Conference Papers , pages 1–9,

  4. [6]

    3d gaussian blendshapes for head avatar animation

    Shengjie Ma, Yanlin Weng, Tianjia Shao, and Kun Zhou. 3d gaussian blendshapes for head avatar animation. In ACM SIGGRAPH 2024 Conference Papers , pages 1–10,

  5. [7]

    Radsplat: Radiance field-informed gaussian splatting for robust real-time rendering with 900+ fps

    Michael Niemeyer, Fabian Manhardt, Marie-Julie Rakotosaona, Michael Oechsle, Daniel Duckworth, Rama Gosula, Keisuke Tateno, John Bates, Dominik Kaeser, and Federico Tombari. Radsplat: Radiance field-informed gaussian splatting for robust real-time rendering with 900+ fps. arXiv:2403.13806,

  6. [8]

    Dynamic Point Fields

    Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Animatable neural radiance fields for human body modeling. In International Conference on Computer Vision (ICCV) , 2021a. Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representatio...

  7. [9]

    Drivable volumetric avatars using texel-aligned features

    Edoardo Remelli, Timur Bagautdinov, Shunsuke Saito, Chenglei Wu, Tomas Simon, Shih-En Wei, Kaiwen Guo, Zhe Cao, Fabian Prada, Jason Saragih, et al. Drivable volumetric avatars using texel-aligned features. In ACM SIGGRAPH 2022 Conference Proceedings,

  8. [10]

    NPC: Neural Point Characters from Video

    Shih-Yang Su, Timur M. Bagautdinov, and Helge Rhodin. Npc: Neural point characters from video. ArXiv, abs/2304.02013,

Show all 18 references
  1. [13]

    Bakedsdf: Meshing neural sdfs for real-time view synthesis

    Lior Yariv, Peter Hedman, Christian Reiser, Dor Verbin, Pratul P Srinivasan, Richard Szeliski, Jonathan T Barron, and Ben Mildenhall. Bakedsdf: Meshing neural sdfs for real-time view synthesis. In ACM SIGGRAPH 2023 Conference Proceedings , pages 1–9,

  2. [14]

    Drivable 3d gaussian avatars

    Wojciech Zielonka, Timur Bagautdinov, Shunsuke Saito, Michael Zollhöfer, Justus Thies, and Javier Romero. Drivable 3d gaussian avatars. arXiv.org, 2311.08581,

  3. [15]

    Gaussian Eigen Models for Human Heads

    Wojciech Zielonka, Timo Bolkart, Thabo Beeler, and Justus Thies. Gaussian Eigen Models for Human Heads. arXiv:2407.04545,

  4. [18]

    This figure evaluates the effect of the losses used to train the convolutional decoder

    Qualitative ablation study. This figure evaluates the effect of the losses used to train the convolutional decoder. Using the losses of Animatable Gaussians [Li et al. 2024] as our baseline, we incorporate an opacity and scale loss to encourage the model to use less and smalle...

  5. [2004]

    Vid2actor: Free- viewpoint animatable person synthesis from video in the wild

    Chung-Yi Weng, Brian Curless, and Ira Kemelmacher-Shlizerman. Vid2actor: Free- viewpoint animatable person synthesis from video in the wild. arXiv preprint arXiv:2012.12884,

  6. [2019]

    Videorf: Rendering dynamic radiance fields as 2d feature video streams

    Liao Wang, Kaixin Yao, Chengcheng Guo, Zhirui Zhang, Qiang Hu, Jingyi Yu, Lan Xu, and Minye Wu. Videorf: Rendering dynamic radiance fields as 2d feature video streams. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 470–481, 2024a. P...

  7. [2021]

    Neural volumes: Learning dynamic renderable volumes from images

    Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. Neural volumes: Learning dynamic renderable volumes from images. arXiv preprint arXiv:1906.07751,

  8. [2022]

    Animatable Gaussians: Learning pose-dependent gaussian maps for high-fidelity human avatar modeling

    Zhe Li, Zerong Zheng, Lizhen Wang, and Yebin Liu. Animatable Gaussians: Learning pose-dependent gaussian maps for high-fidelity human avatar modeling. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 19711–19722,

  9. [2023]

    Sapiens: Foundation for human vision models

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision models. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024...

  10. [2024]

    Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps

    Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang. Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. arXiv preprint arXiv:2311.17245,

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.