Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

GAS: Generative Avatar Synthesis from a Single Image

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Dense 3D reconstruction cues, not sparse templates, give single-image avatars view and temporal consistency.

desk verdict A solid systems paper with a genuinely useful combination, but the evaluation under-measures the upstream reliability it depends on. read the letter →

arxiv 2502.06957 v2 pith:4TGLLSTL submitted 2025-02-10 cs.CV

Yixing Lu , Junting Dong , Youngjoong Kwon , Qin Zhao , Bo Dai , Fernando De la Torre This is my paper · ORCID
classification cs.CV
keywords single-imageavatarsynthesisnovelviewposeanimationvideodiffusionmodelgeneralizableNeRFSMPLnormalmapsconsistencytemporalcoherence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that diffusion-based single-image avatar generation fails because sparse conditioning signals such as depth or SMPL normal maps do not match the subject's true appearance, causing flickering across views and temporal instability. Its proposed method, GAS, first reconstructs the person with a generalizable human NeRF, then feeds that NeRF's dense renderings of the target view or pose, alongside SMPL normal maps, into a video diffusion model. The claim is that this dense appearance-and-geometry conditioning enforces both multi-view and temporal consistency, and the paper reports state-of-the-art PSNR, SSIM, LPIPS, and FVD on THuman, 2K2K, and TikTok benchmarks. If right, a single casually captured photo can produce a consistent, animatable avatar without studio capture or per-subject optimization.

What carries the argument

The central object is the dense appearance cue: renderings produced by a single-view generalizable human NeRF (built on pixel-aligned features and inverse linear blend skinning into SMPL canonical space) of the target novel view or pose, paired with the geometry cue of SMPL normal maps rendered under the same target camera. Both cues are encoded into latent features, element-wise added, and injected into the first convolutional layers of the Stable Video Diffusion UNet, while CLIP features of the reference image enter via cross-attention; a one-hot switcher embedded into the time embedding disentangles static view synthesis from dynamic pose animation. The mechanism's role is to give the diffusion model dense, appearance-rich guidance that stays 3D-consistent across frames, so the generative prior sharpens and refines rather than hallucinating appearance from a sparse signal.

What would settle it

Take an input image in which the SMPL fit is known to be badly wrong, such as a person in a heavy coat with crossed arms recorded from an unusual camera angle, render the NeRF appearance cue for a target view, and generate that view with GAS; if the generated avatar shows the same geometric errors as the bad fit rather than correcting them, the dense cue does not shield the diffusion model from upstream regression failures. A numerical version: compare GAS against an oracle variant whose conditioning renderings use ground-truth geometry from a 3D scan instead of the fitted SMPL; a large quality gap would show that the claimed consistency depends on fitting accuracy rather than on the dense-cue design itself.

Watch

Extended reading notes

Core claim

The central discovery is that the mismatch between sparse conditioning templates and the real appearance of the subject is the root cause of multi-view and temporal inconsistency in generative avatar synthesis, and that replacing the sparse template with dense renderings from a generalizable human NeRF closes that gap. GAS first fits SMPL and trains a single-view generalizable NeRF on multi-view human data, then freezes it and uses its renderings as appearance cues, paired with SMPL normal maps as geometry cues, to condition a Stable Video Diffusion model. A one-hot switcher embedded into the time embedding lets one shared model handle both novel view synthesis and novel pose synthesis, and training on a mix of 3D scans, multi-view videos, and internet videos yields generalization to in-the-wild images. The reported numbers show consistent gains over strong baselines on both tasks.

Load-bearing premise

The whole pipeline rests on the frozen upstream regression stage: for a given input image, the SMPL fit and the generalizable NeRF renderings must be accurate enough to guide the diffusion model, and when they are wrong (poor fitting, occlusions, hands or face), the conditioning misleads the generator and the consistency claim collapses for that input.

Editorial extensions

If this is right

  • Single-image avatar generation can be treated as video generation conditioned on the output of a regression-based 3D reconstruction, so improvements in generalizable human reconstruction translate directly into better view and pose consistency.
  • Because the appearance and geometry cues are rendered offline by frozen modules, large-scale training on internet videos becomes feasible for novel view synthesis, extending studio-trained methods to casual in-the-wild imagery.
  • The switcher result implies that view synthesis and pose animation are distinct modalities that should not be naively mixed in one diffusion model, even when the underlying representation is shared.
  • If the method holds, applications such as telepresence, gaming, virtual try-on, and digital content creation gain a practical route from a single photo to an animatable, view-consistent avatar without per-subject optimization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's diagnosis implies that any sparse or coarse conditioning signal suffers the same fidelity gap, so other generative avatar systems could gain more from densifying their conditioning with 3D reconstruction outputs than from adding more control modalities.
  • The supplementary limitation that SMPL lacks expressiveness in the face and hands points to the next bottleneck: swapping SMPL for more expressive whole-body models or adding regional supervision would likely push the same pipeline further.
  • A clean testable extension is to replace the NeRF appearance cue with a different dense predictor, such as a generalizable Gaussian-splatting renderer, to isolate whether the consistency gain comes from denseness per se or from NeRF's particular rendering properties.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes GAS, a two-stage pipeline for single-image avatar synthesis. In the first stage, a frozen generalizable human NeRF (SHERF) renders a sequence of target-view or target-pose images from a single reference image, conditioned on estimated SMPL parameters. In the second stage, these NeRF renderings (dense appearance cues) together with SMPL normal maps (geometry cues) condition a Stable Video Diffusion model, which is fine-tuned jointly for novel-view and novel-pose synthesis with a one-hot switcher that separates the two tasks. Training combines 3D scans (THuman2.1, 2K2K), multi-view videos (MVHumanNet), and monocular in-the-wild videos (TikTok and a curated set of internet videos). Experiments compare against Champ and Animate Anyone, with and without fine-tuning on the same training data, and report consistent gains in PSNR, SSIM, LPIPS, and FVD on THuman, 2K2K, and TikTok. Ablations isolate the contributions of the geometry cue, appearance cue, diffusion refinement, switcher, and internet-video training.

Significance. If the reported results hold, the paper makes a useful empirical contribution: it demonstrates that dense appearance cues from a generalizable NeRF are a stronger conditioning signal for video-diffusion avatar synthesis than sparse SMPL-based templates, and it shows that joint training with internet videos can improve generalization for both novel-view and novel-pose synthesis. The paper is commendable for fine-tuning both baselines on the same training data, for running a thorough set of internal ablations whose directions are internally consistent, and for including runtime and memory comparisons in the supplementary material. The main limitations are evaluative rather than conceptual: the evidence for state-of-the-art performance and for in-the-wild robustness is not statistically quantified, and the pipeline's dependence on the frozen upstream SMPL/NeRF stage is acknowledged in the text but not analyzed experimentally.

major comments (3)
  1. [4.2 and 4.3, Tables 1 and 2] The central claim of state-of-the-art performance rests on average metric differences without error bars, confidence intervals, or significance tests across subjects. For example, on THuman novel-view synthesis the gap between Ours and the best fine-tuned baseline is 1.57 dB PSNR and 0.005 SSIM, while on TikTok the FVD gap over Champ* is 223 points; given that only 6 subjects are used for TikTok testing, these averages may not be stable. Please report per-subject standard deviations and confidence intervals, run paired significance tests, and provide per-subject breakdowns for the TikTok test set.
  2. [3.3 and Supplementary Limitation A] The pipeline's in-the-wild generalization is conditional on the frozen upstream stage: Section 3.3 concedes that 'inaccurate SMPL fittings or occlusions' can corrupt NeRF guidance, and Supplementary Limitation A notes that SMPL lacks expressiveness in the face and hands. Yet the experiments do not quantify how often or how severely these upstream failures occur, and the TikTok/internet-video results are not stratified by fit quality or annotated difficulty. Since the advertised contribution is generalization to casually captured images, the central claim is currently supported only for inputs where the upstream fit happens to be reliable. Please add a robustness analysis—for instance, correlate output metrics with SMPL fitting error or manual failure labels, and report results on a set of deliberately hard inputs.
  3. [4.1.3 and Tables 1 and 2] The baselines (Champ and Animate Anyone) are fine-tuned for 10,000 iterations, while the proposed model is trained for 150,000 iterations, and no convergence evidence is shown for the baselines. If 10,000 iterations under-trains these models, the reported improvements (e.g., Ours 26.77 vs Champ* 23.89 PSNR on THuman; Ours 19.11 vs Champ* 18.57 on TikTok) could partly reflect an unfair compute budget. Please provide fine-tuning loss curves, train the baselines for a comparable number of iterations, and show that the relative ordering is stable.
minor comments (6)
  1. [4.2, evaluation-protocol paragraph] The sentence 'we apply the ground truth masks to remove backgrounds in the THuman dataset' is ambiguous; it should state explicitly whether masks are applied to all methods or only to Animate Anyone, since this affects the fairness of the comparison.
  2. [Equations (2) and (3)] The noise-prediction loss is written with an unsquared norm; if the standard MSE objective is used, please write the squared norm to avoid confusion.
  3. [Tables 3 and 4] The column headers NVS/NPS do not name the dataset for each number: NPS uses MVHumanNet in Table 3 but TikTok in Table 4, and NVS uses THuman in Table 3 but 2K2K in Table 4. The captions should list the exact dataset per column.
  4. [Supplementary D.1, Table 6] The table caption says '50 consecutive novel poses' while the preceding text says '100 consecutive novel poses'; this is inconsistent and should be corrected.
  5. [Section 3.3, first paragraph] The phrase 'which servers as an input' is a typo for 'serves'.
  6. [Section 4.2, opening sentences] The claim of 'state-of-the-art performance across all evaluation metrics' is stronger than the comparison set supports, since only Champ and Animate Anyone are compared; please either add recent baselines (e.g., MagicMan, Human4DiT) or qualify the claim to the compared methods.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GAS is an externally evaluated conditional generative pipeline; self-citations are contextual, and the target-view NeRF conditioning is a design input, not a definitional reduction.

full rationale

No circular step was found in the derivation chain. GAS proceeds in two trained stages: a generalizable NeRF (SHeRF, an external method) is trained on MVHumanNet, then a video diffusion model initialized from Stable Video Diffusion is trained with the objective in Eq. (3) to predict noise given reference-image latents, NeRF renderings Cnerf, SMPL normal maps Csmpl, and a switcher s. The NeRF rendering of the target view is indeed used as a dense appearance condition, but the output target is the ground-truth latent Z, not the condition; the model is trained by denoising, and Table 4 shows a large change after diffusion (PSNR 24.25 to 28.82 for novel view synthesis), so the output is not identical to the conditioning by construction. The paper's central claim, that dense appearance cues improve view and pose consistency, is supported by controlled ablations (Table 3, Figures 5-6) rather than assumed by definition. Self-citations ([9], [19], [23], [24], [25], [45], [56]) appear only in related-work surveys or supplementary design exploration and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. The manuscript's own limitations—Section 3.3 noting that 'inaccurate SMPL fittings or occlusions' corrupt NeRF guidance, and Limitation A noting SMPL 'lacks expressiveness in regions such as the face and hands'—are honest robustness caveats about upstream modules. They affect generalization and artifact frequency but do not make the central claim reduce to its own inputs. The derivation is self-contained against external benchmarks, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a stack of trained components (SMPL fitting, generalizable NeRF, Stable Video Diffusion, CLIP) plus hand-chosen hyperparameters; none are derived in the paper. The ledger counts these upstream dependencies and chosen constants because the reliability of the claimed result depends on their accuracy.

free parameters (3)
  • CFG scale and schedule = 2 for pose; triangular 1 to 2 to 1 for views
    Chosen by hand per task to trade off sharpness and consistency; affects all qualitative and quantitative outputs (Section 4, Inference).
  • Loss weights (lambda_ssim, lambda_lpips, lambda_mask) = not reported
    Used in the generalizable NeRF training objective in Section 3.5; exact values are not given.
  • Frame count and overlap = 20 frames; 6-frame overlap at inference
    Training batch design and inference window choice described in Section 4.1.2 and Table 6; affects temporal consistency.
assumptions (5)
  • standard math Volume rendering integration and the diffusion forward/reverse process are assumed from NeRF and Stable Video Diffusion.
    Used throughout Section 3.3 without re-derivation; the noise schedule and alpha/sigma terms are inherited from SVD.
  • domain assumption SMPL parameters from 4DHumans are accurate enough on in-the-wild images.
    Invoked in Sections 3.2 and 3.3 and in the data preparation of Section 4.1.1; the authors acknowledge in supplement A that SMPL is limited for face and hands.
  • domain assumption A generalizable human NeRF trained on MVHumanNet generalizes to unseen identities and provides beneficial appearance cues.
    Central to the method (Sections 3.2 and 4.1.2); the ablations in Table 3 support this empirically for the tested benchmarks.
  • ad hoc to paper A one-hot switcher is sufficient to prevent motion leakage between static view synthesis and dynamic pose synthesis.
    Introduced in Section 3.4; the design is supported by the ablation in Table 9 and Figure 9 but has no theoretical justification.
  • domain assumption The TikTok test split (subjects 335 to 340) does not overlap with the 122 internet video sequences selected from Champ's training data.
    Section 4.1.1 defines the splits; no overlap analysis or filtering details are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GAS: Generative Avatar Synthesis from a Single Image." pith.science (2026). https://pith.science/paper/4TGLLSTL

@misc{pith2026250206957,
  author       = {Pith},
  title        = {Pith review of: GAS: Generative Avatar Synthesis from a Single Image},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4TGLLSTL}},
  note         = {Machine review of arXiv:2502.06957}
}
read the original abstract

We present a unified and generalizable framework for synthesizing view-consistent and temporally coherent avatars from a single image, addressing the challenging task of single-image avatar generation. Existing diffusion-based methods often condition on sparse human templates (e.g., depth or normal maps), which leads to multi-view and temporal inconsistencies due to the mismatch between these signals and the true appearance of the subject. Our approach bridges this gap by combining the reconstruction power of regression-based 3D human reconstruction with the generative capabilities of a diffusion model. In a first step, an initial 3D reconstructed human through a generalized NeRF provides comprehensive conditioning, ensuring high-quality synthesis faithful to the reference appearance and structure. Subsequently, the derived geometry and appearance from the generalized NeRF serve as input to a video-based diffusion model. This strategic integration is pivotal for enforcing both multi-view and temporal consistency throughout the avatar's generation. Empirical results underscore the superior generalization ability of our proposed method, demonstrating its effectiveness across diverse in-domain and out-of-domain in-the-wild datasets.

Figures

Figures reproduced from arXiv: 2502.06957 by the authors.

Figure 1
Figure 1. In-the-wild avatar synthesis across views and poses. Starting from a reference image, we generate its novel views and animate the avatar given a pose sequence. Abstract We present a unified and generalizable framework for syn￾thesizing view-consistent and temporally coherent avatars from a single image, addressing the challenging task of single-image avatar generation. Existing diffusion-based methods often conditio… view at source ↗
Figure 2
Figure 2. Overview of GAS. Starting from a single input image, GAS uses a generalizable human NeRF to map the subject into a canonical space, then reposes and renders the 3D NeRF model to extract detailed appearance cues (i.e., NeRF renderings). These are paired with geometry cues (i.e., SMPL normal maps) and fed into a video diffusion model. A switcher module disentangles the tasks, enabling the model to generate either mult… view at source ↗
Figure 3
Figure 3. Qualitative comparisons for novel view synthesis on the THuman dataset. For the first subject, our method generates cleaner garment textures and sharper facial details, achieving better realism and consistency across views (e.g., the hair style in our generated front view is faithful to the reference image). For the second subject, baseline methods exhibit inconsistencies across views, marked with circles (e.g., hai… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons for novel pose synthesis on MVHumanNet dataset. In the first row of Champ [67] results, blue rectangles mark disappearing arms, green rectangles show varying sleeve lengths, and red rectangles indicate inconsistencies in hair appearance. * denot…
Figure 5
Figure 5. Figure 5: Ablation study on the appearance cue. Without the appearance cue, artifacts include incorrect arm raises (red rectan￾gles) and distorted hand placement on the waist (blue circles), both resolved with the appearance cue. Occlusions, which are frequent in in-the-wild vid…
Figure 6
Figure 6. Figure 6: Ablation study on the geometry cue. Without the ge￾ometry cue, occlusion leads human NeRF to misinterpret the arm as clothing texture, which further misleads diffusion generation (red rectangles). References Novel views 1 Novel views 2 [PITH_FULL_IMAGE:figures/full_fi…
Figure 9
Figure 9. Figure 9: Ablation on the switcher for disentangling static view and dynamic motion synthesis. Without the switcher, undesired clothing deformations, such as dress swinging, are involved in the novel view generation, corrupting the view consistency. Effect of switcher for disent…
Figure 10
Figure 10. Figure 10: Network architecture for processing the conditions of the video diffusion model. ϵgeo and ϵappr denote geometry encoder and appearance encoder, respectively. ⊕ denotes element-wise addition. The switcher embedding and time embedding are injected into the diffusion mod…
Figure 11
Figure 11. Figure 11: Novel pose results on THuman and 2K2K. The refer￾ence images are animated by pose sequences derived from MVHu￾manNet dataset. D.5 Robustness to input view angles We train our model using an arbitrary view as input. Thus, we are interested in the novel view synthesis r…
Figure 12
Figure 12. Figure 12: Quantitative and qualitative results before/after diff. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Image-to-image novel view synthesis on MVHuman￾Net dataset by using image diffusion model and appearance cue. Consistency comparisons. Compared to directly using a pretrained video diffusion model, these image-based dif￾fusion methods exhibit significantly inferior pe…
Figure 14
Figure 14. Figure 14: Novel view synthesis on THuman2.1 by a multi-view image-based diffusion model. Inconsistent clothing wrinkles ap￾pear between two adjacent novel view generations. Free-view interpolation. Due to GPU memory limitations, we are restricted to training with approximately …
Figure 15
Figure 15. Figure 15: Qualitative results of the generated novel views for various input views of the same human subject. neither approach achieves satisfactory results: image-based diffusion models show significant inconsistencies, while video diffusion models produce blurry frames. E.2 N…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EgoAnimate: Generating Human Animations from Egocentric top-down Views

    cs.CV 2025-07 conditional novelty 4.0 of 10

    EgoAnimate synthesizes a frontal T-pose image from an egocentric top-down photo using a fine-tuned Stable Diffusion model, then animates it with off-the-shelf image-to-motion methods to produce an animatable avatar.

Reference graph

Works this paper leans on

69 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    Easymocap - make human motion capture easier. Github,

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3, 5

    arXiv 2023

  3. [3]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023. 3

    2023

  4. [4]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators

  5. [5]

    Everybody dance now

    Caroline Chan, Shiry Ginosar, Tinghui Zhou, and Alexei A Efros. Everybody dance now. In IEEE International Confer- ence on Computer Vision (ICCV), 2019. 2

    work page 2019

  6. [6]

    Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion

    Di Chang, Yichun Shi, Quankai Gao, Hongyi Xu, Jessica Fu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mo- hammad Soleymani. Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion. In Forty-first International Conference on Machine Learn- ing, 2023. 2, 5, 12

    work page 2023

  7. [7]

    X-dyna: Expressive dynamic hu- man image animation

    Di Chang, Hongyi Xu, You Xie, Yipeng Gao, Zhengfei Kuang, Shengqu Cai, Chenxu Zhang, Guoxian Song, Chao Wang, Yichun Shi, et al. X-dyna: Expressive dynamic hu- man image animation. arXiv preprint arXiv:2501.10021 ,

    arXiv

  8. [8]

    Totalselfscan: Learning full- body avatars from self-portrait videos of faces, hands, and bodies

    Junting Dong, Qi Fang, Yudong Guo, Sida Peng, Qing Shuai, Xiaowei Zhou, and Hujun Bao. Totalselfscan: Learning full- body avatars from self-portrait videos of faces, hands, and bodies. In Advances in Neural Information Processing Sys- tems, 2022. 2

    work page 2022

Show all 69 references
  1. [9]

    ivs-net: Learning human view synthesis from internet videos

    Junting Dong, Qi Fang, Tianshuo Yang, Qing Shuai, Chengyu Qiao, and Sida Peng. ivs-net: Learning human view synthesis from internet videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22942–22951, 2023. 2

    2023

  2. [10]

    Humans in 4D: Reconstructing and tracking humans with transformers

    Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, and Jitendra Malik. Humans in 4D: Reconstructing and tracking humans with transformers. In ICCV, 2023. 5

    2023

  3. [11]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 2

    2014

  4. [12]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 3, 15

    2023 arXiv

  5. [13]

    High-fidelity 3d hu- man digitization from single 2k resolution images

    Sang-Hun Han, Min-Gyu Park, Ju Hong Yoon, Ju-Mi Kang, Young-Jae Park, and Hae-Gon Jeon. High-fidelity 3d hu- man digitization from single 2k resolution images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR2023), 2023. 5, 12

    2023

  6. [14]

    Magicman: Generative novel view synthesis of humans with 3d-aware diffusion and iterative refinement

    Xu He, Xiaoyu Li, Di Kang, Jiangnan Ye, Chaopeng Zhang, Liyang Chen, Xiangjun Gao, Han Zhang, Zhiyong Wu, and Haolin Zhuang. Magicman: Generative novel view synthesis of humans with 3d-aware diffusion and iterative refinement. arXiv preprint arXiv:2408.14211, 2024. 3

    2024 arXiv

  7. [15]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 5

    2022 arXiv

  8. [16]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. arXiv preprint arXiv:2311.17117, 2023. 2, 5, 6, 15

    2023 arXiv

  9. [17]

    Sherf: Generalizable human nerf from a single image

    Shoukang Hu, Fangzhou Hong, Liang Pan, Haiyi Mei, Lei Yang, and Ziwei Liu. Sherf: Generalizable human nerf from a single image. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , pages 9352–9364,

  10. [18]

    TeCH: Text-guided Reconstruction of Lifelike Clothed Humans

    Yangyi Huang, Hongwei Yi, Yuliang Xiu, Tingting Liao, Ji- axiang Tang, Deng Cai, and Justus Thies. TeCH: Text-guided Reconstruction of Lifelike Clothed Humans. InInternational Conference on 3D Vision (3DV), 2024. 2

    2024

  11. [19]

    Epidiff: Enhancing multi-view synthesis via localized epipolar-constrained diffusion

    Zehuan Huang, Hao Wen, Junting Dong, Yaohui Wang, Yangguang Li, Xinyuan Chen, Yan-Pei Cao, Ding Liang, Yu Qiao, Bo Dai, et al. Epidiff: Enhancing multi-view synthesis via localized epipolar-constrained diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and...

    2024

  12. [20]

    Learning high fi- delity depths of dressed humans by watching social media dance videos

    Yasamin Jafarian and Hyun Soo Park. Learning high fi- delity depths of dressed humans by watching social media dance videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12753– 12762, 2021. 5, 12, 15

    2021

  13. [21]

    Pippo: High-resolution multi-view humans from a single image

    Yash Kant, Ethan Weber, Jin Kyu Kim, Rawal Khirod- kar, Su Zhaoen, Julieta Martinez, Igor Gilitschenski, Shun- suke Saito, and Timur Bagautdinov. Pippo: High-resolution multi-view humans from a single image. arXiv preprint arXiv:2502.07785, 2025. 3

    2025 arXiv

  14. [22]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  15. [23]

    Neural human performer: Learning generalizable ra- diance fields for human performance rendering

    Youngjoong Kwon, Dahun Kim, Duygu Ceylan, and Henry Fuchs. Neural human performer: Learning generalizable ra- diance fields for human performance rendering. Advances in Neural Information Processing Systems, 34:24741–24752,

  16. [24]

    Neural image-based avatars: Generalizable radi- ance fields for human avatar modeling

    Youngjoong Kwon, Dahun Kim, Duygu Ceylan, and Henry Fuchs. Neural image-based avatars: Generalizable radi- ance fields for human avatar modeling. arXiv preprint arXiv:2304.04897, 2023. 2

    2023 arXiv

  17. [25]

    Gen- eralizable human gaussians for sparse view synthesis

    Youngjoong Kwon, Baole Fang, Yixing Lu, Haoye Dong, Cheng Zhang, Francisco Vicente Carrasco, Albert Mosella- Montoro, Jianjin Xu, Shingo Takagi, Daeil Kim, et al. Gen- eralizable human gaussians for sparse view synthesis. arXiv preprint arXiv:2407.12777, 2024. 1, 2

    2024 arXiv

  18. [26]

    Deliffas: Deformable light fields for fast avatar synthesis

    Youngjoong Kwon, Lingjie Liu, Henry Fuchs, Marc Haber- mann, and Christian Theobalt. Deliffas: Deformable light fields for fast avatar synthesis. Advances in Neural Informa- tion Processing Systems, 36, 2024. 2

    2024

  19. [27]

    Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation

    John P Lewis, Matt Cordner, and Nickson Fong. Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the 27th an- nual conference on Computer graphics and interactive tech- niques, pages 165–172, 2000. 2

    2000

  20. [28]

    Synthesizing moving people with 3d control

    Boyi Li, Jathushan Rajasegaran, Yossi Gandelsman, Alexei A Efros, and Jitendra Malik. Synthesizing moving people with 3d control. arXiv preprint arXiv:2401.10889 ,

    arXiv

  21. [29]

    Dispose: Disen- tangling pose guidance for controllable human image anima- tion

    Hongxiang Li, Yaowei Li, Yuhang Yang, Junjie Cao, Zhi- hong Zhu, Xuxin Cheng, and Long Chen. Dispose: Disen- tangling pose guidance for controllable human image anima- tion. arXiv preprint arXiv:2412.09349, 2024. 2

    2024 arXiv

  22. [30]

    Pshuman: Photorealistic single-image 3d human reconstruction using cross-scale multiview diffusion and explicit remeshing

    Peng Li, Wangguandong Zheng, Yuan Liu, Tao Yu, Yang- guang Li, Xingqun Qi, Xiaowei Chi, Siyu Xia, Yan-Pei Cao, Wei Xue, et al. Pshuman: Photorealistic single-image 3d human reconstruction using cross-scale multiview diffusion and explicit remeshing. In Proceedings of the Compu...

    2025

  23. [31]

    Neural rendering and reenactment of human actor videos

    Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, and Christian Theobalt. Neural rendering and reenactment of human actor videos. ACM Transactions on Graphics (TOG), 38(5):1–14, 2019. 2

    2019

  24. [32]

    Neural actor: Neural free-view synthesis of human actors with pose con- trol

    Lingjie Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, and Christian Theobalt. Neural actor: Neural free-view synthesis of human actors with pose con- trol. ACM transactions on graphics (TOG) , 40(6):1–16,

  25. [33]

    Human-vdm: Learning single-image 3d human gaussian splatting from video diffusion models

    Zhibin Liu, Haoye Dong, Aviral Chharia, and Hefeng Wu. Human-vdm: Learning single-image 3d human gaussian splatting from video diffusion models. arXiv preprint arXiv:2409.02851, 2024. 3

    2024

  26. [34]

    Smpl: a skinned multi- person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: a skinned multi- person linear model. ACM Transactions on Graphics (TOG), 34(6):1–16, 2015. 2, 3

    2015

  27. [35]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 2, 3

    2021

  28. [36]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  29. [37]

    Ani- matable neural radiance fields for modeling dynamic human bodies

    Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, and Hujun Bao. Ani- matable neural radiance fields for modeling dynamic human bodies. In ICCV, 2021. 2

    2021

  30. [38]

    Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans

    Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In CVPR,

  31. [39]

    Anigs: Animatable gaussian avatar from a single image with inconsistent gaussian reconstruction

    Lingteng Qiu, Shenhao Zhu, Qi Zuo, Xiaodong Gu, Yuan Dong, Junfei Zhang, Chao Xu, Zhe Li, Weihao Yuan, Liefeng Bo, et al. Anigs: Animatable gaussian avatar from a single image with inconsistent gaussian reconstruction. arXiv preprint arXiv:2412.02684, 2024. 2, 3

    2024 arXiv

  32. [40]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  33. [41]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks,

  34. [42]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  35. [43]

    Pifu: Pixel-aligned implicit function for high-resolution clothed human digitiza- tion

    Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Mor- ishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitiza- tion. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 2304–2314, 2019. 2

    2019

  36. [44]

    Human4dit: Free-view human video generation with 4d diffusion transformer

    Ruizhi Shao, Youxin Pang, Zerong Zheng, Jingxiang Sun, and Yebin Liu. Human4dit: Free-view human video generation with 4d diffusion transformer. arXiv preprint arXiv:2405.17405, 2024. 2, 3

    2024 arXiv

  37. [45]

    Drive: Diffusion-based rigging em- powers generation of versatile and expressive characters

    Mingze Sun, Junhao Chen, Junting Dong, Yurun Chen, Xinyu Jiang, Shiwei Mao, Puhua Jiang, Jingbo Wang, Bo Dai, and Ruqi Huang. Drive: Diffusion-based rigging em- powers generation of versatile and expressive characters. arXiv preprint arXiv:2411.17423, 2024. 2

    2024 arXiv

  38. [46]

    To- wards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018. 5

    2018 arXiv

  39. [47]

    SV3D: Novel multi-view synthesis and 3D generation from a single image using la- tent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitrii Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. SV3D: Novel multi-view synthesis and 3D generation from a single image using la- tent video diffusion. In European Conference on Compute...

    2024

  40. [48]

    Is attention all that nerf needs? arXiv preprint arXiv:2207.13298, 2022

    Peihao Wang, Xuxi Chen, Tianlong Chen, Subhashini Venu- gopalan, Zhangyang Wang, et al. Is attention all that nerf needs? arXiv preprint arXiv:2207.13298, 2022. 2

    2022 arXiv

  41. [49]

    Disco: Disentangled control for realistic human dance generation

    Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung- Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,...

    2024

  42. [50]

    One-shot free-view neural talking-head synthesis for video conferenc- ing

    Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 10039–10049,

  43. [51]

    Holigs: Holistic gaussian splatting for embod- ied view synthesis

    Xiaoyuan Wang, Yizhou Zhao, Botao Ye, Xiaojun Shan, Weijie Lyu, Lu Qi, Kelvin CK Chan, Yinxiao Li, and Ming- Hsuan Yang. Holigs: Holistic gaussian splatting for embod- ied view synthesis. arXiv preprint arXiv:2506.19291, 2025. 2

    2025 arXiv

  44. [52]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 4, 5

    2004

  45. [53]

    SV4D: Dynamic 3d content generation with multi-frame and multi-view consistency

    Yiming Xie, Chun-Han Yao, Vikram V oleti, Huaizu Jiang, and Varun Jampani. SV4D: Dynamic 3d content generation with multi-frame and multi-view consistency. arXiv preprint arXiv:2407.17470, 2024. 3

    2024 arXiv

  46. [54]

    Mvhumannet: A large- scale dataset of multi-view daily dressing human captures

    Zhangyang Xiong, Chenghong Li, Kenkun Liu, Hongjie Liao, Jianqiao Hu, Junyi Zhu, Shuliang Ning, Lingteng Qiu, Chongjie Wang, Shijie Wang, et al. Mvhumannet: A large- scale dataset of multi-view daily dressing human captures. In Proceedings of the IEEE/CVF Conference on Compute...

    2024

  47. [55]

    High quality human image animation using regional supervision and motion blur condi- tion

    Zhongcong Xu, Chaoyue Song, Guoxian Song, Jianfeng Zhang, Jun Hao Liew, Hongyi Xu, You Xie, Linjie Luo, Gu- osheng Lin, Jiashi Feng, et al. High quality human image animation using regional supervision and motion blur condi- tion. arXiv preprint arXiv:2409.19580, 2024. 12

    2024 arXiv

  48. [56]

    Sigman: Scaling 3d human gaussian generation with millions of assets

    Yuhang Yang, Fengqi Liu, Yixing Lu, Qin Zhao, Pingyu Wu, Wei Zhai, Ran Yi, Yang Cao, Lizhuang Ma, Zheng-Jun Zha, et al. Sigman: Scaling 3d human gaussian generation with millions of assets. arXiv preprint arXiv:2504.06982, 2025. 3

    2025 arXiv

  49. [57]

    Effec- tive whole-body pose estimation with two-stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effec- tive whole-body pose estimation with two-stages distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4210–4220, 2023. 3

    2023

  50. [58]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 3, 12

    2024 arXiv

  51. [59]

    pixelnerf: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4578–4587, 2021. 2

    2021

  52. [60]

    Function4d: Real-time human vol- umetric capture from very sparse consumer rgbd sensors

    Tao Yu, Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qiong- hai Dai, and Yebin Liu. Function4d: Real-time human vol- umetric capture from very sparse consumer rgbd sensors. In IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR2021), 2021. 5, 12, 15

    2021

  53. [61]

    The dawn of video generation: Preliminary explorations with sora-like models

    Ailing Zeng, Yuhang Yang, Weidong Chen, and Wei Liu. The dawn of video generation: Preliminary explorations with sora-like models. arXiv preprint arXiv:2410.05227, 2024. 3

    2024 arXiv

  54. [62]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 5

    2018

  55. [63]

    Humannerf: Efficiently gen- erated human radiance field from sparse inputs

    Fuqiang Zhao, Wei Yang, Jiakai Zhang, Pei Lin, Yingliang Zhang, Jingyi Yu, and Lan Xu. Humannerf: Efficiently gen- erated human radiance field from sparse inputs. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7743–7753, 2022. 2

    2022

  56. [64]

    Metric from human: Zero-shot monoc- ular metric depth estimation via test-time adaptation

    Yizhou Zhao, Hengwei Bian, Kaihua Chen, Pengliang Ji, Liao Qu, Shao-yu Lin, Weichen Yu, Haoran Li, Hao Chen, Jun Shen, et al. Metric from human: Zero-shot monoc- ular metric depth estimation via test-time adaptation. In The Thirty-eighth Annual Conference on Neural Information...

    2024

  57. [65]

    Synergistic global- space camera and human reconstruction from videos

    Yizhou Zhao, Tuanfeng Yang Wang, Bhiksha Raj, Min Xu, Jimei Yang, and Chun-Hao Paul Huang. Synergistic global- space camera and human reconstruction from videos. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1216–1226, 2024. 3

    2024

  58. [66]

    Gps- gaussian: Generalizable pixel-wise 3d gaussian splatting for real-time human novel view synthesis

    Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, and Yebin Liu. Gps- gaussian: Generalizable pixel-wise 3d gaussian splatting for real-time human novel view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2024

  59. [67]

    Champ: Controllable and consistent human image animation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision (ECCV), 2024. 2, 3, 4, 5, 6, 7, 12, 14, 15

    2024

  60. [68]

    Idol: Instant photorealistic 3d human creation from a single image

    Yiyu Zhuang, Jiaxi Lv, Hao Wen, Qing Shuai, Ailing Zeng, Hao Zhu, Shifeng Chen, Yujiu Yang, Xun Cao, and Wei Liu. Idol: Instant photorealistic 3d human creation from a single image. arXiv preprint arXiv:2412.14963, 2024. 2 11 GAS: Generative Avatar Synthesis from a Single Imag...

    2024 arXiv

  61. [69]

    We have also attempted to leverage human geometric prior to construct 3D correspondence across different views [19]

    To enable multi-view consistent synthesis for a single subject, we have tried adding an 1D temporal-axis attention layers [12] and only fine-tune these new added layers. We have also attempted to leverage human geometric prior to construct 3D correspondence across different vi...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.