Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Blind face restoration can be cast as a three-step continuation of a latent consistency model, starting from the low-quality image itself rather than from pure noise, and the resulting method restores identity, structure, and color better…

desk verdict A solid, well-ablated face-restoration paper whose 'LQ as 2nd-step LCM state' mechanism is only directly validated on synthetic data; worth engaging, but the evidence for the mechanism in the real world is thinner than the abstract implies. read the letter →

arxiv 2502.02215 v2 pith:SAA5ASLG submitted 2025-02-04 cs.CV

Senmao Li , Kai Wang , Joost van de Weijer , Fahad Shahbaz Khan , Chun-Le Guo , Shiqi Yang , Yaxing Wang , Jian Yang
show 1 more author
Ming-Ming Cheng
This is my paper · ORCID
classification cs.CV
keywords blindfacerestorationlatentconsistencymodeldiffusionpriorfew-stepsamplingperceptuallossadversarialCLIPvisualfeaturesspatialencoder
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that a latent consistency model (LCM) is a better generative prior for blind face restoration than an ordinary diffusion model, because it maps every point on its sampling trajectory back to the same clean origin and therefore keeps identity, structure, and color stable across steps. InterLCM inserts the low-quality image as the state after the first noise addition of a 4-step LCM, runs the remaining three consistency steps, and conditions them with a CLIP-based visual embedding and a ControlNet-style spatial encoder. Because only three denoising steps remain, the method can be trained with an L1 loss, a perceptual loss, and an adversarial loss applied to the final restored image, which the paper argues is how it achieves realistic real-world results. On CelebA-Test, LFW-Test, WebPhoto-Test, and WIDER-Test it reports better or competitive FID and MUSIQ scores than GAN-based and diffusion-based baselines, at 0.42 seconds per image.

What carries the argument

The central object is the latent consistency model, a distilled diffusion model whose network directly predicts the clean origin $z_0$ of an augmented probability-flow ODE trajectory from any point on it, rather than predicting noise step by step. In a 4-step LCM, each iteration alternates between adding noise to the current prediction and predicting the origin again; InterLCM identifies the state after the first noise addition as the insertion point for the low-quality image $x_l$, so that three consistency mappings remain. Two conditioning modules carry the argument: a Visual Module (a CLIP image encoder feeding a trainable Visual Encoder) supplies a face-specific semantic embedding $c_v$ in place of text, and a Spatial Encoder, initialized from the Stable Diffusion U-Net encoder in the style of ControlNet, supplies spatial features $f_v$ that are injected into the cross-attention block with $c_v$ as keys and values. Training uses $\mathcal{L}_1$, a VGG perceptual loss, and an adversarial loss on the final decoded image, with only the two encoders updated; the LCM itself is frozen. The capacity to run the whole pipeline in three steps is what makes the perceptual and adversarial losses usable.

What would settle it

Measure the FID between real-world heavily degraded faces from WIDER-Test after applying the LCM's second-step noise addition and the LCM's own second-step noisy states; if that FID is far above the 2.83 reported for synthetic degradations, the alignment that justifies starting at the second step does not hold in the wild.

Watch

Extended reading notes

Core claim

The discovery is that the low-quality image can be treated as an intermediate state of the LCM itself, not as noise to be removed from scratch. After the first of the LCM's three noise additions, the distribution of synthetically degraded faces is close (FID 2.83) to the model's own noisy states, whereas later steps are far apart (FID 31.83 and 214.40); so starting from the second step lets the consistency mapping complete the restoration while preserving the face's semantics. The paper shows that a naive LCM run from this state changes identity-related details such as hair, so it adds a Visual Module that converts CLIP features of the low-quality image into a face-specific embedding, and a Spatial Encoder copied from the Stable Diffusion U-Net encoder to feed structure into the attention blocks. These modules are trained end-to-end with reconstruction, perceptual, and adversarial losses through the differentiable consistency steps. The result is that the method, on the paper's reported benchmarks, restores faces with better perceptual quality and structure than previous blind face restoration methods while requiring only three LCM sampling steps.

Load-bearing premise

The load-bearing premise is that the degradation in a low-quality face occupies nearly the same region of the model's latent space as the Gaussian noise the LCM adds at its second step, and the paper demonstrates this closeness only with a synthetic degradation pipeline, not with real-world degradations.

Editorial extensions

If this is right

  • Blind face restoration can be reduced to three LCM consistency steps, giving 0.42-second inference that is comparable to CNN/Transformer-based methods and far faster than diffusion baselines needing many iterations.
  • Because only a few steps remain, perceptual and adversarial losses can act on the final restored image; the paper's ablations show those losses are what lift real-world FID and MUSIQ scores.
  • The LCM prior preserves identity, structure, and color better than a conventional diffusion prior (measured with DreamSim, SSIM, and color histogram distance), which is why the restored faces stay faithful to the input.
  • The 2nd-step starting point is the right fidelity-quality balance: starting from the 1st step adds random identity changes, while starting from the 3rd or 4th step leaves the output blurred or too close to the low-quality input.
  • The method inherits the training-data bias of FFHQ: faces with hands are not restored realistically, a limitation the paper explicitly acknowledges.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the latent alignment holds for other degradations, the same 'insert the observation at the right LCM step' recipe could be exported to blind deblurring, super-resolution, or inpainting, since the only face-specific parts are the Visual and Spatial encoders.
  • The starting step acts as a tunable fidelity-quality dial: a practitioner could select the step per degradation type by measuring the FID of the low-quality image at each noise level, rather than fixing the second step for all inputs.
  • The paper's Appendix E.1 shows ordinary diffusion intermediates do not align with low-quality faces even at small noise levels, suggesting that LCM's direct $x_0$-prediction property, not few-step speed alone, is what makes the method work; this predicts that other consistency-trained models would inherit the same advantage.
  • A testable extension would replace the fixed Gaussian schedule in the first noise addition with a learned or degradation-aware mapping that explicitly transports real-world low-quality faces into LCM's 2nd-step manifold, possibly removing the synthetic-versus-real gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes InterLCM, a blind face restoration method built on a frozen 4-step latent consistency model (LCM). The low-quality (LQ) input is encoded by a CLIP-based Visual Module and a ControlNet-style Spatial Encoder, then inserted as the second-step intermediate state of the LCM; three remaining LCM steps produce the restored high-quality image. The authors argue that LCM priors provide better identity, structure, and color consistency than standard diffusion priors, and that few-step LCM sampling makes it possible to train with perceptual and adversarial losses on the final output. Experiments compare InterLCM with CNN/Transformer and diffusion baselines on one synthetic dataset (CelebA-Test) and three real-world datasets (LFW, WebPhoto, WIDER), reporting quality metrics and inference time, along with ablations of the Visual Encoder, Spatial Encoder, starting step, and losses.

Significance. If its central premise holds, the paper makes a useful contribution: it is among the first to show that a latent consistency model, rather than a standard diffusion model, can serve as the generative prior for blind face restoration, and that its few-step formulation permits image-domain perceptual and adversarial losses. The empirical work is extensive: comparisons against twelve baselines, ablation studies for each proposed component, and additional analyses in the appendix (e.g., Appendix E.1 on why LQ images are not intermediate states of SD sampling, and Appendix E.4 on the sufficiency of LQ semantic information). The paper also reports a practical inference speed of 0.42 s per image. These strengths are undermined by three issues: the key 'LQ as 2nd-step LCM state' premise is validated only through a marginal FID measure on synthetic degradations, several headline claims are not supported by the paper's own tables, and no statistical significance or error-bar analysis accompanies any quantitative comparison.

major comments (3)
  1. [Sec. 3.2, Fig. 4, Appendix C.1, Eq. (3)] The load-bearing premise that a real-world LQ image can be inserted as the 2nd-step state of the LCM is supported only by an FID comparison (2.83 versus 31.83 and 214.40 at later steps) computed on CelebA-Test images degraded with the training pipeline of Eq. (3). FID is a marginal distributional measure and a low value does not establish that per-sample LQ latents lie on any LCM ODE trajectory. The comparison is never repeated for real-world degradations such as LFW/WebPhoto/WIDER. Given that Fig. 5 and Table 2 show the Visual Encoder and Spatial Encoder are necessary to avoid identity and hair drift, the paper does not rule out that the learned conditioning modules, rather than the LCM intermediate-state prior, are the operative cause of the real-world gains. Please add a per-sample alignment test (e.g., consistency error of one LCM step starting from an LQ latent versus from a true 2nd-step state) and a real-world latent-distribution comparison.
  2. [Table 1 and Abstract/Conclusion] The abstract and conclusion claim that InterLCM 'outperforms existing approaches in both synthetic and real-world datasets', but Table 1 does not support this uniformly. On CelebA-Test, the proposed method has FID 45.38, which is worse than DifFace (39.23) and VQFR (41.84), and PSNR 25.19, which is below GPEN (25.48). Several winning margins are very small (e.g., LPIPS 0.223 versus 0.227 for CodeFormer). No error bars, confidence intervals, or significance tests are reported anywhere in the paper. The claims should be tempered to the specific metrics where the method is actually best, or supplemented with statistical testing to show that the reported differences are meaningful.
  3. [Table 2, Fig. 8, Sec. 4.2] The ablation for the starting intermediate step does not consistently identify the 2nd step as the best choice. On LFW-Test, the FID for starting from the 2nd step is 51.32, while starting from the 3rd or 4th step gives 50.48 and 50.59, respectively; only MUSIQ is clearly best at the 2nd step. Moreover, the real-world ablations report only FID and MUSIQ, not identity or structure metrics (IDS, SSIM), even though the paper's central argument is about semantic consistency. The claim that 'starting from the second step achieves a balance' is therefore under-supported and should be justified with additional metrics or significance analysis.
minor comments (5)
  1. [Sec. 4.2] In the ablation paragraph, '3 Text+SD+2nd' appears to be a typo; the corresponding entry in Table 2 and Figure 8 is 'Text+SE+2nd'.
  2. [Fig. 4 caption] The caption '2nd-step noise addition1st step' is missing a space and is confusing; it should read '2nd-step noise addition / 1st step' or similar.
  3. [Sec. 4.2, 'Inference time'] The text says 'Quadro RTX 3090 GPU'; no such product exists. This should be corrected to 'NVIDIA RTX 3090' or the actual Quadro model used.
  4. [Table 1] WaveFace is listed as '–' for all synthetic metrics without explanation; a footnote should state that these numbers were not reported or could not be computed with the official release.
  5. [Appendix E.1] The discussion in Appendix E.1 is valuable but would be stronger if it included the analogous experiment for LCM, which is the model actually used in the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is benchmark-evaluated and the LCM-prior premise rests on external consistency-model results plus the paper's own empirical comparisons.

full rationale

The paper's derivation chain is self-contained and benchmark-evaluated rather than circular. The central design decision — inserting the LQ image as the 2nd-step LCM intermediate state — is justified by an empirical FID comparison (Fig. 4, App. C.1: FID 2.83 vs 31.83 and 214.40), which is a design heuristic, not a fitted parameter that is later reported as a prediction. The final restoration quality is measured on held-out test sets (CelebA-Test, LFW, WebPhoto, WIDER) with standard metrics, not derived from the FID choice, so there is no reduction of outputs to inputs by construction. The claim that LCM provides better semantic consistency is supported by an external prior (Luo et al. 2023a) and by the paper's own Fig. 1 measurements across models, not by a self-citation chain; the only overlapping-author citation (VQFR, used as a baseline and as the source of the IDS metric) is not load-bearing. The synthetic test set uses the same degradation pipeline as training (Eq. 3), which is a generalization limitation for real-world claims, but it is not a circular argument. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no ansatz is smuggled in via self-citation.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard consistency-model theory, the domain assumption that LQ degradation resembles LCM noise at the 2nd step, and the assumption that the synthetic degradation model (Eq. 3) transfers to real-world data. Two hand-chosen quantities (adversarial weight lambda=0.1 and the 2nd-step start) affect results; no sensitivity analysis is reported for lambda. No new physical or conceptual entities are postulated.

free parameters (2)
  • Adversarial loss weight lambda = 0.1
    Trade-off parameter in Eq. (2), set by default; no sensitivity analysis is provided.
  • Starting LCM step = 2nd step
    Chosen via FID heuristic in Fig. 4 (FID=2.83 for 2nd step vs 31.83 and 214.40 for later steps); a hand-chosen design point that affects fidelity-quality balance.
assumptions (5)
  • standard math The LCM consistency mapping f_theta maps any point on the augmented PF-ODE trajectory to the trajectory's origin x0.
    Invoked in Sec. 3.1 and used to justify starting from an intermediate state; this is the defining property of consistency models from Song et al. (2023) and Luo et al. (2023a).
  • domain assumption The LQ image lies close to the LCM's 2nd-step noisy state in latent distribution.
    Sec. 3.2 and Fig. 4 justify this with FID=2.83 on the synthetic pipeline; it is the premise that makes the 3-step refinement produce the HQ image.
  • domain assumption The degradation model of Eq. (3) (blur, downsample, noise, JPEG) approximates real-world face degradation.
    Used to generate training pairs from FFHQ in A.2; the synthetic test uses the identical pipeline, so real-world transfer is assumed rather than demonstrated.
  • domain assumption CLIP visual features plus VGG19 perceptual features are sufficient to capture identity and structure for face restoration.
    Sec. 3.2 uses CLIP features for the Visual Module and VGG19 for the perceptual loss; this is standard practice but an assumption about feature adequacy.
  • standard math Adversarial training with a discriminator stabilizes and improves restoration quality.
    Sec. 3.2 training objective; standard GAN assumption from Goodfellow et al. (2014).

how reviews work

0 comments
Cite this review

Pith. "Pith review of InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration." pith.science (2026). https://pith.science/paper/SAA5ASLG

@misc{pith2026250202215,
  author       = {Pith},
  title        = {Pith review of: InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SAA5ASLG}},
  note         = {Machine review of arXiv:2502.02215}
}
read the original abstract

Diffusion priors have been used for blind face restoration (BFR) by fine-tuning diffusion models (DMs) on restoration datasets to recover low-quality images. However, the naive application of DMs presents several key limitations. (i) The diffusion prior has inferior semantic consistency (e.g., ID, structure and color.), increasing the difficulty of optimizing the BFR model; (ii) reliance on hundreds of denoising iterations, preventing the effective cooperation with perceptual losses, which is crucial for faithful restoration. Observing that the latent consistency model (LCM) learns consistency noise-to-data mappings on the ODE-trajectory and therefore shows more semantic consistency in the subject identity, structural information and color preservation, we propose InterLCM to leverage the LCM for its superior semantic consistency and efficiency to counter the above issues. Treating low-quality images as the intermediate state of LCM, InterLCM achieves a balance between fidelity and quality by starting from earlier LCM steps. LCM also allows the integration of perceptual loss during training, leading to improved restoration quality, particularly in real-world scenarios. To mitigate structural and semantic uncertainties, InterLCM incorporates a Visual Module to extract visual features and a Spatial Encoder to capture spatial details, enhancing the fidelity of restored images. Extensive experiments demonstrate that InterLCM outperforms existing approaches in both synthetic and real-world datasets while also achieving faster inference speed.

Figures

Figures reproduced from arXiv: 2502.02215 by the authors.

Figure 1
Figure 1. (Left) The intermediate states in 4-step LCM and SD Turbo models. The network used in LCM maps to the real image space, while SD Turbo progressively denoises the noisy image. (Right) Given the prompt “A headshot of a man with hat and glasses”, we generate 1000 images with both LCM and SD Turbo models. Then we use DreamSim, SSIM, and color histogram distance (HDist) to measure the semantic consistency in the subject … view at source ↗
Figure 2
Figure 2. (Left) The 4-step LCM map its origin at each sampling step: Noise 1st step −−−−→Sampling data add noise −−−−−→Noisy data 2nd step −−−−→Sampling data add noise −−−−−→Noisy data 3rd step −−−−→Sampling data add noise −−−−−→Noisy data 4th step −−−−→Sampling data. In the first step, the origin image is predicted from random noise. In each remaining step, noise is added to the origin image produced in the previous step. (… view at source ↗
Figure 3
Figure 3. Overview of the proposed InterLCM framework. The Visual Module takes LQ images to output the visual embeddings. A Spatial Encoder is used to provide structure information. We consider the LQ image as the intermediate state of LCM. Through standard LCM conditioned with both the visual embedding and spatial features, the LQ input can be reconstructed as a HQ image. 3.1 PRELIMINARIES Latent Diffusion Models. To enable … view at source ↗
Figures from the paper (25 more)
Figure 4
Figure 4. Figure 4: t-SNE visualizations of feature distri￾butions show the first step sampling similarity of LCM and the LQ image (FID=103.70), and their noisy intermediate states after LCM 2nd-step noise diffusion (FID=2.83). 2nd-step intermediate state. To leverage the content consiste…
Figure 5
Figure 5. Figure 5: Naive LCM alters the original semantics of the LQ image (e.g., hair). Visual Encoder. Ideally, the model should re￾construct image quality and align semantic in￾formation with the LQ image. However, noise diffusion introduces randomness, altering the original semantics…
Figure 6
Figure 6. Figure 6: Qualitative comparisons of baselines on the synthetic of CelebA-Test for BFR ( [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparisons of baselines on the real-world images from LFW-Test, WebPhoto [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Visualization of the ablation study for various design variants. ‡ indicates our results. We also experimentally confirm the starting step and present the results in [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: (Left) visualization of the ablation study for both the perceptual and adversarial losses. (Right) visualization of the ablation study comparing the naive ControlNet and our Spatial Encode [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Input LQ images with hands may experience failing restorations. [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: t-SNE (Hinton & Roweis, 2002) visualizations of feature distributions show (Left) the first step sampling result of LCM and the LQ image (FID=103.70) with their noise-added versions (FID=2.83); (Middle) the second step result and the LQ image (FID=157.80) with their n…
Figure 12
Figure 12. Figure 12: Two restoration examples of our InterLCM on the real-world dataset WebPhoto-Test, achieved through random noise addition in the three noise addition step of 4-step LCM. C APPENDIX: ABLATION ANALYSIS C.1 SHOULD WE START FROM THE 2ND, 3RD, OR 4TH STEP IN THE LCM? To lev…
Figure 13
Figure 13. Figure 13: Visualization of the ablation study for both the perceptual and adversarial losses. [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Results using LCM-LoRA and LCM backbone for our method. [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Results of our method using one-step models (as shown in the second and third rows) [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]
Figure 16
Figure 16. Figure 16: The generated results at each timestep of the diffusion sampling process from [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: The generated results are obtained when we regard the LQ image as the intermediate [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]
Figure 18
Figure 18. Figure 18: The super-resolution methods struggle to recover facial details (the second to fourth [PITH_FULL_IMAGE:figures/full_fig_p025_18.png]
Figure 19
Figure 19. Figure 19: (top) In our training dataset FFHQ, there exist images containing festival-style face paint, [PITH_FULL_IMAGE:figures/full_fig_p025_19.png]
Figure 20
Figure 20. Figure 20: (top) Our method with variety inputs. (middle) We find that the semantic information [PITH_FULL_IMAGE:figures/full_fig_p026_20.png]
Figure 21
Figure 21. Figure 21: Results on natural image datasets. E.5 APPLYING THE PROPOSED METHOD TO NATURAL IMAGE DATASETS For the blind face restoration problem, our method InterLCM can efficiently extract facial infor￾mation through the Visual Encoder, as human faces are with less complex seman…
Figure 22
Figure 22. Figure 22: The xˆ0 obtained from the intermediate step (the first to fifth columns) has an appearance gap compared to the x0 (the last column). Input PULSE DFDNet PSFRGAN GFPGAN GPEN RestoreFormer VQFR CodeFormer DR2 DifFace PGDiff Ours Ground Truth [PITH_FULL_IMAGE:figures/ful…
Figure 23
Figure 23. Figure 23: Qualitative comparisons of baselines on the synthetic of CelebA-Test for BFR. [PITH_FULL_IMAGE:figures/full_fig_p028_23.png]
Figure 24
Figure 24. Figure 24: Qualitative comparisons of baselines on the real-world images from LFW-Test, [PITH_FULL_IMAGE:figures/full_fig_p029_24.png]
Figure 25
Figure 25. Figure 25: Qualitative comparison on the synthetic dataset Celeba-Test. [PITH_FULL_IMAGE:figures/full_fig_p030_25.png]
Figure 26
Figure 26. Figure 26: Qualitative comparison on the real-world dataset LFW-Test under mild degradation. [PITH_FULL_IMAGE:figures/full_fig_p031_26.png]
Figure 27
Figure 27. Figure 27: Qualitative comparison on the real-world dataset WebPhoto-Test under medium degra [PITH_FULL_IMAGE:figures/full_fig_p032_27.png]
Figure 28
Figure 28. Figure 28: Qualitative comparison on the real-world dataset WIDER-Test under heavy degradation. [PITH_FULL_IMAGE:figures/full_fig_p033_28.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Persistent Free Volume Governs (Anti)plasticization in Chitosan-Water Mixtures

    cond-mat.soft 2026-04 unverdicted novelty 5.0 of 10

    Dynamically accessible free volume, enabled by connected water-accessible regions, is proposed to govern antiplasticization then plasticization of elastic properties in chitosan–water mixtures.

  2. A Step Forward Towards Trustworthy Risk-Aware Facial Retrieval (RA-FR)

    cs.CV 2026-07 conditional novelty 4.0 of 10

    RA-FR uses blind face restoration, DINOv1 ViT embeddings, and conformal risk control to return adaptive-size match sets with a user-specified miss-rate bound.

Reference graph

Works this paper leans on

43 extracted references · 26 canonical work pages · cited by 2 Pith papers

  1. [3]

    Generative adversarial nets.Advances in neural information processing systems, 27,

    11 Published as a conference paper at ICLR 2025 Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets.Advances in neural information processing systems, 27,

    work page 2025

  2. [4]

    Noise schedule α(t), σ(t), Encoder E, and Decoder D. Initial latent code z0 ← E(xl) for n = 1 to N − 1 do zτn ∼ N(α(τn)z0; σ2(τn)I) z0 ← fθ(zτn , cv, τn, fv) end for xrec ← D(z0) Output: xrec 18 Published as a conference paper at ICLR 2025 Input Random noise addition Figure 12: Two restoration examples of our InterLCM on the real-world dataset WebPhoto-Te...

    work page 2025

  3. [8]

    Controlnet++: Improving conditional controls with efficient consistency feedback

    Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, and Chen Chen. Controlnet++: Improving conditional controls with efficient consistency feedback. arXiv preprint arXiv:2404.07987, 2024a. 12 Published as a conference paper at ICLR 2025 Xiaoming Li, Ming Liu, Yuting Ye, Wangmeng Zuo, Liang Lin, and Ruigang Yang. Learning warped g...

    arXiv 2025

  4. [9]

    When stylegan meets stable diffusion: a W+ adapter for personalized image generation

    Xiaoming Li, Xinyu Hou, and Chen Change Loy. When stylegan meets stable diffusion: a W+ adapter for personalized image generation. Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2024b. Zhenyi Liao, Qingsong Xie, Chen Chen, Hannan Lu, and Zhijie Deng. Fine-tuning diffusion models for enhancing face quality in text-to-i...

    arXiv

  5. [11]

    Diffbir: Towards blind image restoration with generative diffusion prior

    Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Bo Dai, Fanghua Yu, Wanli Ouyang, Yu Qiao, and Chao Dong. Diffbir: Towards blind image restoration with generative diffusion prior. arXiv preprint arXiv:2308.15070,

    arXiv

  6. [12]

    I2sb: Image-to-image schr \” odinger bridge

    Guan-Horng Liu, Arash Vahdat, De-An Huang, Evangelos A Theodorou, Weili Nie, and Anima Anandkumar. I2sb: Image-to-image schr \” odinger bridge. arXiv preprint arXiv:2302.05872 ,

    arXiv

  7. [13]

    Latent consistency models: Synthe- sizing high-resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023a. Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick von Platen, Apolin ´ario Passos, Longbo Huang, Jian Li, and Hang Zhao. Lcm-lora: A universal stable-diffusion a...

    arXiv

  8. [14]

    WaveFace: Authentic Face Restoration with Efficient Frequency Recovery

    Yunqi Miao, Jiankang Deng, and Jungong Han. Waveface: Authentic face restoration with efficient frequency recovery. arXiv preprint arXiv:2403.12760,

    work page Pith review arXiv

Show all 43 references
  1. [15]

    Swiftbrush: One-step text-to-image diffusion model with variational score distillation

    Thuan Hoang Nguyen and Anh Tran. Swiftbrush: One-step text-to-image diffusion model with variational score distillation. arXiv preprint arXiv:2312.05239,

    arXiv

  2. [16]

    Learning transferable visual models from natural language supervision

    13 Published as a conference paper at ICLR 2025 Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ...

    2025

  3. [17]

    Hierarchical text- conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3,

    arXiv

  4. [18]

    U-net: Convolutional networks for biomed- ical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceed- ings, ...

    2015

  5. [19]

    Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512,

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512,

    arXiv

  6. [20]

    Adversarial diffusion dis- tillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion dis- tillation. arXiv preprint arXiv:2311.17042,

    arXiv

  7. [21]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

    arXiv

  8. [22]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469,

    arXiv

  9. [23]

    Image quality assessment: from error visibility to structural similarity

    14 Published as a conference paper at ICLR 2025 Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600– 612,

    2025

  10. [24]

    Controlnet-xs: Designing an effi- cient and effective architecture for controlling text-to-image diffusion models

    Denis Zavadski, Johann-Friedrich Feiden, and Carsten Rother. Controlnet-xs: Designing an effi- cient and effective architecture for controlling text-to-image diffusion models. arXiv preprint arXiv:2312.06573,

    arXiv

  11. [25]

    Chan, Chongyi Li, and Chen Change Loy

    15 Published as a conference paper at ICLR 2025 Shangchen Zhou, Kelvin C.K. Chan, Chongyi Li, and Chen Change Loy. Towards robust blind face restoration with codebook lookup transformer. In NeurIPS,

    2025

  12. [26]

    Scaling the codebook size of vqgan to 100,000 with a utilization rate of 99%

    Lei Zhu, Fangyun Wei, Yanye Lu, and Dong Chen. Scaling the codebook size of vqgan to 100,000 with a utilization rate of 99%. arXiv preprint arXiv:2406.11837,

    arXiv

  13. [27]

    The Spatial Encoder is partially initialized using UNet encoder from the pre-trained Stable Diffusion 1.5, following the approach in (Zhang et al., 2023)

    16 Published as a conference paper at ICLR 2025 APPENDIX A A PPENDIX : I MPLEMENTATION DETAILS A.1 T RAINING DETAILS We mainly use the pre-trained LCM, distilled from StableDiffusion 1.5. The Spatial Encoder is partially initialized using UNet encoder from the pre-trained Stab...

    2025

  14. [28]

    The proposed method is implemented in Pytorch (Paszke et al., 2017)

    serves as the Visual Encoder, with adjustments made to the input and output dimensions to align with our settings. The proposed method is implemented in Pytorch (Paszke et al., 2017). We use Adam (Kingma & Ba,

    2017

  15. [31]

    A.3 T EST DATA. We evaluate our method on onesynthetic dataset and three real-world datasets, which are commonly used for evaluation in blind face restoration tasks (Wang et al., 2021a; Zhou et al., 2022; Yue & Loy, 2024; Yang et al., 2024). The synthetic dataset, CelebA-Test ...

    2022

  16. [32]

    13, and 3https://github.com/krantirk/Self-Supervised-photo 4https://github.com/csxmli2016/DFDNet 5https://github.com/chaofengc/PSFRGAN 6https://github.com/TencentARC/GFPGAN 7https://github.com/yangxy/GPEN 8https://github.com/swz30/Restormer 9https://github.com/TencentARC/VQFR ...

    2025

  17. [34]

    By the quantitative comparison in Tab

    as the backbone to develop our method for the BFR problem. By the quantitative comparison in Tab. 7, we show that consistency model, which directly predict the x0 in each step, better suits the BFR problem. 20 Published as a conference paper at ICLR 2025 Table 6: Quantitative ...

    2025

  18. [36]

    E.3 A DDITIONAL RESULTS WITH TATTOOS OR FESTIVAL -STYLE FACE PAINT As shown in Fig

    adds an additional 5,000 face images from the FFHQ dataset (Karras et al., 2019), it still struggles to recover facial details, such as hair and facial texture (the third column). E.3 A DDITIONAL RESULTS WITH TATTOOS OR FESTIVAL -STYLE FACE PAINT As shown in Fig. 19 (the third...

    2019

  19. [38]

    E.6 A PPLYING PERCEPTUAL LOSS IN DIFFUSION -BASED MODELS Several existing works (Chung et al., 2023; Laroche et al.,

    with 100,000 codebooks to act as the visual encoder for our model in future work. E.6 A PPLYING PERCEPTUAL LOSS IN DIFFUSION -BASED MODELS Several existing works (Chung et al., 2023; Laroche et al.,

    2023

  20. [39]

    have integrated the perceptual loss in to diffusion-based models. The forward process of diffusion-based models is a process that iteratively adds Gaussian noise to the representation using: xt = √αtxt−1 + √ 1 − αtϵ, (4) where αt is the predefined variance, and ϵ ∼ N(0, I). Re...

    2023

  21. [40]

    Our approach uses x0 at the final of the inference steps of the latent consistency model

    lies in how the noise-free real image x0 is obtained. Our approach uses x0 at the final of the inference steps of the latent consistency model. In contrast, existing works (Chung et al., 2023; Laroche et al.,

    2023

  22. [41]

    (5): ˆx0 = 1√¯αt (xt − √ 1 − ¯αtϵ)

    derive x0 from xt at an intermediate step t by directly applying the inversion of forward process using Eq. (5): ˆx0 = 1√¯αt (xt − √ 1 − ¯αtϵ). (6) As shown in Fig. 22, we can observe that theˆx0 obtained from the SD intermediate steps (the first to fifth columns) has an appea...

    2025

  23. [42]

    24, our method can reconstruct more realistic details in forehead wrinkles (first and second rows), eyes and eyebrows (third and fourth rows), and hair (fifth and sixth rows)

    As shown in Fig. 24, our method can reconstruct more realistic details in forehead wrinkles (first and second rows), eyes and eyebrows (third and fourth rows), and hair (fifth and sixth rows). These results demonstrate that our method outperforms the baselines in real-world sc...

    2017

  24. [43]

    The best results are in bold, and the second best results are underlined. Dataset Syntheticdataset Celeba-Test Method MetricsMUSIQ↑ IDS↓ PSNR↑SSIM↑ Input 17.44 37.44 24.24 0.624 CNN/Transformer -based PULSE 71.97 69.90 21.22 0.561 DFDNet 75.96 27.42 25.03 0.620 PSFRGAN 69.85 3...

    2025

  25. [50]

    Intuitively, we regard the LQ image as an intermediate result of SD sampling, especially at small noise levels (see Fig

    The intermediate result of SD sampling lacks a lot of image detail, while the LQ image mainly loses texture detail compared to the HQ image. Intuitively, we regard the LQ image as an intermediate result of SD sampling, especially at small noise levels (see Fig. 16 (red box)). ...

    2025

  26. [512]

    The convolution operation is denoted by ∗, followed by JPEG compression with a quality factor of q ∈ {30 : 90 }

    The resized images are then degraded to generate LQ images following the typical degradation process described in (Zhou et al., 2022): xl = {[(xh ∗ kσ)↓s + nδ]JPEGq }↑s, (3) where xh and xl represent the HQ and LQ images, respectively, kσ is the Gaussian kernel with σ ∈ {1 : 1...

    2022

  27. [2014]

    Boot: Data-free distillation of denoising diffusion models with bootstrapping

    Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Lingjie Liu, and Josh Susskind. Boot: Data-free distillation of denoising diffusion models with bootstrapping. arXiv preprint arXiv:2306.05544,

    arXiv

  28. [2015]

    The models are trained for 15K iterations using eight A40 GPUs (48GB VRAM)

    with a batch size 8, using a learning rate of 2 × 10−5. The models are trained for 15K iterations using eight A40 GPUs (48GB VRAM). A.2 T RAINING DATA We train our models on the FFHQ dataset (Karras et al., 2019), which consists of 70,000 HQ face images with a resolution of 1024 ×

    2019

  29. [2016]

    Progressive growing of gans for im- proved quality, stability, and variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. arXiv preprint arXiv:1710.10196,

    arXiv

  30. [2019]

    19 (the first and second rows), where the festival-style face paints and rich-color hair appear multiple times during training

    is also shown in Fig. 19 (the first and second rows), where the festival-style face paints and rich-color hair appear multiple times during training. E.4 LQ SEMANTIC INFORMATION SUFFICES FOR HQ RECONSTRUCTION In our method (Fig. 20, top(a)), InterLCM, we utilize a Visual Modul...

    2025

  31. [2020]

    Imagen video: High definition video generation with diffusion models

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303,

    arXiv

  32. [2021]

    Stylegan-nada: Clip- guided domain adaptation of image generators

    Rinon Gal, Or Patashnik, Haggai Maron, Gal Chechik, and Daniel Cohen-Or. Stylegan-nada: Clip- guided domain adaptation of image generators. arXiv preprint arXiv:2108.00946,

    arXiv

  33. [2022]

    Face super-resolution guided by 3d facial priors

    Xiaobin Hu, Wenqi Ren, John LaMaster, Xiaochun Cao, Xiaoming Li, Zechao Li, Bjoern Menze, and Wei Liu. Face super-resolution guided by 3d facial priors. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16 , pp. 763–7...

    2020

  34. [2023]

    Vqgan-clip: Open domain image generation and editing with natural language guidance

    Katherine Crowson, Stella Biderman, Daniel Kornis, Dashiell Stander, Eric Hallahan, Louis Cas- tricato, and Edward Raff. Vqgan-clip: Open domain image generation and editing with natural language guidance. arXiv preprint arXiv:2204.08583,

    arXiv

  35. [2024]

    Ctrl-adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model.arXiv preprint arXiv:2404.09967,

    Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal. Ctrl-adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model.arXiv preprint arXiv:2404.09967,

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.