Pith. sign in

REVIEW 3 major objections 4 minor 41 references

Steering Guidance for Personalized Text-to-Image Diffusion Models

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Personalization guidance replaces the CFG weak model with a weight-interpolated pre-trained/fine-tuned null-text model, raising subject fidelity while keeping text fidelity nearly constant.

desk verdict Simple, credible guidance tweak that buys subject fidelity for free; clean-up needed on omega selection, error bars, and an overclaimed abstract. read the letter →

arxiv 2508.00319 v1 pith:ZVNAU4RM submitted 2025-08-01 cs.CV cs.LG

Sunghyun Park , Seokeon Choi , Hyoungwoo Park , Sungrack Yun This is my paper · ORCID
classification cs.CVcs.LG
keywords personalizationguidanceclassifier-freeautoguidanceweightinterpolationsubjectfidelitytext-to-imagediffusionDreamBooth-LoRAViCodataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Personalized text-to-image models face a trade-off: fine-tuning makes a subject recognizable but can erode the model's ability to follow arbitrary text prompts. This paper proposes personalization guidance, a change to classifier-free guidance (CFG) that uses an interpolated weak model conditioned on a null text prompt instead of the fine-tuned model's own null-text score. The authors claim that on Stable Diffusion 1.5, Stable Diffusion 2.1, and SANA, and across DreamBooth-LoRA, DB-LoRA with Textual Inversion, and ClassDiffusion, this raises subject fidelity measured by DINO and CLIP-I while CLIP-T text fidelity changes only slightly. A single interpolation weight $\omega$ then acts as an inference-time dial between subject fidelity and text editability, with no extra forward passes. The practical payoff is a way to tune the personalization trade-off after training, without changing learning rates, steps, or regularization.

What carries the argument

The mechanism is the null-text score of an interpolated model used as the weak branch of CFG, replacing $\epsilon_{\theta'}(x_t|\phi)$ with $\epsilon_{\theta_\omega}(x_t|\phi)$. Because neural networks are nonlinear in their parameters, this weight-interpolated weak model is not the same as an output-space blend of CFG and autoguidance, and it gives the sampler a direction that points away from the pre-trained model's unconditional behavior and toward the fine-tuned conditional behavior. The interpolation weight $\omega$ controls how much of the target concept has been unlearned in the weak branch, and the paper treats this as continuously steering the generation between text-aligned and subject-aligned regions of the noisy data manifold.

What would settle it

On one ViCo concept with a fixed fine-tuned DreamBooth-LoRA, sweep $\omega$ from 0 to 1 in steps of 0.1 with identical seeds and measure DINO, CLIP-I, and CLIP-T; if no $\omega < 1$ beats CFG ($\omega=1$) on subject fidelity at equal or better text fidelity, or if the interpolated null-text score does not lie between the pre-trained and fine-tuned null-text scores, the claimed steering mechanism is not operating as described.

Watch

Extended reading notes

Core claim

The central object is the interpolated weak model $\theta_\omega = \omega \theta' + (1-\omega)\theta$, where $\theta$ is the pre-trained model and $\theta'$ the fine-tuned model. The paper's guidance formula (Eq. 9) reads $\tilde{\epsilon}_{\theta'}(x_t|c) = \epsilon_{\theta_\omega}(x_t|\phi) + \lambda(\epsilon_{\theta'}(x_t|c) - \epsilon_{\theta_\omega}(x_t|\phi))$, with $\phi$ the null text. At $\omega = 1$ this is ordinary CFG; at $\omega = 0$ the pre-trained model plays the weak role. The empirical claim is that intermediate values, especially $\omega$ near $0.6$ for the Stable Diffusion models and $\omega = 0$ for SANA, sit in a region where subject fidelity is higher than CFG and text fidelity is essentially preserved. The paper interprets this as steering the sample path toward a balanced latent space rather than merely scaling the guidance strength.

Load-bearing premise

The load-bearing premise is that the pre-trained model's null-text score remains a better weak model than the fine-tuned model's null-text score, and that straight-line weight interpolation gives a monotonic path of unlearning; the paper itself notes in Appendix D that the method inherits the quality and direction of the fine-tuned model.

Editorial extensions

If this is right

  • Following Eq. 9 at $\omega=0$ improves DINO and CLIP-I over CFG for DB-LoRA on SD 1.5, SD 2.1, and SANA, while CLIP-T falls by at most a few hundredths (Tables 1, 9-11).
  • The interpolation weight gives an inference-time trade-off control: increasing $\omega$ toward 1.0 moves output back toward CFG's text fidelity, while decreasing it toward 0.6 favors subject fidelity, allowing per-concept tuning without retraining.
  • Because the guidance only needs one null-text evaluation per step, the method adds no memory or time overhead compared with CFG, and it combines with CFG++ as well as with plain CFG (Tables 6, 8).
  • The same construction transfers to human-preference optimization (Diffusion-DPO), style personalization (PairCustom), and instruction-based editing (InstructPix2Pix), improving each fine-tuned model's own objective (Tables 4-5, Fig. 9).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the interpolation path in parameter space is approximately straight in score space, then personalization guidance can be viewed as a family of guidance policies interpolating between autoguidance and CFG; the paper's nonlinearity argument leaves open exactly how much of the gain is architecture-specific.
  • A direct test of the 'unlearning' interpretation would measure whether the interpolated null-text score actually moves monotonically from the pre-trained to the fine-tuned prediction as $\omega$ runs from 0 to 1; if it does not, the steering story needs revision even if the metric gains survive.
  • The authors' observation that the method works better when the fine-tuning direction is better suggests personalization guidance could double as a diagnostic: a small gain from the interpolated weak model is evidence that the fine-tune is close to the pre-trained prior, while a large gain indicates the fine-tune has moved into a specialized region.
  • Because the method uses only the pre-trained model's null-text branch, it should transfer to other conditional diffusion architectures such as video or audio personalization whenever a paired pre-trained/fine-tuned model exists, but that transfer is not tested in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes 'Personalization Guidance' (PG), an inference-time modification of classifier-free guidance for personalized text-to-image diffusion models. Instead of using the fine-tuned model's own null-text prediction as the unconditional term, the method uses a 'weak model' obtained by linear interpolation of the pre-trained and fine-tuned weights, θ_ω = ω θ' + (1−ω) θ, and applies guidance as ε̃ = ε_{θ_ω}(x_t|φ) + λ(ε_{θ'}(x_t|c) − ε_{θ_ω}(x_t|φ)). The weight-interpolation scale ω is claimed to steer the output between text fidelity and subject fidelity. Experiments on the ViCo dataset across SD 1.5, SD 2.1, and SANA, combined with DreamBooth-LoRA, DB-LoRA + Textual Inversion, and ClassDiffusion, report improved DINO and CLIP-I subject-fidelity scores compared with CFG, CFG+SAG, and CFG+AG, at no additional computational cost. The paper also applies the idea to Diffusion-DPO, style personalization with PairCustom, and InstructPix2Pix. The central claims are that PG improves subject fidelity while maintaining text fidelity, with a dynamically controllable balance via ω.

Significance. If the empirical claims hold with an honest evaluation protocol, this is a practically useful contribution: it is a simple, inference-only, zero-overhead modification that is orthogonal to many fine-tuning recipes and appears to transfer across U-Net and DiT backbones. The paper deserves credit for disclosing full ω sweeps (Tables 9–13) and for reporting computational cost (Table 8), which shows the method adds no overhead compared with CFG. The application examples to Diffusion-DPO, style LoRAs, and InstructPix2Pix indicate broader generality. However, the headline numbers are weakened by test-set selection of ω and by the absence of any variance or significance assessment; in addition, the quantitative CLIP-T results do not support the abstract's claim of improved text alignment relative to CFG. These issues do not invalidate the method's core idea, but they do require the authors to reposition the claims and provide a fair evaluation protocol.

major comments (3)
  1. [Sec. 4.2 and Tables 9–13] The operating point ω is selected on the same evaluation set used for reporting: Sec. 4.2 states 'We select the optimal ω based on the best DINO and CLIP-I scores,' and Tables 9–13 confirm that the reported rows are the maxima of the DINO/CLIP-I sweeps. This is test-set tuning, so the reported DINO/CLIP-I improvements are optimistic estimates of the method's performance under a fixed recipe. The authors should either fix a default ω (or a validation-based selection rule) and report performance on a disjoint test set, or present the full sweep as the result and clearly label the selected points as tuning outcomes. Without this, the central claim that PG 'improves subject fidelity' at a specific operating point is not a claim about a defined method.
  2. [Abstract, Sec. 4.2, Tables 1–2] The paper claims that the method 'improves text alignment' and 'maintain[s] text fidelity,' but the quantitative CLIP-T scores at the selected ω are consistently below CFG. For example, in Table 1, SD 1.5 DB-LoRA gives CLIP-T 0.3288 with Ours (ω=0.6) versus 0.3345 with CFG; SD 2.1 gives 0.3277 versus 0.3323; SANA with ω=0 gives 0.3357 versus 0.3363. The same pattern appears in Tables 2, 9–13. The user study (Table 3) reports higher subject-fidelity preference for Ours but large 'Undecided' shares (40.31% for SD2.1+ClassDiffusion text fidelity; 85.49% for SANA text fidelity) and no significance tests. The text-fidelity claim should be reworded to reflect the actual quantitative trade-off (e.g., 'small CLIP-T decrease with improved subject fidelity and user-perceived text quality') or supported by an equalized comparison where CLIP-T is matched.
  3. [Sec. 4.2 and Table 3] No standard errors, confidence intervals, or seed-level variance are reported for any of the headline numbers. The DINO/CLIP-I gains are about 0.02–0.03 absolute, and without multiple seeds or a paired statistical test it is impossible to assess whether these gains are reliable. The user study is based on only 15 sets per model, reports no per-participant variance, and does not apply any significance test. Please provide variance estimates or significance testing for the main comparison tables and the user study, or explicitly label the results as preliminary single-run observations.
minor comments (4)
  1. [Sec. 3.2] The sentence beginning 'If θ includes only linear classifier...' is not derived; since this is a theoretical aside, it would help to provide a short derivation or to remove the claim.
  2. [Sec. 3.2 and Fig. 4] The claim that increasing ω 'monotonically' moves toward the fine-tuned model is intuitive but the term 'monotonic' is used loosely; the empirical sweeps show that DINO is not monotonic in ω (it peaks around ω=0.6 in Tables 9–10), so the wording should be softened to 'in general moves toward.'
  3. [Throughout] There are multiple typos and formatting issues: 'AG diretly' (Sec. 3.1), 'previoius' (Sec. 4.1), 'DreamBoooth-LoRA' (Sec. 4.1), 'modifed' (Sec. 3.2), and stray reference markers such as [7] in Table 6 that are not cited in the main text.
  4. [Sec. 4.5] The InstructPix2Pix application (Sec. 4.5(c)) is only qualitative; adding a quantitative measure (e.g., CLIP directional similarity) would strengthen the generalizability claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the proposed guidance formula is a genuine compositional modification of CFG/AG, and the reported gains are empirical outcomes rather than consequences of the definitions.

full rationale

The paper's derivation chain is self-contained and empirically grounded. Equation (9), the core guidance rule, combines the fine-tuned conditional score with a weak-model unconditional score obtained by weight interpolation (Eq. 8); at ω = 1 it reduces to standard CFG, and at ω = 0 it is a modified CFG using the pretrained model's null-text score. This is a substantive algorithmic proposal, not a renaming or a definitional equivalence: the reported DINO/CLIP-I improvements are measured outcomes, not identities forced by construction. The paper does not fit any parameter to reproduce the evaluation metrics; ω is swept in full over [0,1] and the sweep tables (Tables 9–13) are disclosed, with the headline results at ω = 0 or small positive values and no per-prompt fitting. Gains persist across a range of ω, so the central claim does not reduce to selecting the best point of a fitted curve. The method also relies on external baselines (CFG, AG, SAG, CFG++) and standard datasets and metrics (ViCo, DINO, CLIP-I, CLIP-T, PickScore, DreamSim), and there is no load-bearing self-citation chain or imported uniqueness theorem. The choice of ω on the evaluation set and the lack of error bars are legitimate methodological concerns about generalization and statistical reliability, but they are not circularity: the paper's own equations and ablations support a qualitative claim that is not equivalent to its inputs. Therefore no circular step is identifiable, and the appropriate score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are introduced; the 'weak model' is a weight interpolation of existing models. The key free parameter is omega, which is tuned on the evaluation set, and the key ad hoc assumption is that parameter interpolation produces a meaningful 'unlearning' path.

free parameters (3)
  • omega (weight interpolation scale) = 0.0-0.7 per model/method, e.g., 0.6 for SD1.5 DB-LoRA, 0.7 for SD1.5 DB-LoRA+TI, 0.0 for SANA
    Tuned on the evaluation set to maximize DINO/CLIP-I scores; it controls the trade-off between subject and text fidelity. The reported gains at tuned omega are partly a best-case over the ablation sweep.
  • lambda (guidance scale) = 7.5 for SD 1.5/2.1, 4.5 for SANA, 2.0 for AG baseline
    Standard CFG scales from prior work; Fig. 8 shows PG improves over CFG across lambda, so lambda is not the source of the main gain.
  • LoRA rank and fine-tuning hyperparameters = rank 4, lr 1e-4, 500 steps, batch size 1
    Standard DreamBooth-LoRA settings; Table 7 shows the method also works with rank 8-32, so the central result does not depend on a specific rank.
assumptions (4)
  • domain assumption Fine-tuned model epsilon_theta' approximates the target distribution via the denoising objective (Eq. 2).
    All personalization methods assume that optimizing Eq. (2) on a few reference images yields a model faithful to the target concept. Section 3.1.
  • standard math Classifier-free guidance (Eq. 5) and autoguidance (Eq. 6) are valid sampling techniques; the modified guidance (Eq. 9) is assumed to sample from a well-behaved distribution.
    The paper builds on CFG/AG without re-deriving their validity; it assumes the guidance direction improves sample quality. Sections 3.1-3.2.
  • ad hoc to paper Linear interpolation of parameters theta_omega = omega * theta' + (1 - omega) * theta yields a weak model whose 'degree of unlearning' varies monotonically with omega.
    No theoretical guarantee; the paper argues neural networks are nonlinear in parameters so this differs from output interpolation (Section 3.2). The assumption is supported only by empirical ablations (Fig. 4, Tables 9-13).
  • domain assumption Pre-trained model null-text prediction epsilon_theta(xt|phi) serves as a better weak model than the fine-tuned null-text prediction for personalization.
    This is the core heuristic of the method (Eq. 7). The paper motivates it with Fig. 3 but offers no proof; experiments support it on tested models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Steering Guidance for Personalized Text-to-Image Diffusion Models." pith.science (2026). https://pith.science/paper/ZVNAU4RM

@misc{pith2026250800319,
  author       = {Pith},
  title        = {Pith review of: Steering Guidance for Personalized Text-to-Image Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZVNAU4RM}},
  note         = {Machine review of arXiv:2508.00319}
}
read the original abstract

Personalizing text-to-image diffusion models is crucial for adapting the pre-trained models to specific target concepts, enabling diverse image generation. However, fine-tuning with few images introduces an inherent trade-off between aligning with the target distribution (e.g., subject fidelity) and preserving the broad knowledge of the original model (e.g., text editability). Existing sampling guidance methods, such as classifier-free guidance (CFG) and autoguidance (AG), fail to effectively guide the output toward well-balanced space: CFG restricts the adaptation to the target distribution, while AG compromises text alignment. To address these limitations, we propose personalization guidance, a simple yet effective method leveraging an unlearned weak model conditioned on a null text prompt. Moreover, our method dynamically controls the extent of unlearning in a weak model through weight interpolation between pre-trained and fine-tuned models during inference. Unlike existing guidance methods, which depend solely on guidance scales, our method explicitly steers the outputs toward a balanced latent space without additional computational overhead. Experimental results demonstrate that our proposed guidance can improve text alignment and target distribution fidelity, integrating seamlessly with various fine-tuning strategies.

Figures

Figures reproduced from arXiv: 2508.00319 by the authors.

Figure 1
Figure 1. Comparison of generated image quality between Classifier-Free Guidance (CFG) and our Personalization Guidance. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Motivation of Personalization Guidance. Classifier-free [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Comparison between Classifier-Free Guidance (CFG), [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Comparison with other guidance techniques, including without guidance, CFG, CFG+SAG, and CFG+AG. These images are [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Comparison with other guidance techniques. These images are generated by fine-tuned SANA [ [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Generated images by changing ω, where SANA with DB-LoRA is used. Method Subject Fidelity ↑ Text Fidelity ↑ Stable Diffusion 2.1 + ClassDiffusion CFG 10.45% 15.24% CFG + SAG 8.01% 6.20% CFG + AG 12.60% 6.20% Ours 55.22% 32.04% Undecided 13.67% 40.31% SANA + DB-LoRA CFG …
Figure 8
Figure 8. Figure 8: Ablation study on the guidance scale λ ∈ [2.5, 5.0], measuring performance at 0.5 intervals using DB-LoRA based on SANA. When ω is set to 0.2, we can improve both sub￾ject and text fidelity compared to CFG, regardless of guidance scale λ. Note that CLIP-T score is slig…
Figure 9
Figure 9. Figure 9: Qualitative results of our guidance across diverse tasks. [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Ablation study on the number of training steps. No [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Comparison with other guidance techniques. These images are generated by fine-tuned SD 2.1 [ [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Comparison with other guidance techniques. These images are generated by fine-tuned SANA [ [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 14 linked inside Pith

  1. [1]

    Self-rectifying diffu- sion sampling with perturbed-attention guidance

    Donghoon Ahn, Hyoungwon Cho, Jaewon Min, Wooseok Jang, Jungwoo Kim, SeonHwa Kim, Hyun Hee Park, Ky- ong Hwan Jin, and Seungryong Kim. Self-rectifying diffu- sion sampling with perturbed-attention guidance. In Euro- pean Conference on Computer Vision, pages 1–17. Springer,

  2. [2]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 18392–18402, 2023. 3, 8

    2023

  3. [3]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the International Conference on Computer Vi- sion (ICCV), 2021. 5

    2021

  4. [4]

    Improving subject-driven image syn- thesis with subject-agnostic guidance

    Kelvin CK Chan, Yang Zhao, Xuhui Jia, Ming-Hsuan Yang, and Huisheng Wang. Improving subject-driven image syn- thesis with subject-agnostic guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6733–6742, 2024. 3, 5, 6, 1

    2024

  5. [5]

    Para: Personalizing text-to-image diffusion via parameter rank reduction

    Shangyu Chen, Zizheng Pan, Jianfei Cai, and Dinh Phung. Para: Personalizing text-to-image diffusion via parameter rank reduction. arXiv preprint arXiv:2406.05641, 2024. 2, 3

    arXiv 2024

  6. [6]

    Subject-driven text-to-image generation via apprenticeship learning

    Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming-Wei Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning. Advances in Neural Information Processing Sys- tems, 36:30286–30305, 2023. 2, 3

    2023

  7. [7]

    Cfg++: Manifold-constrained clas- sifier free guidance for diffusion models

    Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. Cfg++: Manifold-constrained clas- sifier free guidance for diffusion models. arXiv preprint arXiv:2406.08070, 2024. 1

    arXiv 2024

  8. [8]

    Dream- sim: Learning new dimensions of human visual similar- ity using synthetic data

    Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dream- sim: Learning new dimensions of human visual similar- ity using synthetic data. arXiv preprint arXiv:2306.09344 ,

    arXiv

Show all 41 references
  1. [9]

    An image is worth one word: Personalizing text-to-image gener- ation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image gener- ation using textual inversion. In The Eleventh International Conference on Learning Representations. 2, 3, 1

  2. [10]

    Vico: Plug-and-play visual condition for personalized text-to-image generation

    Shaozhe Hao, Kai Han, Shihao Zhao, and Kwan-Yee K Wong. Vico: Plug-and-play visual condition for personalized text-to-image generation. arXiv preprint arXiv:2306.00971,

    arXiv

  3. [11]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 2, 3, 4, 5, 1

    2021

  4. [12]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. NeurIPS, 33:6840–6851, 2020. 2, 3

    2020

  5. [13]

    Improving sample quality of diffusion models us- ing self-attention guidance

    Susung Hong, Gyuseong Lee, Wooseok Jang, and Seungry- ong Kim. Improving sample quality of diffusion models us- ing self-attention guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7462– 7471, 2023. 3

    2023

  6. [14]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 2, 3, 5, 1

    2022

  7. [15]

    Classdiffusion: More aligned personalization tuning with explicit class guidance

    Jiannan Huang, Jun Hao Liew, Hanshu Yan, Yuyang Yin, Yao Zhao, and Yunchao Wei. Classdiffusion: More aligned personalization tuning with explicit class guidance. arXiv preprint arXiv:2405.17532, 2024. 2, 3, 5, 6, 1

    2024 arXiv

  8. [16]

    In-context lora for diffusion transformers

    Lianghua Huang, Wei Wang, Zhi-Fan Wu, Yupeng Shi, Huanzhang Dou, Chen Liang, Yutong Feng, Yu Liu, and Jin- gren Zhou. In-context lora for diffusion transformers. arXiv preprint arXiv:2410.23775, 2024. 8

    2024 arXiv

  9. [17]

    Spatiotemporal skip guidance for enhanced video diffusion sampling

    Junha Hyung, Kinam Kim, Susung Hong, Min-Jung Kim, and Jaegul Choo. Spatiotemporal skip guidance for enhanced video diffusion sampling. arXiv preprint arXiv:2411.18664,

    arXiv

  10. [18]

    Customizing text-to-image models with a single image pair

    Maxwell Jones, Sheng-Yu Wang, Nupur Kumari, David Bau, and Jun-Yan Zhu. Customizing text-to-image models with a single image pair. In SIGGRAPH Asia 2024 Conference Papers, pages 1–13, 2024. 8

    2024

  11. [19]

    Guiding a diffusion model with a bad version of itself

    Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding a diffusion model with a bad version of itself. Advances in Neural In- formation Processing Systems, 37:52996–53021, 2025. 2, 3, 4, 5, 1

    2025

  12. [20]

    Autolora: Autoguid- ance meets low-rank adaptation for diffusion models

    Artur Kasymov, Marcin Sendera, Michał Stypułkowski, Ma- ciej Zi˛ eba, and Przemysław Spurek. Autolora: Autoguid- ance meets low-rank adaptation for diffusion models. arXiv preprint arXiv:2410.03941, 2024. 6

    2024 arXiv

  13. [21]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1931–1941, 2023. 2, 3

    1931

  14. [22]

    Aligning diffusion mod- els by optimizing human utility

    Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato, and Kazuki Kozuka. Aligning diffusion mod- els by optimizing human utility. Advances in Neural Infor- mation Processing Systems, 37:24897–24925, 2025. 8

    2025

  15. [23]

    Dreammatcher: appearance matching self-attention for semantically-consistent text-to- image personalization

    Jisu Nam, Heesu Kim, DongJae Lee, Siyoon Jin, Seungry- ong Kim, and Seunggyu Chang. Dreammatcher: appearance matching self-attention for semantically-consistent text-to- image personalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024

  16. [24]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,

  17. [25]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 2 9

    2023 arXiv

  18. [26]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  19. [27]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2, 4, 5, 1

    2022

  20. [28]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pages 22500–22510, 2023. 2, 3, 5, 1

    2023

  21. [29]

    Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models. In Proceedings of the IEEE/CVF conference on computer vision and pat...

    2024

  22. [30]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 35:36479–36494, 2022. 2

    2022

  23. [31]

    In- stantbooth: Personalized text-to-image generation without test-time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. In- stantbooth: Personalized text-to-image generation without test-time finetuning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 8543–8552, 2024. 2, 3

    2024

  24. [32]

    Freeu: Free lunch in diffusion u-net

    Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4733–4743, 2024. 3

    2024

  25. [33]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 2, 3, 1

    2010 arXiv

  26. [34]

    Leveraging previous steps: A training-free fast solver for flow diffusion

    Kaiyu Song and Hanjiang Lai. Leveraging previous steps: A training-free fast solver for flow diffusion. arXiv preprint arXiv:2411.07627, 2024. 1

    2024 arXiv

  27. [35]

    Diffusion model align- ment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model align- ment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision a...

    2024

  28. [36]

    Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15943–15953, 2023. 2, 3

    2023

  29. [37]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing in- ference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Re- becca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Ko- rnblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing ...

    2022

  30. [38]

    Robust fine-tuning of zero-shot models

    Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gon- tijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. In Proceedings of the IEEE/CVF conference on computer vi- ...

    2022

  31. [39]

    Stylealign: Analysis and applications of aligned stylegan models

    Zongze Wu, Yotam Nitzan, Eli Shechtman, and Dani Lischinski. Stylealign: Analysis and applications of aligned stylegan models. arXiv preprint arXiv:2110.11323, 2021. 8

    2021 arXiv

  32. [40]

    Sana: Efficient high-resolution image syn- thesis with linear diffusion transformers

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image syn- thesis with linear diffusion transformers. arXiv preprint arXiv:2410.10629, 2024. 2, 5, 6, 1

    arXiv

  33. [41]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 8 10...

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.