Pith. sign in

REVIEW 4 major objections 5 minor 41 references

APT: Adaptive Personalized Training for Diffusion Models with Limited Data

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read APT proposes a per-noise-level overfitting indicator that adapts augmentation and loss weighting, plus feature-statistic and cross-attention regularizers, so diffusion models can be personalized on a few images without losing prior…

desk verdict Solid empirical recipe for few-shot diffusion personalization, but the adaptive core is under-built and the quantitative claims need tightening. read the letter →

arxiv 2507.02687 v1 pith:KYTWC5FG submitted 2025-07-03 cs.CV cs.AI

JungWoo Chae , Jiyoon Kim , JaeWoong Choi , Kyungyul Kim , Sangheum Hwang This is my paper
classification cs.CVcs.AI
keywords diffusionmodelspersonalizationfew-shotfine-tuningoverfittingadaptivedataaugmentationrepresentationregularizationattentionalignmenttext-to-imagegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Adaptive Personalized Training (APT), a fine-tuning recipe for text-to-image diffusion models that must learn a new concept from just a handful of reference images, typically 4–5, without collapsing into memorization. The central idea is to detect where overfitting happens in the denoising process and adapt training on the fly: the method measures how much the fine-tuned model's denoising loss exceeds the pretrained model's loss in each time-step bin, then uses that gap to increase augmentation and down-weight the denoising loss exactly in the affected bins. Two regularizers then anchor the fine-tuned model to the pretrained model: matching the mean and variance of intermediate feature maps, and aligning cross-attention maps for all text tokens. The paper reports that this preserves the base model's text alignment, style, and contextual knowledge while still capturing the reference subject's identity, and that it beats DreamBooth-like and DCO-like baselines in fidelity, diversity, and user preference.

What carries the argument

The load-bearing mechanism is the per-bin overfitting indicator $$\gamma_t = 1 - $e^{{-T(\mathrm{EMA}}$_t[L^\phi_{\mathrm{DM}}]-\mathrm{EMA}_t[L^\theta_{\mathrm{DM}}])},$$ where $\phi$ is the pretrained model, $\theta$ is the fine-tuned model, $L_{\mathrm{DM}}$ is the denoising loss, and the EMA is computed per time-step bin (for example, 10 bins across 1000 steps). The indicator drives augmentation probability $p_{\mathrm{augment}}=\mathrm{clamp}(\gamma_t,0,p_{\max})$ and scales the denoising loss to $L^{\mathrm{apt}}_{\mathrm{DM}}=(1-\gamma_t)L_{\mathrm{DM}}$. Two auxiliary losses reinforce the anchor: $L_\mu+L_\sigma$ matches the mean and variance of the U-Net's intermediate feature maps between $\theta$ and $\phi$, and $L_{\mathrm{attn}}$ aligns the fine-tuned model's summed cross-attention maps with the pretrained model's, so all text tokens keep their original semantic relationships. The total objective is $$L_{\mathrm{total}}=$L^{{\mathrm{apt}}$}_{\mathrm{DM}}+\lambda_{\mathrm{dist}}(L_\mu+L_\$\sigma$)+\lambda_{\mathrm{attn}}L_{\mathrm{attn}},$$ which combines the adaptive denoising loss with the two anchoring regularizers.

What would settle it

Measure the pretrained model's denoising loss on the reference images before any fine-tuning and compare it with the loss on images the base model handles well. If the reference loss is substantially higher, the indicator's baseline is inflated, and any drop in the fine-tuned model's loss will look like overfitting even when the model is only adapting to an unfamiliar subject. A second check is to rerun the SDXL recipe on Stable Diffusion V2.1 while keeping $T$ rather than $T/10$ in the exponent; if training becomes unstable, the indicator's normalization is not portable across base models.

Watch

Extended reading notes

Core claim

APT's claim is that overfitting in few-shot personalization is not uniform across the diffusion process: it appears first and strongest at low-noise time steps, where the model memorizes fine details. The method detects this with a per-bin overfitting indicator $\gamma_t$, computed from the exponential-moving-average gap between the pretrained model's denoising loss and the fine-tuned model's loss on the same batch, and uses $\gamma_t$ both as the probability of applying affine augmentation and as a multiplier $(1-\gamma_t)$ on the denoising loss. Two regularizers then pin the fine-tuned model to the pretrained one: a representation-stabilization loss that matches the mean and variance of intermediate U-Net feature maps, and an attention-alignment loss that matches the summed cross-attention maps, over heads and across all text tokens, to the pretrained model's maps. The paper argues that together these components keep the denoising trajectory close to the original model's, so generated images preserve scene context, style, and text alignment while still capturing the reference subject's identity.

Load-bearing premise

The load-bearing premise is that the gap between the pretrained model's denoising loss and the fine-tuned model's loss on the same training images is a faithful measure of memorization, after one exponential formula turns it into a per-noise-level number; if the reference images are simply unusual for the base model, the indicator mistakes unfamiliarity for overfitting and the adaptive augmentation and loss suppression are aimed at the wrong cause.

Editorial extensions

If this is right

  • Fine-tuning on 4–5 images no longer has to sacrifice the base model's text alignment: the adaptive loss weighting suppresses updates in time bins that are memorizing, while other bins continue learning the subject.
  • Generated images stay contextually coherent, with objects placed in scenes, styles, or human contexts that never appeared in the reference set, because the cross-attention maps stay pinned to the pretrained model's.
  • The per-bin indicator provides an interpretable training-time signal of where overfitting starts, which could be logged during any personalization run to show which noise levels are drifting.
  • The same three-part recipe transfers to a smaller base model, Stable Diffusion V2.1, with only a change in the indicator's exponent, suggesting the mechanism is not specific to SDXL.
  • Prior-preservation evaluation, comparing the personalized model's output distribution with the base model's distribution via FID, precision, and recall, becomes a standard way to measure personalization quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: $\gamma_t$ could serve as a general-purpose memorization diagnostic for other diffusion fine-tuning settings, not just personalization, since it compares the fine-tuned model's per-noise-level loss to the frozen prior's.
  • Beyond the paper: because representation stabilization matches only the mean and variance of feature maps, it may be acting against feature-collapse or rank collapse in the U-Net; a testable extension is to measure effective feature rank under APT versus DreamBooth.
  • Beyond the paper: the paper's own 'monster toy' limitation indicates that the attention-alignment weight $\lambda_{\mathrm{attn}}$ will need to be concept-dependent; a natural follow-up is an automatic per-concept schedule for this weight.
  • Beyond the paper: the paired forward pass through both models could be relaxed if $\gamma_t$ were estimated from the fine-tuned model's loss history alone, which would reduce the memory overhead and widen applicability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes APT, a framework for personalizing text-to-image diffusion models when only a few reference images are available. APT combines three components: (i) adaptive training adjustment, which uses an overfitting indicator gamma_t to modulate data augmentation probability and loss weighting per time-step bin; (ii) representation stabilization, which regularizes the mean and variance of intermediate U-Net activations toward the pretrained model; and (iii) attention alignment, which matches cross-attention maps (averaged over heads) to the pretrained model. The method is evaluated on SDXL using DreamBooth and Textual Inversion datasets, with qualitative comparisons, a user study, and an ablation. The authors claim that APT mitigates overfitting, preserves prior knowledge, and maintains text alignment better than existing methods.

Significance. If the adaptive indicator were a reliable measure of overfitting, the method would be a practical contribution to few-shot personalization, where overfitting and loss of prior knowledge are central problems. The paper provides a fairly extensive ablation and a user study, and the qualitative results are compelling. However, the quantitative support is weakened by two structural issues: the headline FID metric measures closeness to an SDXL-generated source, which is the very property the regularizers enforce, and the adaptive indicator is an uncalibrated raw loss gap that is used without clamping in the loss weighting. The lack of error bars and the small numerical differences in text-alignment metrics make the quantitative superiority claims uncertain. The core idea is still defensible, but these load-bearing points need to be addressed.

major comments (4)
  1. [Section 3.1, Eq. (1)-(3)] The overfitting indicator gamma_t is an uncalibrated raw EMA loss gap scaled by T, and it is used without clamping in Eq. (3). When L_theta exceeds L_phi, gamma_t becomes negative and (1 - gamma_t) amplifies the diffusion loss, which is the opposite of the stated adaptive suppression. The manual change from T to T/10 for SD2.1 reported in Supp. B.4 indicates that the absolute scale of the gap is model-dependent, so the claim that the formulation ensures gamma_t = 0 without overfitting and gamma_t -> 1 with maximal overfitting is not supported. The indicator also cannot distinguish memorization from legitimate learning of a new concept, both of which reduce L_theta below L_phi.
  2. [Section 4.3, Table 1] The FID, Precision, and Recall values are computed between an SDXL-generated source dataset and a target dataset generated by each personalized model. Because the proposed regularizers in Eqs. (4)-(6) explicitly pull the fine-tuned model's features, statistics, and attention maps toward the pretrained SDXL model, this metric largely measures how well the method achieves its own objective. The paper should reposition this as a prior-preservation metric or report FID against the real reference distribution (with appropriate caveats about the small sample size). As it stands, the quantitative claim of 'outperforms existing methods' rests on a partially circular evaluation.
  3. [Table 1 and Section 4.3] No error bars, multiple seeds, or statistical tests are reported. The text-image similarity improvements are very small (CLIP-T: 0.661 to 0.664; HPSv2: 0.272 to 0.288), and without variance estimates it is impossible to judge whether these differences are meaningful. Similarly, the DINOv2 similarity of APT (0.660) is lower than all baselines except the pretrained prior, which the authors attribute to scene context; this trade-off between identity preservation and prior preservation should be quantified (e.g., with object-centric crops or a dedicated identity metric) rather than only explained qualitatively.
  4. [Section 3.2, Eq. (4)-(5)] The representation stabilization losses compare activations of the fine-tuned model conditioned on the identifier c* against activations of the pretrained model conditioned on the class token c. This enforces that the feature statistics for the new concept match those of the general class, which may be too strong a prior when the concept is visually distinct from the class. The paper does not analyze how the choice of the class token affects the identity–prior balance, despite this being central to the personalization task.
minor comments (5)
  1. [Section 2] The text refers to 'Custom Diffision' in the Related Work section; the correct name is 'Custom Diffusion' as used in the references.
  2. [Section 3.1, Eq. (1)] The notation EMA_t is not precise: it should state that a separate exponential moving average is maintained for each time-step bin, and how the bin boundaries (e.g., 10 bins of 100 steps each) are defined in the formula.
  3. [References] References [6] and [7] are duplicated entries for the same CFG++ paper; one should be removed and the citation numbering adjusted.
  4. [Figure 1] The phrase 'lifting a barbel' should be 'barbell'.
  5. [Section 4.4] The user study reports only the overall preference percentages; providing per-prompt agreement or a measure of inter-rater variability would strengthen the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: APT's adaptive indicator and regularizers are explicit control heuristics, and its headline evaluations are held-out comparisons rather than re-statements of the training losses.

full rationale

I walked the derivation chain. The overfitting indicator (Eq. 1) is a heuristic based on the EMA gap between pretrained and fine-tuned denoising losses; it is not derived from a claim about overfitting, and the adaptive weighting (Eq. 3) is a feedback mechanism, not a prediction. The representation and attention regularizers (Eqs. 4-6) explicitly pull the fine-tuned model toward the pretrained model, so the FID-vs-SDXL and user-study 'prior similarity' results are aligned with the objective, but FID is computed on generated images and is not identical to any training loss, so no result is forced by construction. There are no load-bearing self-citations: the cited DCO, DreamBooth, and AttnDreamBooth works are external. The paper's own Limitations section acknowledges hyperparameter sensitivity, and Supp. B.4's T/10 adjustment raises calibration concerns about gamma_t, but these are correctness risks, not circularity. Ablations and the user study provide independent empirical content. Thus the paper's central claims are not definitional or fitted-input-as-prediction, and the derivation is self-contained against external benchmarks.

Assumptions & free parameters 9 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several hand-picked hyperparameters and on the assumption that matching the pretrained model's internal activations under mismatched text conditioning is a sound objective. The reader must accept these choices on trust; the paper does not derive them from first principles.

free parameters (9)
  • Exponential scaling coefficient in gamma (T) = T=1000 for SDXL; T/10=100 for SD V2.1
    Controls how quickly gamma saturates; manually adjusted in Supp B.4 when switching base models, indicating it is not a principled constant.
  • Maximum augmentation probability pmax = 0.8
    Hand-selected clamp for augmentation probability in Eq. (2).
  • Representation stabilization weight lambda_dist = 30
    Hand-selected weight for L_mu + L_sigma in Eq. (7).
  • Attention alignment weight lambda_attn = 3e-4
    Hand-selected weight for L_attn in Eq. (7).
  • EMA smoothing factor alpha = 0.1
    Smoothing for overfitting indicator in Eq. (1); set in Supp A.
  • Number of timestep bins B = 10
    Bins used to compute per-bin gamma; chosen without ablation.
  • Layer choices for RS and AA = Upblocks at 32x32 and 64x64
    Selected layers for mean/variance and attention regularization; no ablation on layer selection.
  • Augmentation ranges = zoom-out 1-3x, rotation +/-15 deg
    Affine augmentation ranges; acknowledged as not fully explored.
  • Loss weighting scheme (1-gamma) = linear downscaling
    Design choice for adaptive loss weighting; no comparison to alternatives.
assumptions (4)
  • domain assumption The difference between the pretrained model's loss and the fine-tuned model's loss, exponentially transformed, is a monotonic measure of overfitting at each timestep bin.
    Used in Eq. (1) to define gamma; if the pretrained loss is high for reasons unrelated to overfitting, gamma is miscalibrated.
  • domain assumption The class-token conditioning c (e.g., 'dog') is a valid semantic reference for the identifier token V* of the specific instance.
    Eqs. (4)-(6) compare fine-tuned model under c* with pretrained model under c; if the two tokens map to very different concepts, the regularization target is inappropriate.
  • domain assumption Preserving the pretrained model's intermediate feature statistics and cross-attention maps preserves 'prior knowledge' and improves text alignment.
    Core assumption behind L_mu, L_sigma, L_attn; not derived, and the paper's own limitations note it can suppress desired concept traits.
  • ad hoc to paper The exponential mapping gamma = 1 - e^{-T*delta} is a reasonable normalization for the loss difference.
    No derivation is provided; the coefficient T is changed manually for a different base model (Supp B.4).

how reviews work

0 comments
Cite this review

Pith. "Pith review of APT: Adaptive Personalized Training for Diffusion Models with Limited Data." pith.science (2026). https://pith.science/paper/KYTWC5FG

@misc{pith2026250702687,
  author       = {Pith},
  title        = {Pith review of: APT: Adaptive Personalized Training for Diffusion Models with Limited Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KYTWC5FG}},
  note         = {Machine review of arXiv:2507.02687}
}
read the original abstract

Personalizing diffusion models using limited data presents significant challenges, including overfitting, loss of prior knowledge, and degradation of text alignment. Overfitting leads to shifts in the noise prediction distribution, disrupting the denoising trajectory and causing the model to lose semantic coherence. In this paper, we propose Adaptive Personalized Training (APT), a novel framework that mitigates overfitting by employing adaptive training strategies and regularizing the model's internal representations during fine-tuning. APT consists of three key components: (1) Adaptive Training Adjustment, which introduces an overfitting indicator to detect the degree of overfitting at each time step bin and applies adaptive data augmentation and adaptive loss weighting based on this indicator; (2)Representation Stabilization, which regularizes the mean and variance of intermediate feature maps to prevent excessive shifts in noise prediction; and (3) Attention Alignment for Prior Knowledge Preservation, which aligns the cross-attention maps of the fine-tuned model with those of the pretrained model to maintain prior knowledge and semantic coherence. Through extensive experiments, we demonstrate that APT effectively mitigates overfitting, preserves prior knowledge, and outperforms existing methods in generating high-quality, diverse images with limited reference data.

Figures

Figures reproduced from arXiv: 2507.02687 by the authors.

Figure 1
Figure 1. Given a few reference images, APT personalizes diffusion models with less overfitting: (Left) By comparing diffusion trajectories [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Our Proposed Method (APT). Illustration of the three key components: (1) Adaptive Training Adjustment with adaptive data augmentation (paugment) and loss weighting (LaptDM) to mitigate overfitting; (2) Representation Stabilization through regularizing intermediate feature maps to stabilize the noise trajectory (Lµ, Lσ); (3) Attention Alignment to preserve prior knowledge by regularizing the cross-attenti… view at source ↗
Figure 3
Figure 3. Cross-Attention Map Comparison. Visualization of cross-attention maps in text-conditioned image generation for (a) SDXL, (b) DreamBooth, and (c) APT. DreamBooth shows changes not only in the class token’s map but also in overall atten￾tion maps, indicating shifts in how the model attends to different tokens after personalization. By applying this regularization to all text tokens, we en￾sure that the model maintains… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparison. We present images generated by the pretrained model, DreamBooth, DreamBooth with Prior Preser￾vation, DCO, and our method (APT) across various data types and styles. Baseline methods tend to memorize textures and generate object-centric images, …
Figure 5
Figure 5. Figure 5: Ablation Study of APT Components. We evaluate the contribution of each component in our method by incrementally adding Adaptive Training Adjustment (ATA), Representation Sta￾bilization (RS), and Attention Alignment (AA) to Base (Dream￾Booth). 4.5. Ablation Study To ass…
Figure 6
Figure 6. Figure 6: Overfitting Indicator The overfitting indicator γt is plotted across training iterations for different time step bins. 5. Limitations While our proposed APT method effectively mitigates over￾fitting and preserves prior knowledge, it has certain lim￾itations. The trade-…
Figure 7
Figure 7. Figure 7: Additional Ablation Study of APT Components. We evaluate the contribution of each component in our method by incrementally adding Adaptive Training Adjustment (ATA), Rep￾resentation Stabilization (RS), and Attention Alignment (AA) to Base (DreamBooth). B.3. Motivation …
Figure 8
Figure 8. Figure 8: Difference in Predicted Noise. The difference in pre￾dicted noise between SDXL (prior) and DreamBooth [27] models is plotted over training iterations. Since the degree of overfitting varies across different datasets, we were motivated to detect over￾fitting during trai…
Figure 9
Figure 9. Figure 9: Additional Qualitative Comparison. We present four images generated by our method and two images from each of the baseline methods, including SDXL, DreamBooth [27], DreamBooth with prior preservation loss, and DCO [17]. Our method demonstrates superior performance in p…
Figure 10
Figure 10. Figure 10: Additional Qualitative Comparison. We present four images generated by our method and two images from each of the baseline methods, including SDXL, DreamBooth [27], DreamBooth with prior preservation loss, and DCO [17]. Our method demonstrates superior performance in …
Figure 11
Figure 11. Figure 11: Additional Qualitative Comparison on Stable Diffusion V2.1. We compare APT with other methods which are based on Stable Diffusion V2.1., including DreamBooth [8, 27], NeTI [2], ViCo [11], OFT [23], and AttnDreamBooth [21]. Two images from each of the baseline methods …
Figure 12
Figure 12. Figure 12: User Study Example. This shows the interface presented to users during the study [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 26 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    A neural space-time representation for text- to-image personalization

    Yuval Alaluf, Elad Richardson, Gal Metzer, and Daniel Cohen-Or. A neural space-time representation for text- to-image personalization. ACM Transactions on Graphics (TOG), 42(6):1–10, 2023. 3, 13, 17

    work page 2023

  3. [3]

    Break-a-scene: Extracting multi- ple concepts from a single image

    Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen- Or, and Dani Lischinski. Break-a-scene: Extracting multi- ple concepts from a single image. In SIGGRAPH Asia 2023 Conference Papers, pages 1–12, 2023. 2

    2023

  4. [4]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 2

    2023

  5. [5]

    Microsoft coco captions: Data collection and evaluation server

    Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedan- tam, Saurabh Gupta, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015. 7

    arXiv 2015

  6. [7]

    Cfg++: Manifold-constrained clas- sifier free guidance for diffusion models

    Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. Cfg++: Manifold-constrained clas- sifier free guidance for diffusion models. arXiv preprint arXiv:2406.08070, 2024. 2

    arXiv 2024

  7. [8]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In ICLR, 2022. 2, 3, 5, 13, 17

    work page 2022

  8. [9]

    Svdiff: Compact param- eter space for diffusion fine-tuning

    Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. Svdiff: Compact param- eter space for diffusion fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7323–7334, 2023. 2, 3

    work page 2023

Show all 41 references
  1. [10]

    Efficient diffu- sion training via min-snr weighting strategy

    Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Baining Guo. Efficient diffu- sion training via min-snr weighting strategy. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 7441–7451, 2023. 12

    2023

  2. [11]

    Vico: Plug-and-play visual condition for personalized text-to-image generation

    Shaozhe Hao, Kai Han, Shihao Zhao, and Kwan-Yee K Wong. Vico: Plug-and-play visual condition for personalized text-to-image generation. arXiv preprint arXiv:2306.00971,

    arXiv

  3. [12]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 2, 11

    2021

  4. [13]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. NeurIPS, 33:6840–6851, 2020. 2, 12

    2020

  5. [14]

    Lora: Low- rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low- rank adaptation of large language models. In International Conference on Learning Representations, 2022. 3, 5, 11

    2022

  6. [15]

    Training generative adver- sarial networks with limited data

    Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adver- sarial networks with limited data. Advances in neural infor- mation processing systems, 33:12104–12114, 2020. 3

    2020

  7. [16]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1931–1941, 2023. 3, 5, 7

    1931

  8. [17]

    Direct consistency optimization for compositional text- to-image personalization

    Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn, and Jinwoo Shin. Direct consistency optimization for compositional text- to-image personalization. NeurIPS, 2024. 2, 3, 5, 7, 11, 15, 16

    2024

  9. [18]

    Decoupled weight de- cay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2019. 11

    2019

  10. [19]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  11. [20]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 7

    2023 arXiv

  12. [21]

    Attndream- booth: Towards text-aligned personalized text-to-image gen- eration

    Lianyu Pang, Jian Yin, Baoquan Zhao, et al. Attndream- booth: Towards text-aligned personalized text-to-image gen- eration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3, 13, 17

    2024

  13. [22]

    Sdxl: Im- proving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, et al. Sdxl: Im- proving latent diffusion models for high-resolution image synthesis. In ICLR, 2024. 5, 11, 12

    2024

  14. [23]

    Controlling text-to-image diffusion by orthogo- nal finetuning

    Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Sch¨olkopf. Controlling text-to-image diffusion by orthogo- nal finetuning. Advances in Neural Information Processing Systems, 36:79320–79362, 2023. 13, 17

    2023

  15. [24]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  16. [25]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  17. [26]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2, 12

    2022

  18. [27]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pages 22500–22510, 2023. 2, 3, 5, 7, 12, 13, 15, 16, 17

    2023

  19. [28]

    Palette: Image-to-image diffusion models

    Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10,

    2022

  20. [29]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Raphael Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 2022. 2

    2022

  21. [30]

    Assessing generative models via precision and recall

    Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. Advances in neural information pro- cessing systems, 31, 2018. 7

    2018

  22. [31]

    Rethinking the spatial inconsistency in classifier- free diffusion guidance

    Dazhong Shen, Guanglu Song, Zeyue Xue, Fu-Yun Wang, and Yu Liu. Rethinking the spatial inconsistency in classifier- free diffusion guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9370–9379, 2024. 2

    2024

  23. [32]

    Face2diffusion for fast and editable face personalization

    Kaede Shiohara and Toshihiko Yamasaki. Face2diffusion for fast and editable face personalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6850–6859, 2024. 2

    2024

  24. [33]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In ICLR, 2020. 2

    2020

  25. [34]

    p+: Extended textual conditioning in text-to- image generation

    Andrey V oynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. p+: Extended textual conditioning in text-to- image generation. arXiv preprint arXiv:2303.09522, 2023. 3

    2023 arXiv

  26. [35]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  27. [36]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 2

    2022 arXiv

  28. [37]

    Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation

    Yu Zeng, Vishal M Patel, Haochen Wang, Xun Huang, Ting- Chun Wang, Ming-Yu Liu, and Yogesh Balaji. Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogni...

    2024

  29. [38]

    Attention calibration for disentangled text-to-image person- alization

    Yanbing Zhang, Mengping Yang, Qin Zhou, and Zhe Wang. Attention calibration for disentangled text-to-image person- alization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4764– 4774, 2024. 2

    2024

  30. [39]

    zoomed-in

    Zhengli Zhao, Sameer Singh, Honglak Lee, Zizhao Zhang, Augustus Odena, and Han Zhang. Improved consistency reg- ularization for gans. In Proceedings of the AAAI conference on artificial intelligence, pages 11033–11041, 2021. 3 APT: Adaptive Personalized Training for Diffusion ...

    2021

  31. [40]

    Text Alignment: Does the generated image align well with the given text prompt?

  32. [41]

    Identity Preservation: Is the generated image similar to the reference images?

  33. [42]

    Figure 12 shows the interface presented to users during the study

    Prior Similarity: Is the generated image similar to the composition of the prior image generated by the pre- trained model? Participants are instructed to choose the image that best met all the criteria. Figure 12 shows the interface presented to users during the study. The re...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.