Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Generating on Generated: An Approach Towards Self-Evolving Diffusion Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Curating a diffusion model's own outputs lets it improve over six rounds of self-training.

desk verdict Useful self-training recipe for diffusion models, but headline preference gains are compromised by same-scorer selection and evaluation. read the letter →

arxiv 2502.09963 v1 pith:CI67QMUI submitted 2025-02-14 cs.CV

Xulu Zhang , Xiaoyong Wei , Jinlin Wu , Jiaxin Wu , Zhaoxiang Zhang , Zhen Lei , Qing Li This is my paper
classification cs.CV
keywords recursiveself-improvementdiffusionmodelstext-to-imagegenerationsyntheticdatamodelcollapsehumanpreferencealignmentpromptfilteringself-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a text-to-image diffusion model can improve through recursive self-improvement—iteratively fine-tuning on its own generated data—if the synthetic data is carefully curated rather than randomly sampled. The authors attribute the usual training collapse to two causes: insufficient perceptual alignment of randomly selected samples, and the accumulation of generative hallucinations such as extra legs or fingers. They propose three strategies: a prompt construction and filtering pipeline for clarity, specificity, and diversity; preference sampling that keeps only images favored by automatic human-preference scorers; and a distribution-based weighting scheme that down-weights out-of-distribution samples. On Stable Diffusion v1.4, repeated rounds of this procedure outperform both the base model and a supervised fine-tuning baseline on HPS v2.1, ImageReward, PickScore, and text-alignment metrics, peaking at round 6 before declining. A small user study also prefers RSIDiff outputs over the base model's for visual appeal and text faithfulness.

What carries the argument

The machinery is a three-stage data curation loop applied at every self-training round. First, a prompt set is built by crawling 172k user prompts from an image-sharing site and filtering them through Llama 3 for clarity and specificity, then selecting a diverse 40k subset via K-means clustering on prompt embeddings. Second, preference sampling ranks the generated images with CLIP text-image alignment, aesthetic quality, and HPS v2 scores, keeping only the top samples. Third, distribution-based weighting encodes samples with the base model's VAE and computes their distance to the base model's generated-image distribution, assigning weight 1 within a threshold $\beta$ and $w_s = \exp(-(d(s,\mathcal{D}_0)-\beta)/\sigma^2)$ outside it. These weights enter the reconstruction loss, so in-distribution samples train fully while hallucinated, out-of-distribution samples are down-weighted.

What would settle it

A new human-preference study on prompts never used during prompt filtering or preference sampling, conducted by raters different from the paper's eight participants and compared against the base model, would settle whether the round-6 gains (7.0% HPS v2.1, 181.6% ImageReward) reflect real human preference; if a majority do not prefer RSIDiff outputs, the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that model collapse during self-training is not inevitable: a diffusion model can be driven to continuously improve by feeding it a filtered subset of its own generations. The paper identifies perceptual alignment and generative hallucinations as the two forces that determine whether self-generated data helps or harms, and shows that both can be controlled. Concretely, RSIDiff constructs a high-quality prompt set from crawled user prompts, filters generated images using automatic preference metrics, and assigns exponentially decaying weights to samples that lie far from the base model's image distribution. With this pipeline, the model surpasses the base model by 7.0% on HPS v2.1 and 181.6% on ImageReward on the HPS test set at round 6, using only 1,200 selected samples, and it also beats a supervised fine-tuning baseline that used 5,000 samples.

Load-bearing premise

The load-bearing premise is that the automatic preference scores (especially HPS v2) used to select training data are faithful proxies for human preference, so optimizing the model toward them genuinely improves image quality rather than merely gaming the scorer.

Editorial extensions

If this is right

  • Self-training on selectively curated synthetic data can improve a diffusion model without any new human-annotated data, suggesting that data scarcity and privacy concerns need not block continued model improvement.
  • Removing any one of the three strategies—prompt filtering, preference sampling, or distribution weighting—measurably degrades the recursion, so all three are necessary components of the recipe.
  • The gains are bounded: performance peaks at round 6 and declines afterward, indicating that residual hallucinations eventually accumulate despite the safeguards.
  • The approach transfers to a newer base model (Stable Diffusion 3 medium) via LoRA fine-tuning, correcting hand rendering, subject interactions, and physical plausibility.
  • A smaller set of tightly selected samples (300 per round) outperforms larger, looser selections (1000–2000), so precision of curation matters more than volume.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the automatic preference scorers used for selection are biased toward the base model's aesthetic distribution, part of the measured gain may reflect alignment with the scorer rather than with general human taste; a large-scale independent human study on held-out prompts would separate these.
  • The distribution-based weighting implicitly treats the base model's output distribution as the reference 'normal' distribution; if the base model has systematic biases, weighting toward it could cap further improvement and limit stylistic exploration.
  • The role of Llama 3 in prompt filtering suggests an unexamined dependency: varying the LLM prompt or using a smaller model could test how much of the quality gain comes from prompt curation versus the image-selection steps.
  • Since the reconstruction loss is used throughout, pairing this filtering scheme with a preference-optimization objective (such as direct preference optimization on the sampled pairs) might extend the number of productive rounds beyond six.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes RSIDiff, a recursive self-improvement procedure for text-to-image diffusion models. At each round, the model generates synthetic images using a curated prompt set; these images are filtered by preference sampling (automatic metrics, including HPS v2), weighted by a distribution-based scheme that penalizes out-of-distribution samples relative to the base model's own output distribution, and used for fine-tuning. Experiments on Stable Diffusion v1.4 report consistent improvements over the base model and SFT across HPS v2.1, PickScore, ImageReward, and CLIP text alignment, up to round 6, followed by a decline. Ablations show each component contributes, and a qualitative study on SD3 with LoRA is included.

Significance. If the reported gains are genuine, the paper makes a useful contribution: a practical recipe for improving text-to-image diffusion models from their own generations, with a small-sample efficiency advantage over SFT and an honest characterization of the performance ceiling. The method is clearly described, and the ablations are helpful. However, the central claim that RSIDiff improves human-preference alignment is not fully supported because the main evaluation metric (HPS v2.1) is from the same family as the training-data selection metric (HPS v2), and the external validation (user study) is underpowered.

major comments (4)
  1. [Section 3.3 and Section 4.2] The preference sampling step selects synthetic samples using 'overall human preference scores' (ref [35], i.e., HPS v2), and the headline evaluation metric in Fig. 5 is HPS v2.1, a close relative of the same scorer. Because the model is fine-tuned on samples that score well under this scorer family, the reported 7.0% HPS v2.1 improvement may be inflated by reward overoptimization rather than reflecting true human preference gains. The paper does not state which exact HPS v2 checkpoint is used inside the loop, nor does it evaluate on a genuinely held-out preference signal. Please report results against a scorer that is provably outside the selection pipeline, state explicitly which metrics are used at the selection step, and provide a human-preference evaluation with sufficient participants and confidence intervals to support the claim.
  2. [Section 4.2 and Fig. 5] All quantitative results are reported as point estimates from a single run, with no error bars or significance tests. The claimed improvements (7.0% on HPS v2.1, 181.6% on ImageReward at round 6) could be within run-to-run variance. Please provide means and standard deviations over multiple seeds (or bootstrapped confidence intervals) for every metric and dataset.
  3. [Section 4.4 and Fig. 9] The SD3 study is qualitative only, and the preference sampling includes 'additional manual selection' after automated filtering, reducing 13,000 images to 1,000. This means the SD3 result is not a demonstration of the fully automatic self-evolving loop, and the manual step may bias the comparison. Please provide quantitative metrics for the SD3 experiments or explicitly discuss the role of manual selection as a limitation.
  4. [Section 4.2 and Table 1] The user study is based on 8 participants and reports only aggregate preference percentages without confidence intervals, details on the number of prompts, or the protocol. With 8 participants, the 69.0% and 59.8% percentages have wide uncertainty intervals, and the study is not powered to validate the large automated-scorer gains. Please provide the full protocol, per-participant consistency, and confidence intervals, or treat the user study as a pilot.
minor comments (6)
  1. [Figure 3] Panels (b) and (c) contain content about knowledge-graph question answering and LLM augmentation (e.g., 'SKIP KG', 'LLM Justification', 'Queryonto') that is unrelated to the preference sampling and distribution-based weighting described in the caption and text. These panels appear to be from a different paper and should be replaced with diagrams of the actual method.
  2. [Section 6] The source code link is given as 'https://open upon acceptance'; please provide a working repository URL or remove the placeholder.
  3. [Eq. (3)] There is a mismatched parenthesis in the reconstruction loss; the final ')' after x0 should be removed.
  4. [Section 4.1] The paper does not specify the sampling steps, guidance scale, or random seed used for generation, nor does it describe the exact protocol for generating 10 images per prompt; please add these details for reproducibility.
  5. [Section 3.2] The clarity and specificity filtering by Llama 3 is described only at a high level; please include the exact instructions or a representative prompt template for reproducibility.
  6. [Table 1 caption] The caption 'The percentage of user preference' is ambiguous; it should state that the percentages indicate the fraction of comparisons in which RSIDiff was preferred over the baseline, and whether ties were allowed.

Circularity Check

2 steps flagged · score 4.0 of 10

The headline HPS v2.1 gain is partly circular because preference sampling selects training data with the same HPS-v2-family scorer used for evaluation, but independent metrics and a small user study provide partial external grounding.

  1. fitted input called prediction [Section 3.3 (Preference Sampling) and Section 4.1/4.2 (Evaluation)]
    "Specifically, we employ automated metrics to evaluate the preference of the synthetic data. These metrics encompass various aspects, including the alignment between text prompts and corresponding generated images [23], aesthetic quality [28], and overall human preference scores [35]. ... HPS v2 [35] is an upgraded version of the human preference scorer [36]. In this paper, we utilize version 2.1."

    Preference sampling (Algorithm 1, Section 3.3) selects the training set using 'overall human preference scores [35]', i.e. the HPS v2 family. The headline evaluation in Section 4.2 reports gains on 'HPS v2.1', citing the same reference [35]. HPS v2 and v2.1 are not independent held-out signals: both are trained on the HPD v2 benchmark family, and the paper does not state that a different checkpoint was used inside the training loop. Thus the reported 7.0% HPS v2.1 improvement is at least in part a selection artifact of fine-tuning on data chosen to score highly under the same scorer family, not an independent confirmation of human preference.

  2. fitted input called prediction [Section 3.3 vs. Section 4.1, Text-alignment metric]
    "These metrics encompass various aspects, including the alignment between text prompts and corresponding generated images [23]. ... Text-alignment evaluates how well the generated images correspond to textual descriptions. We implement it by measuring the similarity between the CLIP image features and the corresponding text features."

    The Text-alignment evaluation metric is the same CLIP image-text similarity that is explicitly one of the automatic metrics used in preference sampling to filter the training data. Improvements on this metric therefore partly reflect selection pressure on the exact quantity being measured, so Text-alignment cannot serve as an independent validation of improved text-image alignment. It does not invalidate the other reported metrics, but it removes Text-alignment as an unbiased confirmation.

full rationale

The central RSI loop is not itself circular: the model is iteratively fine-tuned on its own generated data with prompt filtering, preference sampling, and distribution-based weighting, and the distribution weight in Eq. (5) uses the base model's outputs as a fixed reference rather than as the target being derived. The main circularity is metric overlap: the preference-sampling step in Section 3.3 selects training samples with HPS v2 family scores and CLIP text-image similarity, while the evaluation in Section 4.1/4.2 reports HPS v2.1 and CLIP-based Text-alignment as headline evidence. These are not independent assessments of the claimed improvement. However, the paper also reports gains on PickScore and ImageReward, which were not used in the stated selection metrics, and it includes a small human user study with 4,800 responses, albeit from only 8 participants and without confidence intervals. This external signal keeps the paper from being fully circular, but the strongest numeric claims are contaminated by the scorer-overlap issue. No load-bearing self-citation or imported uniqueness theorem is present; the citations to [35] are to an external scoring method, not the authors' own prior theorems.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central quantitative claim rests on several unvalidated proxy assumptions: preference scorers used for filtering, Llama 3 prompt filtering, K-means diversity, and D0 as a stable reference. The most serious is the HPS v2 selection-evaluation overlap, which inflates the headline metric. No new physical or model entities are introduced.

free parameters (5)
  • beta threshold for in-distribution weighting = 35
    Equation 5; tuned by ablation in Fig 10 on HPS test and PartiPrompts, so the chosen value is fitted to evaluation data.
  • sigma^2 decay rate for OOD weights = 2
    Equation 5; tuned by ablation in Fig 11, another evaluation-fitted hyperparameter.
  • training samples per round = 300
    Selected subset size from 5,000 generated images; chosen by comparing 100, 300, 1000, and 2000 in Fig 8, i.e., fitted to evaluation curves.
  • preference-sampling selection rule = not specified
    The paper does not state how CLIP alignment, aesthetic score, and HPS v2 are combined or thresholded to pick the 300 samples.
  • SD3 manual selection amount = 1,000 high-quality images per round
    Section 4.4 adds a manual selection step, which is a human-fitted choice and cannot be reproduced from the automated recipe.
assumptions (6)
  • domain assumption The diffusion denoising objective in Eqs 3-6 is a valid fine-tuning target for self-improvement.
    The model is assumed to improve by minimizing reconstruction error on its own filtered outputs.
  • domain assumption CLIP, aesthetic, and HPS v2 scores reflect human preference and are safe to use as training filters.
    Preference sampling in Section 3.3 relies on these proxies; the user study is too small to validate them across all rounds.
  • domain assumption Llama 3 instruction-following reliably identifies clear and specific prompts.
    The prompt filtering in Section 3.2 is not validated or quantified beyond selected examples.
  • domain assumption K-means on prompt embeddings yields a diverse and representative prompt set.
    Section 3.2 selects cluster-center prompts; no alternative diversity measure is tested.
  • domain assumption The base model's generated distribution D0 remains a valid reference for in-distribution detection in later rounds.
    Section 3.4 anchors weights to D0; improved later-round outputs that are far from D0 would be unfairly downweighted.
  • domain assumption HPS v2 can be used both to select training data and to evaluate the model without biasing the comparison.
    Section 3.3 selects with HPS v2 and Section 4.2 evaluates with HPS v2; this creates a selection-evaluation overlap.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generating on Generated: An Approach Towards Self-Evolving Diffusion Models." pith.science (2026). https://pith.science/paper/CI67QMUI

@misc{pith2026250209963,
  author       = {Pith},
  title        = {Pith review of: Generating on Generated: An Approach Towards Self-Evolving Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CI67QMUI}},
  note         = {Machine review of arXiv:2502.09963}
}
read the original abstract

Recursive Self-Improvement (RSI) enables intelligence systems to autonomously refine their capabilities. This paper explores the application of RSI in text-to-image diffusion models, addressing the challenge of training collapse caused by synthetic data. We identify two key factors contributing to this collapse: the lack of perceptual alignment and the accumulation of generative hallucinations. To mitigate these issues, we propose three strategies: (1) a prompt construction and filtering pipeline designed to facilitate the generation of perceptual aligned data, (2) a preference sampling method to identify human-preferred samples and filter out generative hallucinations, and (3) a distribution-based weighting scheme to penalize selected samples with hallucinatory errors. Our extensive experiments validate the effectiveness of these approaches.

Figures

Figures reproduced from arXiv: 2502.09963 by the authors.

Figure 1
Figure 1. We introduce RSIDiff, a novel approach that enhances the performance of diffusion models through recursive self-training. By [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Degeneration in RSI. We observe a severe domain shift and decline in image fidelity when fine-tuning diffusion models with self-generated data. The diffusion model gradually loses the ability to generate fine-grained details. performing RSI. To this end, we review the training process of text-to-image diffusion models and find that the quality of the text prompt and its corresponding generated image highly impact th… view at source ↗
Figure 3
Figure 3. Framework of RSIDiff. (a) We crawl Prompts from user-active image synthesis website and filter them based on clarity, specificity, and diversity; (b) We employ preference sampling, which utilizes automatic metrics to identify human-preferred images; (c) We use the distribution-based weighting strategy to penalize out-of-distribution samples; and (d) We fine-tune the diffusion model with the selected samples and star… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Examples generated by SD v1.4 with a simple prompt [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Quantitative Results. Performance comparison with the base model and SFT method across two datasets and 4 evalu￾ation metrics. The results show that RSIDiff significantly outper￾forms the base model and achieves consistent improvements. ws = ( 1 if d (s, D0) ≤ β exp  …
Figure 6
Figure 6. Figure 6: Qualitative Results. Examples generated by the base model (SD v1.4), SFT method, and RSIDiff at 6th round. The results illustrate RSIDiff’s superior performance in several areas: effective centering of concepts (1st row), generation of intricate details (2nd row), bett…
Figure 7
Figure 7. Figure 7: Ablation studies. We compare RSIDiff with four configurations on the HPS test and PartiPrompts datasets. The results demon￾strate that RSIDiff achieves superior recursive improvement while removing any individual strategy significantly degrades performance [PITH_FULL_…
Figure 8
Figure 8. Figure 8: Impact of sample size. We select 100, 300, 1000, and 2000 samples from 5,000 synthetic data per training round and test the impact of sample size on the HPS test dataset. 4.2. Results Quantitative Results. We first validate the effectiveness of our proposed method by c…
Figure 9
Figure 9. Figure 9: Comparison with SD3. We show the effectiveness of RSIDiff on SD3 in generating rational images (1st row), improv￾ing the detail in hand rendering (2nd row), depicting clear separa￾tion between hair and headdress (3rd row), and facilitating coher￾ent interactions betwee…
Figure 10
Figure 10. Figure 10: Ablation Study of β. We assess the effects of β under four different metrics on the HPS test and PartiPrompts datasets. Higher values of β correspond to a more permissive determination of in-distribution samples. We set β as 35 based on the overall performance [PITH_…
Figure 11
Figure 11. Figure 11: Ablation Study of σ 2 . We assess the effects of σ 2 under four different metrics on the HPS test and PartiPrompts datasets. Smaller values of σ 2 correspond to a higher penalization on out-of-distribution samples. We set σ 2 as 2.0 based on the overall performance […
Figure 12
Figure 12. Figure 12: Examples Generated in Rounds. We show examples generated from rounds 1 to 6, alongside outputs from the base model (SD v1.4). The images illustrate a gradual enhancement in visual quality [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: Qualitative Results. Examples generated by the base model (SD v1.4), SFT method, and RSIDiff. The results illustrate RSIDiff’s superior performance in several areas: generation of intricate details, effective centering of concepts, human-aligned aesthetic understandin…
Figure 14
Figure 14. Figure 14: Examples Generated by SD3 and RSIDiff. This comparison highlights that our method RSIDiff enhances the base model by achieving more coherent interactions between subjects [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: Examples Generated by SD3 and RSIDiff. This comparison highlights that our method RSIDiff enhances the base model by improving the detail in hands and feet rendering [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Examples Generated by SD3 and RSIDiff. This comparison highlights that our method RSIDiff enhances the base model by generating physically plausible subjects [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A survey of 1,250 papers organizes AI self-improvement along two axes—what is improved and loop closure—finding that demonstrated self-improvement strength tracks a verification hierarchy from formal verifiers down to...

  2. Epistemic diversity across language models mitigates knowledge collapse

    cs.LG 2025-12 reject novelty 5.0 of 10

    In repeated self-training loops on Wikitext2, ecosystems of four small language models show lower average perplexity than one, two, or sixteen models, but the paper's broader claims about monotonic optima, robustness,...

Reference graph

Works this paper leans on

44 extracted references · 24 canonical work pages · cited by 2 Pith papers

  1. [35]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  2. [1]

    The k-means algorithm: A comprehensive survey and performance evaluation

    Mohiuddin Ahmed, Raihan Seraj, and Syed Mo- hammed Shamsul Islam. The k-means algorithm: A comprehensive survey and performance evaluation. Elec- tronics, 9(8):1295, 2020. 4

    work page 2020

  3. [2]

    Self- consuming generative models go mad

    Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJe- une, Ali Siahkoohi, and Richard G Baraniuk. Self- consuming generative models go mad. arXiv preprint arXiv:2307.01850, 2023. 2, 3

    arXiv 2023

  4. [3]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science., 2(3):8, 2023. 3, 4

    work page 2023

  5. [4]

    Training diffusion models with reinforce- ment learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforce- ment learning. arXiv preprint arXiv:2305.13301, 2023. 3

    arXiv 2023

  6. [5]

    Teaching large language models to self-debug

    Xinyun Chen, Maxwell Lin, Nathanael Sch ¨arli, and Denny Zhou. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128, 2023. 3

    arXiv 2023

  7. [6]

    Self-play fine-tuning converts weak lan- guage models to strong language models

    Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak lan- guage models to strong language models. arXiv preprint arXiv:2401.01335, 2024. 2

    arXiv 2024

  8. [7]

    Emu: Enhanc- ing image generation models using photogenic needles in a haystack

    Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xi- aofang Wang, Abhimanyu Dubey, et al. Emu: Enhanc- ing image generation models using photogenic needles in a haystack. arXiv preprint arXiv:2309.15807, 2023. 3

    arXiv 2023

Show all 44 references
  1. [8]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255. Ieee, 2009. 8

    2009

  2. [9]

    Strong model collapse

    Elvis Dohmatob, Yunzhen Feng, and Julia Kempe. Strong model collapse. arXiv preprint arXiv:2410.04840, 2024. 2, 3

    2024 arXiv

  3. [10]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

  4. [11]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In ICML, 2024. 3, 4, 8

    2024

  5. [12]

    Re- inforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Moham- mad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Re- inforcement learning for fine-tuning text-to-image diffusion models. NIPS, 36, 2024. 3

    2024

  6. [13]

    Critic: Large language models can self-correct with tool-interactive cri- tiquing

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yu- jiu Yang, Nan Duan, and Weizhu Chen. Critic: Large language models can self-correct with tool-interactive cri- tiquing. arXiv preprint arXiv:2305.11738, 2023. 2

    2023 arXiv

  7. [14]

    Small language model can self-correct

    Haixia Han, Jiaqing Liang, Jie Shi, Qianyu He, and Yanghua Xiao. Small language model can self-correct. InAAAI, pages 18162–18170, 2024. 3

    2024

  8. [15]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. NIPS, 33:6840–6851, 2020. 3

    2020

  9. [16]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 8

    2021 arXiv

  10. [17]

    Analyzing and improving the image quality of stylegan

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In CVPR, pages 8110–8119,

  11. [18]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. NIPS, 36:36652–36663, 2023. 3, 6

    2023

  12. [19]

    Self-refine: Itera- tive refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hal- linan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Itera- tive refinement with self-feedback. NIPS, 36, 2024. 3

    2024

  13. [20]

    Selfcheck: Using llms to zero-shot check their own step-by-step reason- ing

    Ning Miao, Yee Whye Teh, and Tom Rainforth. Selfcheck: Using llms to zero-shot check their own step-by-step reason- ing. arXiv preprint arXiv:2308.00436, 2023. 3

    2023 arXiv

  14. [21]

    Bounded recursive self-improvement

    Eric Nivel, Kristinn R Th ´orisson, Bas R Steunebrink, Haris Dindo, Giovanni Pezzulo, Manuel Rodriguez, Carlos Hern´andez, Dimitri Ognibene, J¨urgen Schmidhuber, Ricardo Sanz, et al. Bounded recursive self-improvement. arXiv preprint arXiv:1312.6764, 2013. 1

    2013 arXiv

  15. [22]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 3

    2023 arXiv

  16. [23]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, pages 8748–8763. PMLR, 2021. 5

    2021

  17. [24]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. NIPS, 36, 2024. 3

    2024

  18. [25]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  19. [26]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2, 6

    2022

  20. [27]

    G ¨odel machines: self-referential universal problem solvers making provably optimal self- improvements

    J ¨urgen Schmidhuber. G ¨odel machines: self-referential universal problem solvers making provably optimal self- improvements. arXiv preprint cs/0309048, 2003. 1, 2

    2003 arXiv

  21. [28]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. NIPS, 35:25278–25294,

  22. [29]

    Ai models collapse when trained on recursively generated data

    Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Pa- pernot, Ross Anderson, and Yarin Gal. Ai models collapse when trained on recursively generated data. Nature, 631 (8022):755–759, 2024. 2, 3

    2024

  23. [30]

    Mastering the game of go with deep neu- ral networks and tree search

    David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrit- twieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neu- ral networks and tree search. Nature, 529(7587):484–489,

  24. [31]

    Growing recursive self-improvers

    Bas R Steunebrink, Kristinn R Th ´orisson, and J ¨urgen Schmidhuber. Growing recursive self-improvers. In ICAGI, pages 129–139. Springer, 2016. 1

    2016

  25. [32]

    A survey on self-evolution of large lan- guage models

    Zhengwei Tao, Ting-En Lin, Xiancai Chen, Hangyu Li, Yuchuan Wu, Yongbin Li, Zhi Jin, Fei Huang, Dacheng Tao, and Jingren Zhou. A survey on self-evolution of large lan- guage models. arXiv preprint arXiv:2404.14387, 2024. 1

    2024 arXiv

  26. [33]

    Diffusion model align- ment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model align- ment using direct preference optimization. In CVPR, pages 8228–8238, 2024. 3

    2024

  27. [34]

    Gener- ating sequences by learning to self-correct

    Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. Gener- ating sequences by learning to self-correct. In ICLR. 3

  28. [36]

    Human preference score: Better aligning text-to- image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hong- sheng Li. Human preference score: Better aligning text-to- image models with human preference. InICCV, pages 2096– 2105, 2023. 3, 6

    2023

  29. [37]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. NIPS, 36, 2024. 3, 6

    2024

  30. [38]

    From seed ai to technological singu- larity via recursively self-improving software.arXiv preprint arXiv:1502.06512, 2015

    Roman V Yampolskiy. From seed ai to technological singu- larity via recursively self-improving software.arXiv preprint arXiv:1502.06512, 2015. 1

    2015 arXiv

  31. [39]

    Using human feedback to fine-tune diffusion models without any reward model

    Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In CVPR, pages 8941–8951, 2024. 3

    2024

  32. [40]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 5

    2022 arXiv

  33. [41]

    Self-rewarding language models

    Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason We- ston. Self-rewarding language models. arXiv preprint arXiv:2401.10020, 2024. 2, 3

    2024 arXiv

  34. [42]

    Star: Bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. NIPS, 35: 15476–15488, 2022. 2

    2022

  35. [43]

    Opt: Open pre-trained trans- former language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained trans- former language models. arXiv preprint arXiv:2205.01068,

    arXiv

  36. [44]

    The source code can be accessed at https://open upon acceptance

    Supplementary Materials To provide a more comprehensive understanding of the method, we have included additional details in the fol- lowing sections. The source code can be accessed at https://open upon acceptance. 6.1. Prompt Set Examples This section presents partial prompt ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.