Pith. sign in

REVIEW 3 major objections 3 minor 51 references

The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data

T0 review · 3 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Recursive training on synthetic text widens gender-occupation bias before standard quality metrics show trouble.

desk verdict Fairness collapse is a plausible and cleanly framed empirical claim, but the single-run evidence and in-distribution perplexity cannot yet support the 'silent before collapse' headline; worth refereeing with re-runs required. read the letter →

arxiv 2608.04268 v1 pith:ZRYZW3UC submitted 2026-08-04 cs.CL

Irina Proskurina , Antoine Gourru , Julien Velcin This is my paper · ORCID
classification cs.CL
keywords fairnesscollapsemodelsyntheticdatagenderbiasamplificationrecursivetraininglanguagemodelsinBios
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes and tests a failure mode it calls fairness collapse: when a language model is repeatedly retrained on text it generated itself, gender-occupation associations grow stronger before conventional signs of model collapse appear. Using a balanced subset of the Bias in Bios biographies and a 0.5B-parameter model, the authors run five iterations of continued pretraining under four regimes—seeded or few-shot generation, each in iterative or recursive form—and track fairness metrics (equal-opportunity gaps, likelihood asymmetries, stereotype benchmarks) alongside standard quality metrics (perplexity, MMLU, classification accuracy). They find that the equal-opportunity gap rises from 13.18 to 19.38 under recursive seeded training while perplexity falls from 16.07 to 10.55, meaning the model gets better at predicting its training text at the same time as its demographic fairness worsens. The claim matters because synthetic data is increasingly common in web-scale corpora, and if bias amplifies silently, standard monitoring would not catch it.

What carries the argument

The load-bearing object is the recursive training loop the authors call fairness collapse: a model generates synthetic biographies (either continuing a human-written seed or following a few-shot prompt), the next model is trained on those generations, and the process repeats. The argument is carried by the aggregate Equality of Opportunity gap (EO GAP), the root-mean-square across professions of the difference in true-positive rates for male versus female biographies in a pairwise occupation-prediction task, together with signed gender-conditioned negative log-likelihood gaps and external stereotype benchmarks (CrowS-Pairs and SoFA). These fairness measures move in the direction of greater demographic asymmetry across iterations while perplexity on the training text keeps improving, which is the evidential core that separates fairness collapse from model collapse. The key comparison is between the recursive regime, where the same checkpoint is retrained and regenerates its own data, and the iterative regime, where each new model starts from the human-trained checkpoint; the recursive loop amplifies bias more strongly.

What would settle it

Repeat the recursive seeded training protocol with, say, ten random seeds per iteration and compute confidence intervals on the aggregate EO GAP; if the spread across seeds at any iteration exceeds the reported gap increase (roughly 13 to 19 percentage points by iteration 5), then fairness collapse is not distinguishable from noise. A second check: train the same model on human-written data alone with the same number of repeated runs and show the EO GAP stays within a narrower band than the synthetic conditions.

Watch

Extended reading notes

Core claim

The central claim, stated in the introduction and supported by Tables 1 and 2, is that fairness degradation emerges before severe language-model degradation. In controlled continued-pretraining experiments on the Bias in Bios corpus, exposure to synthetic data consistently amplifies gender-occupation bias—measured by the aggregate equal-opportunity gap rising from 13.18 to 19.38 in recursive seeded training—while perplexity simultaneously improves from 16.07 to 10.55 and MMLU accuracy declines only gradually, from 42.14 to 32.08. The same pattern appears in the few-shot variant and in the iterative regime, though weaker and less monotonic, with SOFA bias scores increasing from 0.509 to 1.020 under recursive seeded training. The authors interpret this as evidence of a distinct early-stage failure mode, fairness collapse, in which biased associations are progressively reinforced by recursive training on model-generated text, independent of and earlier than classical model collapse.

Load-bearing premise

The load-bearing premise is that the 2–7 percentage point movements in the equal-opportunity gap are genuine signal and not run-to-run noise; each training condition appears to have been run once, with no random seeds or significance tests reported, so if single-run variance is that large the claimed ordering could disappear.

Editorial extensions

If this is right

  • Standard language-modeling monitoring—perplexity, MMLU, downstream accuracy—can improve or stay flat while demographic bias is growing, so these metrics are not sufficient early-warning indicators.
  • Recursive or self-consuming training loops are more dangerous for fairness than iterative regeneration from a human checkpoint, because cumulative parameter updates compound the bias.
  • Synthetic data contamination of real training corpora may silently reinforce gender-occupation stereotypes before any model-collapse symptoms are visible.
  • Fairness metrics such as EO GAP and likelihood asymmetry should be tracked alongside quality metrics when training on data that may contain model-generated text.
  • The amplification is directional: female-associated professions like nurse drift toward more negative EO gaps and male-associated professions like professor toward positive ones, showing existing corpus associations are strengthened rather than new noise introduced.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If fairness collapse generalizes beyond Bias in Bios, a testable prediction is that synthetic-data pipelines trained on other stereotype-laden domains (race, age, language) will show measurable bias growth before distributional collapse; the paper does not test this.
  • The results imply that data curation for recursive training should include fairness filtering or balanced regeneration, not just quality or diversity filtering—an extension the authors do not pursue.
  • A direct experimental extension would be to check whether retaining a fixed fraction of human-written data, which prior work shows slows model collapse, also slows or halts fairness collapse; the paper does not run this condition.
  • Because each condition appears to be run once, the quantitative gaps in the tables should be treated as indicative; re-running with multiple seeds is the obvious next step before using these numbers as benchmarks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper studies whether continued pretraining on synthetic text amplifies social biases before standard measures of model collapse degrade. Using Bias in Bios, the authors construct balanced human-written data, generate synthetic biographies by seeded continuation and few-shot prompting with Qwen2.5-0.5B, and train under iterative and recursive feedback regimes. They report EO GAP, NLL gaps, CrowS-Pairs, SoFA, MMLU, and training-set perplexity across five iterations. The central claim is that fairness degradation emerges before severe language-modeling degradation, and that this 'fairness collapse' is a distinct early failure mode. The paper includes controlled comparisons, external fairness benchmarks, held-out evaluation, and anonymous code.

Significance. If the central claim were established, the paper would make a useful contribution: it identifies a failure mode that standard perplexity and accuracy monitoring would miss, and it connects the model-collapse literature with fairness evaluation. The controlled setup, the use of external benchmarks (CrowS-Pairs, SoFA, MMLU, held-out Bias in Bios test data) rather than fitted parameters, and the clear separation of iterative versus recursive regimes are strengths. The core evidence, however, is currently too fragile to support the strength of the claims: the tables report single runs without error bars, the few-shot recursive trajectory is nonmonotonic and ends at baseline, and the perplexity metric is computed on the training distribution, so it cannot establish that standard metrics are blind to the degradation. The phenomenon may well be real, but the manuscript needs additional statistical support and a revision of the central claims to match the evidence.

major comments (3)
  1. [§5.3, Tables 1–2] All conditions appear to be run once, with no random seeds, standard deviations, or significance tests. The EO GAP changes that drive the paper's conclusion are 2–7 percentage points, and some trajectories are nonmonotonic. In particular, Table 1 (few-shot recursive) shows EO GAP rising from 13.18 at Iteration 0 to 16.36 at Iteration 1 and 18.82 at Iteration 2, then falling back to 13.45 at Iteration 5, essentially the baseline value. This directly contradicts the 'consistently amplifies' claim in Section 1 and Section 5.3. The authors should report multiple seeds with confidence intervals or at minimum show that a single-run variance estimate cannot explain the ordering they claim.
  2. [Tables 1–2, captions; §5.3] The perplexity values are computed on the training data used at the corresponding iteration, as the captions state. For recursive training this is model-generated text from the model being trained, so a decreasing training-set PPL from 16.07 to 10.55 is the expected memorization effect and cannot be cited as evidence that standard language-modeling metrics fail to detect the change. The claim that bias increases 'silently before strong indicators of model collapse become apparent' requires a held-out metric, such as perplexity on human-written text or on a fixed validation split, reported across iterations for both synthetic and human-only regimes.
  3. [§5.3, Table 1 vs. Table 15] The temporal claim that fairness degradation precedes classical model collapse is not supported by the reported aggregate trajectories. In Table 1, MMLU drops from 42.14 to 32.08 under seeded recursive training and from 42.14 to 24.63 under few-shot recursive training by Iteration 5, which is a substantial general-capability decline, not a stable baseline. Moreover, the human-only baseline in Table 15 also shows accuracy declining from 80.05 to 74.28 and NLL GAP shifting from -0.093 to -0.103. Without confidence intervals, it is unclear whether the synthetic-data regimes produce effects beyond the human-only training drift or beyond run-to-run variation. The authors should define the 'before' comparison quantitatively, for example, by testing whether fairness metrics change significantly at Iterations 1–2 while held-out language-modeling metrics do not.
minor comments (3)
  1. [Throughout] There are several typographical artifacts, including 'V olume' in references, 'T oken Length' in Figure 5, and 'F oods' in Table 14; these should be corrected.
  2. [Figure 3 caption] The caption is grammatically overloaded; splitting it into separate labeled panels would make the left, center, and right subplots easier to read.
  3. [§5.1] The choice of seed length 30 and temperature 0.9 is motivated by the generated-data evaluation, but the paper does not report whether the qualitative fairness-collapse results are sensitive to these generation hyperparameters; a brief sensitivity discussion would strengthen the claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the fairness-collapse finding is an empirical observation evaluated on external benchmarks, not a consequence of the paper's definitions or fitted quantities.

full rationale

The paper's central claim is that fairness degradation emerges before standard model-collapse indicators under recursive synthetic-data training. This is an empirical claim supported by controlled continued-pretraining runs, with fairness measured on held-out human-written Bias-in-Bios test biographies (EO GAP, NLL-GAP) and on external CrowS-Pairs and SoFA benchmarks. None of these fairness metrics is used as a training objective, and none is fitted to produce the observed outcome; the training and generation hyperparameters are fixed before the fairness evaluation, and the generation hyperparameters (seed length 30, T=0.9) were selected on text-similarity and lexical-diversity criteria, not on fairness outcomes. The term 'fairness collapse' is introduced as a label for the observed pattern rather than as a definition that presupposes the conclusion. The main methodological concern is that perplexity is computed on the training data used at the corresponding iteration, so the reported PPL improvement is partly a memorization effect; this weakens the comparison to conventional collapse metrics but is a measurement-validity issue, not a circular derivation. The paper does not rely on load-bearing self-citations: no reference by the present authors is invoked to justify the central claim. The derivation chain is therefore self-contained with respect to circularity, though the empirical strength of the finding depends on replication and on held-out perplexity evaluation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper names a phenomenon, 'fairness collapse,' but introduces no physical, architectural, or formal entity; the construct is a summary label for observed metric drift. The most important numerical choices are the data-generation hyperparameters and the learning rate, all selected by the authors rather than derived from theory.

free parameters (4)
  • Seed length k = 30 tokens
    All bias experiments use seeded generation with k=30, selected as the best trade-off between similarity to human text and lexical diversity in Section 4.3.
  • Decoding temperature T = 0.9
    All bias experiments use T=0.9, selected from the same generated-data evaluation; the bias results are only reported for this configuration.
  • Learning rate = 5e-5
    Selected by grid search over {1e-5, 2e-5, 5e-5} on the human corpus, then used for all continued-pretraining runs.
  • Few-shot count K = 3
    Few-shot generation uses K=3 human-written examples per profession; this is a design choice not varied in the bias experiments.
assumptions (4)
  • domain assumption Bias in Bios gender labels and profession pairs are accurate enough for measuring fairness drift.
    All EO GAP and NLL gap calculations depend on the dataset's gender annotations and the pairwise profession task.
  • domain assumption The pairwise occupation-prediction task with fixed profession pairs is a valid measure of occupational bias.
    EO GAP is computed from choices between the gold profession and a fixed paired alternative in Table 4; other pairings would likely change the gaps.
  • domain assumption Qwen2.5-0.5B and five iterations of continued pretraining are representative of synthetic-data contamination dynamics.
    The paper acknowledges in Limitations that larger models may behave differently and that only one biased attribute and domain is tested.
  • domain assumption MTLD and embedding Wasserstein distances characterize text quality relevant to generation settings.
    These metrics are used to select seed length and temperature, which in turn determine the synthetic data used for training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data." pith.science (2026). https://pith.science/paper/ZRYZW3UC

@misc{pith2026260804268,
  author       = {Pith},
  title        = {Pith review of: The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZRYZW3UC}},
  note         = {Machine review of arXiv:2608.04268}
}
read the original abstract

Generative models trained on artificially generated data have been shown to exhibit model collapse, resulting in significant performance degradation. As synthetic content increasingly contaminates the training corpora of language models, this raises critical concerns about the use of open data in continued pretraining. Although previous work has demonstrated model collapse in language models, it remains unclear whether exposure to synthetic data amplifies or attenuates the social biases already present in pretrained models. Because language models are known to reproduce and amplify demographic stereotypes, recursive training on self-generated data may create a self-reinforcing feedback loop in which biased associations become progressively stronger across generations. We call this hypothesized phenomenon fairness collapse. In this work, we construct controlled training regimes in which models are repeatedly trained on synthetic data using the Bias in Bios dataset. Across experiments, we observe a consistent and concerning pattern: fairness degradation emerges before substantial degradation is reflected by standard language-modeling metrics. This result highlights a critical risk associated with synthetic data contamination in language model training: bias can increase silently before strong indicators of model collapse become apparent.

Figures

Figures reproduced from arXiv: 2608.04268 by the authors.

Figure 1
Figure 1. Fairness collapse as a recursive bias self-amplification process: model-generated text is reintroduced into [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the three controlled continued-pretraining regimes used in our experiments. We begin with [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Lexical diversity (left), relative continuation Wasserstein distance ( [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Trajectory of profession-level fairness and [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Token length distribution of biographies in the balanced [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Prompt used for pairwise occupation evaluation. Given a biography [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Gender-conditioned lexical-diversity trajectories for selected professions across recursive training iterations. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 17 canonical work pages

  1. [1]

    Collapsed Language Models Promote Fairness

    Collapsed Language Models Promote Fairness , author=. arXiv preprint arXiv:2410.04472 , year=

    work page Pith review arXiv

  2. [2]

    Behavior research methods , volume=

    MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment , author=. Behavior research methods , volume=. 2010 , publisher=

    work page 2010

  3. [3]

    Proceedings of the International Conference on Learning Representations (ICLR) , year=

    Measuring Massive Multitask Language Understanding , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=

  4. [4]

    arXiv preprint arXiv:2604.26965 , year=

    The Impact of AI-Generated Text on the Internet , author=. arXiv preprint arXiv:2604.26965 , year=

    arXiv

  5. [5]

    Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , volume=

    Gender, race, and intersectional bias in resume screening via language model retrieval , author=. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , volume=

    work page

  6. [6]

    J ob F air: A Framework for Benchmarking Gender Hiring Bias in Large Language Models

    Wang, Ze and Wu, Zekun and Guan, Xin and Thaler, Michael and Koshiyama, Adriano and Lu, Skylar and Beepath, Sachin and Ertekin, Ediz and Perez-Ortiz, Maria. J ob F air: A Framework for Benchmarking Gender Hiring Bias in Large Language Models. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/2024.findings-emnlp.184

    doi:10.18653/v1/2024.findings-emnlp.184 2024

  7. [7]

    NAACL , year=

    Gender bias in coreference resolution: Evaluation and debiasing methods , author=. NAACL , year=

    work page

  8. [8]

    EMNLP , year=

    Towards controllable bias in language generation , author=. EMNLP , year=

    work page

Show all 51 references
  1. [9]

    IEEE Transactions on Knowledge and Data Engineering , year=

    Mitigating gender bias in natural language processing: Literature review , author=. IEEE Transactions on Knowledge and Data Engineering , year=

  2. [10]

    The Woman Worked as a Babysitter: On Biases in Language Generation

    Sheng, Emily and Chang, Kai-Wei and Natarajan, Premkumar and Peng, Nanyun. The Woman Worked as a Babysitter: On Biases in Language Generation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on N...

    2019 doi

  3. [11]

    Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization , pages=

    The silicon ceiling: Auditing gpt’s race and gender biases in hiring , author=. Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization , pages=

  4. [12]

    proceedings of the Conference on Fairness, Accountability, and Transparency , pages=

    Bias in bios: A case study of semantic representation bias in a high-stakes setting , author=. proceedings of the Conference on Fairness, Accountability, and Transparency , pages=

  5. [13]

    ACL , year=

    Measuring and reducing gendered correlations in pre-trained models , author=. ACL , year=

  6. [14]

    PNAS nexus , volume=

    Measuring gender and racial biases in large language models: Intersectional evidence from automated resume evaluation , author=. PNAS nexus , volume=. 2025 , publisher=

    2025

  7. [15]

    arXiv preprint arXiv:1910.01244 , year=

    Linking artificial and human neural representations of language , author=. arXiv preprint arXiv:1910.01244 , year=

    1910 arXiv

  8. [16]

    arXiv preprint arXiv:2402.07043 , year=

    A tale of tails: Model collapse as a change of scaling laws , author=. arXiv preprint arXiv:2402.07043 , year=

    arXiv

  9. [17]

    arXiv preprint arXiv:2303.01255 , year=

    Combining generative artificial intelligence (AI) and the Internet: Heading towards evolution or degradation? , author=. arXiv preprint arXiv:2303.01255 , year=

    arXiv

  10. [18]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Will large-scale generative models corrupt future datasets? , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  11. [19]

    L a C o: Large Language Model Pruning via Layer Collapse

    Yang, Yifei and Cao, Zouying and Zhao, Hai. L a C o: Large Language Model Pruning via Layer Collapse. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/2024.findings-emnlp.372

    2024 doi

  12. [20]

    Proceedings of the National Academy of Sciences , volume=

    Prevalence of neural collapse during the terminal phase of deep learning training , author=. Proceedings of the National Academy of Sciences , volume=. 2020 , publisher=

    2020

  13. [21]

    arXiv preprint arXiv:2112.04359 , year=

    Ethical and social risks of harm from language models , author=. arXiv preprint arXiv:2112.04359 , year=

    arXiv

  14. [22]

    Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages=

    On the dangers of stochastic parrots: Can language models be too big? , author=. Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages=

    2021

  15. [23]

    International Conference on Learning Representations , volume=

    Collapsed language models promote fairness , author=. International Conference on Learning Representations , volume=

  16. [24]

    What ' s in a Name? R educing Bias in Bios without Access to Protected Attributes

    Romanov, Alexey and De-Arteaga, Maria and Wallach, Hanna and Chayes, Jennifer and Borgs, Christian and Chouldechova, Alexandra and Geyik, Sahin and Kenthapadi, Krishnaram and Rumshisky, Anna and Kalai, Adam. What ' s in a Name? R educing Bias in Bios without Access to Protecte...

    2019 doi

  17. [25]

    Towards Debiasing Sentence Representations

    Liang, Paul Pu and Li, Irene Mengze and Zheng, Emily and Lim, Yao Chong and Salakhutdinov, Ruslan and Morency, Louis-Philippe. Towards Debiasing Sentence Representations. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.1865...

    2020 doi

  18. [26]

    Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection

    Ravfogel, Shauli and Elazar, Yanai and Gonen, Hila and Twiton, Michael and Goldberg, Yoav. Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/...

    2020 doi

  19. [27]

    F air S teer: Inference Time Debiasing for LLM s with Dynamic Activation Steering

    Li, Yichen and Fan, Zhiting and Chen, Ruizhe and Gai, Xiaotang and Gong, Luqi and Zhang, Yan and Liu, Zuozhu. F air S teer: Inference Time Debiasing for LLM s with Dynamic Activation Steering. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18...

    2025 doi

  20. [28]

    Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP

    Schick, Timo and Udupa, Sahana and Sch. Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP. Transactions of the Association for Computational Linguistics. 2021. doi:10.1162/tacl_a_00434

    2021 doi

  21. [29]

    Social Bias Probing: Fairness Benchmarking for Language Models

    Marchiori Manerba, Marta and Stanczak, Karolina and Guidotti, Riccardo and Augenstein, Isabelle. Social Bias Probing: Fairness Benchmarking for Language Models. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.e...

    2024 doi

  22. [30]

    S tereo S et: Measuring stereotypical bias in pretrained language models

    Nadeem, Moin and Bethke, Anna and Reddy, Siva. S tereo S et: Measuring stereotypical bias in pretrained language models. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Proc...

    2021 doi

  23. [31]

    C row S -Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models

    Nangia, Nikita and Vania, Clara and Bhalerao, Rasika and Bowman, Samuel R. C row S -Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18...

    2020 doi

  24. [32]

    arXiv preprint arXiv:2404.01413 , year=

    Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data , author=. arXiv preprint arXiv:2404.01413 , year=

    arXiv

  25. [33]

    arXiv preprint arXiv:2412.14689 , year=

    How to synthesize text data without model collapse? , author=. arXiv preprint arXiv:2412.14689 , year=

    arXiv

  26. [34]

    , author=

    The Matthew effect in science: The reward and communication systems of science are considered. , author=. Science , volume=. 1968 , publisher=

    1968

  27. [35]

    arXiv preprint arXiv:2410.16713 , year=

    Collapse or thrive? perils and promises of synthetic data in a self-generating world , author=. arXiv preprint arXiv:2410.16713 , year=

    arXiv

  28. [36]

    Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus

    Dodge, Jesse and Sap, Maarten and Marasovi \'c , Ana and Agnew, William and Ilharco, Gabriel and Groeneveld, Dirk and Mitchell, Margaret and Gardner, Matt. Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus. Proceedings of the 2021 Conference ...

    2021 doi

  29. [37]

    arXiv preprint arXiv:2309.11998 , year=

    Lmsys-chat-1m: A large-scale real-world llm conversation dataset , author=. arXiv preprint arXiv:2309.11998 , year=

    arXiv

  30. [38]

    OpenWebText Corpus , author=

  31. [39]

    arXiv preprint arXiv:2001.08361 , year=

    Scaling laws for neural language models , author=. arXiv preprint arXiv:2001.08361 , year=

    2001 arXiv

  32. [40]

    POT Python Optimal Transport (version 0.9.5) , url =

    Flamary, R. POT Python Optimal Transport (version 0.9.5) , url =

  33. [41]

    arXiv preprint arXiv:2502.13595 , year=

    Mmteb: Massive multilingual text embedding benchmark , author=. arXiv preprint arXiv:2502.13595 , year=

  34. [42]

    arXiv preprint arXiv:2305.17493 , year=

    The curse of recursion: Training on generated data makes models forget , author=. arXiv preprint arXiv:2305.17493 , year=

    arXiv

  35. [43]

    arXiv preprint arXiv:2404.05090 , year=

    How bad is training on synthetic data? a statistical analysis of language model collapse , author=. arXiv preprint arXiv:2404.05090 , year=

    arXiv

  36. [44]

    The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text

    Guo, Yanzhu and Shang, Guokan and Vazirgiannis, Michalis and Clavel, Chlo \'e. The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text. Findings of the Association for Computational Linguistics: NAACL 2024. 2024. doi:10.18653/v1/2024.findings-naacl.228

    2024 doi

  37. [45]

    International Conference on Learning Representations , volume=

    Self-consuming generative models go mad , author=. International Conference on Learning Representations , volume=

  38. [46]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

  39. [47]

    arXiv preprint arXiv:2305.17333 , year=

    Weak-to-Strong Generalization: Teaching Large Language Models to Generate Synthetic Data , author=. arXiv preprint arXiv:2305.17333 , year=

    arXiv

  40. [48]

    Advances in neural information processing systems , volume=

    Assessing social and intersectional biases in contextualized word representations , author=. Advances in neural information processing systems , volume=

  41. [49]

    keynote at neurips , author=

    The trouble with bias. keynote at neurips , author=

  42. [50]

    Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods

    Zhao, Jieyu and Wang, Tianlu and Yatskar, Mark and Ordonez, Vicente and Chang, Kai-Wei. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: ...

    2018 doi

  43. [51]

    Advances in neural information processing systems , volume=

    Man is to computer programmer as woman is to homemaker? debiasing word embeddings , author=. Advances in neural information processing systems , volume=

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.