{"id":"acb3dc87-6dbe-4f68-83c8-d2371638b1ad","arxiv_id":"2504.14125","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"User prompts on CivitAI became more lexically repetitive over seven months, and higher prompt similarity was associated with lower visual diversity in generated images.","lead":"This paper tracks over six million text-to-image prompts from the CivitAI community over seven months and finds that users increasingly reuse the same formulaic phrases, while the images they generate become more similar. The study is a large-scale, real-world look at how community habits, not just model biases, shape what AI image generators produce.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Untested confound: high token-overlap prompt clusters share LoRA/checkpoint tags, so §7.2's r=0.33 image-similarity correlation may reflect shared generation settings, not linguistic repetition.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: shared model configuration is a plausible alternative explanation for the correlation between prompt similarity and image similarity. My read strengthens this concern with concrete evidence from the paper itself—the top prompt clusters and 4-gram lists contain explicit LoRA and checkpoint-specific tokens, and the temporal break is tied to Pony Diffusion. Because the central causal claim depends on isolating language from generation settings, and because the dataset metadata appears to include model information, the required stratified analysis is feasible. The descriptive contribution—homogenization of prompt language over time, the dominance of consistent repeaters, and the stability of semantic topics—is solid and supported by multiple metrics and large sample sizes, so no rejection is warranted. The illustrative controlled images in Figures 16 and 17 provide partial support but are not systematic tests of the correlation. The correct remedy is to require the confound-controlled correlation as a condition of acceptance; therefore the reader's CONDITIONAL verdict stands unchanged.","tokens_in":28122,"tokens_out":5120,"duration_ms":45571,"concrete_test":"Recompute §7.2's cluster-level image similarities after matching, within each cluster, on the exact Civiverse model checkpoint and LoRA identifiers (and sampler/CFG scale when present in metadata). Stratify the data into fixed-configuration groups, recompute the Pearson r between prompt token similarity and mean pairwise image-embedding similarity inside each stratum, then pool the within-stratum r's. If the pooled r collapses toward 0 while the between-stratum variation explains the original r=0.33 (and the r=0.43 text-similarity coefficient), the linguistic-repetition driver is not identified; if a positive r remains within fixed configs, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in §7.2—that clusters of prompts with ≥0.8 token overlap produce more visually homogeneous images, so 'linguistic repetition reinforces less diverse representations'—is not identified relative to generation configuration. Civiverse images were generated with different Stable Diffusion checkpoints, LoRA modules, samplers, CFG scales, and seeds, and the paper does not condition on any of these when computing cluster-level image similarities. This matters because the prompt clusters are built from tokens, and the dominant clusters explicitly contain the confound: Table 6 lists clusters with LoRA references such as lora_gothic_outfit06, and Appendix C.2 shows the emergence of 4-grams like 'style sdxllorapony diffusion v6'. The February 2024 shift in lexical and visual diversity is attributed to Pony Diffusion tagging. Users who share a prompt template typically also share the checkpoint/LoRA that makes that template work, so the observed r=0.33 between token similarity and image similarity could be driven by shared model weights rather than by lexical repetition per se. The same confound afflicts the Vendi-score time series: the post-February 2024 drop coincides with Pony adoption, so it cannot separate language from model. The Discussion acknowledges 'model checkpoints, seed values, and hyperparameters' without testing them, so the paper's headline causal wording rests on an uncontrolled correlation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper analyzes over six million prompts from the Civiverse dataset on CivitAI across seven months to study how prompt language evolves and whether it relates to visual diversity. The authors categorize users into consistent repeaters, occasional repeaters, and non-repeaters; compute lexical diversity metrics (TTR, ENW, SRS, CR); track semantic topics via MiniLM embeddings with HDBSCAN/UMAP; and measure visual diversity with Vendi scores. Their main reported findings are that prompt language becomes increasingly homogenized over time, that repeated prompts make up 40–50% of submissions, that semantic topic diversity remains relatively stable, and that higher token overlap in prompts correlates with higher image embedding similarity (r = 0.33, R² = 0.18), which they interpret as linguistic repetition reinforcing visual homogenization.","tokens_in":28389,"tokens_out":3097,"duration_ms":30073,"significance":"If the central claim held, this would be a valuable contribution to the sociotechnical study of text-to-image systems, showing that user behavior—not only model training data—can drive output homogenization. The paper's strengths include the large-scale real-world dataset, the use of multiple complementary lexical diversity metrics, the fixed-size sampling robustness checks for lexical trends, and the transparent description of the pipeline. The public Civiverse dataset also enables further verification. However, the headline claim rests on an uncontrolled observational correlation: image similarity is measured across images generated with different checkpoints, LoRA modules, samplers, CFG scales, and seeds, and the paper does not condition on any of these factors. The temporal post-February 2024 decline also coincides with the adoption of the Pony Diffusion model, so the language–visual diversity link is not isolated from model change. These issues make the central causal-style conclusion currently under-supported, though the descriptive lexical trends appear robust.","major_comments":[{"comment":"The central claim that lexical repetition in prompts drives visual homogeneity is not identified from generation configuration. Civiverse images were generated with different Stable Diffusion checkpoints, LoRA modules, samplers, CFG scales, and seeds, yet the cluster-level analysis in §7.2 correlates prompt token overlap with image embedding similarity without conditioning on any of these settings. This confound is concrete, not hypothetical: the dominant prompt clusters in Table 6 contain LoRA references such as lora_gothic_outfit06, and Appendix C.2 shows the emergence of 4-grams like 'style sdxllorapony diffusion v6'. Users who share a prompt template typically also share the checkpoint or LoRA that makes that template work, so the observed r = 0.33 could reflect shared model weights rather than linguistic repetition per se. The Discussion (§9) acknowledges 'model checkpoints, seed values, and hyperparameters' but does not test them. To support the paper's wording, the analysis should either stratify by generation settings or show that the token–image similarity relationship persists within fixed checkpoint/LoRA/sampler configurations.","section":"§7.2 and §9"},{"comment":"The statistical evidence for the core correlation is weaker than the text suggests. The reported r = 0.33 with R² = 0.18 means only about 18% of variance in image similarity is explained by token similarity, and no confidence intervals or bootstrap estimates are reported. Appendix E shows the cluster-size-weighted R² drops to 0.145 for token similarity, which is a meaningful sensitivity result that is not discussed in the main text. Additionally, the sentence in Appendix E stating that the weighted regression using identical tokens had R² = 0.962 appears to be a typo, presumably 0.0962 or similar; this should be corrected. The paper should report uncertainty intervals and discuss the modest explained variance when framing the result as a 'clear correlation'.","section":"§7.2 and Appendix E"},{"comment":"The temporal alignment between the post-February 2024 decline in lexical diversity and the decline in Vendi scores is presented as suggestive evidence of a language–visual diversity link, but this period coincides with the widespread adoption of Pony Diffusion XL, as the paper itself notes in §5.2. The Vendi score time series therefore cannot separate the effect of changing prompt language from the effect of changing underlying models. A minimal control would be to compute Vendi scores separately for images generated with the same checkpoint before and after the shift, or to restrict the time-series comparison to a stable model cohort. Without such a control, the claimed correspondence between lexical and visual diversity over time remains confounded.","section":"§5.2 and §7.1"},{"comment":"The user categories are defined from duplicate-submission behavior, and the same categories are then used to characterize lexical diversity of unique prompts in §5.3, which risks a form of circularity. For example, consistent repeaters are users who resubmit identical prompts, so it is not surprising that their unique prompts share more formulaic structure. This does not invalidate the descriptive finding, but the paper should clarify that the category-based lexical comparisons are descriptive properties of behaviorally defined groups rather than independent evidence about linguistic experimentation.","section":"§4 and §5.3"}],"minor_comments":[{"comment":"The sentence beginning 'Additionally, categorized users into three distinct groups' is missing a subject and should read 'Additionally, we categorized users into three distinct groups.'","section":"§4"},{"comment":"The description of Vendi score sampling says '10,000 prompts and their corresponding images from each user category, consistent, occasional, and non-repeaters, per month, resulting in a total of 30,000 monthly samples'; the phrase 'and their corresponding images' should be clarified because images and prompts are not one-to-one in the duplicate-inclusive dataset, and it is unclear whether the 10,000 are sampled from unique prompts or from all prompts.","section":"§7.1"},{"comment":"The topic modeling paragraph states that GPT-4o proposes labels based on c-TF-IDF keywords, but no details are given about the prompt template, number of labels, or validation of the labels; adding this information would improve reproducibility.","section":"§6.1"},{"comment":"In Table 25, the 4-gram 'style sdxllorapony diffusion v6' appears with percentages that seem inconsistent (e.g., 36.85% in April 2024 while adjacent rows show 38.32% and 32.12%); these values should be checked for internal consistency.","section":"Appendix C.2, Table 25"},{"comment":"The paper does not state whether analysis code or computed metric values are available; providing a reproducibility link or specifying data/software availability would strengthen the contribution.","section":"General"},{"comment":"The captions for Figures 8a and 8b refer to 'token-based' and 'semantic-based' image similarity, but the text also discusses CLIP text embedding similarity; the captions should be expanded to indicate which embedding spaces are compared in each panel.","section":"Figure 8"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid descriptive study of prompt evolution on CivitAI, but the central causal-style claim about linguistic repetition driving visual homogenization is not yet established because generation settings are not controlled. The confound is concrete—LoRA and checkpoint names appear inside the very prompt clusters that drive the correlation. I would be willing to reconsider after the authors add stratified or conditional analyses, report confidence intervals, and temper the language to match the observational design. I do not see grounds for rejection, since the lexical homogenization findings are well supported and the visual-diversity question is addressable with additional analyses."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The descriptive finding is real: over seven months, prompts on CivitAI became measurably more lexically homogeneous, and the trend survives a fixed-size sampling check. The headline causal claim — linguistic repetition drives visual homogeneity — is not established, because the analysis never separates shared prompt language from shared generation settings.\n\nThe paper does something genuinely useful. It takes six million prompts from Civiverse and tracks lexical diversity over time with TTR, self-repetition score, compression ratio, and effective number of words, plus a MinHash near-duplicate analysis. The user categorization into consistent, occasional, and non-repeaters is a nice addition, and the finding that 47% of users account for 72% of prompts is notable. The temporal break after February 2024, coinciding with Pony Diffusion adoption, is a plausible and well-documented explanation for the lexical shift. For the descriptive claims, the evidence is solid.\n\nThe soft spot is the bridge from language to images. The cluster-level correlation between token similarity and image embedding similarity (r=0.33, R^2=0.18) does not control for model checkpoint, LoRA modules, sampler, CFG scale, or seed. That matters because the clusters are built from tokens that literally include LoRA identifiers and Pony Diffusion tags — the 4-gram 'style sdxllorapony diffusion v6' appears in the top patterns, and Table 6 lists clusters with 'lora_gothic_outfit06'. Users who share a prompt template usually share the model and LoRA that makes that template work. So the correlation could be driven by shared weights, not by lexical overlap per se. The Vendi-score time series has the same problem: the post-February drop coincides with Pony adoption, so it cannot separate language from model. The authors acknowledge checkpoints and hyperparameters in the Discussion but do not test them. That is the difference between a solid descriptive study and a supported causal claim.\n\nThere are minor numeric inconsistencies between Section 4 and Appendix B: the user-category prompt totals in Table 2 sum to 2,776,218 against the stated 2,747,380 unique prompts, and the non-repeater user count differs between Table 2 (20,756) and Table 9 (21,835). Fixable, but the paper should be tidied.\n\nWho is this for: HCI and generative-AI researchers interested in how community norms shape prompting behavior. It deserves serious peer review because the descriptive part is a useful large-scale contribution, but the causal framing needs to be softened or, better, the confound addressed with a matched or controlled analysis.\n\nRecommendation: send it to review; the reviewers should push hard on generation settings as a competing explanation.","headline":"Solid large-scale evidence of prompt homogenization on CivitAI, but the visual-diversity link is confounded by shared generation settings and needs a controlled analysis.","tokens_in":28900,"tokens_out":4218,"would_cite":true,"duration_ms":33541,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Repeated prompt language measurably shrinks visual diversity in AI image generation.","keywords":["text-to-image generation","prompt engineering","visual diversity","lexical diversity","Vendi score","user repetition","community tags","homogenization"],"falsifier":"Regenerate the images in the high- and moderate-overlap prompt clusters while holding model version, style add-on modules, random seed, sampler, and guidance scale fixed across all clusters; if the correlation between token overlap and image-embedding similarity falls to near zero, the claim that linguistic repetition drives visual uniformity fails.","tokens_in":27916,"feed_emoji":"🎨","tokens_out":7318,"duration_ms":64739,"temperature":0.7,"pith_summary":"An analysis of over six million text-to-image prompts collected over seven months finds that the language users type is steadily narrowing, and that narrowing shows up in the images themselves. The paper classifies users as consistent repeaters, occasional repeaters, and non-repeaters, and shows that a small group of consistent repeaters produces most prompts, with exact duplicates making up 40-50% of submissions. It then measures lexical diversity with standard text metrics and visual diversity with Vendi scores, a count of effectively distinct images, and reports that higher lexical overlap in prompts correlates with higher image similarity. The central claim is that linguistic repetition is itself a measurable driver of visual homogenization, distinct from biases baked into model training data. A sympathetic reader would care because it suggests user behaviour and community norms, not just model weights, determine how creatively inclusive AI-generated imagery becomes.","feed_headline":"Repeated prompts narrow the visual range of AI images","feed_subtitle":"Seven months of six million prompts show token overlap tracks image sameness.","key_machinery":"The central mechanism is token overlap: the fraction of shared vocabulary between two prompts is treated as the operating cause of visual similarity. To measure it, the paper truncates prompts to 20 tokens, applies MinHash-based Jaccard similarity to form high- and moderate-overlap clusters, and compares the mean pairwise cosine similarity of the corresponding image embeddings. Visual diversity itself is measured with the Vendi score, the effective number of distinct images in a set, which combines how many clusters of images exist and how evenly they are populated. The user taxonomy of consistent repeaters, occasional repeaters, and non-repeaters does the explanatory work of identifying who produces the repeated language.","core_discovery":"On its own terms, the paper establishes three linked facts. First, over seven months the platform's prompt vocabulary contracts: type-token ratio and effective vocabulary fall, compression rises, self-repetition rises, and high-similarity near-duplicates grow from about 68-73% to over 83% of submissions. Second, this contraction is driven largely by consistent repeaters, users who resubmit identical prompts, who are about 47% of users but contribute about 72% of prompts and over three-quarters of near-duplicate clusters. Third, textual overlap predicts visual sameness: prompt clusters sharing at least 16 of 20 tokens yield more similar image embeddings, and Vendi scores of generated images decline as lexical diversity falls after February 2024. The paper reads these facts as evidence that linguistic repetition reinforces less diverse representations, adding a user-driven layer of bias on top of model training bias.","pith_inferences":["A decisive confound remains untested: high-overlap prompt clusters may share model versions, style add-on modules, seeds, samplers, or guidance scales, so a controlled regeneration experiment holding those fixed is the direct way to confirm the language effect.","The same repetition-to-homogenization loop likely occurs in other generative systems, including LLM writing tools, where template reuse and 'best practice' phrases could narrow output diversity independently of model bias.","A practical extension: randomly substitute rare or novel descriptors into the dominant prompt templates and monitor Vendi scores; if rare-token variants measurably increase visual diversity, community tag recommendations could be designed as an antidote to homogenization."],"forward_implications":["If token overlap drives image sameness, then prompting tools that suggest fresh words or rare descriptors should measurably increase the visual spread of generated images.","Because exact duplicates account for 40-50% of submissions and consistent repeaters dominate near-duplicate clusters, interventions aimed at heavy repeaters could change corpus-wide visual diversity more than general interventions.","Text-only lexical metrics, especially the self-repetition score and compression ratio, can serve as cheap early-warning signals for expected drops in visual diversity, since they correlate with Vendi scores without needing to render images.","The post-February 2024 drop tracks adoption of a new model's structured rating tags, so the homogenization effect is tied to community adoption of fixed prompt formulas and is likely to recur when major new models appear."],"supporting_citations":[{"why":"Supplies the Civiverse dataset of over six million prompts and generated images across seven months that all analyses use.","marker":"[38]"},{"why":"Provides the contrastive language-image embedding approach that underlies the semantic similarity and image similarity measurements.","marker":"[39]"},{"why":"Defines the MinHash-based Jaccard similarity method used to identify near-duplicate prompt clusters above the 0.8 threshold.","marker":"[24]"},{"why":"Precedent for using Vendi scores to quantify diversity in generated images, supplying the paper's visual-diversity measure.","marker":"[54]"},{"why":"Provides the compression-ratio and text-diversity measurement approach adopted for the lexical metrics.","marker":"[46]"},{"why":"Supplies the self-repetition score that detects formulaic multi-word patterns in prompts.","marker":"[44]"},{"why":"Earlier analysis of prompting practice showing community formulaic tokens, which motivates the homogenization hypothesis.","marker":"[45]"}],"fun_headline_variants":["Repeated prompts tied to less diverse AI images","Six million prompts show repetition shrinks visual variety","Prompt repetition drives visual sameness in AI art","Same words, same images: how users narrow AI art","User prompt habits narrow the diversity of AI imagery"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim assumes that overlapping words in prompts are what make generated images look similar, even though images can differ in model version, style add-on modules, random seed, sampler, and guidance scale, all of which may be shared within prompt clusters.","fun_headline_variants_meta":{"raw":{"variants":["Repeated prompts tied to less diverse AI images","Six million prompts show repetition shrinks visual variety","Prompt repetition drives visual sameness in AI art","Same words, same images: how users narrow AI art","User prompt habits narrow the diversity of AI imagery"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000699,"raw_usage":{"total_tokens":3167,"prompt_tokens":965,"completion_tokens":2202,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":2129}},"tokens_in":581,"tokens_out":2202,"duration_ms":14003,"temperature":1.0,"reasoning_tokens":2129,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:55:41.271042+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Regenerate the images in the high- and moderate-overlap prompt clusters while holding model version, style add-on modules, random seed, sampler, and guidance scale fixed across all clusters; if the correlation between token overlap and image-embedding similarity falls to near zero, the claim that linguistic repetition drives visual uniformity fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the MinHash-based Jaccard similarity method used to identify near-duplicate prompt clusters above the 0.8 threshold."},{"cited_title":"hair, \"","cited_arxiv_id":null,"evidence_quote":"Precedent for using Vendi scores to quantify diversity in generated images, supplying the paper's visual-diversity measure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the self-repetition score that detects formulaic multi-word patterns in prompts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier analysis of prompting practice showing community formulaic tokens, which motivates the homogenization hypothesis."}],"review_version":1}