{"id":"f03d501e-419a-4f51-9c48-214cade00bdd","arxiv_id":"2506.18706","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey of 157 fanfiction community members shows strong attachment to human-centered creativity, widespread demand for AI transparency, and a spectrum of attitudes from cautious acceptance to active rejection.","lead":"This paper surveys 157 active fanfiction readers and writers about their attitudes toward generative AI, finding a community that strongly values human creativity but is split on AI tools. It matters because fanfiction platforms are early testbeds for how creative communities will absorb or reject AI.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bot-removal rule removes 68% of respondents; if some exclusions are false positives, the headline threat-perception percentages could be materially biased.","rationale":"The reader's weakest_assumption identifies both sample representativeness and bot detection as potential issues. I agree that these are the load-bearing assumptions for the central claim, and I narrow the concern to the bot-removal rule in Section 3.2.2, where 68% of respondents were excluded. This rule is based on behavioral heuristics (response time, absence of open-ended text, email server type) that can plausibly admit false positives, and the paper provides no validation of the rule's accuracy. Because the final sample is both small and self-selected, a false-positive rate of even modest size in the 111 'suspected bot' group could shift the headline percentages enough to weaken the 'predominantly' characterization. I do not think this rises to rejection: the survey is transparent about recruitment and limitations, the central descriptive pattern is plausible and qualitatively echoed in the open-ended responses, and the data availability statement makes a robustness check feasible. The reader's CONDITIONAL verdict remains appropriate; the condition should explicitly include a sensitivity analysis of the bot-removal criteria. I also note secondary numerical inconsistencies (e.g., Section 4.2.2 reports 74.4% used at least one AI tool and 33% never used any, summing to over 100%; Section 5.1 reports 90 'No' and 25 'Maybe' on Q19 yet later uses 63 as the 'read or might have read' group), but these concern supporting statistics rather than the three headline threat items. Overall, the central claim is not invalidated, but its strength depends on the robustness of the cleaning step, which is currently untested.","tokens_in":31384,"tokens_out":8678,"duration_ms":85333,"concrete_test":"Request the raw de-identified survey data from the corresponding author (as promised in the Data Availability statement) and re-run the analysis with the 111 suspected bots either included or subjected to a relaxed exclusion threshold (e.g., answer time 4–7 seconds per question). Then recompute the three headline percentages (Q48_5, Q48_6, Q21). If any of these shifts by more than 3 percentage points, or if any drops below an absolute majority, the central claim is not robust to the bot-removal criteria. As an auxiliary check, statistically compare the final sample's age, gender, geography, and years-of-engagement distributions against the 2024 AO3 Census to quantify the representativeness limitation beyond the paper's qualitative comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2.2 reports that 332 of 489 respondents (68%) were removed as bots: 221 due to a 'consecutive bot attack' identified by a single geo-location, and 111 because they answered only the minimal 44 required questions, each within 5–6 seconds, provided no open-ended responses, and used email servers that 'do not require authentication.' The final sample is therefore 157 self-selected volunteers. The exclusion rule is not validated against a gold standard. A fast human reader completing Likert-scale items can plausibly spend 5–6 seconds per item and skip open-ended questions; a group of legitimate respondents sharing a single IP/location (e.g., a fan convention, a university cohort, a shared Wi-Fi) could match the geo-location pattern. If any nontrivial share of the 111 suspected bots were human, the denominators for the central percentages change. For example, Q48_5 (76.4%, 120/156) and Q48_6 (83.4%, 131/157) could shift by several points if even 10–15% of the excluded respondents were retained, because the excluded group might have different attitudes than the retained sample. Since the paper's central claim is that 'the fanfiction community predominantly perceives' AI as a threat to human-centered values, the bot-removal rule is load-bearing. The paper acknowledges sample skew toward North American and English-speaking participants (Section 6.6), but it does not address the possibility of false-positive exclusions in the cleaning step, which is a distinct and testable threat to the validity of the reported magnitudes.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a survey-based study of 157 self-identified fanfiction community members, examining their attitudes toward generative AI (GenAI) in fanfiction writing and community dynamics. The authors recruited participants through fanfiction platforms, cleaned the data by removing 332 of 489 initial respondents whom they classified as bots, and analyzed the remaining 157 responses using descriptive statistics, group comparisons, and qualitative coding of open-ended questions. The central findings are that a majority of respondents perceive AI-authored fanfiction as a threat to the social and human-centered values of the community (e.g., 76.4% agreed that AI-authored fanfiction endangers social aspects, Q48_5; 83.4% worried about AI stories overshadowing human ones, Q48_6), and that more experienced writers are more resistant to AI integration than newer writers. The paper also proposes design interventions for fanfiction platforms, including AI-use disclosure, consent mechanisms for AI training, and ways to enrich community engagement. The manuscript is framed as contributing to HCI research on participatory creative communities and human-AI co-creation.","tokens_in":31809,"tokens_out":3265,"duration_ms":33599,"significance":"If the findings are valid, the paper makes a useful empirical contribution to the growing literature on how creative communities respond to generative AI. The survey covers both readers and writers, includes a substantial qualitative component with reported inter-coder reliability (86.6%, 99.3%, and 86.3% for Q29, Q36, and Q46), and connects the results to participatory culture and data-ethics frameworks. The design implications for AI-use labeling, consent, and community engagement are concrete and actionable. The experience-based differences between newer and veteran writers, if replicable, would be a valuable finding for platform designers. The paper also provides a demographic comparison against AO3 census data, which helps contextualize the sample, and it explicitly acknowledges the sample's skew toward North American and English-speaking respondents.","major_comments":[{"comment":"The bot-removal rule is load-bearing for the paper's central quantitative claims but is not validated. The authors removed 332 of 489 respondents (68%) based on a geographic cluster (221 respondents) and a behavioral heuristic (111 respondents who answered only the minimal 44 required questions in 5–6 seconds, provided no open-ended responses, and used email servers that 'do not require authentication'). The paper does not report any validation of this rule against a gold standard, such as manual inspection or a pilot with known human respondents. A fast human reader can plausibly answer Likert items in 5–6 seconds each and skip open-ended questions, and a group of legitimate respondents using shared Wi-Fi or attending an event could match the geographic pattern. If even a modest fraction of the 111 excluded respondents were human, the reported percentages (e.g., Q48_5, 76.4%, 120/156; Q48_6, 83.4%, 131/157; Q21, 72.2%, 112/155) could shift materially, and the excluded group may hold systematically different attitudes. Given that the abstract and conclusions generalize to 'the fanfiction community,' the authors should either validate the exclusion rule, provide a sensitivity analysis showing the robustness of headline percentages under different exclusion assumptions, or substantially soften the generalizations. This issue is distinct from the acknowledged sample skew and is not addressed in the Limitations section.","section":"Section 3.2.2"},{"comment":"Several reported denominators are inconsistent with the stated sample size and with each other, which undermines confidence in the quantitative results. In Section 5.1.2, the text describes responses to Q42 and Q43 using denominators of 72 (e.g., '36%, /72', '65% (47/72)'), while the paper states there are 90 writers in the final sample; no explanation is given for the missing 18 respondents. In Section 4.2.3, the figure '64.74%, 101/159' for Q15 exceeds the total sample of 157 respondents, which is arithmetically impossible; the correct denominator may be 156, but 101/156 is also inconsistent with the stated N. Similarly, Section 4.2.3 reports 'The writers among our participants (90/156)' although the total sample is 157 and the paper states there are 90 writers and 67 exclusive readers (summing to 157). These inconsistencies affect the credibility of the descriptive statistics. The authors should reconcile all reported denominators against the actual number of valid responses per question, and state the number of missing responses for each key item.","section":"Section 5.1.2 and Section 4.2.3"},{"comment":"There is an internal inconsistency in the reporting of ethical concerns that makes the prevalence unclear. The text states '68.6% (107/156) of participants (Q51)' and then reports 'A few participants, 6.4% (10/156), also indicated concerns about data privacy and AI environmental impact, with 7.6% (7/91) of the responses expressing such concern.' The relationship between the 107 participants and the subcategories is not specified, and the denominator 91 for the 7.6% is unexplained. More importantly, the sentence immediately before this (about 'potential infringement and originality') does not give a numeric prevalence, so the reader cannot tell whether the '68.6%' refers to any ethical concern or to a specific subcategory. Please clarify the coding and the exact basis for each percentage.","section":"Section 5.2.2"}],"minor_comments":[{"comment":"The keyword line contains a typo: 'Storytelling, , GenAI, Participatory community' has a double comma and the list is oddly phrased. Please revise to a clean comma-separated list.","section":"Abstract and Keywords"},{"comment":"The sentence 'Most writers the use of AI for editing acceptable (36%, /72)' is grammatically incomplete and displays a missing number before the comma. It should read, for example, 'Most writers found the use of AI for editing acceptable (36%, 26/72).'","section":"Section 5.1.2"},{"comment":"The authors report multiple Mann-Whitney U tests comparing experience groups across five Likert items (Q41_1 through Q41_5) without any correction for multiple comparisons. Given the number of tests, some significant p-values near 0.04 may not survive adjustment. Please state whether corrections were considered and, if not, acknowledge this as a limitation.","section":"Section 5.1.3"},{"comment":"The discussion of the inconsistency between participants' uncertainty about identifying AI-generated text (Q49) and their reported confidence in having never read AI-generated fic (Q19) is interesting, but it would benefit from a more explicit reference to the possibility of undetected AI content, which the authors only gesture at.","section":"Section 6.2.1"},{"comment":"Reference [3] is listed as 'Anonymous' with no author name; this is a valid citation for a pseudonymous AO3 census, but the entry should include a note clarifying the pseudonymous nature of the source (similar to how the 2013 AO3 Census is cited via 'centreforthelights').","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely topic and provides a rich dataset, but the quantitative foundation needs consolidation. The bot-removal validation and denominator inconsistencies are the main barriers to acceptance. The data availability statement says the data are available from the corresponding author upon reasonable request; for an empirical paper of this type, the editor may wish to encourage the authors to deposit an anonymized dataset in a public repository to allow independent verification of the reported percentages. Additionally, the authors should be asked to clarify whether the study was preregistered and to present the survey instrument in full in the appendix, as only a subset of questions is shown in Tables 2–7."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a solid, readable survey of fanfiction readers' and writers' attitudes toward generative AI, and it fills a real gap. Fanfiction is an important site for participatory culture, and the paper gives us one of the first direct, community-level looks at how members are reacting to GenAI. The sample is modest and self-selected, but the authors are honest about that, and the subgroup analysis by writers' experience level is a genuine addition. The questionnaire is detailed, the qualitative coding reports inter-coder reliability, and the limitations section says the right things about sample skew. The central descriptive finding—broad concern about AI eroding the human-centered, social values of the community, with newer writers more open to AI than veterans—is credible and consistent with the quoted responses.\n\nThe soft spots are real but not load-bearing. First, the data-cleaning section is a bit blunt. Removing 221 respondents from a single geo-location as a bot attack is defensible, and the additional 111 flagged for fast minimal responses and no open-ended text are plausible bots, but the rule could catch a fast human or two. That matters only for the precision of the percentages, not for the direction of the findings. The sample is already self-selected and heavily North American, so the paper's claims are about this sample anyway. Second, there are a few numerical inconsistencies that need fixing before publication: the writer-only questions show denominators of 72 in the text (e.g., Q42/Q43) while the stated writer count is 90, and Q15 reports 101/159 when the final sample is 157. These look like copy-paste or recoding errors, and they need a careful pass. Third, the design implications section goes beyond the data in places—the badge and recommendation ideas are reasonable but speculative, and the paper does not fully acknowledge that they are suggestions rather than findings. That is a minor issue, not a flaw.\n\nOverall, this paper deserves a serious referee. It is empirically useful, clearly presented, and honest about its limits. The central claims hold up. I would send it out with a request for minor revision: correct the numbers, add a sentence acknowledging the residual uncertainty in bot removal, and label the design suggestions as speculative. The reader is right on all three counts.","headline":"A useful and careful survey of fanfiction community attitudes toward GenAI, with a few data-reporting wrinkles that warrant minor revision rather than rejection.","tokens_in":32173,"tokens_out":1407,"would_cite":true,"duration_ms":17193,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fanfiction community members predominantly perceive AI-generated content as a threat to the human-centered, participatory values of their space, with veteran writers more resistant than newer ones.","keywords":["generative AI","fanfiction","creative communities","participatory culture","human-AI co-creation","AI transparency","creative labor","survey study"],"falsifier":"A larger, more representative, or multilingual survey that found a majority of fanfiction members welcoming AI-authored stories under transparent labeling would contradict the paper's central claim, as would behavioral evidence that AI-labeled stories receive equal or greater readership on major fanfiction platforms.","tokens_in":31218,"feed_emoji":"📚","tokens_out":6069,"duration_ms":63093,"temperature":0.7,"pith_summary":"This paper tries to establish that fanfiction readers and writers mostly see generative AI as a threat to the human-centered, participatory values of their community, even as a minority of early adopters and newer writers view it as a useful tool. Surveying 157 active community members, it finds large majorities who fear AI stories will flood platforms, overshadow human work, weaken social bonds, and undermine authenticity, and who demand transparent disclosure of AI involvement. The authors interpret these responses as evidence of an early resistance-and-adoption phase, in which more experienced writers resist AI more strongly than writers with one to five years of involvement. This matters because fanfiction communities are early, vocal testbeds for how participatory creative communities can integrate AI without losing the values that define them.","feed_headline":"83% of fanfiction fans fear AI stories will drown out human ones","feed_subtitle":"A 157-member survey shows a creative community in early resistance; newer and AI-experienced writers are the exception.","key_machinery":"The argument is carried by a graduated AI-involvement scale: the survey asked readers (Q27) and writers (Q41) to rate their willingness to engage with stories produced under eight levels of AI involvement, from grammar and spell-checking only to fully AI-generated drafts with minimal editing. This scale produces the paper's core gradient, showing that acceptance is high for low-level assistance (82.7% would read work that used AI only for spelling and grammar) and collapses for full generation (about 10% of writers supported AI producing an entire story). The same instrument anchors the statistical comparisons between experience cohorts using Mann-Whitney U and chi-square tests, revealing the consistent resistance gradient among veteran writers.","core_discovery":"The central claim is that the fanfiction community currently perceives GenAI-authored content predominantly as a threat to its participatory, human-centered culture rather than as a neutral enhancement. The paper grounds this in survey figures: 76.4% (120/156) agreed that AI-authored fanfiction endangers the community's social aspects, 83.4% (131/157) worried that an influx of AI-created stories would overshadow human-authored ones, and 72.2% (112/155) reported negative feelings upon discovering that a story they read was AI-written. At the same time, 86% (135/157) insisted that authors be transparent about AI involvement, and 57.7% (90/156) said they had never knowingly read an AI-generated story even though most were unsure they could detect one. Writers were already using AI tools (74.4% reported using at least one), yet acceptance dropped sharply as AI involvement rose, and writers with over ten years of experience were significantly more resistant than newer writers at every level of AI involvement.","pith_inferences":["Because the survey cannot establish causation, a natural extension is a randomized experiment in which identical stories are labeled human-written or AI-assisted; if rejection tracks the label rather than the text, then threat perception is driven by provenance disclosure rather than by content quality.","The experience gradient implies a generational shift: if veteran members disproportionately leave or lose influence, community norms could move toward accepting AI assistance faster than the current aggregate resistance suggests.","The same participatory-value concerns likely apply to other non-commercial creative communities, such as collaborative writing circles, zine scenes, and amateur art spaces, but the paper does not test that generalization.","The small group of highly AI-familiar, heavy-usage writers may act as the diffusion bridge to wider adoption, but with only about one in ten writers in that group, their long-term influence remains uncertain."],"forward_implications":["Platform designers can address the transparency demand with user-driven, fine-grained self-disclosure of AI involvement, since 86% of participants want disclosure and acceptance varies sharply by level of AI involvement.","Opt-in and opt-out mechanisms for allowing fanfiction to be used in AI training would respond to the ethical concerns and the sense of theft that participants reported.","Recommendation systems that label and filter by AI involvement could preserve the visibility of human-authored stories while using AI to revive dormant fandoms.","If the experience gradient persists, community turnover will likely raise baseline acceptance of AI assistance over time, even if veteran resistance remains strong.","The gap between participants' low confidence in detecting AI and their strong quality objections suggests that disclosure, not content inspection, is the main lever on acceptance."],"supporting_citations":[{"why":"Provides the participatory-culture framework that defines fanfiction as collaborative, low-barrier creative practice, grounding the study's central value concerns.","marker":"[51]"},{"why":"Supplies the three scenarios for how GenAI could disrupt creative work, which the paper uses to frame readers' and writers' views on the future of storytelling.","marker":"[22]"},{"why":"Evidence that GenAI boosts individual creativity while reducing collective diversity, cited to explain fears of homogenized stories.","marker":"[26]"},{"why":"Empirical finding that writers feel disempowered when their online work is absorbed into training data, used to interpret participants' consent and authorship concerns.","marker":"[36]"},{"why":"Prior analysis of AI in fandom as human-community-machine interaction, used to position the study and discuss threats to participatory culture.","marker":"[60]"},{"why":"Diffusion-of-innovations theory used to interpret early adopters among writers as potential drivers of wider acceptance.","marker":"[76]"},{"why":"A widely read account of AI's limits in fiction writing, cited alongside scholarly sources for concerns about emotional depth and authenticity.","marker":"[84]"},{"why":"The recent demographic census of a major fanfiction platform used to compare the sample's representativeness against known community attributes.","marker":"[3]"}],"fun_headline_variants":["83% of fanfiction fans fear AI will eclipse human-written stories","Survey: 83% of fanfiction enthusiasts fear AI stories will crowd out human ones","Fanfiction community: 86% want AI use disclosed, survey shows","72% of fanfiction readers feel negative when they learn a story is AI-made","Newer fanfiction writers more open to AI than veterans, survey finds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported percentages stand or fall on whether the 157 self-selected, mostly North American, English-speaking respondents represent the broader fanfiction community, and on whether the bot filter removed only automated responses rather than real participants.","fun_headline_variants_meta":{"raw":{"variants":["83% of fanfiction fans fear AI will eclipse human-written stories","Survey: 83% of fanfiction enthusiasts fear AI stories will crowd out human ones","Fanfiction community: 86% want AI use disclosed, survey shows","72% of fanfiction readers feel negative when they learn a story is AI-made","Newer fanfiction writers more open to AI than veterans, survey finds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001261,"raw_usage":{"total_tokens":5138,"prompt_tokens":895,"completion_tokens":4243,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":4143}},"tokens_in":511,"tokens_out":4243,"duration_ms":30060,"temperature":1.0,"reasoning_tokens":4143,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:42:51.426863+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A larger, more representative, or multilingual survey that found a majority of fanfiction members welcoming AI-authored stories under transparent labeling would contradict the paper's central claim, as would behavioral evidence that AI-labeled stories receive equal or greater readership on major fanfiction platforms.","supporting_citations":[{"cited_title":"2016.Participatory Culture in a Networked Era: A Conversation on Youth, Learning, Commerce, and Politics","cited_arxiv_id":null,"evidence_quote":"Provides the participatory-culture framework that defines fanfiction as collaborative, low-barrier creative practice, grounding the study's central value concerns."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior analysis of AI in fandom as human-community-machine interaction, used to position the study and discuss threats to participatory culture."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Diffusion-of-innovations theory used to interpret early adopters among writers as potential drivers of wider acceptance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"A widely read account of AI's limits in fiction writing, cited alongside scholarly sources for concerns about emotional depth and authenticity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The recent demographic census of a major fanfiction platform used to compare the sample's representativeness against known community attributes."}],"review_version":1}