{"id":"3096828b-cd3e-466b-8f6c-e654fd76af3e","arxiv_id":"2607.23941","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Reddit's user-recognized bot population grew until roughly 2021, then declined before the 2023 API change, while its 18 behavioral 'species' kept a stable composition.","lead":"This paper sorted 3,389 Reddit accounts that other users had voted as bots into 18 behavioral types and tracked how those types changed from 2007 to 2025. Recognized-bot numbers grew into the COVID era, then fell before Reddit's 2023 API fees, while the mix of types stayed compositionally stable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Time-dependent vote-threshold censoring may produce the post-2022 decline; need to test with lower thresholds or cohort vote-accrual analysis.","rationale":"The reader identified sample representativeness as the weakest assumption, focusing on changing recognition conventions. My concern is a more concrete and testable mechanism within the same sample-selection problem: the lifetime 10-vote threshold creates right-censoring that mechanically depresses counts of recent bots. This is independent of whether users recognize bots well; even under perfectly stable recognition, a bot created in 2024 has less time to accumulate votes than one created in 2018. The paper does not address this directly, and its own limitation statement (though honest) does not mention the threshold's temporal effect. This concern is load-bearing because the temporal trend is a central contribution, and it is directly checkable from data the authors must possess. The verdict remains CONDITIONAL: the paper's methods are otherwise reasonable, and the concern could be resolved with additional analysis. I do not think it requires rejection, but the abstract's unqualified temporal claim should be tempered or supported by such a test.","tokens_in":12947,"tokens_out":7033,"duration_ms":72008,"concrete_test":"Re-run the temporal analyses in Fig. 1B/1C and Fig. 3A using progressively lower vote thresholds (e.g., ≥1, ≥3, ≥5 votes instead of ≥10) on the same botranks/botrank data, and plot active-bot counts by year. If the post-2022 decline flattens or disappears under the lower thresholds, the decline is caused by the 10-vote censoring rather than a true population trend. In addition, for each bot in the full voting dataset, compute the time from account creation to the 10th vote; if recent cohorts (post-2021) have not had enough time to reach 10 votes, the censoring is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is that the sample selection rule (≥10 'good bot'/'bad bot' votes) introduces a time-dependent censoring bias that directly threatens the central temporal claim. Bots created after ~2021 have had less time to accumulate the 10-vote threshold before the dataset was assembled (data sources active 2020–23 and ongoing to 2026). Consequently, the number of active bots and new bot accounts will appear to decline in recent years even if the true bot population is stable or growing. The paper acknowledges a bias toward 'long-living, active and benevolent bots' but does not quantify or correct for this right-censoring. Without such a correction, the abstract's claim that bot numbers 'expanded rapidly before peaking around the COVID-19 period, then started declining even before Reddit's 2023 API policy changes' is not established: the decline may be an artifact of the vote threshold interacting with the observation window.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a population-level study of Reddit bots identified through crowdsourced 'good bot'/'bad bot' votes. The authors compile 3,389 bot accounts with full activity histories, extract 33 temporal, community, linguistic, and semantic features, reduce these to 23 features via PCA, and apply hierarchical clustering to obtain a taxonomy of 18 bot 'species'. They further analyze the temporal evolution of bot counts, activity, and species diversity from 2005 to 2025, claiming rapid growth peaking around the COVID-19 period, a decline beginning in 2022 before Reddit's API policy changes, and a remarkably stable diversity of bot types. The paper also examines co-posting networks and discusses ecological and evolutionary interpretations.","tokens_in":13192,"tokens_out":4079,"duration_ms":44933,"significance":"If the results hold, this would be the first comprehensive empirical taxonomy of Reddit bots and a valuable longitudinal description of an online bot ecosystem. The paper draws on a relatively large sample of overt, user-recognized bots and attempts to link behavioral features to functional roles, which is a useful contribution to the growing literature on machine behavior. The explicit discussion of limitations—including the exclusion of covert bots and the possibility of recognition bias—is commendable. However, the central temporal claims rest on a sample selection rule that introduces time-dependent censoring, and the paper does not provide code, data, or robustness checks to support the key findings. The abstract presents the decline as an established fact without the important caveats that appear later in the text. With appropriate sensitivity analyses and a more cautious framing, the paper could make a solid descriptive contribution, but in its current form the population-level claims are not fully supported.","major_comments":[{"comment":"The ≥10-vote inclusion rule creates a time-dependent censoring problem. Bots active after ~2021 have had less time to accumulate the required votes before the botranks.com data collection (2020–23) and botrank.net (ongoing to 2026) were assembled. The paper itself acknowledges that the analyses 'mainly capture relatively long-living, active and benevolent bots.' Consequently, the observed decline in active bot counts and new bot accounts after 2022 may be an artifact of this right-censoring rather than a real population trend. The abstract states that 'bot numbers and activity expanded rapidly before peaking around the COVID-19 period, then started declining' without this caveat. To support the temporal claim, the authors should provide sensitivity analyses using lower vote thresholds (e.g., ≥1, ≥5), cohort-based analyses of vote accrual rates, or some explicit model of the censoring pro","section":"Data and Methods, vote threshold; Discussion, limitations"},{"comment":"The paper states: 'However, this decline in commenting activity disappears if we account for AutoModerator, which today is responsible for more activity on the platform than the rest of the bot population combined.' This is a direct qualification of the abstract's claim that 'bot numbers and activity ... started declining even before Reddit's 2023 API policy changes.' If the commenting-activity decline is not robust to including the platform's largest official bot, the abstract overstates the result. Please clarify what 'account for AutoModerator' means (include, exclude, or control for) and explicitly reconcile this statement with the headline claim. At minimum, the abstract should be revised to state that the decline holds for the sampled non-AutoModerator bot population.","section":"Results, Fig. 1C and surrounding text"},{"comment":"The 18-type taxonomy is derived from a single hierarchical clustering run, with k chosen by silhouette scores. The paper mentions that results were 'similar' to k-means but provides no quantitative comparison. There is no assessment of cluster stability (e.g., bootstrap resampling, subsampling, alternative linkage methods, or varying the number of features retained after PCA). The taxonomy underpins the '18 distinct bot types' and the 'stable diversity' claims, so some evidence that the clusters are reproducible and not an artifact of the specific algorithmic choices is essential. Additionally, the paper does not provide code or data, making it impossible for readers to verify the clustering or reproduce the taxonomy.","section":"Data and Methods, hierarchical clustering"},{"comment":"The 13 macro-domain topic frequencies are constructed by manually grouping fine-grained BERTopic topics. This introduces a subjective layer into the semantic features. The paper does not report intercoder reliability, alternative groupings, or sensitivity analyses. Because the taxonomy and the interpretation of 'content-specialized' bot types rely on these semantic features, the authors should demonstrate that the conclusions are not sensitive to the particular manual grouping decisions.","section":"Data and Methods, BERTopic macro-domain grouping"}],"minor_comments":[{"comment":"The abstract says 'started declining even before Reddit's 2023 API policy changes', but the Results section (Fig. 1) shows the decline beginning in early 2022. This is consistent, but the phrase 'even before' could be interpreted as surprising; consider rewording to 'the decline began in 2022, predating the April 2023 API announcement.'","section":"Abstract and Results"},{"comment":"The description of Subreddit specialization says the Herfindahl–Hirschman Index is '0 if activity is equally distributed among 10+ subreddits', but the index formula sum of squared shares is positive for any distribution; the statement should clarify that it would be near zero, not exactly zero, for a uniform distribution over 10+ subreddits.","section":"Data and Methods, Table 1"},{"comment":"There are numerous typographical artifacts such as 'T witter', 'V ariance', 'T o', and inconsistent spacing in the PDF. Please proofread for these issues before publication.","section":"General formatting"},{"comment":"The references include URLs with access dates. Reference [42] and [43] describe 'botranks' and 'botrank' but the main text uses 'botranks.com' and 'botrank.net'. Ensure names are consistent. Also, reference [8] (Moltbook) is an arXiv preprint dated Feb 2026; please confirm it is publicly available and correctly cited.","section":"References"},{"comment":"The t-SNE plot is noted to be stochastic, but the visual separation of clusters is not quantified. A metric such as silhouette width per cluster or a validation measure would help readers assess cluster cohesion. This is a presentation issue, not a central claim.","section":"Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of a computational social science journal and addresses a timely topic. The main obstacle is not the taxonomy itself but the gap between the abstract's strong population-level claims and the acknowledged selection and censoring biases. The authors should be asked to provide robustness analyses for the temporal decline and to soften the abstract accordingly. Lack of code/data sharing is also a concern for reproducibility; I would suggest the journal require a data/code availability statement. There is no evidence of misconduct or circularity; the analysis is descriptive and the limitations are noted in the Discussion."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's real contribution is the first data-driven, population-level taxonomy of community-recognized Reddit bots. The 18 archetypes from 33 features are plausible and useful, and the longitudinal account from 2007 to 2025 — including the shift toward AutoModerator centralization — is a genuinely new empirical picture. The authors engage honestly with the Wikipedia bot literature and are unusually candid about limits in the Discussion.\n\nThe soft spots are in proportion. The biggest one: the ≥10-vote inclusion threshold right-censors younger bots. A bot created in 2022 has had far less time to accumulate ten \"good bot\"/\"bad bot\" votes than one created in 2015, so the observed post-2022 decline in active accounts may be an artifact of the sampling rule interacting with the observation window. The stress-test note is right that the paper never tests this with lower thresholds or a cohort vote-accrual analysis. The abstract states the decline without the caveat that appears later in the Discussion — that this is about overt, recognized bots and should not be read as a drop in automation. The Results section also leans on the pattern as evidence. That is a real gap between the abstract and the qualified claims.\n\nOther issues are minor: no code or data shipped, cluster stability not quantified beyond noting similarity to k-means, and the silhouette-based choice of k=18 is presented without sensitivity analysis. The \"remarkably stable diversity\" finding is also partly tied to the same sample bias — long-lived bots dominate, so composition stability is expected.\n\nThat said, the taxonomy itself is descriptive and not circular. The authors do not fit constants to reach conclusions; the clustering is an unsupervised summary. The absence of cluster validation is a weakness but not a fatal one.\n\nThe paper is for computational social scientists and platform governance researchers. It deserves a serious referee: the taxonomy is reusable, the longitudinal dataset is valuable, and the censoring problem is fixable with additional robustness checks. I'd recommend sending it out with requests for cohort analysis and lower-threshold sensitivity tests, and for code and data release.","headline":"First population-level taxonomy of overt Reddit bots, with a plausible pre-API decline that is partly an artifact of vote-threshold censoring; worth reviewing carefully.","tokens_in":13654,"tokens_out":1280,"would_cite":true,"duration_ms":15129,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reddit's recognized bot population forms an ecosystem of 18 behavioral species, and it peaked around 2021 before declining.","keywords":["Reddit bots","bot taxonomy","bot ecosystem","machine behavior","clustering analysis","population dynamics","AutoModerator","online communities"],"falsifier":"A platform-side audit that counts all automated accounts, including those that never post, post privately, or evade 'good bot' votes, would settle the population claim: if total automation was flat or rising after 2022, the paper's decline is a visibility artifact. Alternatively, re-clustering the same bots while excluding trigger-dependent and moderation bots would test robustness: if the 18 groups collapse, the taxonomy is not stable.","tokens_in":12860,"feed_emoji":"🤖","tokens_out":3525,"duration_ms":41313,"temperature":0.7,"pith_summary":"The paper tries to show that Reddit's community-recognized bots are not an undifferentiated crowd but a structured ecosystem with at least 18 behaviorally distinct types, and that this ecosystem has a measurable life cycle: rapid growth, a peak during the COVID-19 period, and a decline that began in early 2022, before Reddit's paid API changes. The decline applies to publicly visible, crowd-vetted bots; the authors themselves warn it does not prove overall automation on Reddit has fallen, because covert bots and non-posting automation are invisible to their data. What persists through the contraction is diversity: the 18 bot types mostly appeared by 2016, none has gone extinct as of 2026, and their relative proportions stay stable even as total numbers fall. The paper also documents increasing centralization, with Reddit's official AutoModerator now responsible for more activity than all other bots combined.","feed_headline":"18 bot species populate Reddit—none have gone extinct","feed_subtitle":"3,000 crowd-voted bots split into 18 behavioral types, and the population shrank before Reddit's paid API.","key_machinery":"The central object is the behavioral fingerprint: each bot is represented by 23 normalized features in four dimensions—temporal (mean inter-post time, hourly entropy, response-time variance), community (number of subreddits, subreddit specialization, trigger dependence, similarity to parent posts), linguistic (lexicon size, lexical diversity, sentiment), and semantic (13 macro topic frequencies derived from BERTopic/Sentence-BERT embeddings). Hierarchical clustering over these fingerprints, with k=18 chosen by silhouette scores, produces the taxonomy; co-posting networks built from one-mode projections of bipartite bot–subreddit graphs map the ecosystem's community structure and its change o","core_discovery":"Analyzing 3,389 crowd-voted Reddit bots through 33 behavioral features spanning temporal rhythms, community focus, linguistic style, and semantic topics, and clustering on 23 retained features, the authors identify 18 distinct bot archetypes: content-specialized types (technology and programming, gaming, politics, adult content), behavior-driven types (conversational, triggered response, meme), and infrastructural roles (platform governance, moderation support, on-demand utility). Longitudinal tracking shows bot account creation accelerating from 2017, active accounts peaking during the COVID-19 period, and activity declining from the start of 2022—before Reddit's April 2023 API announcement","pith_inferences":["A plausible reading the paper leaves implicit is that the post-2022 decline in recognized bots may reflect a migration of automation from visible to invisible forms, as newer LLM-based bots become harder for users to recognize; this can be tested with independent bot-detection applied to the same time window.","If the covert-bot undercount grew over time, the observed 'stable diversity' of the 18 recognized species may be a property of the recognition process rather than of the full bot population.","The AutoModerator centralization result suggests a fragility dynamic: a platform-managed single point of failure could disrupt moderation more severely than a distributed population of independent bots; a simulation-based robustness test would clarify this.","The taxonomy likely captures benevolent, long-lived, publicly disclosed automation; a complementary dataset of covert or ephemeral bots would likely add new cluster types and change the ecosystem-level conclusions."],"forward_implications":["The 18-type taxonomy gives researchers a shared, empirically grounded language for studying bot diversity on Reddit and a template for comparing automation on other platforms.","The documented decline beginning in early 2022 means the contraction of recognized bots cannot be blamed solely on Reddit's 2023 API pricing change; other factors were already at work.","Stable species diversity alongside shrinking numbers implies that ecological roles persist even as the population thins, suggesting a resilient functional core.","The rise of AutoModerator to dominant activity signals a structural shift from decentralized community-developed bots to centralized platform tooling, with potential trade-offs for innovation and vulnerability.","The temporary GPT-2 bot communities around 2021 illustrate how generative-AI advances create novel variants within existing ecological roles before contracting as conditions change."],"fun_headline_variants":["Reddit's bot ecosystem: 18 species, population in decline","18 bot types on Reddit, numbers fell before API paywall","Bot diversity holds on Reddit even as population shrinks","Reddit bots: 18 species, peak activity came pre-paywall"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The 3,389 crowd-voted, publicly recognized bots are a faithful enough sample of Reddit's bot population that the 18-type structure and the early-2022 decline reflect real bot ecology rather than changing user recognition habits or the hiding of newer AI bots.","fun_headline_variants_meta":{"raw":{"variants":["Reddit's bot ecosystem: 18 species, population in decline","18 bot types on Reddit, numbers fell before API paywall","Bot diversity holds on Reddit even as population shrinks","Reddit bots: 18 species, peak activity came pre-paywall"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"total_tokens":1356,"prompt_tokens":654,"completion_tokens":702,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":398,"completion_tokens_details":{"reasoning_tokens":639}},"tokens_in":398,"tokens_out":702,"duration_ms":7566,"temperature":1.0,"reasoning_tokens":639,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T23:26:44.914526+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A platform-side audit that counts all automated accounts, including those that never post, post privately, or evade 'good bot' votes, would settle the population claim: if total automation was flat or rising after 2022, the paper's decline is a visibility artifact. Alternatively, re-clustering the same bots while excluding trigger-dependent and moderation bots would test robustness: if the 18 groups collapse, the taxonomy is not stable.","supporting_citations":[],"review_version":1}