{"id":"f28049fd-7d00-4fc2-8c02-179fcc6d0801","arxiv_id":"2601.11128","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new public dataset aligns pre- and post-intervention user activity for 25 Reddit/Voat moderation actions, enabling comparative moderation research.","lead":"This paper introduces TBBT, a dataset of 25 content-moderation actions on Reddit and Voat, with pseudonymized user activity from three months before and after each intervention, covering 339K users and 38.7M messages. It is aimed at researchers who want to compare moderation effects across bans, quarantines, and post removals without recollecting data from scratch.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Two largest post-removal rows list '1-30 Jun 22' as t0, contradicting the discrete timestamp required for the claimed t0 ± 3-month window alignment; this weakens the central 'consistent and comparable windows' claim.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern I find: the discrete-t0 data model is contradicted by the month-long date range assigned to the two post-removal interventions. This is not a minor formatting issue; it affects the two largest rows by user count and undermines the central claim of consistent, comparable pre/post windows across all 25 interventions. The paper explicitly states in the methodology that t_before and t_after are t0 ± 3 months, so the table's range cannot be reconciled without either redefining t0 for those rows or disclosing a different alignment rule. The concern is concrete and testable: inspecting the actual timestamps in the released files will show whether the windows are contiguous around a single day or separated/overlapping by ~30 days. I do not see a more fundamental problem; the dataset appears substantial, the aggregate counts are internally consistent, and the paper is transparent about other limitations such as the ten-post threshold and the migration username-matching. Thus the appropriate verdict remains conditional: the resource is valuable, but the claims of uniform alignment must be corrected or scoped before downstream users rely on those two rows.","tokens_in":15063,"tokens_out":6686,"duration_ms":70121,"concrete_test":"For each of the two post-removal rows (2022-AskReddit and 2022-science), download the archive from the Zenodo DOI and compute the minimum and maximum created_utc in the IN-BEFORE and IN-AFTER JSON files. Then check whether the union of the two slices leaves a gap or overlap of approximately the month of June 2022. If the gap/overlap is ~30 days (or if the boundary is not a single day), the fixed t0 ± 3m alignment is not implemented for these rows; if the boundary is a single day (e.g., pre ends 31 May, post starts 1 Jun, with June excluded), then a discrete t0 was used despite the table range, and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The dataset's core promise is that every intervention is modeled as a discrete, time-stamped event t0 and that all slices are aligned to fixed windows t0 ± 3 months (Methodology, Data Collection step 1). Table 1 violates this for two of the 25 rows: both post-removal interventions, r/AskReddit and r/science, list '1-30 Jun 22' as the date, i.e., a month-long period, not a point timestamp. For the claim to hold, each row would need a single t0; a range makes the pre/post window boundaries undefined. Concretely, for r/AskReddit (273,792 users, 11.66M IN-BEFORE comments) and r/science (5,028 users), the IN-BEFORE and IN-AFTER slices cannot both be exactly three months around the same t0. If t0 is taken as 1 Jun, the pre window ends 31 May and the post window starts 1 Jun; if t0 is 30 Jun, the pre window includes all of June and the post window starts 1 Jul. Either choice misaligns one slice by a month. Because these two rows account for roughly 278K of the 339K total distinct users, the 'systematically aligned observations from fixed windows' claim is materially weakened for the largest portion of the dataset. The paper does not discuss this discrepancy in the methodology or limitations, so it is an internal inconsistency rather than a disclosed design choice. This is the load-bearing assumption: the whole value proposition is comparability across interventions, and comparability fails at the point where the intervention date is not a point.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces TBBT, a dataset of 25 content-moderation interventions on Reddit and Voat, spanning 2015–2023. For each intervention, the authors provide standardized metadata and pseudonymized user activity in four slices (IN/OUT × BEFORE/AFTER) aligned to fixed three-month windows around an intervention timestamp t0. The dataset reportedly contains 38,700,732 comments from 339,125 distinct users. A six-step collection pipeline is described, and the dataset is released on Zenodo with a DOI. The paper also presents descriptive exploratory analyses and discusses use cases for moderation research.","tokens_in":15458,"tokens_out":6041,"duration_ms":65506,"significance":"If the dataset is as described, it is a valuable multi-intervention resource for studying moderation effects, biases, and spillovers, enabling comparative and reproducible research that existing single-intervention datasets do not support. Strengths include the explicit and mostly transparent pipeline, the public DOI, pseudonymization, FAIR compliance, and the inclusion of cross-platform migration data for selected interventions. The paper is honest about limitations such as platform skew, observational design, and user-selection bias. However, the central value proposition—consistent and comparable pre/post windows across all interventions—is materially compromised for the two largest rows by a mismatch between the data model's discrete-t0 assumption and the month-long date range reported for those interventions.","major_comments":[{"comment":"The data model assumes a discrete, time-stamped intervention t0, with pre/post windows defined as t0 ± 3 months (Figure 1; Data Collection step 1). However, Table 1 lists the date for the two post-removal interventions, r/AskReddit and r/science, as '1-30 Jun 22' — a month-long range, not a single timestamp. For these rows, the IN-BEFORE and IN-AFTER slices cannot both be aligned to the same t0: taking t0 as June 1 excludes June from the pre-window; taking it as June 30 includes June in the pre-window and shifts the post-window. Either way, the claim of 'systematically aligned observations from fixed windows' fails for these two interventions, which together cover roughly 278K of the 339K distinct users. The paper does not disclose or discuss this discrepancy in Methodology or Limitations. The authors must either provide exact enforcement timestamps for these post-removal cases or redefi","section":"Methodology, Data Model and Data Collection step 1; Table 1 rows 1 and 6"},{"comment":"The total of 339,125 distinct users is presented as a headline number, but the row-level user counts in Table 1 sum to 352,461 in the IN-BEFORE column alone, indicating that users appear in multiple interventions. The paper does not state whether the 339K figure is de-duplicated across the four slices and across interventions, or how a user active in multiple rows is counted. This is relevant for any cross-intervention analysis and should be clarified, even if the figure is correct.","section":"Dataset Description, Table 1 and Limitations"}],"minor_comments":[{"comment":"The column headings contain apparent artifacts: '/user-friends' and '/commen◎s' likely should read 'users' and 'comments'. Please correct these rendering issues.","section":"Table 1"},{"comment":"The descriptive analysis reports that activity decreases 'across nearly all interventions' but provides no statistical tests or uncertainty measures. For a dataset paper this is acceptable, but adding paired tests or effect sizes would strengthen the claim.","section":"Dataset Overview and Descriptive Statistics; Figure 5"},{"comment":"For migration cases, the matching of usernames across Reddit and Voat is asserted but not described in detail. Please specify the matching procedure and any validation performed, since this determines the integrity of the OUT-AFTER slices for those interventions.","section":"Data Collection and Preparation, step 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a solid dataset contribution, and the arithmetic and pipeline checks reported by the reader are consistent. The main blocker is the unresolved t0 range for the two largest post-removal rows, which undermines a central claim. I recommend major revision; if the authors provide precise timestamps or explicitly re-define the windows and revise the alignment claims, the paper could be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"TBBT is worth a serious look. It assembles 25 moderation interventions across Reddit and Voat, with pseudonymized user activity in four standardized slices (IN/OUT × BEFORE/AFTER), totaling 38.7M comments from 339K users. That is a real resource: prior datasets are mostly single-intervention or general-purpose archives, and this one standardizes the temporal and spatial slicing across nine years. The pipeline is explicit and reproducible, the aggregate counts check out (I summed Table 1 myself), and the limitations section is honest about selection bias, bot filtering, and observational scope. I would use it as a benchmark or comparative testbed.\n\nThe soft spot is exactly what the stress-test flagged. The data model assumes each intervention has a discrete timestamp t0, and all windows are t0 ± 3 months. But Table 1 lists the two post-removal interventions (r/AskReddit and r/science) with date '1-30 Jun 22' — a month-long period, not a point. Those two rows cover roughly 278K of the 339K distinct users. If t0 is ambiguous, the IN-BEFORE and IN-AFTER slices cannot both be exactly three months around the same event. The paper doesn't mention this. It's not a fatal flaw: researchers can still use those slices with care, and the rest of the dataset is aligned. But the 'consistent and comparable windows' claim is weakened for the largest part of the dataset, and the paper should either re-define those interventions with a specific t0 or add a caveat.\n\nThe other issues are minor and mostly disclosed: migration slices depend on exact username matches, and the ten-post threshold filters out low-activity users. Both are stated in the methodology. The intervention selection leans on prior literature and the authors' own case studies, which limits novelty but is not circular — the dataset is a resource, not a test of a hypothesis.\n\nOverall, this deserves a serious referee. I would accept it conditional on addressing the t0 ambiguity for the two post-removal rows, either by picking a defined date or by releasing separate pre/post windows for those cases. The dataset will be useful to anyone studying moderation effects, and the descriptive analysis is appropriately hedged.\n\nRecommendation: send to peer review with a request for a revision note on the date-range issue. It's a solid dataset paper that needs one clarification.","headline":"TBBT is a genuinely useful standardized multi-intervention moderation dataset, but the two largest rows use a month-long date range as t0, which breaks the paper's claim of uniform ±3-month alignment.","tokens_in":15927,"tokens_out":2574,"would_cite":true,"duration_ms":26633,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces The Big Ban Theory (TBBT), a reusable dataset that aligns three months of user activity before and after each of 25 content-moderation interventions across Reddit and Voat.","keywords":["content moderation","dataset","Reddit","Voat","community bans","quarantine","pre-post intervention","pseudonymization"],"falsifier":"Look at the timestamps of removed posts in the two post-removal cases spanning '1-30 Jun 22'; if removal timestamps are spread across many days, then the fixed t0 ± 3-month windows are not aligned to a single intervention, which would contradict the paper's claim of uniform and comparable pre/post windows across all 25 interventions.","tokens_in":14964,"feed_emoji":"📊","tokens_out":4660,"duration_ms":48707,"temperature":0.7,"pith_summary":"The paper introduces The Big Ban Theory (TBBT), a dataset built around 25 content-moderation interventions—community bans, quarantines, post removals, and ban-triggered migrations—carried out between 2015 and 2023. It claims to cover pseudonymized activity from 339,125 users and 38.7 million messages, with each intervention's data organized into fixed three-month windows before and after the event. The aim is to let researchers study moderation's effects, fairness, and spillovers across many interventions at once, instead of assembling one-off collections for each study. The central design choice is a four-slice model separating time (before/after) and space (inside/outside the moderated community), which also handles cases where the moderated space disappears after a ban.","feed_headline":"One dataset frames 25 moderation actions with pre/post windows","feed_subtitle":"Pre/post snapshots around each intervention let researchers compare moderation's effects across Reddit and Voat.","key_machinery":"The load-bearing device is the four-slice data model (IN-BEFORE, IN-AFTER, OUT-BEFORE, OUT-AFTER), built on the idea that a moderation intervention is a discrete event at time t0 with a defined moderated space. Fixed windows t_before = t0 − 3 months and t_after = t0 + 3 months make interventions commensurable; the IN/OUT split separates activity inside the moderated community from activity in other spaces where affected users participate. The model determines which slices exist for each intervention type—post removals and quarantines have IN-AFTER; community bans substitute OUT-BEFORE and OUT-AFTER; migrations pair IN-BEFORE on Reddit with OUT-AFTER on Voat.","core_discovery":"TBBT is a standardized, intervention-centered corpus: for each of 25 moderation events, the paper provides the same kinds of fields (timestamps, text, user hashes, subreddit, scores), collected from three months before and, when possible, three months after the intervention. For community bans, where post-ban activity inside the community is structurally unavailable, the dataset instead tracks the same affected users' activity elsewhere on the platform or on Voat, including matched usernames for migration cases. The paper's claim is that this unified structure turns scattered moderation events into a comparable, reusable resource for descriptive, quasi-experimental, predictive, and governanc","pith_inferences":["Our reading: the uniform-window claim should be tested against the two post-removal cases whose reported date is a range ('1-30 Jun 22') rather than a single day; if removals were distributed across the month, the IN-BEFORE and IN-AFTER slices for those cases are not centered on one intervention event, and pooled comparisons should treat them as a separate condition.","Our reading: the rule requiring at least ten pre-intervention messages and the bot filter mean the dataset describes the most active users in a community, not all affected users; any effect estimates generalize to that subpopulation, and replication with relaxed thresholds would show how sensitive conclusions are to this choice.","Our reading: the matched-username migration cases open a route to studying the same individuals' behavior before a ban on one platform and after migration to another; extending this design to other banned communities would require similar cross-platform archives with overlapping identities."],"forward_implications":["Because every intervention is stored in the same four-slice format with the same fields, a single analysis script can be run across all 25 events, making cross-intervention comparisons and robustness checks feasible.","The dataset provides pre-moderation behavioral baselines for hundreds of thousands of users, allowing tests of whether similar communities are treated consistently by moderators and whether outcomes vary by intervention type, community size, or time period.","Pre-intervention activity can serve as input features and post-intervention behavior as ground-truth labels, supporting predictive models of user abandonment, migration, and toxicity change.","The OUT slices for community bans make within-platform and cross-platform spillover measurable, including migration to Voat where usernames were matched across platforms.","The dataset can anchor method comparisons: researchers can benchmark before-after estimators, interrupted time series, and difference-in-differences on the same intervention set, testing sensitivity to window length and aggregation."],"fun_headline_variants":["25 bans, 39M posts: one dataset for moderation studies","Compare 25 moderation actions with pre/post user traces","Tracking banned users across Reddit and Voat post-ban","39M messages from 339K users across 25 interventions","Standardized moderator intervention data for reproducible research"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The three-month windows are centered on a single, well-defined intervention timestamp t0 for every case; if an intervention unfolded over days or weeks instead of at one moment, the 'before' and 'after' slices are not comparing around the same event.","fun_headline_variants_meta":{"raw":{"variants":["25 bans, 39M posts: one dataset for moderation studies","Compare 25 moderation actions with pre/post user traces","Tracking banned users across Reddit and Voat post-ban","39M messages from 339K users across 25 interventions","Standardized moderator intervention data for reproducible research"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000157,"raw_usage":{"total_tokens":1052,"prompt_tokens":731,"completion_tokens":321,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":241}},"tokens_in":475,"tokens_out":321,"duration_ms":4368,"temperature":1.0,"reasoning_tokens":241,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T10:05:33.598003+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Look at the timestamps of removed posts in the two post-removal cases spanning '1-30 Jun 22'; if removal timestamps are spread across many days, then the fixed t0 ± 3-month windows are not aligned to a single intervention, which would contradict the paper's claim of uniform and comparable pre/post windows across all 25 interventions.","supporting_citations":[],"review_version":1}