REVIEW 4 major objections 6 minor 9 references
Political Fact-Checking Efforts are Constrained by Deficiencies in Coverage, Speed, and Reach
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Institutional fact-checking during the 2022 U.S. midterms covered fewer than half of prominent election rumors, lagged four days behind them, and reached barely one percent of the conversation.
desk verdict A genuinely new measurement of fact-checking limits built on an independent rumor corpus; the headline numbers are plausible but the fact-check identification needs a sensitivity check before they are treated as exact. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing comparison is between two corpora built independently: ElectionRumors2022 (a real-time, manually curated dataset of 135 false or misleading election narratives on X/Twitter during the 2022 midterms) and a manually collected set of 164 fact-checks found by Google searches for each narrative. To that is added a partisanship assignment for users and narratives built from co-engagement networks of reposts, plus a logistic regression of which narratives get fact-checked. Together these allow the paper to measure coverage (share of narratives with at least one fact-check), speed (delay from a narrative's first post to its first fact-check), and reach (share of posts containing fact-check links and the partisanship of users sharing them).
What would settle it
A fully independent census of 2022 midterm election misinformation—assembled without real-time observer judgment, for example by mining a complete archive of election-related posts with a different narrative-clustering method or by using post-election retrospective media reviews—that found more than half of prominent narratives were fact-checked, or that the median lag was under 24 hours, would contradict the core coverage and speed claims. A narrower test: locate a widely shared voter-suppression narrative from the period that was quickly fact-checked by multiple organizations; the paper's finding that suppression narratives are checked at 19% and largely ignored would need revision.
Extended reading notes
Core claim
The paper's central discovery is that the three practical constraints—coverage, speed, and reach—compound to make institutional fact-checking a marginal force in a live U.S. election. Only 63 of 135 prominent election rumors (47%) received any fact-check; the median first fact-check came four days after the rumor's first appearance and three days after peak activity, with 79% of related posts already published; and among posts about even fact-checked narratives, only 1.2% contained or linked to a fact-check. Sharing of fact-checks was heavily partisan: left-leaning users posted 85% of fact-checks for left-leaning narratives and 80% for right-leaning narratives, while right-leaning users posted about 10%; only 0.30% of right-leaning users and 4.15% of left-leaning users shared any fact-check. The paper also shows that selection into fact-checking tracks claim type and timing more than virality or influencer involvement: all improbable-result narratives were checked, while only 19% of voter-suppression narratives were, and post-election narratives were checked more often. It reads the near-zero partisan coefficient after controls as evidence against the claim that fact-checking is systematically biased against the political right.
Load-bearing premise
The results depend on the 135 narratives in ElectionRumors2022 being a complete and unbiased record of prominent election misinformation; if the real-time observation missed major narratives or systematically excluded certain communities, every headline rate—47% coverage, four-day delay, and the partisan comparisons—would shift.
Editorial extensions
If this is right
- If the 47% coverage figure is correct, reliance on institutional fact-checks alone leaves more than half of prominent election rumors publicly unrebutted in the critical window.
- A median four-day delay with 79% of posts already out implies the modal fact-check functions as a record of a rumor rather than a timely correction.
- Fact-check posts being 1.2% of the conversation and mostly within one partisan community means the audiences most exposed to a rumor rarely encounter the correction.
- Using fact-check archives as a sampling frame for misinformation research inherits a bias toward easily debunkable, post-election narratives and overstates partisan asymmetry.
- Coverage decisions driven by claim type rather than virality or influencer engagement suggest that important rumors, such as voter-suppression claims, are systematically under-served.
Reading between the lines
- Editorial inference: the same coverage-speed-reach triad is a ready-made template for evaluating platform-embedded corrections, such as community notes, in the same 2022 data or in later elections.
- Editorial inference: because suppression narratives are the least-covered type, a concrete reform—dedicating a share of fact-checking capacity to first-person, hard-to-verify claims—would directly target the largest coverage gap.
- Editorial inference: the reach figure was measured on X/Twitter only; corrections attached to posts rather than circulated as links might cross partisan lines differently, so transferring the 1.2% number to other platforms is an extrapolation, not a finding.
- Editorial inference: the archive-bias result implies that prior misinformation studies sampling from fact-check databases may have overestimated effect sizes conditional on claim type; re-running such studies on fact-check-independent corpora would show whether their conclusions survive.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates real-world constraints on institutional fact-checking during the 2022 U.S. midterm elections by combining three data sources: ElectionTweets2022 (446 million election-related posts), ElectionRumors2022 (135 manually curated false/misleading narratives), and a new corpus of 164 fact-checks identified by Google search with Media Bias Fact Check (MBFC) source filtering. The central empirical claims are that only 47% of prominent narratives were fact-checked, that the median first fact-check appeared four days after a narrative's first post and after 79% of its misinformation posts, and that fact-check-linked posts constitute only 1.2% of the narrative conversation and circulate almost exclusively within partisan communities. The paper also argues that partisan asymmetry in coverage attenuates once narrative type is controlled, using logistic and OLS regressions with robustness checks on virality and influencer thresholds.
Significance. If the headline estimates hold, the paper makes an important contribution: it measures fact-checking effectiveness in a naturalistic setting using a rumor corpus that is not built from fact-check archives, thereby avoiding the selection/circularity problem that affects much of this literature. The study is also transparent in reporting manual validation of narrative timelines, robustness checks on variable cut-offs, and a stated intention to release code and data, all of which strengthen confidence in its descriptive findings. The coverage, speed, and reach magnitudes, should they survive sensitivity analysis, would be directly relevant to debates about whether institutional fact-checking can serve as a primary correction mechanism.
major comments (4)
- [Data & Methods, Fact-check Identification (p. 9)] The headline coverage figure of 47% is sensitive to the completeness of fact-check identification, which relies on the first ten Google results, a requirement that pages use the term "fact-check," and MBFC ratings of unbiased/high credibility. A debunk that does not surface in that search, is published by an organization MBFC rates as biased, or is formatted as a news correction or platform label would be missed, and if any such debunk corresponds to a narrative coded as "not fact-checked," the 47% figure is inflated and the four-day median delay is lengthened. The manuscript needs a sensitivity analysis that varies these inclusion rules, for example by (a) matching narratives against IFCN/International Fact-Checking Network signatory databases or the Duke Reporter's Lab list, (b) relaxing the MBFC filter to include all fact-checking organizations regardless of MBFC bias rating, and (c) reporting how many "not fact-checked" narratives have at least one candidate correction from these broader sources. This is load-bearing for the paper's central claim.
- [Data & Methods, False and Misleading Narratives (p. 8)] The 47% coverage rate and the four-day median delay are conditional on the completeness of the ElectionRumors2022 denominator, which was constructed in real time by manual observation and which the authors state eliminated "very low-spread" cases. If the curation missed prominent narratives or systematically underrepresented particular communities or types of claims, the coverage, speed, and partisan-comparison results all shift, and the direction of the bias is not obvious a priori. The paper should state this limitation explicitly in the main text and provide an external validation check, such as comparing the 135-narrative corpus against an independently constructed list of prominent election-claims (e.g., from the Election Integrity Partnership or media casebooks), including how many narratives overlap and whether the coverage rate changes on the union of the two lists.
- [Results, Coverage; SI Table 3 (logistic regression)] The logistic regression in Table 3 shows signs of quasi-complete separation: the coefficients for "Partisanship: Neutral" (-16.036, SE 1.239) and "Classification: Improbable" (16.484, SE 0.832) are implausibly large with small standard errors, which is a classic separation artifact rather than a meaningful estimate. Consequently, the claim in the Results that partisanship has "limited evidence" of influencing coverage rests on unstable model estimates, especially given only 3 neutral narratives and 9 improbable narratives. The authors should re-estimate with Firth's penalized likelihood or a Bayesian model, or present the OLS specification in Table 6 as the primary model, and should note in the text that the "Improbable" and "Neutral" coefficients are not interpretable as finite odds ratios.
- [Results, Reach (pp. 16-18)] The reach analysis defines a fact-check post only as a post containing a link to a URL of an identified fact-check. This measure excludes quotes, replies, and text-based corrections that name or describe the fact-check without a URL, and the Discussion later acknowledges that "textual corrections which do not link to an external fact-check" are outside the analysis. As a result, the 1.2% conversation share and the 4.15%/0.30%/0.38% sharing rates are lower bounds, not point estimates. The main text should state this directional bias explicitly, and ideally the authors should hand-code a sample of posts that discuss fact-checks without linking to quantify how much the reach estimate would change. The Discussion's statement that "fewer than 2% of users ... also shared a fact-check" should also be reconciled with the Table 2 percentages, which use different denominators.
minor comments (6)
- [p. 3, Introduction] The citation "election period Schafer et al., 2024" is missing an opening parenthesis; it should read "election period (Schafer et al., 2024)."
- [p. 12, Results, Coverage] The text says "at least 100,00 followers" and should read "100,000."
- [p. 15, Results, Speed] The sentence beginning "While this outcome likely comes of no surprise to fact-checkers, who prioritize ... based on their as their human and technological resources" contains a duplicated "as their" and should be rephrased.
- [Figure 2 caption] The caption refers to the "modal fact-check (red)" in one place and to the "aggregated (median) fact-check response time (vertical line)" in another, which is confusing about whether the vertical line marks the mean, median, or mode; the text and legend should use one consistent statistic.
- [Abstract and Discussion] The abstract's phrase "most comprehensive assessment to date" is an overclaim for a single-platform, single-election study with a manually curated corpus; I suggest tempering it to something like "a comprehensive assessment" or "one of the first campaign-wide assessments."
- [References] Several author names appear with a spacing artifact (e.g., "V ogels" for Vogels, "V osoughi" for Vosoughi, "V ." for V.). These should be corrected for a publication version.
Circularity Check
No circularity: fact-check coverage, speed, and reach are computed by matching an independently curated rumour corpus against a new fact-check collection, with no fitted parameter renamed as a prediction.
full rationale
The derivation chain is self-contained. Coverage (63/135 narratives), speed (median four-day delay), and reach (1.2% of posts; 0.30-4.15% of users sharing) are all computed by matching the ElectionRumors2022 corpus against a fact-check set newly collected for this paper via Google queries and Media Bias Fact Check filtering. The decisive anti-circularity design choice is explicit: "Unlike prior studies on online misinformation, we start from a baseline set of false and misleading election claims that are not determined by their prior fact-check status." The outcome variables - whether a fact-check exists, when it was published, and how often it was shared - were not inputs to the corpus construction, which was real-time observation with duplicate, over-broad, and very low-spread cases removed. The one self-citation overlap (Duskin and Wack also appear on the ElectionRumors2022 paper, and the Acknowledgements thank Joseph Schafer for data help) supplies the denominator but qualifies as independent support rather than circularity: the dataset is a codified, externally auditable artifact whose construction did not include the target results, and the present conclusions do not reduce to the citation. The logistic regression (SI Equation 1) is descriptive and in-sample; "predicted probability" is standard marginal-effects language, and no parameter is fitted to a subset and then reported as a prediction of a closely related quantity. The reach operationalization, counting as fact-check posts only those linking to identified fact-check URLs, is a construct definition, and the paper candidly bounds it in the Discussion: the analyses "do not address the full set of corrective interventions available to online platforms, including most notably textual corrections which do not link to an external fact-check as well as crowd-sourced alternatives." That is a measurement-coverage caveat (a possible undercount), which belongs under correctness risk rather than circularity, as do the Google/MBFC filtering choices and reliance on the completeness of the 135-narrative corpus. No uniqueness theorem is imported, no ansatz is smuggled in via citation, and no known result is renamed; the Table 1 versus SI inconsistency in the count of "improbable" narratives (9 versus 10) is a data-consistency issue, not a circular step.
Assumptions & free parameters
free parameters (3)
- Partisan label threshold =
80% of reposts to a single partisan influencer cluster
- Influencer follower threshold =
100,000 followers
- Virality window =
24 hours (with a 10-hour robustness check)
assumptions (5)
- domain assumption ElectionRumors2022 is a complete and accurate enumeration of prominent election misinformation narratives on X/Twitter in the 2022 US midterm period.
- domain assumption Google search with a 'fact check' query on the first ten results is a valid proxy for the accessibility of every relevant institutional fact-check.
- domain assumption MBFC bias and credibility ratings are a valid filter for 'unbiased' fact-check sources.
- domain assumption Influencer partisanship labels derived from co-engagement networks and the 80% repost threshold are accurate enough for the reach analyses.
- domain assumption Manual validation that the earliest narrative-linked post is really related to the narrative correctly establishes first-appearance times.
Cite this review
Pith. "Pith review of Political Fact-Checking Efforts are Constrained by Deficiencies in Coverage, Speed, and Reach." pith.science (2026). https://pith.science/paper/BIM4M672
@misc{pith2026241213280,
author = {Pith},
title = {Pith review of: Political Fact-Checking Efforts are Constrained by Deficiencies in Coverage, Speed, and Reach},
year = {2026},
howpublished = {\url{https://pith.science/paper/BIM4M672}},
note = {Machine review of arXiv:2412.13280}
}
read the original abstract
Fact-checking has been promoted as a key method for combating political misinformation. Comparing the spread of election-related misinformation narratives along with their relevant political fact-checks, this study provides the most comprehensive assessment to date of the real-world limitations faced by political fact-checking efforts. To examine barriers to impact, this study extends recent work from laboratory and experimental settings to the wider online information ecosystem present during the 2022 U.S. midterm elections. From analyses conducted within this context, we find that fact-checks as currently developed and distributed are severely inhibited in election contexts by constraints on their i. coverage, ii. speed, and, iii. reach. Specifically, we provide evidence that fewer than half of all prominent election-related misinformation narratives were fact-checked. Within the subset of fact-checked claims, we find that the median fact-check was released a full four days after the initial appearance of a narrative. Using network analysis to estimate user partisanship and dynamics of information spread, we additionally find evidence that fact-checks make up less than 1.2\% of narrative conversations and that even when shared, fact-checks are nearly always shared within,rather than between, partisan communities. Furthermore, we provide empirical evidence which runs contrary to the assumption that misinformation moderation is politically biased against the political right. In full, through this assessment of the real-world influence of political fact-checking efforts, our findings underscore how limitations in coverage, speed, and reach necessitate further examination of the potential use of fact-checks as the primary method for combating the spread of political misinformation.
Figures
Figures from the paper (11 more)
Reference graph
Works this paper leans on
-
[1]
Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. Journal of economic perspectives, 31(2), 211–236. Amazeen, M. A. (2016). Checking the fact-checkers in 2008: Predicting political ad scrutiny and assessing consistency. Journal of Political Marketing, 15(4), 433–464. Aslett, K., Godel, W., Sanderson, Z., Nagler, J., Bonne...
work page 2017
-
[2]
Kim, J. (2009). Describing and predicting information-seeking behavior on the web.Journal of the american society for information science and technology, 60(4), 679–693. Koch, T. K., Frischlich, L., & Lermer, E. (2023). Effects of fact-checking warning labels and social endorsement cues on climate change fake news credibility and engagement on social medi...
arXiv 2009
-
[7]
M., Pennycook, G., Berinsky, A
Brashier, N. M., Pennycook, G., Berinsky, A. J., & Rand, D. G. (2021). Timing matters when correcting fake news.Proceedings of the National Academy of Sciences, 118(5), e2020043118. Bruns, A. (2021). After the ‘APIcalypse’: Social media platforms and their fight against critical scholarly research. Disinformation and Data Lockdown on Social Platforms, 14–...
work page 2021
-
[9]
=β0 + β1 × Partisanship + β2 × log(Virality) + β3 × log(Influencers) + β4 × Classification + β5 × Timing (1) where the output represents the binary outcome of whether a fact-check was produced or the specified narrative remained un-fact-checked (0/1), β0 is the intercept, β1 is a categorical vari- able with three levels denoting variations partisan leanin...
work page 2022
-
[19]
V osoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. science, 359(6380), 1146–1151. Vraga, E. K., & Bode, L. (2020). Correction as a solution for health misinformation on social media. American Journal of Public Health, 110(S3), S278–S280. Walter, N., Cohen, J., Holbert, R. L., & Morag, Y . (2020). Fact-checking: A meta-ana...
work page 2018
-
[115]
Jiang, S., Robertson, R. E., & Wilson, C. (2019). Bias misperceived: The role of partisanship and misinformation in youtube comment moderation. Proceedings of the International AAAI Conference on Web and Social Media, 13, 278–289. Kennedy, I., Wack, M., Beers, A., Schafer, J. S., Garcia-Camargo, I., Spiro, E. S., & Starbird, K. (2022). Repeat spreaders an...
work page 2019
-
[150]
Friggeri, A., Adamic, L., Eckles, D., & Cheng, J. (2014). Rumor cascades.Eighth International AAAI Conference on Weblogs and Social Media. Garrett, R. K., & Weeks, B. E. (2013). The promise and peril of real-time corrections to political misperceptions, 1047–1058. Godel, W., Sanderson, Z., Aslett, K., Nagler, J., Bonneau, R., Persily, N., & Tucker, J. A. ...
arXiv 2014
-
[237]
Rosenfeld, N., Szanto, A., & Parkes, D. C. (2020). A kernel of truth: Determining rumor ve- racity on twitter by diffusion pattern alone. Proceedings of The Web Conference 2020, 1018–1028. Saeed, M., Traub, N., Nicolas, M., Demartini, G., & Papotti, P. (2022). Crowdsourced Fact- Checking at twitter: How does the crowd compare with experts?Proceedings of t...
arXiv 2020
Show all 9 references
-
[2021]
https://www.pewresearch.org/internet/2021/04/07/social-media-use-in-2021/ Azevedo, L
Pew Research Center, 1, 1–4. https://www.pewresearch.org/internet/2021/04/07/social-media-use-in-2021/ Azevedo, L. (2018). Truth or lie: Automatically fact checking news. Companion Proceedings of the The Web Conference 2018, 807–811. Bardon, A. (2019). The truth about denial: ...
2018
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.