pith. sign in

arxiv: 2606.04289 · v1 · pith:Y73Q76KZnew · submitted 2026-06-02 · 💻 cs.SI

Characterizing Online Criticism of Partisan News Media Using Weakly Supervised Learning

Karthik Shivaram , Mustafa Bilgic , Matthew Shapiro , Aron Culotta This is my paper

Pith reviewed 2026-06-28 07:31 UTC · model grok-4.3

classification 💻 cs.SI
keywords weakly supervised learningsocial mediamedia criticismpartisan newsTwitter analysishyperpartisanshipinformation ecosystem
0
0 comments X

The pith

Weakly supervised learning identifies tweets criticizing partisan news and links them to spikes during polarizing events.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a weakly supervised approach to detect tweets criticizing partisan news sources when labeled data is scarce. It builds a classifier from multiple noisy signals drawn from tweet wording and each user's history of sharing news links. The resulting labels support analysis of how criticism changes across users, sources, and time periods. The study reports clear rises in such criticism during major political events and after users encounter unreliable outlets. These patterns matter because criticism and distrust of media are thought to contribute to hyperpartisanship and fragmented information environments.

Core claim

We develop a weakly supervised learning approach that leverages multiple noisy labeling functions based on both the content of the tweet as well as the historical news sharing behavior of the user. Using this classifier, we explore how tweets expressing criticism vary by user, news source, and time, finding substantial spikes in media criticism during politically polarizing events, such as the investigation into Russian interference in the 2016 U.S. elections and the 2017 unite the right rally in Charlottesville. This type of media-targeting criticism is also more likely to occur after users have been exposed to unreliable and hyperpartisan media.

What carries the argument

A classifier trained via multiple noisy labeling functions that combine tweet content features with users' past news-sharing records.

If this is right

  • Media criticism shows sharp increases during events such as the 2016 election interference probe and the 2017 Charlottesville rally.
  • Users become more likely to post media criticism after they have shared links from unreliable or hyperpartisan sources.
  • Criticism rates differ systematically across individual users, specific news outlets, and calendar periods.
  • Tracking these patterns supplies one concrete way to monitor the health of the broader information ecosystem.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same labeling strategy could be reused on other platforms or languages to compare media-distrust patterns across countries.
  • If the post-exposure effect holds in longitudinal data, it would suggest a self-reinforcing cycle between unreliable sources and subsequent criticism.
  • Testing whether interventions that reduce hyperpartisan exposure also lower criticism rates would provide a direct check on the observed correlation.

Load-bearing premise

The noisy labeling functions drawn from tweet content and user news-sharing history supply enough reliable signal to train a classifier whose outputs can be used for temporal and behavioral analysis.

What would settle it

A hand-labeled test set of several thousand tweets in which the classifier's positive and negative predictions match human judgments at rates no better than random chance would falsify the method.

Figures

Figures reproduced from arXiv: 2606.04289 by Aron Culotta, Karthik Shivaram, Matthew Shapiro, Mustafa Bilgic.

Figure 1
Figure 1. Figure 1: Example tweet critical of a news source. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: ROC curves of network models trained using dif [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Criticism shown towards the most mentioned news sources from each partisan stance. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Criticism ratio by partisan stance of the user (left [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of Normalized Stance Entropy before [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Criticism of Popular News Sources through Time [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
read the original abstract

We propose novel methods to identify tweets that criticize partisan news sources. Prior work suggests that criticism, ridicule, and distrust of news media all play important roles in hyperpartisanship, misinformation, and filter bubble formation. Thus, understanding the prevalence and temporal dynamics of media-targeted criticism can provide us with updated tools to assess the health of the information ecosystem. There is a scarcity of labeled data for this task, and we develop a weakly supervised learning approach that leverages multiple noisy labeling functions based on both the content of the tweet as well as the historical news sharing behavior of the user. Using this classifier, we explore how tweets expressing criticism vary by user, news source, and time, finding substantial spikes in media criticism during politically polarizing events, such as the investigation into Russian interference in the 2016 U.S.~elections and the 2017 ``unite the right'' rally in Charlottesville. This type of media-targeting criticism is also more likely to occur after users have been exposed to unreliable and hyperpartisan media.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper develops a weakly supervised learning approach to detect tweets criticizing partisan news sources, using multiple noisy labeling functions derived from tweet content and users' historical news-sharing behavior. The resulting classifier is applied to characterize how such criticism varies by user, news source, and time, with reported findings of substantial spikes during polarizing events (e.g., 2016 Russian interference investigation, 2017 Charlottesville rally) and higher likelihood following user exposure to unreliable or hyperpartisan media.

Significance. If the classifier outputs prove reliable, the work could contribute tools for quantifying media-targeted criticism and its links to hyperpartisanship and filter bubbles, addressing data scarcity via weak supervision. The temporal and exposure analyses, if validated, would offer concrete evidence on information ecosystem dynamics during key events.

major comments (2)
  1. [Methods] Methods section: No validation metrics, baseline comparisons, error analysis, or details on labeling function construction/evaluation against gold-standard labels are provided. This is load-bearing for the central claim, as downstream conclusions on temporal spikes and post-exposure effects rest entirely on unverified classifier outputs.
  2. [Results] Results/Discussion: The labeling functions rely on the same tweet content and user news-sharing signals analyzed downstream, creating risk that systematic correlations or biases in weak labels could artifactually generate the reported patterns (e.g., event spikes) without independent verification of label quality.
minor comments (1)
  1. [Abstract] Abstract and introduction could more explicitly state the scope of the Twitter dataset (time period, volume, collection method) to ground the scale of the analysis.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their detailed and constructive report. We address each major comment below, indicating the revisions we will make to strengthen the validation and discussion of potential biases in our weakly supervised approach.

read point-by-point responses

  1. Referee: [Methods] Methods section: No validation metrics, baseline comparisons, error analysis, or details on labeling function construction/evaluation against gold-standard labels are provided. This is load-bearing for the central claim, as downstream conclusions on temporal spikes and post-exposure effects rest entirely on unverified classifier outputs.

    Authors: We agree that the current Methods section would benefit from expanded details on validation. In the revised manuscript, we will add a dedicated subsection describing the construction of each labeling function (including the specific content heuristics such as criticism keywords and user-history rules based on known media sources), their estimated accuracies via comparison to a gold-standard set of 400 manually labeled tweets, baseline comparisons against a keyword-only matcher and a supervised logistic regression model, and an error analysis of common misclassifications. These additions will directly support the reliability of the classifier outputs used for the temporal and exposure analyses. revision: yes

  2. Referee: [Results] Results/Discussion: The labeling functions rely on the same tweet content and user news-sharing signals analyzed downstream, creating risk that systematic correlations or biases in weak labels could artifactually generate the reported patterns (e.g., event spikes) without independent verification of label quality.

    Authors: We acknowledge the risk of circularity highlighted here. While the labeling functions apply targeted, rule-based signals (specific phrases and historical sharing from predefined unreliable sources) that differ from the aggregate downstream metrics (e.g., alignment with external event dates), we will address this by adding an ablation study in the revision that retrains the model using only content-based labeling functions and confirms that the reported spikes during events like the 2016 election investigation and Charlottesville rally, as well as the exposure effects, persist. We will also expand the Discussion to explicitly note this limitation of weak supervision and the steps taken to mitigate it. revision: partial

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper proposes a weakly supervised classifier trained on noisy labeling functions derived from tweet content and user news-sharing history, then applies the resulting labels to analyze temporal spikes, user/source variation, and post-exposure effects. No equations, fitted parameters, or self-referential definitions are present in the abstract or described method. The labeling functions and downstream analyses operate on overlapping data sources but do not reduce the reported patterns (e.g., event spikes or exposure correlations) to the inputs by construction; the classifier outputs remain an intermediate step whose quality is not claimed to be tautological. No self-citations or uniqueness theorems are invoked as load-bearing. The derivation is self-contained as a standard weak-supervision pipeline with exploratory analysis.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no explicit free parameters, axioms, or invented entities are described. The approach implicitly assumes that noisy labeling functions derived from content and user history are adequate proxies for ground truth.

pith-pipeline@v0.9.1-grok · 5715 in / 1110 out tokens · 28926 ms · 2026-06-28T07:31:40.531292+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    InProceedings of the fifteenth ACM international conference on web search and data min- ing, 241–251

    Semi-supervised stance detection of tweets via dis- tant network supervision. InProceedings of the fifteenth ACM international conference on web search and data min- ing, 241–251. Eady, G.; Bonneau, R.; Tucker, J.; and Nagler, J. 2021. News Sharing on Social Media: Mapping the Ideology of News Media Content, Citizens, and Politicians. OSF Preprints. Ferra...

    2021

  2. [2]

    Jurkowitz, M.; Mitchell, A.; Shearer, E.; and Walker, M

    Emotions: The unexplored fuel of fake news on social media.Journal of Management Information Systems, 38(4): 1039–1066. Jurkowitz, M.; Mitchell, A.; Shearer, E.; and Walker, M. 2020. U.S. Media Polarization and the 2020 Election: A Nation Divided. https://www.pewresearch.org/journalism/2020/01/24/u- s-media-polarization-and-the-2020-election-a-nation- div...

    work page arXiv 2020

  3. [3]

    Adam: A Method for Stochastic Optimization

    PMLR. Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980. Li, C.; and Goldwasser, D. 2021. MEAN: Multi-head Entity Aware Attention Networkfor Political Perspective Detection in News Media. InProceedings of the Fourth Workshop on NLP for Internet Freedom: Censorship, Disinformation, and Propaganda, 66–...

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  4. [4]

    For most authors... (a) Would answering this research question advance sci- ence without violating social contracts, such as violat- ing privacy norms, perpetuating unfair profiling, exac- erbating the socio-economic divide, or implying disre- spect to societies or cultures? Yes, see Ethics section. (b) Do your main claims in the abstract and introduction...

  5. [5]

    (a) Did you clearly state the assumptions underlying all theoretical results? Yes, see Sections 6 and 7

    Additionally, if your study involves hypotheses testing... (a) Did you clearly state the assumptions underlying all theoretical results? Yes, see Sections 6 and 7. (b) Have you provided justifications for all theoretical re- sults? Yes, see Sections 6 and 7. (c) Did you discuss competing hypotheses or theories that might challenge or complement your theor...

  6. [6]

    (a) Did you state the full set of assumptions of all theoret- ical results? NA (b) Did you include complete proofs of all theoretical re- sults? NA

    Additionally, if you are including theoretical proofs... (a) Did you state the full set of assumptions of all theoret- ical results? NA (b) Did you include complete proofs of all theoretical re- sults? NA

  7. [7]

    Additionally, if you ran machine learning experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (ei- ther in the supplemental material or as a URL)? No, but have released code and data. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? Ye...

  8. [8]

    (a) If your work uses existing assets, did you cite the cre- ators? Yes, see Data

    Additionally, if you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the cre- ators? Yes, see Data. (b) Did you mention the license of the assets? NA (c) Did you include any new assets in the supplemental material or as a URL? Yes, we have released data and code a...

    2024

  9. [9]

    Additionally, if you used crowdsourcing or conducted re- search with human subjects... (a) Did you include the full text of instructions given to participants and screenshots? NA (b) Did you describe any potential participant risks, with mentions of Institutional Review Board (IRB) ap- provals? NA (c) Did you include the estimated hourly wage paid to part...

    2015

  10. [10]

    Follower Size (≤1000)

  11. [11]

    Following Size (≤1000)

  12. [12]

    Daily Tweet Activity (≤10)

  13. [13]

    hey” and “dear

    Total Tweets authored during the life of the account (≥ 1000and≤30000) A.1 Additional Validation To further validate the quality of the predictions the set of 1.2M tweets, we performed two additional checks. First, we manually annotated a random sample of 150 tweets pre- dicted as criticism and 150 predicted as non-criticism, find- ing an accuracy score o...

    2016