Forecasting Political News Engagement on Social Media

Aron Culotta; Karthik Shivaram; Matthew Shapiro; Mustafa Bilgic

REVIEW 2 major objections 2 minor 16 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

A neural network that combines past Twitter engagements with tweet content can forecast the political lean of news articles users will engage with next.

2026-06-28 07:28 UTC pith:S4DB5Z6N

load-bearing objection The paper assembles a 60M-tweet corpus from politically engaged users and trains a neural forecaster for engagement lean before clustering, but the reported patterns rest on unvalidated annotations of news outlets in only 10% of the data. the 2 major comments →

arxiv 2606.04293 v1 pith:S4DB5Z6N submitted 2026-06-02 cs.SI

Forecasting Political News Engagement on Social Media

Karthik Shivaram , Mustafa Bilgic , Matthew Shapiro , Aron Culotta This is my paper

classification cs.SI

keywords political newssocial medianeural networksforecastinghyperpartisanshipTwittermisinformationfilter bubbles

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper assembles over 60 million tweets from politically engaged users across seven years and annotates roughly 10 percent with the political leanings of mentioned news outlets. It trains a neural network to predict the leaning of future news engagements by using both a user's history of interactions and the text of their tweets. The learned user representations are then clustered to surface long-term patterns. These patterns show hyperpartisan users engage with more news overall, right-leaning users reach across the aisle more often than left-leaning users, and certain topics mark higher engagement with low-quality sources. The work supplies a concrete way to track how political news consumption evolves at scale.

Core claim

A neural network trained to forecast the political lean of news articles that Twitter users will engage with, given sequences of prior engagements and tweet content, produces user representations whose clusters reveal that hyperpartisan users are more engaged with news, right-leaning users engage with contra-partisan sources more than left-leaning users, and topics such as immigration, COVID-19, Islamaphobia, and gun control indicate engagement with low-quality news sources.

What carries the argument

Neural network that learns user representations from past news engagements and tweet content for forecasting political lean and subsequent clustering.

Load-bearing premise

The manual or semi-automatic labels assigning political leanings to news outlets in roughly 10 percent of the tweets are accurate enough that forecasting accuracy and cluster patterns are not driven by labeling errors.

What would settle it

Retraining the model on an independently verified set of outlet leanings and finding that the reported differences in engagement volume, cross-aisle behavior, or topic associations disappear or reverse.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Hyperpartisan users engage with more news than other politically engaged users.
Right-leaning users engage with sources from the opposite side more often than left-leaning users.
Topics including immigration, COVID-19, Islamaphobia, and gun control mark higher engagement with low-quality news sources.
The learned representations support clustering that surfaces these long-term consumption patterns over seven years.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The asymmetry in cross-aisle engagement could be tested by measuring whether right-leaning users' contra-partisan exposures actually change their expressed views over time.
The same forecasting setup could be applied to detect early shifts toward hyperpartisanship before cluster membership becomes stable.
Extending the model to include engagement with non-news content might clarify whether news consumption alone drives the observed topic associations with low-quality sources.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The paper assembles a 60M-tweet corpus from politically engaged users and trains a neural forecaster for engagement lean before clustering, but the reported patterns rest on unvalidated annotations of news outlets in only 10% of the data.

read the letter

The core of this work is a seven-year collection of 60 million tweets from a filtered set of politically active Twitter users, with roughly 10% labeled by news outlet lean. They train a neural network that takes past engagements plus tweet text to predict the lean of the next news item a user will interact with, then use the model's internal representations to cluster users and surface patterns such as higher overall engagement among hyperpartisan accounts and greater cross-partisan reading on the right.

The dataset scale and the longitudinal span are the clearest strengths. Pulling together that volume of tweets and linking them to outlet leans gives a usable resource for anyone studying how engagement evolves. Turning the forecasting model into a representation for clustering is a straightforward but practical move that lets them extract user groups without hand-crafted features.

The main weakness is the annotation step. The abstract gives no numbers on how the 10% of tweets were labeled, what agreement was achieved, or whether the process was validated on a held-out set. Because both the supervised forecasting task and all the cluster interpretations flow from those labels, any systematic bias in which outlets were chosen or how leans were assigned would affect the downstream claims about hyperpartisanship, asymmetric consumption, and topic correlations. The pre-selection of politically engaged users adds another layer of selection that is not quantified.

This paper is for researchers who work with social-media traces and want observational patterns on polarization or low-quality news. A reader already familiar with Twitter data pipelines will see incremental extensions rather than new machinery, but the concrete findings on engagement differences could still be cited as descriptive results.

It is worth sending to peer review. The data effort is large enough that referees can usefully check the labeling protocol and model performance numbers that are missing from the abstract.

Referee Report

2 major / 2 minor

Summary. The paper curates a dataset of over 60M tweets from politically engaged users spanning seven years, annotates roughly 10% of them with news-outlet mentions and political leanings, trains a neural network to forecast the political lean of future news engagements using both historical engagement patterns and tweet content, and applies clustering to the learned user representations to identify long-term engagement patterns. The reported findings are that hyperpartisan users engage more with news, right-leaning users consume more contra-partisan sources than left-leaning users, and topics such as immigration, COVID-19, Islamaphobia, and gun control correlate with engagement with low-quality sources.

Significance. If the annotation quality and model validation hold, the work supplies a large-scale empirical view of temporal shifts in political news consumption on Twitter and demonstrates the utility of learned representations for discovering engagement archetypes. The scale of the data collection and the dual use of supervised forecasting plus unsupervised clustering on the same representations constitute a methodological contribution that could be cited in studies of polarization and misinformation.

major comments (2)

[Abstract / Data Annotation] The annotation procedure for the ~10% of tweets labeled with news-outlet political leanings (described in the abstract and presumably detailed in the data section) supplies no inter-annotator agreement statistics, held-out accuracy figures, or error analysis. Because these labels constitute the sole supervision signal for the forecasting model and the sole basis for all cluster interpretations (hyperpartisan engagement, asymmetric contra-partisan consumption, topic–low-quality correlations), any systematic labeling bias or low consistency directly undermines the central claims.
[Abstract / Model Training] No baseline comparisons, validation metrics, or error bars are referenced for the neural forecasting model (abstract). Without these, it is impossible to assess whether the reported forecasting performance exceeds trivial predictors (e.g., majority-class or recency-based) or whether the learned representations are meaningfully richer than simpler user-history features.

minor comments (2)

[Abstract] The spelling 'Islamaphobia' in the abstract should be corrected to 'Islamophobia'.
[Abstract] The politically-engaged user filter is mentioned but its precise definition (e.g., minimum tweet volume or keyword criteria) is not stated in the abstract; this detail should be supplied early to allow readers to judge selection effects.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their insightful comments. Below we provide point-by-point responses to the major comments and indicate the revisions we plan to make.

read point-by-point responses

Referee: [Abstract / Data Annotation] The annotation procedure for the ~10% of tweets labeled with news-outlet political leanings (described in the abstract and presumably detailed in the data section) supplies no inter-annotator agreement statistics, held-out accuracy figures, or error analysis. Because these labels constitute the sole supervision signal for the forecasting model and the sole basis for all cluster interpretations (hyperpartisan engagement, asymmetric contra-partisan consumption, topic–low-quality correlations), any systematic labeling bias or low consistency directly undermines the central claims.

Authors: The referee correctly notes that the abstract lacks these details. In the revised manuscript, we will expand the abstract to summarize the annotation validation and add a dedicated subsection in the data section with inter-annotator agreement statistics, held-out accuracy on a labeled sample, and error analysis. This will directly address the concern about the reliability of the supervision signal. revision: yes
Referee: [Abstract / Model Training] No baseline comparisons, validation metrics, or error bars are referenced for the neural forecasting model (abstract). Without these, it is impossible to assess whether the reported forecasting performance exceeds trivial predictors (e.g., majority-class or recency-based) or whether the learned representations are meaningfully richer than simpler user-history features.

Authors: We agree that the abstract should reference the model validation. The experiments section of the manuscript includes baseline comparisons and validation metrics. We will update the abstract to mention these comparisons and the use of error bars, making it clear that the neural model outperforms trivial predictors. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical ML pipeline is self-contained

full rationale

The paper's workflow consists of data curation (60M tweets), manual/semi-automatic annotation of ~10% with outlet political leanings, supervised training of a neural network to forecast future engagement leans from history plus content, and clustering on the resulting user representations. Reported patterns (hyperpartisan engagement levels, asymmetric contra-partisan consumption, topic-low-quality correlations) are extracted post-training from the learned embeddings and are not equivalent by construction to any fitted parameter or input label. No self-definitional equations, fitted-input-called-prediction, or load-bearing self-citation chains appear; the forecasting task is a standard supervised prediction evaluated on data, and clustering is an unsupervised discovery step on the trained model outputs. The derivation chain therefore remains independent of its own fitted values.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The abstract supplies no explicit free parameters, mathematical axioms, or newly postulated entities; all elements described are standard data-collection and modeling steps.

pith-pipeline@v0.9.1-grok · 5692 in / 1246 out tokens · 31440 ms · 2026-06-28T07:28:36.617958+00:00 · methodology

0 comments

read the original abstract

Understanding how political news consumption changes over time can provide insights into issues such as hyperpartisanship, filter bubbles, and misinformation. To investigate long-term trends of news consumption, we curate a collection of over 60M tweets from politically engaged users over seven years, annotating ~10% with mentions of news outlets and their political leaning. We then train a neural network to forecast the political lean of news articles Twitter users will engage with, considering both past news engagements as well as tweet content. Using the learned representation of this model, we cluster users to discover salient patterns of long-term news engagement. Our findings include the following: (1) hyperpartisan users are more engaged with news; (2) right-leaning users engage with contra-partisan sources more than left-leaning users; (3) topics such as immigration, COVID-19, Islamaphobia, and gun control are salient indicators of engagement with low quality news sources.

Figures

Figures reproduced from arXiv: 2606.04293 by Aron Culotta, Karthik Shivaram, Matthew Shapiro, Mustafa Bilgic.

**Figure 1.** Figure 1: Train-test splits for D1 (2015-2018), showing the time ranges used for each instance. Analogous time ranges are used for D2-D4. given user engages with news sources across all stances (−3 to +3) in subsequent time steps. 5 Methods We discuss in this section our data pre-processing techniques, a number of forecasting models employed via multiple neural network architectures, and our baseline method. We al… view at source ↗

**Figure 2.** Figure 2: Mean absolute error by quarter. The rising error is [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Figure 3: True vs. predictied values for users with the highest errors, according to the SFN+C model. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

**Figure 4.** Figure 4: Error by transition points based on cosine distance [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

**Figure 5.** Figure 5: Visualization of 20 discovered clusters of users based on the learned representation of the forecasting model. Over [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

16 extracted references · 2 canonical work pages

[1]

InConference Companion Publication of the 2019 on Computer Supported Coopera- tive Work and Social Computing, 151–155

Gobo: A System for Exploring User Control of Invis- ible Algorithms in Social Media. InConference Companion Publication of the 2019 on Computer Supported Coopera- tive Work and Social Computing, 151–155. Bhuiyan, M. M.; Horning, M.; Lee, S. W.; and Mitra, T

2019
[2]

oil spill

NudgeCred: Supporting News Credibility Assessment on Social Media Through Nudges.Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2): 1–30. Brady, W. J.; Wills, J. A.; Jost, J. T.; Tucker, J. A.; and Bavel, J. J. V . 2017. Emotion shapes the diffusion of moral- ized content in social networks.Proceedings of the National Academy of Sciences, 114...

2017
[3]

Climate change

Research note: Lies and presidential debates: How political misinformation spread across media streams during the 2020 election.Harvard Kennedy School Misinformation Review. Hasan, M.; Orgun, M. A.; and Schwitter, R. 2018. A survey on real-time event detection from the Twitter data stream. Journal of Information Science, 44(4): 443–463. Hasell, A. 2021. S...

2020
[4]

Lutzke, L.; Drummond, C.; Slovic, P.; and ´Arvai, J

Factors predicting willingness to share COVID-19 misinformation.Frontiers in psychology, 11: 2413. Lutzke, L.; Drummond, C.; Slovic, P.; and ´Arvai, J. 2019. Priming critical thinking: Simple interventions limit the in- fluence of fake news about climate change on Facebook. Global Environmental Change, 58: 101964. Masrour, F.; Wilson, T.; Yan, H.; Tan, P....

2019
[5]

InInternational Conference on Learning Representations

N-BEATS: Neural basis expansion analysis for inter- pretable time series forecasting. InInternational Conference on Learning Representations. Osmundsen, M.; Bor, A.; Vahlstrup, P. B.; Bechmann, A.; and Petersen, M. B. 2021a. Partisan Polarization Is the Pri- mary Psychological Motivation behind Political Fake News Sharing on Twitter.American Political Sci...

work page arXiv 2021
[6]

InProceedings of the International Conference on Web and Social Media

Characterizing Online Criticism of Partisan News Me- dia Using Weakly Supervised Learning. InProceedings of the International Conference on Web and Social Media. Shivaram, K.; Liu, P.; Shapiro, M.; Bilgic, M.; and Culotta, A. 2022. Reducing Cross-Topic Political Homogenization in Content-Based News Recommendation. InProceedings of the 16th ACM Conference ...

work page arXiv 2022
[7]

For most authors... (a) Would answering this research question advance sci- ence without violating social contracts, such as violat- ing privacy norms, perpetuating unfair profiling, exac- erbating the socio-economic divide, or implying disre- spect to societies or cultures? Yes, see Discussion and Limitations. (b) Do your main claims in the abstract and ...
[8]

(a) Did you clearly state the assumptions underlying all theoretical results? Yes, see Model Accuracy

Additionally, if your study involves hypotheses testing... (a) Did you clearly state the assumptions underlying all theoretical results? Yes, see Model Accuracy. (b) Have you provided justifications for all theoretical re- sults? Yes, see Model Accuracy (c) Did you discuss competing hypotheses or theories that might challenge or complement your theoretica...
[9]

(a) Did you state the full set of assumptions of all theoret- ical results? NA (b) Did you include complete proofs of all theoretical re- sults? NA

Additionally, if you are including theoretical proofs... (a) Did you state the full set of assumptions of all theoret- ical results? NA (b) Did you include complete proofs of all theoretical re- sults? NA
[10]

(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (ei- ther in the supplemental material or as a URL)? Yes

Additionally, if you ran machine learning experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (ei- ther in the supplemental material or as a URL)? Yes. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? Yes, see Methods and Experimental ...
[11]

(a) If your work uses existing assets, did you cite the cre- ators? Yes, see Data

Additionally, if you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the cre- ators? Yes, see Data. (b) Did you mention the license of the assets? NA (c) Did you include any new assets in the supplemental material or as a URL? Yes. (d) Did you discuss whether and ...
[12]

Additionally, if you used crowdsourcing or conducted re- search with human subjects... (a) Did you include the full text of instructions given to participants and screenshots? NA (b) Did you describe any potential participant risks, with mentions of Institutional Review Board (IRB) ap- provals? NA (c) Did you include the estimated hourly wage paid to part...

2015
[13]

Follower Size (≤1000)
[14]

Following Size (≤1000)
[15]

Daily Tweet Activity (≤10)
[16]

Total Tweets authored during the life of the account (≥ 1000and≤30000) Hyperparameters We search over the hyperparameters shown in Table 9 to train our models, picking the best settings based on mean absolute error on the validation set. Parameter Values Hidden Units - LSTM 32, 64, 128, 256, 512 Hidden Units - Linear 128, 512 Bidirectional True, False LST...

[1] [1]

InConference Companion Publication of the 2019 on Computer Supported Coopera- tive Work and Social Computing, 151–155

Gobo: A System for Exploring User Control of Invis- ible Algorithms in Social Media. InConference Companion Publication of the 2019 on Computer Supported Coopera- tive Work and Social Computing, 151–155. Bhuiyan, M. M.; Horning, M.; Lee, S. W.; and Mitra, T

2019

[2] [2]

oil spill

NudgeCred: Supporting News Credibility Assessment on Social Media Through Nudges.Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2): 1–30. Brady, W. J.; Wills, J. A.; Jost, J. T.; Tucker, J. A.; and Bavel, J. J. V . 2017. Emotion shapes the diffusion of moral- ized content in social networks.Proceedings of the National Academy of Sciences, 114...

2017

[3] [3]

Climate change

Research note: Lies and presidential debates: How political misinformation spread across media streams during the 2020 election.Harvard Kennedy School Misinformation Review. Hasan, M.; Orgun, M. A.; and Schwitter, R. 2018. A survey on real-time event detection from the Twitter data stream. Journal of Information Science, 44(4): 443–463. Hasell, A. 2021. S...

2020

[4] [4]

Lutzke, L.; Drummond, C.; Slovic, P.; and ´Arvai, J

Factors predicting willingness to share COVID-19 misinformation.Frontiers in psychology, 11: 2413. Lutzke, L.; Drummond, C.; Slovic, P.; and ´Arvai, J. 2019. Priming critical thinking: Simple interventions limit the in- fluence of fake news about climate change on Facebook. Global Environmental Change, 58: 101964. Masrour, F.; Wilson, T.; Yan, H.; Tan, P....

2019

[5] [5]

InInternational Conference on Learning Representations

N-BEATS: Neural basis expansion analysis for inter- pretable time series forecasting. InInternational Conference on Learning Representations. Osmundsen, M.; Bor, A.; Vahlstrup, P. B.; Bechmann, A.; and Petersen, M. B. 2021a. Partisan Polarization Is the Pri- mary Psychological Motivation behind Political Fake News Sharing on Twitter.American Political Sci...

work page arXiv 2021

[6] [6]

InProceedings of the International Conference on Web and Social Media

Characterizing Online Criticism of Partisan News Me- dia Using Weakly Supervised Learning. InProceedings of the International Conference on Web and Social Media. Shivaram, K.; Liu, P.; Shapiro, M.; Bilgic, M.; and Culotta, A. 2022. Reducing Cross-Topic Political Homogenization in Content-Based News Recommendation. InProceedings of the 16th ACM Conference ...

work page arXiv 2022

[7] [7]

For most authors... (a) Would answering this research question advance sci- ence without violating social contracts, such as violat- ing privacy norms, perpetuating unfair profiling, exac- erbating the socio-economic divide, or implying disre- spect to societies or cultures? Yes, see Discussion and Limitations. (b) Do your main claims in the abstract and ...

[8] [8]

(a) Did you clearly state the assumptions underlying all theoretical results? Yes, see Model Accuracy

Additionally, if your study involves hypotheses testing... (a) Did you clearly state the assumptions underlying all theoretical results? Yes, see Model Accuracy. (b) Have you provided justifications for all theoretical re- sults? Yes, see Model Accuracy (c) Did you discuss competing hypotheses or theories that might challenge or complement your theoretica...

[9] [9]

(a) Did you state the full set of assumptions of all theoret- ical results? NA (b) Did you include complete proofs of all theoretical re- sults? NA

Additionally, if you are including theoretical proofs... (a) Did you state the full set of assumptions of all theoret- ical results? NA (b) Did you include complete proofs of all theoretical re- sults? NA

[10] [10]

(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (ei- ther in the supplemental material or as a URL)? Yes

Additionally, if you ran machine learning experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (ei- ther in the supplemental material or as a URL)? Yes. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? Yes, see Methods and Experimental ...

[11] [11]

(a) If your work uses existing assets, did you cite the cre- ators? Yes, see Data

Additionally, if you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the cre- ators? Yes, see Data. (b) Did you mention the license of the assets? NA (c) Did you include any new assets in the supplemental material or as a URL? Yes. (d) Did you discuss whether and ...

[12] [12]

Additionally, if you used crowdsourcing or conducted re- search with human subjects... (a) Did you include the full text of instructions given to participants and screenshots? NA (b) Did you describe any potential participant risks, with mentions of Institutional Review Board (IRB) ap- provals? NA (c) Did you include the estimated hourly wage paid to part...

2015

[13] [13]

Follower Size (≤1000)

[14] [14]

Following Size (≤1000)

[15] [15]

Daily Tweet Activity (≤10)

[16] [16]

Total Tweets authored during the life of the account (≥ 1000and≤30000) Hyperparameters We search over the hyperparameters shown in Table 9 to train our models, picking the best settings based on mean absolute error on the validation set. Parameter Values Hidden Units - LSTM 32, 64, 128, 256, 512 Hidden Units - Linear 128, 512 Bidirectional True, False LST...