Cross-Subreddit Behavior as Open-Source Indicators of Coordinated Influence: A Case Study of r/Sino & r/China

Ian McCulloh; Manon Pilaud

REVIEW 4 major objections 5 minor 21 references

Cross-Subreddit Behavior as Open-Source Indicators of Coordinated Influence: A Case Study of r/Sino & r/China

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Users active in both r/Sino and r/China show sentiment spikes, flat affect, and behavioral flags consistent with coordinated influence.

desk verdict A decent exploratory snapshot of dual-subreddit users, but the central 'strategic amplifier' claim needs a null model before it can support coordinated-influence inferences. read the letter →

arxiv 2507.16857 v1 pith:7JZM52KY submitted 2025-07-21 cs.SI

Manon Pilaud , Ian McCulloh This is my paper · ORCID

classification cs.SI

keywords BotDetectionChinaCoordinatedMessagingOnlinePoliticalDiscourseRedditSentimentAnalysisTopicModelingOSINT

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the small group of Reddit users who participate in both r/Sino and r/China—two ideologically opposed communities about Chinese politics—carry signs of coordinated influence activity. It tries to show that these dual-subreddit users do not simply mirror the wider Reddit discourse, but display sentiment outliers, unusually flat affect, and behavioral anomalies consistent with strategic narrative amplification. The study builds a user–topic sentiment matrix from topic modeling and sentiment analysis, compares each user's scores to global topic baselines, and layers on behavioral flags such as low lexical diversity, karma imbalance, and account-age irregularities. If the interpretation holds, cross-community participation itself becomes an open-source indicator for spotting possible influence operations in contested information spaces.

What carries the argument

The central object is the dual-subreddit user: any account with at least three posts or comments in each of r/Sino and r/China (63 users in this corpus). The argument is carried by three linked instruments: a user–topic sentiment matrix built from Latent Dirichlet Allocation (LDA) topic assignments and per-post sentiment scores; a comparison of each user's topic-level sentiment against both a dual-user average and a global all-user baseline; and a behavioral flag framework that counts anomalies in account age, karma distribution, email verification, lexical diversity, and dominant language. A subreddit co-participation network then maps where flagged users sit in the larger Reddit ecosystem. The load-bearing comparison is the deviation of individual sentiment and behavior from the global baseline: that deviation is what turns ordinary cross-community browsing into a possible indicator of strategic engagement.

What would settle it

Apply the same flag framework to dual users in an ideologically opposed subreddit pair with no suspected influence operation; if the outlier rates (low lexical diversity, karma imbalance, sentiment deviation) match those seen in the r/Sino–r/China pair, the indicators are not diagnostic of coordinated influence.

Watch

Extended reading notes

Core claim

Dual-subreddit users—people who post or comment at least three times in both r/Sino and r/China—exhibit linguistic, affective, and behavioral patterns that deviate from the general r/Sino and r/China populations. The global topic model yields well-separated thematic clusters, while the dual-user model produces diffuse, emotionally tinted topics that blend policy terms with informal rhetoric. Sentiment analysis finds a subset of 7 users with sharply positive sentiment (average 0.427) on trade and economic topics whose global baseline is near-neutral (0.094), a group of 5 low-variance users with nearly constant tone across topics, and 42 negative outliers. Behavioral flags—low lexical diversity in 51 of 63 users, karma imbalances, and two accounts suspended by Reddit—converge on a small set of elevated-risk participants. The paper concludes that these users may not act as neutral ideological bridges but could function as strategic amplifiers, modulating tone and diffusing narratives across communities.

Load-bearing premise

The paper assumes that its heuristic flags and sentiment deviations—measured against a baseline that includes the very same users—are valid indicators of inauthentic or strategically structured participation, rather than organic variation in how people talk across communities.

Editorial extensions

If this is right

Dual-subreddit users are not merely neutral bridges: a subset appears to inject unusually positive tone into otherwise neutral or negative trade and economic discourse, potentially reframing China's economic narrative.
A small number of accounts can occupy structurally important bridging positions across high-traffic subreddits (r/worldnews, r/technology, r/economics, r/AskReddit), giving them outsized reach for narrative diffusion.
Behavioral heuristics such as lexical diversity, karma distribution, email verification, and account age can be aggregated into a flag-count score that surfaces a small elevated-risk set (5 of 63 users with two or more flags, 2 suspended).
The flagged users can serve as weak labels for training supervised classifiers for coordinated-inauthentic-behavior detection.
Combining topic sentiment deviation with behavioral flags offers a modular, open-source pipeline for monitoring politically contested Reddit spaces without platform cooperation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A cleaner test of the amplifier hypothesis would exclude dual users from the global baseline before computing outliers; because the baseline includes them, the current deviation estimates are conservatively biased, and excluding them could make the outliers even more pronounced.
The method could be transferred to other ideologically opposed subreddit pairs (for example, r/ukraine and r/russia) to see whether the same anomaly structure emerges where state-linked influence has been alleged.
The sentiment classifier used in the paper does not reliably detect sarcasm or irony; the 0.427 positive-outlier cluster on trade topics could in part reflect mocking or hyperbolic positive language, so a qualitative reading of those posts would sharpen the interpretation.
The network analysis identifies where flagged users sit but does not establish that their activity changes anyone else's sentiment; a temporal diffusion analysis (do posts by flagged users precede sentiment shifts in the wider subreddit?) would test the amplifier mechanism directly.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a competent exploratory case study of the 63 users who post in both r/Sino and r/China, and the descriptive statistics are fine as far as they go. But the paper's central inference—that these users may be 'strategic amplifiers'—rests on sentiment deviations that are never tested against a null model. That's a load-bearing gap, not a stylistic quibble.

What's genuinely useful: the focus on cross-subreddit participants is a reasonable niche that most prior Reddit bot-detection work ignores. The data collection is described concretely (999+930 posts, 6k+7.6k comments), and the finding that only 1.2% of users meet the dual-participation threshold, with a high-variance activity distribution, is a clean descriptive result. The authors also deserve credit for hedging their language; they say 'patterns consistent with' rather than claiming proof. The paper is honest about the exploratory nature.

The soft spot is where the stress-test lands. Section 4.3's 'positive outlier' finding—7 users with average sentiment 0.427 on Topic 4 vs. a global 0.094—sounds impressive, but with a 63×6 user-topic matrix, selecting the most extreme cells will produce large deviations even from pure noise. There's no permutation test, no confidence interval, no multiple-comparison correction. The baseline includes the flagged users, though that actually shrinks the deviation; the real problem is the absence of any null distribution. The same applies to the low-variance and negative-outlier groups. The behavioral flags in §4.4 are heuristic and unvalidated; flagging users with low lexical diversity or karma imbalance and then treating the flag count as evidence of inauthentic behavior is circular unless those flags are independently validated against known coordinated accounts. The paper doesn't do that. Also, the §4.2 comparison of dual-user vs. all-user topic structure confounds user type with document type (posts+comments vs posts only) and corpus size, so the 'fragmented discourse' conclusion is not clean.

None of this means the paper is worthless. It's a reasonable descriptive case study that could serve as a starting point for more rigorous work. But the main conclusion currently exceeds the evidence. A revision that adds permutation testing, validates the flags, and releases code/data would make it a genuinely useful methodological contribution.

For a reader: this is for people working on OSINT and social cybersecurity who want a concrete example of how someone might approach cross-community influence detection. It's not a methodological advance. I wouldn't cite it in its current form, but I'd send it to review with a clear request for major revisions—the data collection and framing are solid enough that the authors deserve a chance to fix the statistics.

Referee Report

4 major / 5 minor

Summary. The paper investigates potential indicators of coordinated influence among users who participate in both r/Sino and r/China, two ideologically opposed Reddit communities. Using Reddit API data, the authors identify 63 dual-subreddit users, apply LDA topic modeling and TextBlob sentiment analysis to construct a user–topic sentiment matrix, compare individual sentiment to global baselines, and profile users with heuristic behavioral flags (lexical diversity, karma distribution, email verification, account age, language). They report positive and negative sentiment outliers, low-variance users, and five users flagged with two or more behavioral anomalies, and they examine these users' positions in a subreddit co-participation network. The conclusion is that dual-subreddit users 'may not simply act as ideological bridges, but could function as strategic amplifiers,' with the findings framed as 'patterns consistent with' inauthentic or strategically structured participation.

Significance. If the central claim were supported by rigorous evidence, this paper would contribute a useful open-source method for detecting subtle, persona-driven influence activity on Reddit, an area with relatively little prior work compared to Twitter and Facebook. The study has several strengths: it uses publicly available data, offers a modular framework combining content and activity signals, and is carefully hedged in its language, acknowledging limitations such as the non-definitive nature of account suspension. However, the current quantitative analysis is largely descriptive, and the inferential gap between observed anomalies and coordinated influence is not closed by the presented evidence. The findings would be more significant if the sentiment-outlier and behavioral-flag analyses were validated against null models and calibrated baselines.

major comments (4)

[§4.3, Table 3] The central evidence for sentiment divergence is the report that 7 dual users had an average sentiment of 0.427 on Topic 4 versus a global average of 0.094, a +0.333 deviation. This claim is not accompanied by any null model, confidence interval, or multiple-comparison correction. With 63 users and 6 topics, there are 378 user–topic cells, and selecting extreme cells post hoc will produce large deviations even under pure noise. Given the reported mean of 8.7 posts/comments per user, many cells are based on one or two posts, making the averages unstable. Without a permutation test or activity-matched baseline, the observed deviations cannot support an inference from descriptive anomaly to coordinated influence.
[§4.3, baseline definition] The 'global average sentiment for that topic' is computed from the full corpus that includes the dual users' own posts and comments. Since each user's sentiment contributes to the baseline, the comparison is not independent; for highly active users, the baseline may be pulled toward their own values, biasing the reported deviation. While this bias may be conservative, it means the baseline is not an external reference. A leave-one-out or out-of-sample baseline should be used to make the comparison self-consistent.
[§4.2] The comparison of the dual-user LDA model with the all-user LDA model confounds user type with corpus composition. The dual-user model is trained on all posts and comments from the 63 users, while the global model is trained on posts only from the full population. The conclusion that dual users exhibit 'fragmented' and 'diffuse' discourse relative to a 'cohesive' global structure may be an artifact of including comments, which are typically more informal and stylistically varied than posts. A matched comparison using the same document types and comparable corpus sizes is needed before interpreting this as a distinctive trait of dual users.
[§4.4] The behavioral anomaly framework is not validated. The paper does not specify the thresholds for 'low lexical diversity,' 'karma imbalance,' or 'high activity paired with low link karma,' nor does it report baseline rates of these properties among regular Reddit users. The flag-count threshold of 'two or more' anomalies is arbitrary. Most importantly, low lexical diversity is flagged in 51 of 63 users, so this feature has little discriminative power. Without calibration against a known ground-truth set of inauthentic accounts or a null distribution of flag counts from the general user population, the statement that flagged users show 'elevated risk for inauthentic or coordinated activity' is unsupported.

minor comments (5)

[§3] The LDA hyperparameters (number of passes, alpha, beta, random seed) and the sentiment aggregation method (e.g., how multiple sentences within a post are combined) are not reported, which limits reproducibility.
[Table 3] Table 3 uses topic indices 0–5, while the text and Tables 1–2 refer to Topics 1–6; this inconsistency should be fixed.
[§4.5] The statement that flagged users occupy 'bridging positions' in the co-participation network is qualitative; it should be supported by quantitative network measures such as betweenness centrality or a comparison against a null network model.
[References] Several references are incomplete or inconsistently formatted (e.g., missing journal names in refs 2, 4, 8, 9, 10); the list should be checked against a standard style.
[§4.3] The phrase 'markedly more positive sentiment' is used without a statistical test; adding a permutation-based p-value or confidence interval would make the language precise.

Circularity Check

1 steps flagged · score 6.0 of 10

The sentiment-outlier evidence in §4.3 reduces to its own selection criterion; the rest of the analysis is not formally circular.

self definitional [Section 4.3, Sentiment Divergence Across Topics]
"However, among 7 dual users flagged as positive outliers, the average sentiment was 0.427, a sharp +0.333 deviation."

The group labeled 'positive outliers' is selected because its topic-level sentiment is far above the global topic mean; no independent threshold or null model is reported. Reporting that these users' average sentiment is +0.333 above the global mean is therefore a restatement of the selection criterion, not an independent empirical finding. The same applies to the 42 'negative sentiment outliers' and the 5 'low-variance users,' whose reported deviations and near-zero standard deviation are the statistics used to define the groups. Using these self-selected groups as evidence that 'affective profiles ...

full rationale

The paper's other pillars are not formally circular. The topic-model comparison (§4.2) is a descriptive contrast between two LDA models trained on different corpora; the confounds (posts+comments vs posts, corpus size) are validity problems, not reductions. The behavioral flags (§4.4) are heuristics whose link to inauthenticity is assumed, not derived, so flag counts are not tautological. The network analysis (§4.5) is independent descriptive evidence. Self-citations [8]-[11] appear only in background framing and are not load-bearing. The central coordinated-influence conclusion is partly supported by the self-definitional sentiment-outlier step above, so the score reflects partial circularity; the remaining evidence would need proper null models to be persuasive, but that is a statistical-correctness concern rather than a circularity concern.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on several free parameters and unvalidated assumptions. The topic count, participation threshold, and flag threshold are chosen by the authors without empirical justification. The interpretation of sentiment and behavioral deviations as indicators of coordinated influence is an ad hoc assumption that is not tested against ground truth.

free parameters (3)

LDA topic count = 6
The number of topics was set to six without a coherence-based or held-out likelihood justification (Section 3).
Dual participation threshold = at least 3 posts/comments per subreddit
Users were retained if they posted or commented at least three times in each subreddit (Section 3), which excludes the majority of cross-community participants.
Behavioral flag threshold = 2 or more flags
Users with two or more behavioral anomalies were flagged for scrutiny (Section 4.4); the threshold is not justified by data.

assumptions (3)

domain assumption LDA topics represent coherent discourse themes that can be meaningfully labeled.
Section 4.2 interprets topics based on top keywords without quantitative validation such as coherence scores.
domain assumption TextBlob sentiment polarity scores are a valid measure of affective stance in this context.
Used as the sole sentiment measure (Section 3), without calibration to political discourse.
ad hoc to paper Deviations from the global topic baseline indicate anomalous, possibly coordinated behavior.
Section 5 interprets sentiment and behavioral deviations as suggestive of influence, an assumption not independently validated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Cross-Subreddit Behavior as Open-Source Indicators of Coordinated Influence: A Case Study of r/Sino & r/China." pith.science (2026). https://pith.science/paper/7JZM52KY

@misc{pith2026250716857,
  author       = {Pith},
  title        = {Pith review of: Cross-Subreddit Behavior as Open-Source Indicators of Coordinated Influence: A Case Study of r/Sino & r/China},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7JZM52KY}},
  note         = {Machine review of arXiv:2507.16857}
}

read the original abstract

This study investigates potential indicators of coordinated influence activity among users participating in both r/Sino and r/China, two ideologically divergent Reddit communities focused on Chinese political discourse. Topic modeling and sentiment analysis are applied to all posts and comments authored by dual-subreddit users to construct a user-topic sentiment matrix. Individual sentiment patterns are compared to global topic baselines derived from the broader r/Sino and r/China populations. Behavioral profiling is performed using full user activity histories and metadata, incorporating measures such as lexical diversity, language consistency, account age, posting frequency, and karma distribution. Users exhibiting multiple behavioral anomalies are identified and examined within a subreddit co-participation network to assess structural overlap. The combined linguistic and behavioral analysis enables the identification of patterns consistent with inauthentic or strategically structured participation. These findings demonstrate the utility of integrating content and activity-based signals in the analysis of online influence behavior within contested information environments.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 17 canonical work pages

[1]

Computational and Mathematical Organization Theory 26(4), 365-381 (2020)

Carley, K.M.: Social cybersecurity: an emerging science. Computational and Mathematical Organization Theory 26(4), 365-381 (2020)

work page 2020
[2]

lol", "dont

LDA topic model of all users, active in r/Sino or r/China To establish a baseline for comparison, a second LDA model was trained on the full corpus of posts from r/Sino and r/China, including users who did not participate in both subreddits. Compared to the more diffuse and stylistically heterogeneous dual-user model, the global model exhibited greater th...

work page 2012
[3]

The Dual Personas of Social Media Bots

Ng, L.H.X., Carley, K.M.: The dual personas of social media bots. arXiv2504.12498. (2025)

work page Pith review arXiv 2025
[4]

Scientific Reports, 15(1) (2025)

Ng, L.H.X., Carley, K.M.: A global comparison of social media bot and human characteristics. Scientific Reports, 15(1) (2025)

work page 2025
[5]

Ng, L.H.X., Carley, K.M.: Analyzing digital propaganda and conflict rhetoric: a study on Russia’s bot-driven campaigns and counter-narratives during the Ukraine crisis

Marigliano, R. Ng, L.H.X., Carley, K.M.: Analyzing digital propaganda and conflict rhetoric: a study on Russia’s bot-driven campaigns and counter-narratives during the Ukraine crisis. J. of Social Network Analysis and Mining 14(1) (2024)

work page 2024
[6]

In: Proc

Carragher, P., Williams, E.M., Carley, K.M.: Detection and discovery of misinformation sources using attributed webgraphs. In: Proc. Of Int’l AAAI Conference on Web and Social Media 18 (2024). 13

work page 2024
[7]

Human Factors 66(1) 88-102 (2024)

Kenny, R., Fischhoff, B., Davis, A., Carley, K.M., Canfield, C.: Duped by bots: why some are better than others at detecting fake social media personas. Human Factors 66(1) 88-102 (2024)

work page 2024
[8]

EPJ Data Science 12(1) (2023)

Ng, L.H.X., Carley, K.M.: Deflating the Chinese balloon: types of Twitter bots in US-China balloon incident. EPJ Data Science 12(1) (2023)

work page 2023

Show all 21 references

[9]

In: Proc

Lowetz, C., McCulloh, I.: Russian Invaders on the Internet’s Front Page – A Survey of Behaviors in Ukraine-Related Subreddits. In: Proc. 2024 IEEE/ACM Foundations of Open Source Intelligence and Security Informatics. (2024)

2024
[10]

McCulloh, I., Bergamini, R., Mackey, C.: Echoes of War: How Reddit Narratives Shape Sectarian Views of the Israel-Hamas War. In Proc. 2024 IEEE/ACM Foundations of Open Source Intelligence and Security Informatics. (2024)

2024
[11]

In: Proc 2019 IEEE/ACM ASONAM (2019)

Takacs, R., McCulloh, I.: Dormant bots in social media: Twitter and the 2018 US Senate election. In: Proc 2019 IEEE/ACM ASONAM (2019)

2019
[12]

McCulloh, I., Armstrong, H., Johnson, A.N.: Social Network Analysis with Applications. Wiley. (2013)

2013
[13]

In: Proceedings of the International AAAI Conference on Web and Social Media (ICWSM), pp

Albanese, J., Morgan, D., Goldberg, D.: Detecting bots on Reddit using posting behavior and content features. In: Proceedings of the International AAAI Conference on Web and Social Media (ICWSM), pp. 1–8. AAAI Press (2020)

2020
[14]

Cinelli, M., Morales, G.D.F., Galeazzi, A., Quattrociocchi, W., Starnini, M.: The echo chamber effect on social media. Proc. Natl. Acad. Sci. 118(9), e2023301118 (2021). https://doi.org/10.1073/pnas.2023301118

2021 doi
[15]

Starbird, K., Arif, A., Wilson, T.: Disinformation as collaborative work: Surfacing the participatory nature of strategic information operations. Proc. ACM Hum.-Comput. Interact. 3(CSCW), 127 (2019). https://doi.org/10.1145/3359229

2019 doi
[16]

ICWSM (2017)

Varol, O., Ferrara, E., Davis, C.A., Menczer, F., Flammini, A.: Online human-bot interactions: detection, estimation, and characterization. ICWSM (2017)

2017
[17]

In: Proceedings of the 10th ACM Conference on Web Science (WebSci '19), pp

Zannettou, S., Caulfield, T., Setzer, W., Blackburn, J.: Who let the trolls out? Towards understanding state-sponsored trolls. In: Proceedings of the 10th ACM Conference on Web Science (WebSci '19), pp. 353–362. ACM, Boston (2019). https://doi.org/10.1145/3292522.3326042

2019
[18]

In: The Web Conference 2018 (WWW ’18), pp

Kumar, S., Hamilton, W.L., Leskovec, J., Jurafsky, D.: Community interaction and conflict on the web. In: The Web Conference 2018 (WWW ’18), pp. 933–943. ACM, Lyon (2018). https://doi.org/10.1145/3178876.3186141

2018
[19]

Keller, T.R., Klinger, U.: Social bots in election campaigns: Theoretical, empirical, and methodological implications. J. Political Communication 36(1) 171-189 (2019)

2019
[20]

Ferrara, E., Varol, O., Davis, C., Menczer, F., Flammini, A.: The rise of social bots. J. Communications of the ACM 59(7) 96-104 (2016)

2016
[21]

Cresci, S.: A decade of social bot detection. Commun. ACM 63(10), 72–83 (2020). https://doi.org/10.1145/3363184

2020 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Computational and Mathematical Organization Theory 26(4), 365-381 (2020)

Carley, K.M.: Social cybersecurity: an emerging science. Computational and Mathematical Organization Theory 26(4), 365-381 (2020)

work page 2020

[2] [2]

lol", "dont

LDA topic model of all users, active in r/Sino or r/China To establish a baseline for comparison, a second LDA model was trained on the full corpus of posts from r/Sino and r/China, including users who did not participate in both subreddits. Compared to the more diffuse and stylistically heterogeneous dual-user model, the global model exhibited greater th...

work page 2012

[3] [3]

The Dual Personas of Social Media Bots

Ng, L.H.X., Carley, K.M.: The dual personas of social media bots. arXiv2504.12498. (2025)

work page Pith review arXiv 2025

[4] [4]

Scientific Reports, 15(1) (2025)

Ng, L.H.X., Carley, K.M.: A global comparison of social media bot and human characteristics. Scientific Reports, 15(1) (2025)

work page 2025

[5] [5]

Ng, L.H.X., Carley, K.M.: Analyzing digital propaganda and conflict rhetoric: a study on Russia’s bot-driven campaigns and counter-narratives during the Ukraine crisis

Marigliano, R. Ng, L.H.X., Carley, K.M.: Analyzing digital propaganda and conflict rhetoric: a study on Russia’s bot-driven campaigns and counter-narratives during the Ukraine crisis. J. of Social Network Analysis and Mining 14(1) (2024)

work page 2024

[6] [6]

In: Proc

Carragher, P., Williams, E.M., Carley, K.M.: Detection and discovery of misinformation sources using attributed webgraphs. In: Proc. Of Int’l AAAI Conference on Web and Social Media 18 (2024). 13

work page 2024

[7] [7]

Human Factors 66(1) 88-102 (2024)

Kenny, R., Fischhoff, B., Davis, A., Carley, K.M., Canfield, C.: Duped by bots: why some are better than others at detecting fake social media personas. Human Factors 66(1) 88-102 (2024)

work page 2024

[8] [8]

EPJ Data Science 12(1) (2023)

Ng, L.H.X., Carley, K.M.: Deflating the Chinese balloon: types of Twitter bots in US-China balloon incident. EPJ Data Science 12(1) (2023)

work page 2023