REVIEW 3 major objections 6 minor 23 references
Structure and Context of Retweet Coordination in the 2022 U.S. Midterm Elections
T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper claims that Cramer's phi values for three-nearest-neighbor retweet pairs are bimodal, with the upper mode supplying a data-driven threshold for coordinated users; the resulting candidates split into music-award vote promotion…
desk verdict The paper's central threshold rests on an incorrect formula for Cramer's phi, and the cutoff is selected in-sample; the case study itself is useful but needs correction and validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two linked objects carry the argument. The first is a latent sharing space: the binary retweeter-tweet matrix is double-centered by subtracting expected counts under independence and then decomposed by singular value decomposition; the first three dimensions of user scores are clustered with HDBSCAN, a density-based clustering algorithm, into four user clusters. The second is a three-nearest-neighbor association graph: each user is connected to their three most cosine-similar users, and each edge is weighted by Cramer's $\phi$, defined from a $2\times2$ retweet contingency table. The load-bearing mechanism is the bimodality of the $\phi$ distribution over these edges: the lower mode corresponds to ordinary shared interests and the upper mode to the dense, internally connected components that the paper labels coordinated. The threshold $\phi \geq 0.67$ is chosen from the gap between these modes rather than from significance testing.
What would settle it
Run the same pipeline on retweet data from a large event with no known coordinated campaigns, such as a week of breaking-news coverage of a natural disaster, and check whether a strong mode near $\phi \geq 0.67$ still appears in the three-nearest-neighbor association distribution. If the upper mode persists without any organized actors, the distribution shape by itself cannot certify coordination; a complementary test would apply the threshold to a ground-truth set of known coordinated accounts and measure how many fall above it.
Extended reading notes
Core claim
The central discovery is that association values among three-nearest-neighbor retweet pairs are bimodal, with a smaller mode of strong associations around and above roughly $\phi = 0.67$ that the paper identifies as coordinated activity. A conventional significance-based cutoff would mark nearly all users as coordinated, while an arbitrarily high similarity threshold discards meaningful structure. The bimodal shape, corroborated by a network-connectedness analysis in which the strong-association mode forms dense components that dissolve as the threshold rises, supplies what the paper calls a natural place to threshold. Applied to 73,097 users who retweeted at least 20 times during November 5–11, 2022, this identifies 2,275 coordination candidates: 1,212 in a K-pop award-voting cluster, 424 in a right-wing political cluster, 381 in a left-leaning cluster largely devoted to Taylor Swift award voting, and 258 in a Democratic mobilization cluster. The latent sharing space, built from the singular value decomposition of the double-centered retweeter-tweet matrix, yields four clusters in three dimensions and shows that entertainment-fandom coordination and political mobilization can overlap, so a high-threshold-only approach would fragment groups that share a common motivation.
Load-bearing premise
The assumption that one very high association with any of a user's three nearest neighbors is both necessary and sufficient for coordinated activity; if such an edge can arise from ordinary shared enthusiasm or homophily, the candidate labels will include many organic users.
Editorial extensions
If this is right
- Analysts can estimate coordination thresholds from the data's own distribution instead of choosing arbitrary cosine or Jaccard cutoffs, by locating the valley between the two modes of a k-nearest-neighbor association distribution.
- Coordination detection should cover entertainment and award-vote manipulation alongside political influence campaigns, because in this dataset the largest coordinated-candidate set is K-pop promotion rather than political mobilization.
- Latent-space context can prevent content misclassification: Taylor Swift fans who write 'Vote Red' share wording with Republican mobilization but sit in a different region of the sharing space, so keyword-only labeling would conflate the two.
- Even with a natural-looking threshold, coordinated and organic users in the same cluster (as in the K-pop cluster) can perform the same visible actions, so binary coordinated/organic labels are a simplification even if the cutoff itself is justified.
- Network connectedness offers a second structural signal: components dissolve at characteristic $\phi$ levels, so the threshold can be cross-checked by graph decomposition rather than by distribution shape alone.
Reading between the lines
- The bimodal-threshold recipe may transfer to other event-centered retweet corpora without retuning, but the paper demonstrates it on a single midterm-election week, so its generality is untested.
- A direct testable extension would be to apply the same k-nearest-neighbor $\phi$ pipeline to nonpolitical corpora (sports, product launches) and see whether the upper mode appears only when organized campaigns are present.
- The paper's 'Vote Red' example implies that text-based coordination labels should be combined with latent-space position, since the same phrase can signal opposite motivations.
- If the sufficiency assumption holds, coordination should be detectable from purely local graph structure; a ground-truth study of known influence operations could verify whether every coordinated account has at least one high-$\phi$ neighbor.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper analyzes retweet data collected around the 2022 U.S. midterm elections. It constructs a 3-nearest-neighbor association network in which edge weights are claimed to be Cramer's phi of bivariate retweet overlap, then uses a latent sharing space (SVD plus HDBSCAN) to group users. The central claim is that the distribution of phi values among nearest-neighbor pairs is bimodal and that its upper mode provides a natural coordination threshold at phi >= 0.67, yielding 2,275 coordination candidates across four clusters: K-pop award promotion, right-wing political mobilization, left-wing political commentary, and Democratic get-out-the-vote activity. The paper emphasizes that the threshold is data-driven rather than arbitrary and that the latent space contextualizes groups that would otherwise appear fragmented.
Significance. If the quantitative foundation held, the paper would be a useful descriptive case study and a plausible proposal for selecting coordination thresholds from distributional shape rather than from an arbitrary high-similarity cutoff. Its strengths are the clearly specified descriptive pipeline, the use of a published dataset, the explicit treatment of the difficulty of separating coordinated actors from organic homophily, and the honest acknowledgment of several limitations. However, the central quantitative result is not yet established: the printed phi formula is not the standard Cramer's phi, the threshold is selected in-sample without validation, and a key sufficiency assumption is asserted rather than tested. The contributions of the paper are therefore conditional on corrections to the measure and on additional sensitivity or validation analysis.
major comments (3)
- [§2.2, Eq. (1)] Equation (1) as printed is not Cramer's phi. The standard phi coefficient for a 2x2 table is (ad - bc) / sqrt((a+b)(c+d)(a+c)(b+d)), not |ad - bc| / sqrt(abcd). The printed formula omits the row and column marginals and is unbounded; for a=100, d=100, b=c=1 it gives approximately 99.99 rather than a value between -1 and 1. Because Figures 1-3 and the 0.67 threshold are all defined on values of this measure, the bimodality used to justify the coordination threshold could be an artifact of the incorrect normalization. Please clarify whether Eq. (1) is a typo, correct the formula or implementation throughout, and release the code so the computation can be independently checked. As written, the statement that phi may be computed equivalently from the chi-squared test statistic is also inconsistent with Eq. (1), since the standard relation uses the marginals.
- [§3.1, Figs. 1-3] The 0.67 threshold (abstract, §3.1, and the yellow region in Fig. 1) is selected from the same empirical distribution that it is then used to label. The paper provides no ground-truth comparison, random baseline, or sensitivity analysis showing that the 'natural' break is stable under changes to k, the activity filters, the number of SVD dimensions, or the HDBSCAN parameters. The claim that the upper mode corresponds to coordinated actors, rather than to homophilous fans or to users with high overlap for other reasons, is therefore unsupported. Please add a null-model comparison (for example, randomized retweet assignments that preserve user activity and tweet popularity) and a sensitivity analysis over the free parameters, or explicitly reframe the paper as a descriptive analysis of distributional structure rather than a validated detector of coordination.
- [§2.2] The statement that 'having even one very high association with a neighbor is both necessary and sufficient for coordinated activity' is a strong load-bearing assumption that is neither derived nor tested. Since the k-nearest-neighbor edges are based on cosine similarity of retweet vectors, a high association on such an edge can arise from organic homophily or from users who simply retweet many of the same highly popular posts. The sufficiency assumption directly inflates the candidate counts, for example the 1,212 candidates in Cluster A, if ordinary fans or voters can have one high-phi edge. Please validate against accounts known to be coordinated from prior work, or at least show that the identified candidates are not explained by retweet volume and by the popularity of the shared posts.
minor comments (6)
- [Abstract] The phrase 'thresholded with a k-nearest neighbor criterion' is potentially misleading; the k-NN criterion is used to sparsify the association network, while the phi threshold is a separate choice.
- [§2.3] The HDBSCAN parameters (for example min_cluster_size and min_samples) are not reported, although the four-cluster solution is central to the interpretation.
- [§3.2] The scree plot from the SVD is mentioned but not shown; please include it or report the eigenvalues that motivate the three-dimension choice.
- [Fig. 1 caption] The caption contains a typo ('the we treat' should be 'we treat') and refers to 'Black and yellow region' without defining the colors in the figure.
- [Cluster C, §3.2] The text says 'Another prominently shard post'; this should be 'shared post'.
- [Fig. 6 caption] The caption reads 'See 7 to compare'; this should be 'See Figure 7'.
Circularity Check
No circular derivation found; the threshold is an in-sample descriptive choice and the main risk is a misstated phi formula, which is a correctness issue rather than circularity.
full rationale
The paper's derivation is not circular. The phi values on 3-nearest-neighbor edges are computed from retweet co-occurrence counts, and the 0.67 threshold is read off the resulting empirical distribution (Sec. 3.1); the paper explicitly declines to treat the significance-based critical phi as a threshold and labels the outcome "coordination candidates" rather than confirmed coordination, so no fitted parameter is renamed as a prediction. The Section 2.2 statement that a high neighbor association is necessary and sufficient for coordination is a stated modeling assumption, not a derived result, and the subsequent cluster interpretations rest on qualitative inspection of the retweeted content (Sec. 3.2), which is independent of the threshold. Self-citations [1] and [22] provide the dataset and prior context, but the core quantitative and qualitative findings do not reduce to accepting those citations. A separate correctness concern, not a circularity, is that Eq. (1) in Sec. 2.2 omits the row and column marginals of the standard phi coefficient and is therefore not Cramer's phi as claimed; this threatens the numerical threshold values but does not make the argument circular.
Assumptions & free parameters
free parameters (5)
- k for nearest neighbor graph =
3
- Coordination threshold phi =
0.67
- Activity filters =
20+ retweets per user; tweet shared by 10+ unique users
- Number of latent dimensions =
3
- HDBSCAN parameters =
not reported
assumptions (5)
- ad hoc to paper Having even one very high association with a k-nearest neighbor is necessary and sufficient for coordinated activity
- domain assumption Bimodality of the association distribution corresponds to a substantive distinction between organic and coordinated retweeting
- domain assumption SVD of the binarized, double-centered retweeter-tweet matrix yields latent dimensions that reflect meaningful sharing motivations
- domain assumption HDBSCAN clusters in the latent space correspond to genuine user groups
- standard math Cramer's phi and chi-square properties are valid for binary retweet association after conditioning on kNN similarity
Cite this review
Pith. "Pith review of Structure and Context of Retweet Coordination in the 2022 U.S. Midterm Elections." pith.science (2026). https://pith.science/paper/MVHWEXHK
@misc{pith2026250111165,
author = {Pith},
title = {Pith review of: Structure and Context of Retweet Coordination in the 2022 U.S. Midterm Elections},
year = {2026},
howpublished = {\url{https://pith.science/paper/MVHWEXHK}},
note = {Machine review of arXiv:2501.11165}
}
read the original abstract
The ability to detect coordinated activity in communication networks is an ongoing challenge. Prior approaches emphasize considering any activity exceeding a specific threshold of similarity to be coordinated. However, identifying such a threshold is often arbitrary and can be difficult to distinguish from grassroots organized behavior. In this paper, we investigate a set of Twitter retweeting data collected around the 2022 US midterm elections, using a latent sharing-space model, in which we identify the main components of an association network, thresholded with a k-nearest neighbor criterion. This approach identifies a distribution of association values with different roles in the network at different ranges, where the shape of the distribution suggests a natural place to threshold for coordinated user candidates. We find coordination candidates belonging to two broad categories, one involving music awards and promotion of Korean pop or Taylor Swift, the other being users engaged in political mobilization. In addition, the latent space suggests common motivations for different coordinated groups otherwise fragmented by using an appropriately high threshold criterion for coordination.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Aiyappa, R., DeVerna, M.R., Pote, M., Truong, B.T., Zhao, W., Axelrod, D., Pes- sianzadeh, A., Kachwala, Z., Kim, M., Seckin, O.C., et al.: A multi-platform collec- tion of social media posts about the 2022 us midterm elections. In: Proceedings of 14 D. Axelrod, et al. the international AAAI conference on web and social media. vol. 17, pp. 981–989 (2023)
work page 2023
-
[2]
Political analysis23(1), 76–91 (2015)
Barberá, P.: Birds of the same feather tweet together: Bayesian ideal point esti- mation using twitter data. Political analysis23(1), 76–91 (2015)
work page 2015
-
[3]
Barberá, P., Jost, J., Nagler, J., Tucker, J., Bonneau, R.: Tweeting from left to right: Is online political communication more than an echo chamber? Psychological Science 26, 1531–1542 (2015)
work page 2015
-
[4]
Bastian, M., Heymann, S., Jacomy, M.: Gephi: An open source software for ex- ploring and manipulating networks. In: International AAAI Conference on We- blogs and Social Media (2009),http://www.aaai.org/ocs/index.php/ICWSM/09/ paper/view/154
work page 2009
-
[5]
Journal of the Korean Statistical Society42(3), 323–328 (2013)
Bergsma, W.: A bias-correction for cramér’s v and tschuprow’s t. Journal of the Korean Statistical Society42(3), 323–328 (2013)
2013
-
[6]
Cima, L., Mannocci, L., Avvenuti, M., Tesconi, M., Cresci, S.: Coordinated behav- ior in information operations on Twitter. IEEE Access (2024)
work page 2024
-
[7]
Conover, M.D., Gonçalves, B., Ratkiewicz, J., Flammini, A., Menczer, F.: Pre- dicting the political alignment of twitter users. In: 2011 IEEE third international conference on privacy, security, risk and trust and 2011 IEEE third international conference on social computing. pp. 192–199. IEEE (2011)
work page 2011
-
[8]
Nature Communications 14(62) (2023)
Eady, G., Paskhalis, T., Zilinsky, J., Bonneau, R., Nagler, J., Tucker, J.: Exposure to the russian internet research agency foreign influence campaign on twitter in the 2016 us election and its relationship to attitudes and voting behavior. Nature Communications 14(62) (2023)
work page 2023
Show all 23 references
-
[9]
Nature Human Behaviour 7, 904–916 (2023)
Flamino, J., Galeazzi, A., Feldman, S., Macy, M., Cross, B., Zhou, Z., Serafino, M., Bovet, A., Makse, H., Szymanski, B.: Political polarization of news media and influencers on twitter in the 2016 and 2020 us presidential elections. Nature Human Behaviour 7, 904–916 (2023)
2023
-
[10]
Gabriel, N.A., Broniatowski, D.A., Johnson, N.F.: Inductive detection of influence operations via graph learning (2023)
2023
-
[11]
Journal of Online Trust and Safety1(2) (2022)
Magelinski, T., Ng, L., Carley, K.: A synchronized action framework for detection of coordination on social media. Journal of Online Trust and Safety1(2) (2022)
2022
-
[12]
In: Data Mining Workshops (ICDMW), 2017 IEEE International Conference on
McInnes, L., Healy, J.: Accelerated hierarchical density based clustering. In: Data Mining Workshops (ICDMW), 2017 IEEE International Conference on. pp. 33–42. IEEE (2017)
2017
-
[13]
In: Proceedings of the International AAAI Conference on Web and Social Media (2021)
Nizzoli, L., Tardelli, S., Avvenuti, M., Cresci, S., Tesconi, M.: Coordinated behavior on social media in 2019 uk general election. In: Proceedings of the International AAAI Conference on Web and Social Media (2021)
2021
-
[14]
EPJ Data Science12(1), 33 (2023)
Nwala, A.C., Flammini, A., Menczer, F.: A Language Framework for Modeling Social Media Account Behavior. EPJ Data Science12(1), 33 (2023)
2023
-
[15]
In: Companion Proceedings of the Web Conference 2020 (2020).https://doi.org/10.1145/3366424.3385775, https://doi.org/10
Pacheco, D., Flammini, A., Menczer, F.: Unveiling coordinated groups behind white helmets disinformation. In: Companion Proceedings of the Web Conference 2020 (2020).https://doi.org/10.1145/3366424.3385775, https://doi.org/10. 1145/3366424.3385775
2020
-
[16]
Pacheco, D., Hui, P.M., Torres-Lugo, C., Truong, B.T., Flammini, A., Menczer, F.: Uncovering coordinated networks on social media: Methods and case studies. Proc. International AAAI Conference on Web and Social Media15(1), 455–466 (2021)
2021
-
[17]
In: Proceedings of the 15th ACM Web Science Conference 2023
Pierri, F., Luceri, L., Jindal, N., Ferrara, E.: Propaganda and misinformation on facebook and twitter during the russian invasion of ukraine. In: Proceedings of the 15th ACM Web Science Conference 2023. p. 65–74. WebSci ’23, Association for Structure of Coordination 15 Comput...
2023
-
[18]
arXiv preprint arXiv:2407.18098 (2024)
Saeed, M.H., Ali, S., Paudel, P., Blackburn, J., Stringhini, G.: Unraveling the Web of Disinformation: Exploring the Larger Context of State-Sponsored Influence Campaigns on Twitter. arXiv preprint arXiv:2407.18098 (2024)
2024 arXiv
-
[19]
EPJ Data science13 (2024)
Tardelli, S., Nizzoli, L., Avvenuti, M., Cresci, S., Tesconi, M.: Multifaceted online coordinated behavior in the 2020 us presidential election. EPJ Data science13 (2024)
2024
-
[20]
arXiv preprint arXiv:2301.06774 (2023)
Tardelli, S., Nizzoli, L., Tesconi, M., Conti, M., Nakov, P., Martino, G.D.S., Cresci, S.: Temporal dynamics of coordinated online behavior: Stability, archetypes, and influence. arXiv preprint arXiv:2301.06774 (2023)
2023 arXiv
-
[21]
IEEE transactions on knowledge and data engi- neering 28(8), 2158–2172 (2016)
Wong, F.M.F., Tan, C.W., Sen, S., Chiang, M.: Quantifying political leaning from tweets, retweets, and retweeters. IEEE transactions on knowledge and data engi- neering 28(8), 2158–2172 (2016)
2016
-
[22]
Big Data & Society 8(1), 20539517211013861 (2021).https://doi.org/10.1177/20539517211013861, https://doi.org/10.1177/20539517211013861
Yang, K.C., Pierri, F., Hui, P.M., Axelrod, D., Torres-Lugo, C., Bryden, J., Menczer, F.: The covid-19 infodemic: Twitter versus facebook. Big Data & Society 8(1), 20539517211013861 (2021).https://doi.org/10.1177/20539517211013861, https://doi.org/10.1177/20539517211013861
2021 doi
-
[23]
In: Companion Proc
Zannettou,S.,Caulfield,T.,DeCristofaro,E.,Sirivianos,M.,Stringhini,G.,Black- burn,J.:DisinformationWarfare:UnderstandingState-sponsoredTrollsonTwitter and their Influence on the Web. In: Companion Proc. WWW Conf. pp. 218–226 (2019). https://doi.org/10.1145/3308560.3316495
2019
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.