{"id":"de248e73-3c49-47ef-955b-cbbdda3675e0","arxiv_id":"2412.13459","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A global scan of GitHub data identifies about six million suspected fake stars and 18,617 repositories running fake-star campaigns, with a major surge in 2024 and weak long-term promotional value.","lead":"This paper builds StarScout, a tool that scans GitHub's public event archive and flags repositories that received coordinated or low-activity stars, and reports about six million suspected fake stars between 2019 and 2024. It shows fake-star activity surged in 2024, often around malware or spam repositories, and that bought stars give only a short-term popularity boost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 6M/18,617 counts rest on hand-set thresholds and an untested low-activity definition; without sensitivity analysis or a direct precision audit, the headline numbers are not anchored to the true fake-star population.","rationale":"Agreeing with the Reader's conditional verdict. The paper's phenomena are plausibly real: the artifact is public, the Stargazer Ghost recall check is a genuine external anchor, and the deletion-rate gap is consistent with platform enforcement. But those supports establish recall and criterion correlation, not precision of the threshold-defined population. The low-activity signature is especially fragile because its definition is an absence-of-activity test, so it cannot distinguish a throwaway bot from a real user who starred once; the 50-star/50%/10% postprocessing rules are also motivated only by gray-market seller minimums. Since Section 3.5 explicitly declines a sensitivity analysis, the headline magnitudes are uncalibrated. A grid sweep is cheap relative to the original BigQuery scan and would directly show whether the six-million figure and 2024 surge are robust. Until then, the honest verdict stays CONDITIONAL (UNCHANGED); if the sweep shows stability, the concern is retired.","tokens_in":28331,"tokens_out":7428,"duration_ms":69059,"concrete_test":"Re-run StarScout on two randomly selected six-month GHArchive chunks (one from 2023, one from 2024) under a small parameter grid: low-activity repo cutoff ∈ {20, 50, 100}; lockstep n ∈ {25, 50, 100}; ρ ∈ {0.3, 0.5, 0.7}; monthly fake-ratio threshold ∈ {0.3, 0.5, 0.7}. Compare per-chunk fake-star counts, campaign counts, and the 2024/2023 ratio across the grid. If the counts vary by more than roughly 2x, or if the 2024 surge ratio is not preserved, the reported 6.0M / 18,617 / 301k numbers are threshold artifacts rather than stable measurements of fake-star prevalence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines the low-activity signature as any account with exactly one WatchEvent and at most one other event in the five-year GHArchive window, and the lockstep signature with n=50, m=10, Δt=30 days, ρ=0.5; Section 3.5 concedes these parameters are ad hoc and that no thorough sensitivity tests were run. The central counts in Section 3.3 — 6.0M fake stars before postprocessing, 18,617 campaign repositories, 301k accounts, 3.81M postprocessed fake stars — are therefore point estimates at a single arbitrary operating point. This matters concretely for the low-activity branch (1.06M stars): it flags any dormant but real user who starred one repository and then did nothing else, and a repository with 50 such users in a month plus a >50% monthly fake ratio is classified as a campaign even if all 50 users are legitimate. The Section 3.4 precision evidence is indirect: deletion rates may reflect spam/malware takedown rather than fake-star status, and recall is measured only against one known campaign. A plausible false-positive population could change both the level and the 2024-surge conclusion. A sensitivity sweep over the hand-set thresholds is the missing experiment.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents StarScout, a scalable BigQuery-based detector for fake GitHub stars, and applies it to all GHArchive event data from July 2019 to December 2024. StarScout combines a low-activity signature (accounts with exactly one WatchEvent and at most one other event) and a lockstep signature (clusters of at least n=50 accounts repeatedly starring at least m=10 repositories within Δt=30 days, with ρ=0.5 density), followed by postprocessing thresholds (monthly fake stars >50, monthly fake-star ratio >50%, and all-time fake-star ratio >10%). The detector identifies 6.0 million suspected fake stars across 26,254 repositories before postprocessing, and 18,617 repositories, 301k accounts, and 3.81 million fake stars after postprocessing. Evaluation against the Stargazer Ghost Network yields 81% repository recall and 76% account recall, and detected repositories/accounts show deletion ratios up to 90%, about 16x higher than random baselines. The measurement study reports a 2024 surge in fake-star campaigns, characterizes the involved accounts and repositories as having trivial activity patterns and skew toward spam/phishing and hyped domains, and uses panel autoregression to argue that fake stars provide only a short-term promotional effect and a long-term negative effect on real star gain.","tokens_in":28634,"tokens_out":5599,"duration_ms":50243,"significance":"If the headline numbers are robust, this is the first systematic, global, longitudinal measurement of fake GitHub stars, and it is a valuable contribution to software supply-chain security and platform moderation research. The paper has several genuine strengths: the detector is implemented at scale on public GHArchive data; recall is anchored to an external, independently documented malware campaign; the deletion-ratio evidence is suggestive and consistently higher than baselines; the artifact, data, and scripts are publicly released; and the responsible-disclosure process is clearly described. The RQ4 result that fake stars have only a short-term promotional effect, if credible, would directly inform practitioner guidance. However, the central counts are point estimates produced by hand-set thresholds that the authors themselves describe as ad hoc, and precision is only assessed indirectly through deletion rates. These issues do not invalidate the study, but they make the absolute prevalence numbers and the 2024-surge conclusion less anchored than the paper's framing suggests.","major_comments":[{"comment":"","section":"Sections 3.2, 3.3, 3.5"},{"comment":"","section":"Section 3.4"},{"comment":"","section":"Section 4.4, Table 6"}],"minor_comments":[{"comment":"","section":"Section 3.4"},{"comment":"","section":"Section 3.4"},{"comment":"","section":"Section 4.2, Table 3"},{"comment":"","section":"Section 4.4, Table 6"},{"comment":"","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":""},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here is my take. This is the first global longitudinal measurement of fake GitHub stars that I know of, and it is a solid contribution despite the caveats. The authors build StarScout, run it over GHArchive from 2019 to 2024, and report 6M suspected fake stars before postprocessing, 18,617 repositories with campaigns, 301k accounts, and 3.81M fake stars after postprocessing. They validate recall against an external confirmed malware campaign (81% of repositories, 76% of accounts), show deletion ratios about 16x baseline, ship code and data, and report 130 phishing repositories to GitHub. Those are real strengths. The RQ4 promotional-effect result is a fitted regression, and the authors themselves say it is not causal; the finding that fake stars help for only two months and then become a liability is plausible but should be treated as an association.\n\nThe main soft spot is the one the stress-test note flags: the headline numbers depend on hand-set thresholds (50-star minimum, lockstep n=50/m=10/Δt=30/ρ=0.5, postprocessing rules), and Section 3.5 admits no thorough sensitivity analysis. The low-activity signature—one WatchEvent plus at most one other event over five years—will catch dormant but real users, and the postprocessing rules could call a repository a campaign if 50 such users star it in one month. I do not think this invalidates the overall picture. The deletion differential and the ground-truth recall argue that the detector is finding something real. But the 6M/18,617 numbers are operating-point estimates, not firm population counts. A sensitivity sweep would materially strengthen the paper.\n\nPrecision is only estimated indirectly through deletion rates. Deletion could reflect spam or malware takedowns rather than fake-star status. That is a legitimate concern, though with 90% deletion for postprocessed repositories and 16x baseline, it is enough to support the detector's usefulness, just not exact precision. The \"majority are phishing/malware\" claim rests partly on names of deleted repositories and open coding with an LLM as the second coder (κ=0.71); it is reasonable but not airtight, and the authors are appropriately careful in wording.\n\nThe citation pattern looks sound: prior black-market work, gray literature, CopyCatch, and supply-chain attack research are all represented. The artifact availability is a plus.\n\nWho is this for: platform moderators, supply-chain security researchers, and anyone studying popularity metrics. It deserves a serious referee. A good referee should push for threshold sensitivity analysis and clearer precision evidence, but desk rejection would be wrong.","headline":"First solid global measurement of fake GitHub stars, but the headline counts are operating-point estimates until someone runs a sensitivity analysis.","tokens_in":29172,"tokens_out":2259,"would_cite":true,"duration_ms":29302,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that six million suspected fake stars accumulated on GitHub between 2019 and 2024, that fake-star campaigns surged in 2024, and that most served to promote short-lived malware or hype repositories.","keywords":["GitHub","fake stars","star fraud","supply chain security","longitudinal measurement","malware","popularity signals","GHArchive"],"falsifier":"Re-run StarScout over the same GHArchive window while sweeping the thresholds (e.g., lockstep n between 20 and 100, rho between 0.3 and 0.7, the 50-star minimum between 20 and 200, and the postprocessing cutoffs around their set values): if the 2024 surge and the 18,617-campaign count do not persist across most of the parameter space, the headline numbers are artifacts of the chosen cutoffs. Independently, take a random sample of accounts flagged as low-activity and check GitHub's deletion, suspension, or IP-level telemetry records: if a large share of flagged accounts remain active for years with organic behavior, the low-activity signature overcounts real users.","tokens_in":28132,"feed_emoji":"⭐","tokens_out":8341,"duration_ms":69056,"temperature":0.7,"pith_summary":"This paper tries to establish that fake GitHub stars are not a fringe phenomenon but a systemic, growing one visible in the platform's public event stream: an estimated six million suspected fake stars were cast between July 2019 and December 2024, and the coordinated campaigns behind them surged sharply in 2024. The authors build StarScout, a detector that runs as SQL queries over the GHArchive event dataset and flags two anomalous starring patterns: accounts that star once and then go silent, and lockstep clusters of accounts starring the same repositories inside short time windows. They argue that if the counts are right, star counts are a partially corrupted popularity signal, a security hazard because a large share of fake-star repositories turn out to be phishing or malware vehicles, and a weak growth-hacking strategy because fake stars attract real attention for under two months and then become a liability. The paper's contribution is the first systematic global longitudinal measurement of fake-star campaigns on GitHub, with downstream stakes for how practitioners, platforms, and supply-chain security tools treat star counts.","feed_headline":"Six million suspected fake stars found on GitHub","feed_subtitle":"A scan of all GitHub events since 2019 ties most fake stars to malware and short-lived hype campaigns.","key_machinery":"The carrying mechanism is StarScout, which detects two signatures of anomalous starring in the stargazer bipartite graph of accounts and repositories. The low activity signature flags accounts with a single WatchEvent plus at most one other event, i.e., accounts that star one repository and then go stale. The lockstep signature flags groups of at least 50 accounts that repeatedly star at least 10 repositories such that each repository receives at least 25 stars from the group within a 30-day window, found by running the CopyCatch algorithm expressed as SQL on the GHArchive data. A postprocessing step then keeps only repositories with an anomalous monthly spike of fake stars (more than 50 fake stars and over 50% fake in one month, plus over 10% fake all-time), converting noisy per-star signals into a repository-level claim about coordinated fake star campaigns.","core_discovery":"On the paper's own terms, the central discovery is that fake-star activity is large, growing, and increasingly malicious: StarScout identifies 6.0 million suspected fake stars across 26,254 repositories before postprocessing, and 3.81 million fake stars within 18,617 repositories judged to be running fake star campaigns, involving 301,000 accounts. The lockstep signature alone accounts for 4.93 million of those stars. The paper further reports that campaigns surged in 2024; that most participating accounts and repositories have highly trivial activity patterns; that the largest category of still-accessible fake-star repositories is spam or phishing, while deleted repositories carry names suggesting pirated software, cryptocurrency bots, and game cheats; and that panel autoregression models show fake stars have a short-lived promotional effect roughly five times weaker than real stars, with a negative long-run association between accumulated fake stars and later real-star gains.","pith_inferences":["The six-million figure is a lower bound by construction, because StarScout only sees public GHArchive events and ignores repositories with fewer than 50 fake stars; if merchants shift to smaller batches or higher-activity accounts, the measured volume would understate the true market.","The same lockstep signature could be ported to other popularity surfaces such as npm download counts, PyPI mirrors, model-download counts, or social-media view counts, turning the paper's method into a generic coordinated-burst fraud detector rather than a GitHub-specific study.","A testable extension of the promotional-effect result is to follow repositories after GitHub strips their fake stars: if traffic and attention drop sharply, the boost was cosmetic, whereas steady traffic would suggest campaigns do convert into real adoption.","The parameter set matches merchant batch sizes observed in 2024; re-running the detector on later data would reveal whether the arms race pushes merchants toward slower, smaller, or more realistic patterns, which is exactly the sensitivity the paper could not run at 20 TB scale."],"forward_implications":["If the counts are right, raw star counts are a materially distorted popularity and trust signal on GitHub, since in July 2024 about one in six popular repositories (3,499 of those with 50+ monthly stars) had a fake star campaign.","Fake stars and malware are linked: 90.42% of campaign repositories were deleted by January 2025, and open coding of the survivors puts spam or phishing as the largest category at roughly 30%, so star inflation should be treated as a supply-chain security indicator.","Buying stars is counterproductive for growth hacking: a 1% increase in fake stars is associated with only about 0.07% more real stars one month later (versus 0.36% for real stars), and the accumulated fake-star stock is associated with lower future real-star gains.","Current ranking mechanisms already filter much of the fraud: only 78 fake-star campaign repositories (0.42%) ever appeared in GitHub Trending during the study window.","Supply-chain exposure is real but narrow: 229 fake-star campaign repositories were linked to 738 packages in registries, most with no dependent packages or repositories, so the main current risk is the malware subset rather than widespread downstream adoption."],"supporting_citations":[{"why":"Supplies the complete GHArchive event dataset (all GitHub events from July 2019 to December 2024) that StarScout scans.","marker":"[1]"},{"why":"Provides the CopyCatch lockstep-detection algorithm that StarScout implements to find clusters of accounts starring repositories in synchrony.","marker":"[30]"},{"why":"Supplies the Stargazer Ghost Network dataset, the confirmed malware campaign used as ground truth to measure StarScout's recall.","marker":"[21]"},{"why":"Documents the GitHub black-market accounts phenomenon and the prior machine-learning account-fraud detection approach this study extends.","marker":"[44]"},{"why":"Source of the low-activity heuristic (accounts that star a single repository) that StarScout simplifies and applies at platform scale.","marker":"[10]"},{"why":"Reports the GitHub star black market as a real phenomenon, motivating the existence of purchasable fake stars.","marker":"[13]"},{"why":"Provides the historical GitHub Trending archive used to measure how many fake-star campaign repositories reach developers.","marker":"[25]"}],"fun_headline_variants":["6M fake GitHub stars, most tied to malware campaigns","Fake GitHub stars surge to 6M in 2024, study finds","StarScout: 6M fake GitHub stars, most serve malware","Six million fake GitHub stars, with malware at the core","GitHub fake stars: 6M suspected, campaigns surge in 2024"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that StarScout's hand-set thresholds — the 50-star minimum, the one-star-plus-one-event low-activity rule, the lockstep parameters (50 accounts, 10 repositories, half the accounts co-starring within 30 days), and the postprocessing cutoffs (monthly fake stars >50, fake ratio >50%, all-time fake ratio >10%) — cleanly separate fake from authentic starring, and the paper itself notes these parameters were chosen by judgment without thorough sensitivity tests.","fun_headline_variants_meta":{"raw":{"variants":["6M fake GitHub stars, most tied to malware campaigns","Fake GitHub stars surge to 6M in 2024, study finds","StarScout: 6M fake GitHub stars, most serve malware","Six million fake GitHub stars, with malware at the core","GitHub fake stars: 6M suspected, campaigns surge in 2024"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001134,"raw_usage":{"total_tokens":4732,"prompt_tokens":989,"completion_tokens":3743,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":605,"completion_tokens_details":{"reasoning_tokens":3649}},"tokens_in":605,"tokens_out":3743,"duration_ms":25432,"temperature":1.0,"reasoning_tokens":3649,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:06:20.295716+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run StarScout over the same GHArchive window while sweeping the thresholds (e.g., lockstep n between 20 and 100, rho between 0.3 and 0.7, the 50-star minimum between 20 and 200, and the postprocessing cutoffs around their set values): if the 2024 surge and the 18,617-campaign count do not persist across most of the parameter space, the headline numbers are artifacts of the chosen cutoffs. Independently, take a random sample of accounts flagged as low-activity and check GitHub's deletion, suspension, or IP-level telemetry records: if a large share of flagged accounts remain active for years with organic behavior, the low-activity signature overcounts real users.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CopyCatch lockstep-detection algorithm that StarScout implements to find clusters of accounts starring repositories in synchrony."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the GitHub black-market accounts phenomenon and the prior machine-learning account-fraud detection approach this study extends."}],"review_version":1}