Pith. sign in

REVIEW 3 major objections 6 minor 37 references

Weak Links in LinkedIn: Enhancing Fake Profile Detection in the Age of LLMs

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read Existing LinkedIn detectors accept GPT-generated fake profiles 42-52% of the time, and adversarial training on GPT-generated profiles restores the false accept rate to 1-7%.

desk verdict Useful, honest empirical study of LLM-generated LinkedIn fake profiles; the countermeasure works in-distribution, but the abstract overstates cross-LLM generality. read the letter →

arxiv 2507.16860 v1 pith:DCAA3JQQ submitted 2025-07-21 cs.SI cs.CVcs.CY

classification cs.SIcs.CVcs.CY
keywords fakeprofiledetectionlargelanguagemodelsadversarialtrainingLinkedInGPT-4SectionTagEmbeddingsfalseacceptrateauthenticity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that text-based LinkedIn fake-profile detectors have a specific weak point: they catch manually written fakes (false accept rate 6-7%) but let GPT-3.5- and GPT-4-written profiles through as legitimate 42-52% of the time. It argues that the fix is GPT-assisted adversarial training, adding LLM-generated profiles to the training data, which restores the false accept rate to 1-7% while keeping false reject rates around 1.5-2.4%. The study also claims that fusing numerical profile features with text embeddings gives the most robust detector, that numerical features alone are sturdier against attack than text alone, and that neither human evaluators (F1 58.9%) nor prompted GPT-4 (F1 up to 85.7%) match the retrained automated detectors.

What carries the argument

The mechanism is GPT-assisted adversarial training: synthetic fake profiles generated with GPT-3.5 and GPT-4-Turbo are mixed into the training set alongside genuine profiles and manually created fakes, forcing the boosted tree classifiers to learn the statistical footprint of LLM-written profiles. The input representation is the Section Tag Embedding (STE), computed as the mean over a profile's sections of the difference between the section-text embedding and the section-tag embedding, then reduced with PCA to 150 dimensions and concatenated with 17 normalized numerical features into a 167-dimensional vector. The classifiers that carry the result are gradient-boosted trees, specifically XGBoost and CatBoost. The combined GPT3.5+4 training mixture is the part that yields cross-variant generalisation; single-family training leaves the other GPT variant's false accept rate elevated in double digits.

What would settle it

Generate a fresh test set of fake LinkedIn profiles with non-GPT LLMs such as Gemini, Claude, or Llama, or with humans editing GPT output, and run the Flair+XGBoost detector trained on the combined GPT3.5+4 data; if its false accept rate rises well above the reported 1-7%, the robustness claim holds only within the GPT model family rather than for LLM-generated fakes in general.

Watch

Extended reading notes

Core claim

The central claim is that the vulnerability created by LLM-generated fake profiles is largely a training-data gap: detectors trained only on genuine profiles and manual fakes fail on GPT profiles because those profiles closely mimic real ones, with a mean textual similarity of 88.9%. Adding GPT-3.5- and GPT-4-generated profiles to training closes the gap. Training on either family alone reduces that family's false accept rate but leaves the other family elevated; training on the combined GPT3.5+4 set yields false accept rates between 1.34% and 2.6% on all attacks, F1 scores above 97.5%, and false reject rates between 1.48% and 2.41%, without sacrificing performance on genuine profiles. The best performing configuration pairs Flair text embeddings with XGBoost. Feature-removal experiments show that numerical profile features resist attack better than text features alone, while combining the two types is the strongest configuration.

Load-bearing premise

The load-bearing premise is that the fake profiles used to train the countermeasure are representative of the fake profiles an attacker will actually deploy, since both the attack vectors and the adversarial training data came from the same GPT-3.5/GPT-4 model family; if real-world fakes are written by other LLMs or edited by humans, the restored detection rates may not transfer.

Editorial extensions

If this is right

  • Platforms can restore robustness by periodically retraining detectors on LLM-generated profiles rather than relying only on datasets of manually created fakes.
  • Keeping structural numerical features such as counts of jobs, education entries, skills, and connections matters, because text-only models degrade most sharply under LLM attack.
  • Task-specifically trained automated detectors currently outperform both human moderators and general-purpose LLM prompting on this detection task.
  • Robustness does not fully generalise across GPT versions, so training data should combine multiple LLM generations; training on only one variant leaves the other with double-digit false accept rates.
  • Calibration quality tracks robustness: lower Brier scores co-occur with lower false accept rates, so monitoring calibration can flag when a detector is losing its edge against new synthetic profiles.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the attack profiles and the adversarial training data were generated by the same GPT-3.5/GPT-4 family, the restored 1-7% false accept rate is best read as a same-family result; an attacker using a different LLM family could still evade, and this transfer is the paper's own stated limitation.
  • The reported correlation between calibration and false accept rate suggests a practical drift alarm: platforms could track the Brier score of their detector and trigger retraining when calibration worsens, without waiting for confirmed fake-profile labels.
  • As legitimate users increasingly use LLMs to write or polish their own profiles, the assumption that 'looks LLM-written' equals 'fake' will erode; detectors will likely need to shift from stylistic signals toward behavioural or relational signals, a consequence the paper leaves implicit.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper investigates the robustness of text-based fake profile detectors on LinkedIn against fake profiles generated by LLMs. The authors augment a public dataset with 1,200 GPT-3.5 and 600 GPT-4 generated profiles, extract Section Tag Embeddings (STE) from four transformer encoders plus 17 numerical features, and evaluate six classifiers, focusing on XGBoost and CatBoost. Baseline models perform well on manual fakes (F1~97%, FAR 6-7%) but degrade sharply on GPT-generated profiles (FAR up to 52%). The proposed GPT-assisted adversarial training, especially with combined GPT-3.5+GPT-4 generated profiles, restores FAR to 1.3-2.6% and F1 above 97.5% on held-out profiles from the same model family, while keeping FRR below 2.5%. The paper also benchmarks human and GPT-4 evaluators, showing both underperform the trained detectors, and reports ablation results on text-only, numerical-only, and fused features.

Significance. If the reported results hold beyond the evaluated model family, the paper makes a practical contribution: a simple retraining strategy that uses LLM-generated profiles to restore detector accuracy, along with a publicly available dataset extension and an analysis of feature modality robustness. The paper's strengths include the release of code and data, a careful hyperparameter tuning procedure, calibration analysis, and the explicit recognition in Section 5 that the attack and defense are from the same LLM family. The within-family asymmetry (GPT3.5-trained detectors leave GPT4P FAR at 16.9-19.3%; GPT4-trained detectors leave GPT3.5P FAR at 36-38%) is honestly reported in Figure 2 and is important evidence that the headline 1-7% FAR is an in-distribution result, not a demonstrated property across LLM families.

major comments (3)
  1. [Section 5 (third limitation); Section 4.2 and Figure 2(b)] The central claim 'restoring the False Accept Rate to between 1-7%' is supported only for test profiles generated by the same OpenAI GPT model family used for adversarial training. The paper's own results in Figure 2(b) show that GPT3.5-assisted training leaves GPT4P FAR at 16.9-19.3%, and GPT4-assisted training leaves GPT3.5P FAR at 36.1-38.5%; only the combined GPT3.5+4 training closes the gap, and only on OpenAI-generated profiles. Since Section 5 acknowledges that the system has not been evaluated against other LLM families, the abstract and conclusion should qualify the 1-7% FAR as an in-distribution result, and ideally the authors should add a transfer experiment with profiles from a different LLM family (e.g., Gemini, Claude, Llama) before claiming practical robustness.
  2. [Section 4.2; Figure 2] All key results are reported as point estimates without confidence intervals, standard deviations, or repeated runs. The selection of Flair+XGBoost as the best setup rests on differences of a few tenths of a percent (e.g., F1 98.2 vs 98.0, FAR 1.34 vs 1.5), which may be within experimental noise. Please provide variance estimates (e.g., multiple seeds or bootstrap) for the headline FAR/FRR comparisons in Figure 2, or explicitly state that these differences are not statistically evaluated.
  3. [Section 4.2 (baseline comparison)] The claim of outperforming Ayoobi et al. [4] compares the authors' STE models with numbers reported in that paper, rather than with a controlled re-run using the same train/test splits and preprocessing. The paper does not specify whether the original dataset splits were preserved after augmenting the data. Please clarify the comparison protocol or provide a matched re-implementation, since the reported improvement could be an artifact of different splits or feature preprocessing.
minor comments (6)
  1. [Section 3.5] The composition of the 360 profiles given to GPT-4 (how many FLPs vs LLM-generated fakes) and the selection of the 15 profiles used for human evaluation are not specified; please state the sampling procedure and class mix.
  2. [Section 3.2] The sentence 'PCA was applied to reduce embeddings to 150 dimensions from 786' does not match the varying dimensionalities of the encoders (e.g., Flair is typically 1024-dimensional); please report the input dimension for each encoder.
  3. [Figure 2] The matrix-style display is dense; please add axis labels to the heatmaps and consider moving the full numeric table for all encoder/classifier combinations to an appendix.
  4. [Section 2] The phrase 'LLM-assited' should be 'LLM-assisted'.
  5. [References] Reference [32] points to an anonymous p5js editor URL, which is not a stable archival source; please provide a persistent link or institutional page.
  6. [Abstract and Section 5] The abstract reports FAR '1-7%' while Section 4.2 reports 1.34-2.6% and Section 5 reports 1.34-2.28%; please harmonize these ranges and ensure the chosen range matches the presented results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the countermeasure is evaluated on held-out GPT profiles from the same generator family used for training, and the paper explicitly flags this as a limitation rather than presenting it as a cross-LLM guarantee.

full rationale

The paper's main derivation is empirical rather than definitional: baseline detectors are trained on genuine and manually fake profiles and evaluated on held-out LLM-generated profiles; the same held-out protocol is used after GPT-assisted adversarial training. The reported FAR restoration (1-7%) is an in-distribution held-out result on GPT-3.5/GPT-4 profiles, and the paper does not fit a parameter to the test set and then relabel it as a prediction. The only scope concern is Section 5, which states that 'both the creation of fake profiles for generating attack vectors and adversarial training used models from the same LLM family (OpenAI GPT),' and recommends evaluation 'against a wide variety of advanced LLMs.' This is an honest generalization limitation, not a circular derivation: the training and test profiles are disjoint, and the paper does not claim cross-family robustness. No load-bearing self-citation, imported uniqueness theorem, or ansatz-smuggling is present. The comparison against prior work [4] is external, and the implementation details are self-contained. Therefore no circular step meets the evidentiary bar.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on (1) the inherited dataset's validity, (2) the representativeness of the GPT-generated attack profiles, (3) several hand-chosen numbers (PCA dimension, augmentation size, few-shot count), and (4) tuned classifier hyperparameters whose values are not disclosed. No new theoretical entities are introduced; the paper is an empirical study.

free parameters (4)
  • PCA dimension = 150
    Hand-chosen based on variance explained (93.6-98.9%). The final 167-dimensional feature vector depends on this reduction, and the reported FAR values are sensitive to it.
  • GPT-4P augmentation count = 600
    Chosen by the authors; determines the sizes of the GPT4 train/test splits (420/180), directly affecting the FAR estimates.
  • GPT-4 few-shot exemplar count = 3
    The GPT-4 evaluation used 3 random labeled examples; the resulting F1 (85.7%) is likely sensitive to this number and the random choice.
  • Classifier hyperparameters (XGBoost, CatBoost) = not reported
    Tuned via Bayesian optimization and genetic algorithms on validation splits; exact parameter values are not listed in the paper.
assumptions (4)
  • domain assumption The Ayoobi et al. [4] dataset is a valid ground-truth corpus of genuine and manually fake LinkedIn profiles.
    The entire evaluation builds on this inherited dataset; the paper does not validate label quality or representativeness (Section 3.1).
  • domain assumption GPT-4-Turbo profiles generated with few-shot prompting from curated legit exemplars are representative of real-world LLM-generated fake profiles.
    The attack and countermeasure rely on this representativeness; Section 5 acknowledges only OpenAI models were used.
  • domain assumption The STE feature formula (mean of section-embedding minus tag-embedding) captures discriminative signal for profile authenticity.
    Adopted as a simplification of [4]; no ablation shows this exact formula is optimal compared to simpler alternatives (Section 3.2).
  • standard math A single fixed split yields stable performance estimates for the reported F1/FAR/FRR.
    Table 1 defines one split per scenario; the paper does not use repeated resampling or bootstrap, so stability is assumed (Section 3.4).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Weak Links in LinkedIn: Enhancing Fake Profile Detection in the Age of LLMs." pith.science (2026). https://pith.science/paper/DCAA3JQQ

@misc{pith2026250716860,
  author       = {Pith},
  title        = {Pith review of: Weak Links in LinkedIn: Enhancing Fake Profile Detection in the Age of LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DCAA3JQQ}},
  note         = {Machine review of arXiv:2507.16860}
}
read the original abstract

Large Language Models (LLMs) have made it easier to create realistic fake profiles on platforms like LinkedIn. This poses a significant risk for text-based fake profile detectors. In this study, we evaluate the robustness of existing detectors against LLM-generated profiles. While highly effective in detecting manually created fake profiles (False Accept Rate: 6-7%), the existing detectors fail to identify GPT-generated profiles (False Accept Rate: 42-52%). We propose GPT-assisted adversarial training as a countermeasure, restoring the False Accept Rate to between 1-7% without impacting the False Reject Rates (0.5-2%). Ablation studies revealed that detectors trained on combined numerical and textual embeddings exhibit the highest robustness, followed by those using numerical-only embeddings, and lastly those using textual-only embeddings. Complementary analysis on the ability of prompt-based GPT-4Turbo and human evaluators affirms the need for robust automated detectors such as the one proposed in this study.

Figures

Figures reproduced from arXiv: 2507.16860 by the authors.

Figure 1
Figure 1. (Left) PCA variance curves highlight that Flair and RoBERTa achieve [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Performance of STE-based models with Flair and DeBERTa embeddings [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Confusion matrices comparing human and GPT-4 performance on [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 35 canonical work pages

  1. [4]

    The looming threat of fake and llm- generated linkedin profiles: Challenges and opportunities for detection and preven- tion,

    N. Ayoobi, S. Shahriar, and A. Mukherjee, “The looming threat of fake and llm- generated linkedin profiles: Challenges and opportunities for detection and preven- tion,” inACM HT ’23, 2023

  2. [1]

    Linkedin statistics 2025: Active users data (worldwide),

    DemandSage, “Linkedin statistics 2025: Active users data (worldwide),” December

  3. [2]

    Aiding the detection of fake accounts in large scale social online services,

    Q. Cao, M. Sirivianos, X. Yang, and T. Pregueiro, “Aiding the detection of fake accounts in large scale social online services,” inNSDI 2012

  4. [3]

    Identifying Fake Profiles in LinkedIn

    S. Adikari and K. Dutta, “Identifying fake profiles in linkedin,” 2020. [Online]. Available: https://arxiv.org/abs/2006.01381

  5. [5]

    Fake accounts are getting way more common on linkedin (data),

    J. Strandell, “Fake accounts are getting way more common on linkedin (data),”

  6. [6]

    Linkedin fake profile detection,

    A. Gulati, R. Kumar, and V. Agarwal, “Linkedin fake profile detection,” https: //github.com/apoorva106/linkedin fake profile detection, 2024

  7. [7]

    Available: https://besedo.com/blog/linkedin-fake-accounts/

    [Online]. Available: https://besedo.com/blog/linkedin-fake-accounts/

  8. [8]

    Friend or faux: Graph-based early detec- tion of fake accounts on social networks,

    A. Breuer, R. Eilat, and U. Weinsberg, “Friend or faux: Graph-based early detec- tion of fake accounts on social networks,” inACM-WWW, 2020

Show all 37 references
  1. [9]

    Sybilbelief: A semi-supervised learning approach for structure-based sybil detection,

    N. Z. Gong, M. Frank, and P. Mittal, “Sybilbelief: A semi-supervised learning approach for structure-based sybil detection,”IEEE TIFS, 2014

  2. [10]

    Authorship verification applied to detection of compromised accounts on online social networks: A continuous approach,

    S. Barbon, R. A. Igawa, and B. Bogaz Zarpel˜ ao, “Authorship verification applied to detection of compromised accounts on online social networks: A continuous approach,”Multimedia Tools and Applications, 2016

  3. [11]

    Sybilflyover: Heterogeneous graph-based fake account de- tection model on social networks,

    S. Li and J. Y. at al., “Sybilflyover: Heterogeneous graph-based fake account de- tection model on social networks,”Knowledge-Based Systems, 2022

  4. [12]

    Llm-assisted cheating detection in korean lan- guage via keystrokes,

    D. Roh, R. Kumar, and A. Ngo, “Llm-assisted cheating detection in korean lan- guage via keystrokes,” inIEEE-IJCB, 2025

  5. [13]

    Keystroke dynamics against academic dishonesty in the age of llms,

    D. K. et al., “Keystroke dynamics against academic dishonesty in the age of llms,” inIEEE-IJCB, 2024. 10 Apoorva Gulati, Rajesh Kumar, Vinti Agarwal, and Aditya Sharma

  6. [14]

    Bhattasali and K

    T. Bhattasali and K. Saeed,Typing Pattern Analysis for Fake Profile Detection in Social Media, 2021

  7. [15]

    Spotting fake profiles in social networks via keystroke dynamics,

    A. Kuruvilla, R. Daley, and R. Kumar, “Spotting fake profiles in social networks via keystroke dynamics,” inIEEE-CCNC, 2024

  8. [16]

    Available: https://sites.google.com/view/weaklinksinlinkedin

    [Online]. Available: https://sites.google.com/view/weaklinksinlinkedin

  9. [17]

    Unmasking fake social network accounts with explainable intelligence,

    E. Alnagi, A. Ahmad, Q. A. Al-Haija, and A. Aref, “Unmasking fake social network accounts with explainable intelligence,”IJACSA, 2024

  10. [18]

    Roberta: A robustly optimized bert pretraining ap- proach,

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettle- moyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining ap- proach,” inICLR, 2020

  11. [19]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    J. Devlin, M.-W. Chang, and et al., “BERT: Pre-training of deep bidirectional transformers for language understanding.” ACL, 2019

  12. [20]

    Smarter, better, faster, longer: A modern bidirectional en- coder for fast, memory efficient, and long context finetuning and inference,

    B. W. et al., “Smarter, better, faster, longer: A modern bidirectional en- coder for fast, memory efficient, and long context finetuning and inference,” arXiv:2412.13663, 2024

  13. [21]

    Deberta: Decoding-enhanced bert with disentangled attention,

    P. He, X. Liu, J. Gao, and W. Chen, “Deberta: Decoding-enhanced bert with disentangled attention,” inICLR, 2021

  14. [22]

    Glove: Global vectors for word representation,

    J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for word representation,” inEMNLP, 2014

  15. [23]

    Contextual string embeddings for sequence labeling,

    A. Akbik, D. Blythe, and R. Vollgraf, “Contextual string embeddings for sequence labeling,” inNAACL-HLT 2018, 2018

  16. [24]

    Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks,

    A. Demontis and M. e. a. Melis, “Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks,” inUSENIX Security, 2019

  17. [25]

    Enhancing robustness of machine learning systems via data transformations,

    A. N. Bhagoji, D. Cullina, C. Sitawarin, and P. Mittal, “Enhancing robustness of machine learning systems via data transformations,” 2017. [Online]. Available: https://arxiv.org/abs/1704.02654

  18. [26]

    Catboost: unbiased boosting with categor- ical features,

    L. Prokhorenkova and G. e. a. Gusev, “Catboost: unbiased boosting with categor- ical features,” inNeurIPS, 2018

  19. [27]

    Xgboost: A scalable tree boosting system,

    T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,”SIGKDD, 2016

  20. [28]

    Algorithms for hyper-parameter optimization,

    J. Bergstra and R. e. a. Bardenet, “Algorithms for hyper-parameter optimization,” inNeurIPS, 2011

  21. [29]

    Practical bayesian optimization of machine learning algorithms,

    J. Snoek, H. Larochelle, and R. P. Adams, “Practical bayesian optimization of machine learning algorithms,” 2012

  22. [30]

    Understanding model calibration – a gentle introduction and visual exploration of calibration and the expected calibration error,

    M. Pavlovic, “Understanding model calibration – a gentle introduction and visual exploration of calibration and the expected calibration error,” 2025

  23. [31]

    Optimizing deep learning hyperparameters through an evolutionary algorithm,

    S. R. Young and D. C. e. a. Rose, “Optimizing deep learning hyperparameters through an evolutionary algorithm,” inMLHPC. ACM, 2015

  24. [32]

    https://editor.p5js.org/anonymous icwsm/full/N2Ait Cwo, 2025

  25. [33]

    Verification of forecasts expressed in terms of probability,

    G. W. Brier, “Verification of forecasts expressed in terms of probability,”Monthly Weather Review, 1950

  26. [34]

    Improving calibration through the relationship with adversarial robustness,

    Y. Qin, X. Wang, A. Beutel, and E. H. Chi, “Improving calibration through the relationship with adversarial robustness,”NeurIPS, 2020

  27. [35]

    Enriching tabular data with contextual llm embed- dings: A comprehensive ablation study for ensemble classifiers,

    G. Kasneci and E. Kasneci, “Enriching tabular data with contextual llm embed- dings: A comprehensive ablation study for ensemble classifiers,” 2024

  28. [37]

    Towards certification of uncertainty calibration under adversarial attacks,

    C. Emde, F. Pinto, T. Lukasiewicz, P. H. Torr, and A. Bibi, “Towards certification of uncertainty calibration under adversarial attacks,”ICLR, 2025

  29. [2024]

    Available: https://www.demandsage.com/linkedin-statistics/

    [Online]. Available: https://www.demandsage.com/linkedin-statistics/

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.