Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

PEEK: Phishing Evolution Framework for Phishing Generation and Evolving Pattern Analysis using Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims its iterative GAN-plus-LLM loop generates phishing emails that are realistic enough (84.8% usable) and diverse enough to measurably harden detectors against text-perturbation attacks.

desk verdict PEEK is a serious attempt at a real problem—diverse, current phishing data—but the GAN mechanics are underspecified and the headline quality numbers rest on a circular ChatGPT judge; worth a major revision, not a reject. read the letter →

arxiv 2411.11389 v2 pith:MKD62HA7 submitted 2024-11-18 cs.CR

Fengchao Chen , Tingmin Wu , Van Nguyen , Shuo Wang , Alsharif Abuadbba , Carsten Rudolph This is my paper · ORCID
classification cs.CR
keywords phishinggenerationlargelanguagemodelsadversarialtrainingGANpersuasionprinciplesdatasetaugmentationdetectionrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes PEEK, a recurrent framework that combines large language models with adversarial training to generate phishing emails that are both realistic and diverse. The authors claim PEEK raises the fraction of generated samples judged usable by a phishing analyzer from 21.4% to 84.8%, surpassing prompt-only and fine-tuning-only LLM baselines, and that detectors fine-tuned on the resulting corpus are markedly harder to evade, with attack success rates falling by up to 70% (e.g., RoBERTa's ASR drops from 14.81% to 8.76% under DeepWordBug). The motivation is practical: public phishing data are scarce, outdated, and privacy-restricted, so detectors trained on static corpora miss evolving tactics. If the claims are right, PEEK supplies both a scalable data-augmentation route and a running analysis of how LLM-driven phishing persuasion changes over time.

What carries the argument

The load-bearing mechanism is the iterative adversarial loop. A Llama 3.1 8B generator, fine-tuned with LoRA under a Chain-of-Thought style chat prompt, is trained against a Bi-LSTM discriminator with the GAN objective, so the discriminator's realism judgment pushes the generator toward human-like output. A ChatGPT-based analyzer, using the phishing-intent prompt of reference [5], then filters the outputs and assigns each a phishing authentication score; this same analyzer's judgment is what supports the 84.8% usability figure. The retained samples are fed into isolation forest and LDA clustering to find statistically distinct anomalies, which are annotated through six persuasion principles using LLOOM and LIWC-22; the extracted topic keywords are inserted back into the prompt to seed the next iteration. Everything downstream—the diversity analysis and the detector-robustness gains—rests on that loop.

What would settle it

Take a random sample of the PEEK-generated corpus and have independent human phishing analysts (or a second, independently calibrated analyzer) label it; if the proportion judged realistic falls well below 84.8% or agreement with the PEEK analyzer is weak, the quality and downstream robustness claims built on the filtered corpus lose their support.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that an iterative generation-extraction loop produces phishing emails that are simultaneously more realistic and more diverse than what prior LLM-based generators produce, and that training detectors on these emails substantially improves robustness to text-perturbation attacks. The authors find that PEEK-generated emails cluster into new topic families beyond banking (shipment, university services, login failures) and that their persuasive style shifts from blunt urgency toward concrete, context-rich Authority and Reciprocity cues; 84.8% of generated samples receive a phishing authenticity score of 6 or higher from the analyzer, and 71.3% score 8 or higher. Fine-tuning experiments on six detectors across four perturbation frameworks show consistent gains, with F1-scores above 0.88 and best-case attack success rates near 1%. The paper also reports zero-shot topic transfer with 88.1% of cross-topic samples judged realistic.

Load-bearing premise

The ChatGPT-based analyzer is an accurate and unbiased judge of whether a generated email is realistic phishing, because the same analyzer filters the corpus and produces the PAS scores behind the headline 84.8% and the robustness claims.

Editorial extensions

If this is right

  • Fine-tuning existing detectors on PEEK-generated phishing should lower attack success rates under TextFooler, PWWS, Pruthi, and DeepWordBug compared with the IWSPA 2023 and DeepAI datasets.
  • The claimed 88.1% PAS on seven zero-shot topics implies the loop can generate fresh topics on demand, reducing the need to collect new real-world phishing data.
  • The observed shift from urgency to concrete authority and reciprocity phrasing suggests detectors that over-weight words like 'urgent' or 'immediately' will miss the next wave of LLM-crafted emails.
  • Because the extracted patterns are fed back into training, the framework is explicitly designed to track tactic drift across iterations, giving defense teams an observable signal of how LLM phishing changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper does not spell out: because the same analyzer both filters the corpus and generates the PAS scores, the 84.8% figure is partially self-consistent; an independent judge could revise the number, and the ablation already shows the IBM NLU filter reports different rates.
  • Connected neighboring problem: the same generate-filter-extract-feedback loop should transfer to other text-based social engineering (SMS, chat, voicemail) as long as a judge of realism exists for that modality; the paper only demonstrates email.
  • Testable extension: the reported robustness gains are measured against four fixed perturbation tools; an adaptive adversary that knows PEEK was used in training could reveal whether the improvement persists under attack-specific tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. PEEK proposes an iterative framework for generating phishing emails with an LLM (Llama 3.1) trained under a GAN-style objective with a Bi-LSTM discriminator, followed by a ChatGPT-based analyzer that filters generated samples. The retained PEEK phishing corpus is then analyzed with isolation forests, LDA clustering, and LIWC/persuasion-principle tools to identify evolving phishing patterns, which are fed back into the generator prompts in a recurrent loop. The paper claims that PEEK raises the proportion of usable phishing samples from 21.4% to 84.8%, improves detector robustness after fine-tuning, and maintains 70% detection accuracy under adversarial attacks.

Significance. If the central claims hold, PEEK would be a valuable practical contribution: it provides a scalable method for generating diverse, realistic phishing emails and demonstrates that such data can improve the robustness of downstream detectors. The study evaluates multiple detectors, multiple adversarial perturbation attacks, and multiple benchmark datasets, and it includes an ablation comparing generator choices and prompt designs, which is commendable. The framework also incorporates an interpretable analysis of persuasion strategies, which is useful for understanding LLM-driven phishing evolution. However, the significance is currently conditional because the key quality metric is produced by the same analyzer used for filtering, and the GAN training mechanism for discrete text is not specified.

major comments (4)
  1. [Section III-A and Appendix B-A, Eq. (2)] The GAN training procedure is underspecified for discrete text. The generator is Llama 3.1, which produces discrete tokens, and the discriminator is a Bi-LSTM, but the paper does not describe how discriminator gradients propagate through the discrete sampling process. The objective in Eq. (2) is the standard continuous GAN objective; without a concrete mechanism such as Gumbel-softmax, policy gradient, or a differentiable relaxation, the claim that adversarial training improves the generator is not established. The ablation in Table VII may reflect the benefit of fine-tuning plus filtering rather than GAN-specific training. Please specify the exact training algorithm, including how G(z) is sampled and how dD/dG is computed.
  2. [Section III-B and Section VI-A1] The headline usability rate of 84.8% is circular because the same ChatGPT analyzer (gpt-3.5-turbo with Koide's prompt) that filters the corpus in Phase B is used to compute the PAS scores reported in Section VI-A1. The analyzer's validation (Appendix B-C, Table VI) is on 1,000 human-labeled emails, not on PEEK-generated emails, where the paper itself argues the language differs (e.g., downplayed urgency). The IBM NLU result in Table VII (88.2%) provides partial external support for existing topics, but the extra-topic row shows a large discrepancy (57.4% IBM NLU vs. 88.1% analyzer), so the cross-topic PAS claim in Section VI-C2 is not independently validated. Please add an external validation of the analyzer on PEEK-generated samples and report per-topic agreement.
  3. [Abstract and Section VI-C1, Tables X and XIV] The abstract's statement that PEEK 'still maintain[s] 70% detection accuracy against adversarial attacks' is contradicted by the reported tables. After PEEK fine-tuning, ALBERT achieves EVA-Acc of 0.31 under PWWS (Table X, and Table XIV), and 0.55 under TextFooler (Table IX). The robustness gain is real for several model-attack combinations, but the headline number is not the measured worst case. The abstract and Section VI-C1 should either report the full range or explicitly state that the 70% figure is the best case or applies only to specific configurations.
  4. [Section VI-C1 and Appendix D-C, Tables II-III, IX-XIV] The robustness results are reported without error bars or repeated-run variance, and the fine-tuning learning rate is selected separately for each attack type (Appendix D-C). This makes it difficult to assess whether the ASR improvements are statistically significant or partially due to per-attack hyperparameter tuning. Please report means and standard deviations over multiple random seeds and clarify whether the attack-specific hyperparameter search introduces optimism in the reported robustness gains.
minor comments (5)
  1. [Table VII caption and column headers] The column headers 'FT-NLU%' and 'FT-Llama3.1%' are ambiguous; the text should clarify that these are fine-tuned IBM NLU and Llama 3.1 classifiers used as independent filters, not generator variants.
  2. [Section VI-A1 and Introduction] The paper switches between 'PAS analysis identifies 84.8% realistic phishing mimicry' (Introduction) and '84.8% of PEEK-generated samples received PAS scores of 6 or higher' (Section VI-A1); please use one consistent definition and make the PAS threshold explicit wherever the number is cited.
  3. [Appendix B-A] There is a typo: 'Bi-LISM Attention' should be 'Bi-LSTM Attention'. Similar small typos appear in Table II ('Comapred') and in the text ('P EEKanml').
  4. [Section VII, 'Towards Future Phishing Analyzers'] The discussion acknowledges that analyzers will evolve but does not address the circularity concern that the same analyzer both filters and evaluates the corpus. A sentence noting this limitation and the need for independent evaluators would strengthen the discussion.
  5. [Appendix A-A and Table V] The Cambridge dataset is described as private, which limits reproducibility of the full pipeline. Please state explicitly which subsets of the results depend on the private dataset and whether the released corpus will include the processed Cambridge-derived samples.

Circularity Check

1 steps flagged · score 6.0 of 10

The 84.8% usable-samples claim is the ChatGPT analyzer's own acceptance rate, because the same analyzer filters the corpus and defines the PAS quality metric.

  1. self definitional [Section III-B (Phase B filter), Section IV (PAS definition), Section VI-A1, Table VII]
    "We deploy ChatGPT as an analyzer to validate and retain PEEK-generated phishing samples that are classified as “phishing” based on the prompt designed by Koide [5]. ... PAS is determined through a ChatGPT analyzer employing targeted prompts to classify samples, assign phishing authenticity scores, and provide explanatory rationales [5]. ... overall, 84.8% of PEEK-generated phishing samples received PAS scores of 6 or higher, indicating realistic phishing attempts."

    The Phase B filter retains exactly the samples the ChatGPT analyzer labels phishing, and the PAS quality metric is computed by the same ChatGPT analyzer using the same Koide prompt. Thus the headline 84.8% 'usable/realistic' figure is the filter's own acceptance rate on PEEK outputs: the evaluation instrument and the selection instrument are identical, so the claim reduces by construction to the analyzer's self-consistency rather than to an independent property of the corpus. The independent IBM NLU check appears only in the ablation (Table VII, 88.2%), while the abstract and main text cite the ChatGPT number, and the extra-topic 88.1% PAS has no independent check.

full rationale

The circularity is localized to the quality/usability headline. Section VI-C's robustness evaluation is externally grounded: detectors are fine-tuned on PEEK phishing and then tested against TextFooler, PWWS, Pruthi, and DeepWordBug, so those ASR/F1 improvements are not outputs of the ChatGPT analyzer. The abstract's 'maintaining 70% detection accuracy' is inconsistent with Table X (e.g., ALBERT EVA-Acc 0.31 under PWWS), but that is a correctness/consistency issue, not circularity. I found no load-bearing self-citation chain: the authors' own prior works cited (e.g., [72], [75], [78]) are peripheral related work, not the justification for PEEK's core claims. The central 84.8% claim, however, is self-referential because the same gpt-3.5-turbo analyzer both selects PEEK phishing (Phase B) and produces the PAS scores used to declare the samples usable (Section VI-A1), with only a partial independent IBM NLU check in the ablation. This warrants a partial-circularity score rather than a full one, since the downstream robustness results retain independent content.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The framework does not introduce new physical or mathematical entities. Its central claims rest on the reliability of the ChatGPT analyzer, the effectiveness of an underspecified GAN-style training loop, and the representativeness of filtered public datasets. The headline quality metric is circular because the filter and the metric use the same analyzer, though an independent IBM NLU check in the ablation provides partial external support.

free parameters (4)
  • PAS usability threshold = 6 (on 0-10 scale)
    Samples with analyzer score at least 6 are counted as usable phishing; this cutoff directly determines the reported 84.8% usable-sample rate. Changing the threshold changes the headline result.
  • Email length filter = 64 to 512 tokens
    Dataset preprocessing removes emails shorter than 64 tokens; this manual choice affects what the generator learns and may bias the distribution.
  • GAN training epochs = 5
    The generator is saved after 5 epochs of GAN training; no convergence analysis is provided, so the chosen epoch count could influence generation quality.
  • LoRA hyperparameters = r=16, alpha=16, lr=3e-4, 4-bit quantization
    Standard fine-tuning knobs; not fitted to the central claim, but needed to reproduce the generator.
assumptions (6)
  • domain assumption The GAN objective (Eq. 2) with min-max training is a valid procedure for improving a discrete-output LLM generator.
    The paper states GAN-based training but does not describe how gradients pass through sampled text or how the discriminator loss updates the Llama generator; the entire quality improvement over fine-tuning is attributed to this mechanism.
  • domain assumption gpt-3.5-turbo with Koide's prompt is a reliable and unbiased phishing analyzer.
    The analyzer is validated on 1,000 samples (F1=0.95, F2=0.97), but the same analyzer is then used both to filter the corpus and to compute PAS; any label bias propagates into all quality and robustness results.
  • domain assumption The six public phishing datasets after deduplication, similarity removal, and length filtering are representative of real-world phishing.
    The final 13,800 emails are a heavily filtered subset of roughly 70,000 collected emails; representativeness is not analyzed.
  • domain assumption Text perturbation attacks (TextFooler, PWWS, Pruthi, DeepWordBug) produce realistic adversarial phishing emails.
    Robustness claims rest on these four attack frameworks; no validation that perturbed emails are semantically meaningful phishing attempts.
  • domain assumption Cialdini's persuasion principles and the tools LLooM and LIWC-22 validly operationalize deceptive strategies.
    The evolution analysis and DPS scores depend on these tools' outputs; no manual annotation validation is reported.
  • domain assumption Fine-tuning detectors on generated data transfers to improved robustness on real-world adversarial examples.
    The paper assumes gains on the benchmark test set generalize; no out-of-distribution evaluation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PEEK: Phishing Evolution Framework for Phishing Generation and Evolving Pattern Analysis using Large Language Models." pith.science (2026). https://pith.science/paper/MKD62HA7

@misc{pith2026241111389,
  author       = {Pith},
  title        = {Pith review of: PEEK: Phishing Evolution Framework for Phishing Generation and Evolving Pattern Analysis using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MKD62HA7}},
  note         = {Machine review of arXiv:2411.11389}
}
read the original abstract

Phishing remains a pervasive cyber threat, as attackers craft deceptive emails to lure victims into revealing sensitive information. While Artificial Intelligence (AI), in particular, deep learning, has become a key component in defending against phishing attacks, these approaches face critical limitations. The scarcity of publicly available, diverse, and updated data, largely due to privacy concerns, constrains detection effectiveness. As phishing tactics evolve rapidly, models trained on limited, outdated data struggle to detect new, sophisticated deception strategies, leaving systems and people vulnerable to an ever-growing array of attacks. We propose the first Phishing Evolution FramEworK (PEEK) for augmenting phishing email datasets with respect to quality and diversity, and analyzing changing phishing patterns for detection to adapt to updated phishing attacks. Specifically, we integrate large language models (LLMs) into the process of adversarial training to enhance the performance of the generated dataset and leverage persuasion principles in a recurrent framework to facilitate the understanding of changing phishing strategies. PEEK raises the proportion of usable phishing samples from 21.4% to 84.8%, surpassing existing works that rely on prompting and fine-tuning LLMs. The phishing datasets provided by PEEK, with evolving phishing patterns, outperform the other two available LLM-generated phishing email datasets in improving detection robustness. PEEK phishing boosts detectors' accuracy to over 88% and reduces adversarial sensitivity by up to 70%, still maintaining 70% detection accuracy against adversarial attacks.

Figures

Figures reproduced from arXiv: 2411.11389 by the authors.

Figure 1
Figure 1. Structure, phases and steps of the Phishing Evolution Framework (PEEK). The PEEK integrates a GAN model in Phase [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Chain of Thought Designed Prompts. Training prompts [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 5
Figure 5. Latent Phishing Topics Distribu￾tion. Topic circle size denotes weight, distance shows similarity. Dispersion re￾flects phishing diversity. The Phishing Authentication Score (PAS) further validates the realism of PEEK phishing samples. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figures from the paper (8 more)
Figure 6
Figure 6. Figure 6: Examples of PEEK-generated Email Sample (T1), password updates (T2), and restricted account access (T4); and 2) business-oriented communications and extending company-related interactions (T3). While the majority of the PEEK phishing emails align closely with these est…
Figure 8
Figure 8. Figure 8: Deceptive Persuasion Score (DPS) for Normal vs. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 7
Figure 7. Figure 7: Identified PEEK Anomalies. Though PEEK anomalies [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: Linguistic Feature Differences Aligned with Persuasion Principles. Statistically significant features illustrate how specific [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Example of PEEK-generated phishing email with [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Persuasion Principle Usage Across Phishing Themes. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Examples of PEEK-generated Email Sample APPENDIX B OBJECT FUNCTION AND MODEL CONFIGURATION A. Object Function In this work, we denote the dataset as D = {(x1, y1), · · · ,(xn, yn)}, where xi represents the email con￾tent, yi is the label of email xi where yi ∈ {0, 1} …
Figure 13
Figure 13. Figure 13: Persuasion Principle Usage Across Phishing Themes. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Can We End the Cat-and-Mouse Game? Simulating Self-Evolving Phishing Attacks with LLMs and Genetic Algorithms

    cs.CR 2025-07 conditional novelty 6.0 of 10

    A closed-loop LLM simulation with genetic algorithms suggests that phishing strategies can evolve to bypass simulated victims' defenses, but the result has not been validated against real humans.

Reference graph

Works this paper leans on

86 extracted references · 59 canonical work pages · cited by 1 Pith paper

  1. [1]

    Fighting against phishing attacks: state of the art and future challenges,

    B. B. Gupta, A. Tewari, A. K. Jain, and D. P. Agrawal, “Fighting against phishing attacks: state of the art and future challenges,” Neural Computing and Applications , vol. 28, pp. 3629–3654, 2017

    2017

  2. [2]

    Apwg 2024 phishing report,

    “Apwg 2024 phishing report,” https://apwg.org/ trendsreports/, 2024

    2024

  3. [3]

    A systematic literature review on phishing email detection using natural language processing techniques,

    S. Salloum, T. Gaber, S. Vadera, and K. Shaalan, “A systematic literature review on phishing email detection using natural language processing techniques,” IEEE Access, vol. 10, pp. 65 703–65 727, 2022

    2022

  4. [4]

    Applying machine learn- ing and natural language processing to detect phishing email,

    A. Alhogail and A. Alsabih, “Applying machine learn- ing and natural language processing to detect phishing email,” Computers & Security, vol. 110, p. 102414, 2021

    2021

  5. [5]

    Chatspamdetector: Leveraging large language models for effective phishing email detection,

    T. Koide, N. Fukushi, H. Nakano, and D. Chiba, “Chatspamdetector: Leveraging large language models for effective phishing email detection,” arXiv preprint arXiv:2402.18093, 2024

    arXiv 2024

  6. [6]

    A survey of large language models for cyber threat detection,

    Y . Chen, M. Cui, D. Wang, Y . Cao, P. Yang, B. Jiang, Z. Lu, and B. Liu, “A survey of large language models for cyber threat detection,” Computers & Security , p. 104016, 2024

    2024

  7. [7]

    Towards security threats of deep learning systems: A survey,

    Y . He, G. Meng, K. Chen, X. Hu, and J. He, “Towards security threats of deep learning systems: A survey,” IEEE Transactions on Software Engineering , vol. 48, no. 5, pp. 1743–1770, 2020

    2020

  8. [8]

    Privacy engineering in the wild: Understanding the practitioners’ mindset, organizational aspects, and current practices,

    L. H. Iwaya, M. A. Babar, and A. Rashid, “Privacy engineering in the wild: Understanding the practitioners’ mindset, organizational aspects, and current practices,” IEEE Transactions on Software Engineering , vol. 49, no. 9, pp. 4324–4348, 2023

    2023

Show all 86 references
  1. [9]

    Sok: a comprehensive reexamination of phishing research from the security perspective,

    A. Das, S. Baki, A. El Aassal, R. Verma, and A. Dun- bar, “Sok: a comprehensive reexamination of phishing research from the security perspective,” IEEE Communi- cations Surveys & Tutorials, vol. 22, no. 1, pp. 671–708, 2019

    2019

  2. [10]

    Phish- ing email detection using natural language processing techniques: a literature survey,

    S. Salloum, T. Gaber, S. Vadera, and K. Shaalan, “Phish- ing email detection using natural language processing techniques: a literature survey,” Procedia Computer Sci- ence, vol. 189, pp. 19–28, 2021

    2021

  3. [11]

    Text data aug- mentation: Towards better detection of spear-phishing emails,

    M. Regina, M. Meyer, and S. Goutal, “Text data aug- mentation: Towards better detection of spear-phishing emails,” arXiv preprint arXiv:2007.02033 , 2020

    2007 arXiv

  4. [12]

    Ad- versarial sampling attacks against phishing detection,

    H. Shirazi, B. Bezawada, I. Ray, and C. Anderson, “Ad- versarial sampling attacks against phishing detection,” in Data and Applications Security and Privacy XXXIII: 33rd Annual IFIP WG 11.3 Conference, DBSec 2019, Charleston, SC, USA, July 15–17, 2019, Proceedings 33 . Springe...

    2019

  5. [13]

    Enhancing detection of arabic social spam using data augmentation and machine learning,

    A. M. Alkadri, A. Elkorany, and C. Ahmed, “Enhancing detection of arabic social spam using data augmentation and machine learning,” Applied Sciences, vol. 12, no. 22, p. 11388, 2022

    2022

  6. [14]

    Data augmenta- tion methods for enhancing robustness in text classifica- tion tasks,

    H. Tang, S. Kamei, and Y . Morimoto, “Data augmenta- tion methods for enhancing robustness in text classifica- tion tasks,” Algorithms, vol. 16, no. 1, p. 59, 2023

    2023

  7. [15]

    Adversarial examples generation method for chinese text classification,

    E.-H. Xu, X.-L. Zhang, Y .-P. Wang, S. Zhang, L.-X. Liu, and L. Xu, “Adversarial examples generation method for chinese text classification,” Int. J. Netw. Secur , vol. 24, pp. 587–596, 2022

    2022

  8. [16]

    Rule-based adversarial sample generation for text classification,

    N. Zhou, N. Yao, J. Zhao, and Y . Zhang, “Rule-based adversarial sample generation for text classification,” Neural Computing and Applications , vol. 34, no. 13, pp. 10 575–10 586, 2022

    2022

  9. [17]

    An empirical survey of data augmentation for limited data learning in nlp,

    J. Chen, D. Tam, C. Raffel, M. Bansal, and D. Yang, “An empirical survey of data augmentation for limited data learning in nlp,” Transactions of the Association for Computational Linguistics, vol. 11, pp. 191–211, 2023

    2023

  10. [18]

    Adversarial robustness of phishing email detection models,

    P. Mehdi Gholampour and R. M. Verma, “Adversarial robustness of phishing email detection models,” in Pro- ceedings of the 9th ACM International Workshop on Security and Privacy Analytics , 2023, pp. 67–76

    2023

  11. [19]

    Data augmentation in classification and segmentation: A survey and new strategies,

    K. Alomar, H. I. Aysel, and X. Cai, “Data augmentation in classification and segmentation: A survey and new strategies,” Journal of Imaging, vol. 9, no. 2, p. 46, 2023

    2023

  12. [20]

    Analysis and prevention of ai-based phishing email attacks,

    C. S. Eze and L. Shamir, “Analysis and prevention of ai-based phishing email attacks,” Electronics, vol. 13, no. 10, p. 1839, 2024

    2024

  13. [21]

    Phishing or not phishing? a survey on the detection of phishing websites,

    R. Zieni, L. Massari, and M. C. Calzarossa, “Phishing or not phishing? a survey on the detection of phishing websites,” IEEE Access , vol. 11, pp. 18 499–18 519, 2023

    2023

  14. [22]

    Devising and detecting phishing: Large lan- guage models vs. smaller human models,

    F. Heiding, B. Schneier, A. Vishwanath, J. Bernstein, and P. S. Park, “Devising and detecting phishing: Large lan- guage models vs. smaller human models,” arXiv preprint arXiv:2308.12287, 2023

    2023 arXiv

  15. [23]

    Large lan- guage model lateral spear phishing: A comparative study in large-scale organizational settings,

    M. Bethany, A. Galiopoulos, E. Bethany, M. B. Karke- vandi, N. Vishwamitra, and P. Najafirad, “Large lan- guage model lateral spear phishing: A comparative study in large-scale organizational settings,” arXiv preprint arXiv:2401.09727, 2024

    2024 arXiv

  16. [24]

    A survey on explainable anomaly detection,

    Z. Li, Y . Zhu, and M. Van Leeuwen, “A survey on explainable anomaly detection,” ACM Transactions on Knowledge Discovery from Data , vol. 18, no. 1, pp. 1– 54, 2023

    2023

  17. [25]

    Detection of ai-generated emails-a case study,

    P. Gryka, K. Grado ´n, M. Kozłowski, M. Kutyła, and A. Janicki, “Detection of ai-generated emails-a case study,” in Proceedings of the 19th International Confer- ence on Availability, Reliability and Security , 2024, pp. 1–8

    2024

  18. [26]

    Phishing and social engineering attack prevention with llms,

    H. J. D. S. De Queiroz, “Phishing and social engineering attack prevention with llms,” in Revolutionizing Cyberse- curity With Deep Learning and Large Language Models . IGI Global Scientific Publishing, 2025, pp. 133–164

    2025

  19. [27]

    ClaudeAI, https://claude.ai/, 2024

    2024

  20. [28]

    Gemini: a family of highly capable multi- modal models,

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Mil- lican et al. , “Gemini: a family of highly capable multi- modal models,” arXiv preprint arXiv:2312.11805 , 2023

    2023 arXiv

  21. [29]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  22. [30]

    Bi-lstm model to increase accuracy in text classification: Combining word2vec cnn and attention mechanism,

    B. Jang, M. Kim, G. Harerimana, S.-u. Kang, and J. W. Kim, “Bi-lstm model to increase accuracy in text classification: Combining word2vec cnn and attention mechanism,” Applied Sciences, vol. 10, no. 17, p. 5841, 2020

    2020

  23. [31]

    An explainable transformer-based model for phishing email detection: A large language model approach,

    M. A. Uddin and I. H. Sarker, “An explainable transformer-based model for phishing email detection: A large language model approach,” arXiv preprint arXiv:2402.13871, 2024

    2024 arXiv

  24. [32]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

    2022

  25. [33]

    Isolation forest,

    F. T. Liu, K. M. Ting, and Z.-H. Zhou, “Isolation forest,” in 2008 eighth ieee international conference on data mining. IEEE, 2008, pp. 413–422

    2008

  26. [34]

    Latent dirichlet allocation,

    D. M. Blei, A. Y . Ng, and M. I. Jordan, “Latent dirichlet allocation,” Journal of machine Learning research, vol. 3, no. Jan, pp. 993–1022, 2003

    2003

  27. [35]

    The science of persuasion,

    R. B. Cialdini, “The science of persuasion,” Scientific American, vol. 284, no. 2, pp. 76–81, 2001

    2001

  28. [36]

    Concept induction: Analyzing unstructured text with high-level concepts using lloom,

    M. S. Lam, J. Teoh, J. A. Landay, J. Heer, and M. S. Bernstein, “Concept induction: Analyzing unstructured text with high-level concepts using lloom,” in Proceed- ings of the CHI Conference on Human Factors in Com- puting Systems, 2024, pp. 1–28

    2024

  29. [37]

    The development and psychometric properties of liwc-22,

    R. L. Boyd, A. Ashokkumar, S. Seraj, and J. W. Pen- nebaker, “The development and psychometric properties of liwc-22,” Austin, TX: University of Texas at Austin , vol. 10, 2022

    2022

  30. [38]

    Optimizing semantic coherence in topic models,

    D. Mimno, H. Wallach, E. Talley, M. Leenders, and A. McCallum, “Optimizing semantic coherence in topic models,” in Proceedings of the 2011 conference on empirical methods in natural language processing, 2011, pp. 262–272

    2011

  31. [39]

    Class-based n-gram models of natural language,

    P. F. Brown, V . J. Della Pietra, P. V . Desouza, J. C. Lai, and R. L. Mercer, “Class-based n-gram models of natural language,” Computational linguistics, vol. 18, no. 4, pp. 467–480, 1992

    1992

  32. [40]

    Interpreting tf-idf term weights as making relevance decisions,

    H. C. Wu, R. W. P. Luk, K. F. Wong, and K. L. Kwok, “Interpreting tf-idf term weights as making relevance decisions,” ACM Transactions on Information Systems (TOIS), vol. 26, no. 3, pp. 1–37, 2008

    2008

  33. [41]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014

    2014 arXiv

  34. [42]

    Machine literature searching viii. operational criteria for designing information retrieval systems,

    K. Allen, M. M. Berry, F. U. Luehrs Jr, and J. W. Perry, “Machine literature searching viii. operational criteria for designing information retrieval systems,” American Documentation (pre-1986), vol. 6, no. 2, p. 93, 1955

    1986

  35. [43]

    Iwspa phishing dataset,

    “Iwspa phishing dataset,” https://goo.su/gSzA V, 2021

    2021

  36. [44]

    Nazario phishing dataset,

    “Nazario phishing dataset,” https://shorturl.asia/HfID5, 2021

    2021

  37. [45]

    Miller smiles phishing dataset,

    “Miller smiles phishing dataset,” https://shorturl.asia/ lbHx1, 2021

    2021

  38. [46]

    Phish bowl phishing dataset,

    “Phish bowl phishing dataset,” https://shorturl.asia/ NSycm, 2021

    2021

  39. [47]

    Nigerian fraud dataset,

    “Nigerian fraud dataset,” https://shorturl.asia/l9p8r, 2021

    2021

  40. [48]

    An improved transformer-based model for detecting phishing, spam and ham emails: A large language model approach,

    S. Jamal, H. Wimmer, and I. H. Sarker, “An improved transformer-based model for detecting phishing, spam and ham emails: A large language model approach,” Security and Privacy , p. e402, 2024

    2024

  41. [49]

    The psychological meaning of words: Liwc and computerized text analysis methods,

    Y . R. Tausczik and J. W. Pennebaker, “The psychological meaning of words: Liwc and computerized text analysis methods,” Journal of language and social psychology , vol. 29, no. 1, pp. 24–54, 2010

    2010

  42. [50]

    Changing others’ beliefs online: Online comments’ persuasiveness,

    L. Xiao and T. Khazaei, “Changing others’ beliefs online: Online comments’ persuasiveness,” in Proceedings of the 10th International Conference on social media and Society, 2019, pp. 92–101

    2019

  43. [51]

    Cognitive triaging of phishing attacks,

    A. Van Der Heijden and L. Allodi, “Cognitive triaging of phishing attacks,” in 28th USENIX Security Symposium (USENIX Security 19) , 2019, pp. 1309–1326

    2019

  44. [52]

    The development and psychometric properties of liwc2015,

    J. W. Pennebaker, R. L. Boyd, K. Jordan, and K. Black- burn, “The development and psychometric properties of liwc2015,” University of Texas at Austin, Technical Report, 2015

    2015

  45. [53]

    Infobert: Improving robustness of language models from an information theoretic perspective,

    B. Wang, S. Wang, Y . Cheng, Z. Gan, R. Jia, B. Li, and J. Liu, “Infobert: Improving robustness of language models from an information theoretic perspective,” arXiv preprint arXiv:2010.02329, 2020

    2010 arXiv

  46. [54]

    Generating nat- ural language adversarial examples through probability weighted word saliency,

    S. Ren, Y . Deng, K. He, and W. Che, “Generating nat- ural language adversarial examples through probability weighted word saliency,” in Proceedings of the 57th annual meeting of the association for computational linguistics, 2019, pp. 1085–1097

    2019

  47. [55]

    Combating adversarial misspellings with robust word recognition,

    D. Pruthi, B. Dhingra, and Z. C. Lipton, “Combating adversarial misspellings with robust word recognition,” arXiv preprint arXiv:1905.11268 , 2019

    1905 arXiv

  48. [56]

    Black-box generation of adversarial text sequences to evade deep learning classifiers,

    J. Gao, J. Lanchantin, M. L. Soffa, and Y . Qi, “Black-box generation of adversarial text sequences to evade deep learning classifiers,” in 2018 IEEE Security and Privacy Workshops (SPW). IEEE, 2018, pp. 50–56

    2018

  49. [57]

    Chatgpt and a new academic reality: Arti- ficial intelligence-written research papers and the ethics of the large language models in scholarly publishing,

    B. D. Lund, T. Wang, N. R. Mannuru, B. Nie, S. Shimray, and Z. Wang, “Chatgpt and a new academic reality: Arti- ficial intelligence-written research papers and the ethics of the large language models in scholarly publishing,” Journal of the Association for Information Science ...

    2023

  50. [58]

    Social engineering in cybersecurity: Effect mechanisms, human vulnerabilities and attack methods,

    Z. Wang, H. Zhu, and L. Sun, “Social engineering in cybersecurity: Effect mechanisms, human vulnerabilities and attack methods,” Ieee Access , vol. 9, pp. 11 895– 11 910, 2021

    2021

  51. [59]

    Udh: Universal deep hiding for steganography, water- marking, and light field messaging,

    C. Zhang, P. Benz, A. Karjauv, G. Sun, and I. S. Kweon, “Udh: Universal deep hiding for steganography, water- marking, and light field messaging,” Advances in Neural Information Processing Systems , vol. 33, pp. 10 223– 10 234, 2020

    2020

  52. [60]

    Towards adversarial phishing detection,

    T. K. Panum, K. Hageman, R. R. Hansen, and J. M. Ped- ersen, “Towards adversarial phishing detection,” in 13th USENIX Workshop on Cyber Security Experimentation and Test (CSET 20) , 2020

    2020

  53. [61]

    Hooked: A real-world study on qr code phishing,

    M. Geisler and D. P ¨ohn, “Hooked: A real-world study on qr code phishing,” arXiv preprint arXiv:2407.16230 , 2024

    2024 arXiv

  54. [62]

    An image is worth a thousand toxic words: A metamorphic testing framework for content moderation software,

    W. Wang, J. Huang, J.-t. Huang, C. Chen, J. Gu, P. He, and M. R. Lyu, “An image is worth a thousand toxic words: A metamorphic testing framework for content moderation software,” in 2023 38th IEEE/ACM Interna- tional Conference on Automated Software Engineering (ASE). IEEE, 20...

    2023

  55. [63]

    {KnowPhish}: Large lan- guage models meet multimodal knowledge graphs for enhancing {Reference-Based} phishing detection,

    Y . Li, C. Huang, S. Deng, M. L. Lock, T. Cao, N. Oo, H. W. Lim, and B. Hooi, “ {KnowPhish}: Large lan- guage models meet multimodal knowledge graphs for enhancing {Reference-Based} phishing detection,” in 33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 793–810

    2024

  56. [64]

    From chatbots to phishbots?: Phishing scam generation in commercial large language models,

    S. S. Roy, P. Thota, K. V . Naragam, and S. Nilizadeh, “From chatbots to phishbots?: Phishing scam generation in commercial large language models,” in 2024 IEEE Symposium on Security and Privacy (SP) . IEEE Com- puter Society, 2024, pp. 221–221

    2024

  57. [65]

    Chatgpt’s security risks and benefits: of- fensive and defensive use-cases, mitigation measures, and future implications,

    M. Charfeddine, H. M. Kammoun, B. Hamdaoui, and M. Guizani, “Chatgpt’s security risks and benefits: of- fensive and defensive use-cases, mitigation measures, and future implications,” IEEE Access, 2024

    2024

  58. [66]

    A survey on dataset quality in machine learning,

    Y . Gong, G. Liu, Y . Xue, R. Li, and L. Meng, “A survey on dataset quality in machine learning,” Information and Software Technology, vol. 162, p. 107268, 2023

    2023

  59. [67]

    Generating optimal attack paths in generative adversarial phishing,

    R. Al-Qurashi, A. AlEroud, A. A. Saifan, M. Alsmadi, and I. Alsmadi, “Generating optimal attack paths in generative adversarial phishing,” in 2021 IEEE Interna- tional Conference on Intelligence and Security Informat- ics (ISI). IEEE, 2021, pp. 1–6

    2021

  60. [68]

    Weaponizing data science for social engineering: Automated e2e spear phishing on twitter,

    J. Seymour and P. Tully, “Weaponizing data science for social engineering: Automated e2e spear phishing on twitter,” Black Hat USA , vol. 37, pp. 1–39, 2016

    2016

  61. [69]

    Is bert really robust? a strong baseline for natural language attack on text classification and entailment,

    D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits, “Is bert really robust? a strong baseline for natural language attack on text classification and entailment,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 05, 2020, pp. 8018–8025

    2020

  62. [70]

    Social engineering in cybersecurity: The evolution of a concept,

    J. M. Hatfield, “Social engineering in cybersecurity: The evolution of a concept,” Computers & Security , vol. 73, pp. 102–113, 2018

    2018

  63. [71]

    Defining social engineer- ing in cybersecurity,

    Z. Wang, L. Sun, and H. Zhu, “Defining social engineer- ing in cybersecurity,” IEEE Access , vol. 8, pp. 85 094– 85 115, 2020

    2020

  64. [72]

    Email summarization to assist users in phishing iden- tification,

    A. Kashapov, T. Wu, S. Abuadbba, and C. Rudolph, “Email summarization to assist users in phishing iden- tification,” in Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security , 2022, pp. 1234–1236

    2022

  65. [73]

    Email phishing and signal detection: How persuasion principles and personality influence response patterns and accuracy,

    P. Lawson, C. J. Pearson, A. Crowson, and C. B. Mayhorn, “Email phishing and signal detection: How persuasion principles and personality influence response patterns and accuracy,” Applied ergonomics, vol. 86, p. 103084, 2020

    2020

  66. [74]

    Detection method of phish- ing email based on persuasion principle,

    X. Li, D. Zhang, and B. Wu, “Detection method of phish- ing email based on persuasion principle,” in 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), vol. 1. IEEE, 2020, pp. 571–574

    2020

  67. [75]

    Utilizing large language models with human feedback integration for generating dedicated warning for phishing emails,

    Q. H. Nguyen, T. Wu, V . Nguyen, X. Yuan, J. Xue, and C. Rudolph, “Utilizing large language models with human feedback integration for generating dedicated warning for phishing emails,” in Proceedings of the 2nd ACM Workshop on Secure and Trustworthy Deep Learning Systems, 202...

    2024

  68. [76]

    Digital deception: Gen- erative artificial intelligence in social engineering and phishing,

    M. Schmitt and I. Flechais, “Digital deception: Gen- erative artificial intelligence in social engineering and phishing,” arXiv preprint arXiv:2310.13715 , 2023

    2023 arXiv

  69. [77]

    Social engineering at- tacks: A survey,

    F. Salahdine and N. Kaabouch, “Social engineering at- tacks: A survey,” Future internet, vol. 11, no. 4, p. 89, 2019

    2019

  70. [78]

    Ai2tale: An innovative information theory-based approach for learning to localize phishing attacks,

    V . Nguyen, T. Wu, M. Grobler, S. Nepal, C. Rudolph et al., “Ai2tale: An innovative information theory-based approach for learning to localize phishing attacks,” in The Thirteenth International Conference on Learning Representations, 2025

    2025

  71. [79]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” Advances in neural infor- mation processing systems , vol. 27, 2014

    2014

  72. [80]

    Conditional generative ad- versarial nets,

    M. Mirza and S. Osindero, “Conditional generative ad- versarial nets,” arXiv preprint arXiv:1411.1784 , 2014

    2014 arXiv

  73. [81]

    Meta-AI, https://shorturl.asia/LaHr7, 2024

    2024

  74. [82]

    Evaluating chatgpt’s performance for multilingual and emoji-based hate speech detection,

    M. Das, S. K. Pandey, and A. Mukherjee, “Evaluating chatgpt’s performance for multilingual and emoji-based hate speech detection,” arXiv preprint arXiv:2305.13276, 2023

    2023 arXiv

  75. [83]

    Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech,

    F. Huang, H. Kwak, and J. An, “Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech,” in Companion pro- ceedings of the ACM web conference 2023 , 2023, pp. 294–297

    2023

  76. [84]

    Cross-validation methods,

    M. W. Browne, “Cross-validation methods,” Journal of mathematical psychology , vol. 44, no. 1, pp. 108–132, 2000

    2000

  77. [85]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019

    2019

  78. [86]

    Nigerian

    IBM, https://cloud.ibm.com/, 2024. APPENDIX A RELATED DATASET INFORMATION A. Existing Phishing Dataset Information We present information on our collected datasets in Ta- ble IV, accompanied by brief descriptions. Specific acquisition methods can be found through relevant refe...

    2018

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.