Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

MultiPhishGuard: An Explainable and Adaptive Multi-Agent LLM System for Phishing Email Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MultiPhishGuard claims that a five-agent LLM ensemble with reinforcement-learned fusion weights and an adversarial training loop detects phishing at 97.89% accuracy, beating single-agent, chain-of-thought, and transformer baselines.

desk verdict Multi-agent LLM phishing detection that reads well but needs the split protocol disclosed before the headline numbers are credible. read the letter →

arxiv 2505.23803 v2 pith:24LEBKIK submitted 2025-05-26 cs.CR cs.AI

Yinuo Xue , Eric Spero , Meng Wai Woo , Wei Gao , Giovanni Russello This is my paper
classification cs.CRcs.AI
keywords PhishingEmailDetectionLargeLanguageModelMulti-AgentSystemReinforcementLearningAdversarialTrainingProximalPolicyOptimizationExplainabilitySecurity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that phishing detection improves when the task is split among specialized large-language-model agents rather than given to one model or one prompt. MultiPhishGuard uses a text agent, a URL agent, and a metadata agent, each producing a verdict, confidence score, and rationale; a reinforcement-learning module learns when to trust each agent for each email; an adversarial agent rewrites real emails into subtle variants during training; and a simplifier turns the rationales into plain language. Evaluated on roughly 4,000 emails pooled from six public corpora, the system reports 97.89% accuracy with a 2.73% false positive rate and a 0.20% false negative rate. The paper also reports that every component earns its place, with the URL agent, the learned weights, and the adversarial loop each contributing measurable gains in ablations. A sympathetic reader would take away that coordinated, modality-specialized LLM agents with per-email weighting are a workable route to accurate and explainable phishing defense.

What carries the argument

The load-bearing mechanism is the learned fusion rule. The text, URL, and metadata agents each output a phishing probability and confidence; the PPO policy $\pi_\theta(w|x)$ draws the weight vector $w$ conditioned on email features such as URL count, phishing keywords, sender reputation, authentication results, and the agents' own confidences, and the ensemble verdict is the weighted sum $y = \sum_i w_i p_i$. PPO's clipped surrogate objective keeps the weight updates inside a bounded trust region, which is what allows the system to adapt per email without destabilizing. Around this fusion rule sit the adversarial training loop, in which an LLM-based generator rewrites existing emails through synonym substitution, sentence rewriting, content modification, homoglyph replacement, and polymorphic variation, and the explanation simplifier, which condenses the three rationales into a single plain-language explanation for human users.

What would settle it

Re-run the pipeline with a documented disjoint split: train the PPO weight policy and run the adversarial loop only on a training fold, then evaluate on the 2024 phishing subset and the hardest legitimate subset. If accuracy on that held-out fold lands near the static-weight ablation level of about 95% instead of 97.89%, or if the gap over the RoBERTa baseline shrinks, the headline result partly reflects adaptation to the evaluation set rather than generalization to new email.

Watch

Extended reading notes

Core claim

The central claim is that a coordinated team of LLM agents with learned per-email weighting outperforms any single model or prompting strategy at phishing detection. Each specialized agent emits a phishing probability $p_i$; a Proximal Policy Optimization (PPO) module maps email features to weights $w_i$; and the final phishing score is $y = \sum_i w_i p_i$. The adversarial agent generates context-aware variants of both phishing and legitimate emails to harden the ensemble, and the explanation simplifier merges the three technical rationales into one non-technical summary. On its pooled evaluation the paper reports 97.89% accuracy ($F_1$ 95.88%, false positive rate 2.73%, false negative rate 0.20%), and reports that it beats chain-of-thought prompting, a single-agent model, and a fine-tuned RoBERTa baseline on most corpora under discordant-pair significance tests. Ablations show F1 dropping from 95.88% to 84.20% without the URL agent, 90.36% without metadata, 91.43% with static instead of learned weights, and 90.97% without the adversarial loop.

Load-bearing premise

The results stand only if the scored emails were never used to update the learned weights or the adversarial loop, but the paper does not disclose a train/test split for either component. If the pooled evaluation set contributed reward updates to the PPO policy or feedback to the adversarial agent, the reported 97.89% accuracy and the margins over fixed baselines partly measure in-sample adaptation rather than generalization.

Editorial extensions

If this is right

  • On the pooled six-corpus testbed, the system reports 97.89% accuracy and a 95.88% $F_1$, with a 2.73% false positive rate, versus 86.60%/78.45% for chain-of-thought, 84.42%/75.84% for the single agent, and 95.73%/91.89% for the RoBERTa baseline.
  • Removing the URL agent drops $F_1$ from 95.88% to 84.20% and raises the false positive rate from 2.73% to 11.33%, showing URL inspection is the strongest single modality.
  • Replacing the learned weights with static weights drops $F_1$ to 91.43%, and removing the adversarial loop drops it to 90.97%, evidence that both adaptation and adversarial hardening contribute beyond the agent set.
  • The explanation simplifier produces text with perplexity 25 and a reading-ease score of 41, which the paper reports as more fluent and readable than chain-of-thought outputs, and it matches an expert's rationale with ROUGE-1 0.59 and cosine similarity 0.82.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same architecture should port to other multi-modal security classification tasks, such as malicious website detection or malware triage, where distinct evidence channels need context-dependent weighting; the port would need its own adversarial generator for each channel.
  • A strict temporal-split experiment is the natural test: train on the older corpora and evaluate only on the 2024 phishing emails, which would directly measure whether dynamic weighting generalizes forward in time.
  • The dual-generation adversarial loop (perturbing both phishing and legitimate email) points to a general recipe for shrinking false positives in imbalanced detection tasks, but its benefit can only be cleanly measured with adversarial examples held out from the final training epoch.
  • The explanation-quality evidence rests on automated readability metrics and a single expert's rationales; a multi-participant user study across expertise levels is the missing test of the claim that simpler explanations build user trust.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents MultiPhishGuard, a multi-agent LLM system for phishing email detection. Three specialized agents (text, URL, metadata) produce verdicts, confidence scores, and rationales; a PPO module learns weights that fuse these outputs based on email features; an adversarial agent generates subtle variants of phishing and legitimate emails during training; and an explanation simplifier converts technical rationales into plain language. Evaluations on six public corpora (979 phishing and 3,000 legitimate emails) report 97.89% accuracy, 95.88% F1, 99.80% recall, 2.73% FPR, and statistically significant McNemar comparisons against CoT prompting, single-agent GPT-4o, and RoBERTa-base, with ablations showing contributions of each component.

Significance. The multi-agent architecture and the combination of RL-based fusion with adversarial training are timely and the evaluation design is mostly sensible: the confusion-matrix arithmetic is internally consistent, and the use of mid-p McNemar tests with Benjamini-Hochberg control is appropriate. The adversarial loop and the explanation simplifier address real gaps in phishing detection. However, the central empirical claims depend on the scored emails being out-of-sample for the PPO weights and the adversarial feedback loop, and the manuscript never states or demonstrates this isolation. Section 5.1 candidly acknowledges the single-baseline comparison and the lack of explanation ground truth, which tempers the scope of those claims. With a clean held-out evaluation protocol, MultiPhishGuard could be a useful contribution; as written, the generalization claims are not yet fully supported.

major comments (4)
  1. [Section 3.2, 3.4, 4.3.1] Section 3.2 defines the PPO reward as 'the accuracy of the final classification' and says the weights are 'continuously updated,' while Section 3.4 states that during 'periodic evaluation cycles' newly generated adversarial examples are reintroduced into the system. Section 4.3.1 describes the primary evaluation as a pooled analysis of all datasets with no mention of a train/test split, cross-validation, or seed protocol. If any of the 3,979 scored emails contributed reward updates or served as inputs to adversarial example generation before the final scoring, the headline accuracy (97.89%), F1 (95.88%), and the McNemar p-values in Tables 4 and 7 measure in-sample fit rather than generalization. The authors need to state explicitly that the evaluation set was never used for reward updates or adversarial generation, or re-run the experiments with a held-out split (or nested cross-validation) and report the resulting metrics.
  2. [Section 4.3.4 and 4.1] RoBERTa-base is a fitted classifier, but the paper does not describe the train/validation/test split used to fit it; Section 4.3.4 only says 'we used the same training settings as in [50].' If RoBERTa-base was trained and evaluated on the same pooled 3,979 emails, the comparison is in-sample and the reported margin over this baseline is not trustworthy. The authors should provide the dataset split, hyperparameter selection procedure, and evaluation folds for RoBERTa-base, and ensure they match the protocol used for MultiPhishGuard.
  3. [Section 4.4.3] The static-weight ablation is not a fair baseline: the weights (0.3 text, 0.4 URL, 0.3 metadata) are justified 'from our previous experimental results' and chosen to reflect observed relative importance. Tuning the ablation weights on the same data that defines the comparison inflates the apparent benefit of the PPO module. Use a prespecified scheme (e.g., equal weights) or a validation set separate from the test set, and report performance for both the full and ablated systems.
  4. [Section 4.5 and 5] The human evaluation uses a single cybersecurity expert and an unspecified 'small subset' of phishing emails, and no inter-annotator reliability can be computed with one rater. Section 5's statement that 'human evaluation and automated metrics show that these explanations align well with expert reasoning' is therefore stronger than the evidence supports. The authors should report the number of emails and experts, the selection procedure, and ideally multiple annotators with agreement measures; otherwise the claim should be scoped as a case study.
minor comments (5)
  1. [Section 4.3.5] The text says p-values were adjusted using Benjamini-Hochberg, but Table 4 reports only unadjusted p-values. Please report the adjusted p-values or clearly mark which comparisons remain significant after the adjustment.
  2. [Section 4.1 and Table 3] The pooled evaluation set has a 1:3 phishing-to-legitimate ratio with only 979 phishing emails; reporting confidence intervals for the point estimates in Table 1 would help readers assess the stability of the 92.26% precision and 95.88% F1.
  3. [Section 3.2] Details of the PPO implementation are missing: the policy network architecture, the state/action representation, the number of episodes, the learning rate, and the convergence criterion are not specified, which makes the 'continuously adjusting' mechanism difficult to reproduce or compare.
  4. [Abstract and Section 5.1] Given that Section 5.1 acknowledges comparison with only a single state-of-the-art baseline, the abstract's phrase 'state-of-the-art detectors' in the plural overstates the comparative evaluation; the authors should align the wording with the actual baseline set.
  5. [Section 3.4] Phrases such as 'the system becomes increasingly resilient' and 'generalizes more effectively' are forward-looking claims that go beyond the experimental evidence; please replace them with hedged statements tied to the reported results.

Circularity Check

3 steps flagged · score 6.0 of 10

Headline 97.89% accuracy is the PPO reward on the pooled emails, the static-weight baseline is fitted from observed results, and the adversarial loop reintroduces variants of evaluation emails; the central comparisons are partly in-sample.

  1. fitted input called prediction [Section 3.2 'Dynamic Weight Adjustment'; Section 4.3 'Comparative Evaluation'; Section 4.3.1 'MultiPhishGuard']
    "The objective is to maximize the expected reward E[r], and the reward function is the accuracy of the final classification. ... While our primary analysis pools all datasets to evaluate overall performance, Table 3 and Section 4.3.5 show comparisons broken down by individual dataset."

    The only trainable parameters, the PPO weights, are updated to maximize the accuracy of the final classification. The paper's sole reported evaluation pools all 3,979 emails and discloses no train/test split, cross-validation scheme, or seed protocol for the PPO policy. Under the paper's own description, the reported 97.89% accuracy is the value of the PPO reward evaluated on the same pooled data that could have supplied reward updates, so the headline 'measured' performance is, by construction, the fitted training objective rather than an out-of-sample prediction. The statistically significant McNemar margins over baselines inherit the same in-sample bias.

  2. fitted input called prediction [Section 4.4.3 'Static Weight']
    "From our previous experimental results, we observed that the URL agent had a greater impact on overall accuracy than the metadata agent. Removing the URL agent led to a more significant decline in detection performance, indicating its higher importance in phishing detection. Consequently, we assigned a weight of 0.3 to the text agent, 0.4 to the URL agent, and 0.3 to the metadata agent."

    The static-weight ablation baseline is not an independent, pre-specified weighting scheme: its 0.3/0.4/0.3 weights are selected after inspecting the same experimental results on the same pooled data to reflect which agent seemed more important. The subsequent comparison of 'Static Weight' against PPO therefore measures PPO against a baseline that was tuned to the test outcome. The observed F1 gap (91.43 vs. 95.88) is partly manufactured by the baseline construction and does not independently establish that learned weighting is superior.

1 more flagged steps
  1. other [Section 3.4 'Adversarial Training Module'; Figure 8 and Appendix B]
    "This feedback loop is realized through periodic evaluation cycles, during which newly generated adversarial examples are reintroduced into the system."

    The adversarial agent modifies existing phishing emails from the same corpora later used in the pooled evaluation; Appendix B's Figure 8 shows a January 2024 Nazario corpus email (To: jose@monkey.org) transformed into an adversarial variant. If such variants are reintroduced during the periodic evaluation cycles, the full system is trained on perturbed copies of the test emails, while the No-Adversarial ablation is not. The robustness improvement attributed to the adversarial loop is therefore at least partly an in-sample artifact of training on test-derived variants rather than evidence of generalization to unseen emails.

full rationale

The paper's strongest empirical claims rest on a single pooled evaluation of 3,979 emails, but the paper never defines a held-out set for the two learned/adaptive components (the PPO weight policy and the adversarial training loop). Section 3.2 defines the PPO reward as the accuracy of the final classification, and Section 4.3 pools all datasets for the headline numbers; hence the 97.89% accuracy is, as described, the PPO objective value on the evaluation pool rather than a demonstrated out-of-sample prediction. The static-weight ablation is explicitly fitted from the same observed results, further weakening the ablative evidence for dynamic weighting. Most damaging, Section 3.4's adversarial loop reintroduces generated examples into the system, and Figure 8 shows such a variant created from a Nazario test email, so the full model may have been trained on perturbed copies of emails that later appear in the evaluation. These issues are not self-citation or definitional circularity, and the fixed GPT-4o multi-agent prompts retain independent content, but the central 'significant outperformance' claim is partially constructed from the evaluation data itself. Score 6 reflects partial circularity via fitted inputs being reported as measured predictions.

Assumptions & free parameters 4 free parameters · 6 assumptions · 2 invented entities

The central claim rests on one fitted component, the 3-parameter PPO policy whose final weights are unreported, plus hand-designed prompts, transformation rules, and benchmark subsampling. No new physical or mathematical entities are introduced; the invented components are software agents whose value is demonstrated only through in-paper ablations. The dominant background axiom is that GPT-4o's zero-shot, modality-restricted judgments are accurate and calibrated, since neither the base model nor the prompts are trained on the benchmark data.

free parameters (4)
  • PPO agent weight vector (w_text, w_url, w_metadata) = not disclosed
    Learned by PPO to maximize classification accuracy (Section 3.2); final weights and the train/test split are not reported, so the contribution of these fitted weights to the headline metrics cannot be audited.
  • Static ablation weights (text, URL, metadata) = 0.3, 0.4, 0.3
    Chosen in Section 4.4.3 'from our previous experimental results', i.e., fit to observed performance on the same datasets, then compared against the learned policy.
  • Adversarial transformation parameters and prompt adjustments = not disclosed
    Section 3.4 tunes 'transformation parameters and adjusting prompt instructions' based on detector errors; the tuning schedule and final configuration are unspecified.
  • Benchmark subsampling choices = 979 phishing / 3,000 legitimate (1:3)
    Only 2024 Nazario and 2007 Nigerian phishing are kept; Enron, TREC, CEAS ham are sampled; 500 'hard ham' SpamAssassin emails are deliberately selected (Section 4.1). These hand-made choices set test difficulty and class prior.
assumptions (6)
  • domain assumption GPT-4o with the quoted JSON-mode prompts yields accurate, calibrated per-modality phishing judgments for text, URL, and metadata.
    The entire system inherits this capability, which is invoked throughout Section 3.1; no calibration or per-modality validation is provided.
  • domain assumption PPO converges to a useful weighting policy given the accuracy reward and the listed email-feature state.
    Section 3.2 justifies PPO qualitatively; no learning curves, hyperparameters, or learned weight values are shown.
  • domain assumption The email feature vector (URL counts, phishing keywords, sender reputation, SPF/DKIM/DMARC results) is well-defined and computable for every tested email.
    Section 3.2 lists these features but does not specify the extraction procedure or data sources.
  • domain assumption The Adversarial Agent's generated variants resemble real-world phishing evasions, so robustness gains transfer to practice.
    Section 3.4 asserts improved robustness; generated emails are not released and no external validation of their realism is provided.
  • domain assumption Labels of the six public corpora are accurate ground truth and the pooled 1:3 phishing/legitimate mix is a meaningful deployment prior.
    Section 4.1 constructs the testbed from dataset provenance without re-verification of labels.
  • standard math One-sided McNemar tests with mid-p correction and Benjamini-Hochberg adjustment are valid for these discordant-pair counts.
    Section 4.3.5; standard practice and correctly applied given the small discordant counts on some datasets.
invented entities (2)
  • Adversarial Agent (GPT-4o-based email variant generator)
    purpose: Generates subtle variants of phishing and legitimate emails during training to expose detector weaknesses and drive prompt/weight adjustments (Section 3.4).
    The claimed robustness benefit is shown only by an in-paper ablation (No Adversarial F1 90.97 vs 95.88, Table 5); generated variants are withheld, so fidelity to real-world phishing is not independently validated.
  • Explanation Simplifier with Expert Mode
    purpose: Consolidates the three technical rationales into a plain-language explanation, with a technical 'Expert Mode' variant (Section 3.3).
    User-facing value is asserted via readability statistics (perplexity 25, coherence 0.35, FRES 41) and comparison against one expert's analysis (Table 8); planned user studies are explicitly out of scope.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MultiPhishGuard: An Explainable and Adaptive Multi-Agent LLM System for Phishing Email Detection." pith.science (2026). https://pith.science/paper/24LEBKIK

@misc{pith2026250523803,
  author       = {Pith},
  title        = {Pith review of: MultiPhishGuard: An Explainable and Adaptive Multi-Agent LLM System for Phishing Email Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/24LEBKIK}},
  note         = {Machine review of arXiv:2505.23803}
}
read the original abstract

Phishing email detection faces significant challenges due to evolving adversarial tactics and heterogeneous attack patterns. Traditional approaches, such as rule-based filters and denylists, often struggle to keep pace, leading to missed detections and security risks. While machine learning methods have improved detection performance, they remain limited in adapting to novel and rapidly changing phishing strategies. We present MultiPhishGuard, an LLM-based multi-agent detection framework with learned coordination across specialized agents. The system consists of five cooperative agents (text, URL, metadata, explanation simplifier, and adversarial agents), with agent contributions dynamically weighted using Proximal Policy Optimization. To address emerging threats, the framework incorporates an adversarial training loop in which an LLM-based agent generates subtle, context-aware email variants to expose potential model weaknesses and improve robustness to ambiguous phishing cases. Experimental evaluations on public datasets show that MultiPhishGuard achieves stronger performance than established baselines, including Chain-of-Thought prompting and single-agent variants, as supported by ablation studies and comparative analyses. The system achieves an accuracy of 97.89%, with a false positive rate of 2.73% and a false negative rate of 0.20%. In addition, an explanation simplifier agent transforms technical model outputs into plain-language rationales intended for human users. Overall, these results suggest that multi-agent LLM architectures with adaptive coordination and adversarial training represent a promising direction for phishing email detection.

Figures

Figures reproduced from arXiv: 2505.23803 by the authors.

Figure 1
Figure 1. MultiPhishGuard Detection Pipeline. Incoming [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. MultiPhishGuard Architecture. An Adversarial Agent generates subtle variants of both phishing and legitimate emails. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Text Agent’s Prompt 3.2 Dynamic Weight Adjustment Our framework employs an RL-based mechanism to dynamically adjust the weights of outputs from various specialized agents, ensur￾ing that each email is evaluated according to its unique characteris￾tics. Instead of relying on fixed weights, our system continuously learns to assign optimal importance to the analyses provided by the text, URL, and metadata agents. For e… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Explanation Simplifier Agent’s Prompt into a cohesive summary. We explicitly require that explanations be “truthful ... based solely on factual evidence—do not include any fabricated details,” ensuring factual integrity. Directives such as “avoid technical jargon” and …
Figure 5
Figure 5. Figure 5: Adversarial Agent Prompt discriminator, whose goal is to correctly classify each email as phishing or legitimate. The adversarial agent seeks to maximize the discriminator’s error by producing emails that are difficult to detect, thereby encouraging the discriminator t…
Figure 6
Figure 6. Figure 6: URL Agent’s Prompt You are a cybersecurity expert specializing in phishing, with a particular focus on email metadata. Your task is to scrutinize the provided email header—including the subject, sender address, reply-to, return-path, and received fields—for any signs o…
Figure 7
Figure 7. Figure 7: Metadata Agent’s Prompt B Example result of adversarial agent email variant generation From: “Monkey Support Notification” <info@creditloiuse.com> Subject: Important Password Validation To: <jose@monkey.org> Date: Wed, 10 Jan 2024 16:00:51 -0800 Monkey Support Notifica…
Figure 8
Figure 8. Figure 8: Example result of adversarial agent email variant [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Account-History Features for Social Bot Detection in the Era of Large Language Models

    cs.SI 2026-06 conditional novelty 5.0 of 10

    Account-history features achieve ROC-AUC 0.977 for bot detection and remain stable under LLM text rewriting while content features drop to 0.466 AUC.

  2. An LLM-based Chain-of-Response Counter-Scam System

    cs.CY 2026-05 unverdicted novelty 5.0 of 10

    Counter Scam is a multiagent LLM system that integrates safe data handling, nine role-specific NLP tasks, and a 185k-case scam corpus, with fine-tuned small models beating commercial LLMs by over 10% on those tasks.

  3. Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats

    cs.CR 2026-06 unverdicted novelty 4.0 of 10

    A joint prompt-response verification framework using intent analysts, harm analysts, and a judge improves average F1 to 0.95 and cuts attack success rate to 4.1% across jailbreaks, prompt injection, phishing, cyber ab...

Reference graph

Works this paper leans on

64 extracted references · 37 canonical work pages · cited by 3 Pith papers

  1. [50]

    Sayak Saha Roy, Poojitha Thota, Krishna Vamsi Naragam, and Shirin Nilizadeh

  2. [1]

    Abdullah M Almuhaideb, Nida Aslam, Almaha Alabdullatif, Sarah Altamimi, Shooq Alothman, Amnah Alhussain, Waad Aldosari, Shikah J Alsunaidi, and Khalid A Alissa. 2022. Homoglyph attack detection model using machine learning and hash function.Journal of Sensor and Actuator Networks11, 3 (2022), 54

    work page 2022

  3. [2]

    2024.Unifying the Global Response To Cybercrime

    APWG. 2024.Unifying the Global Response To Cybercrime. Phishing Activity Trends Report, 3rd Quarter 2024. APWG

    work page 2024

  4. [3]

    Mohammad Asfour and Juan Carlos Murillo. 2023. Harnessing large language models to simulate realistic human responses to social engineering attacks: A case study.International Journal of Cybersecurity Intelligence & Cybercrime6, 2 (2023), 21–49

    work page 2023

  5. [4]

    Sultan Asiri, Yang Xiao, Saleh Alzahrani, and Tieshan Li. 2024. PhishingRTDS: A real-time detection system for phishing attacks using a Deep Learning model. Computers & Security141 (2024), 103843

    work page 2024

  6. [5]

    Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing.Journal of the Royal Statistical Society: Series B (Methodological)57, 1 (1995), 289–300

    1995

  7. [6]

    Battista Biggio and Fabio Roli. 2018. Wild patterns: Ten years after the rise of adversarial machine learning. InProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 2154–2156

    work page 2018

  8. [7]

    Santosh Kumar Birthriya, Priyanka Ahlawat, and Ankit Kumar Jain. 2025. De- tection and Prevention of Spear Phishing Attacks: A Comprehensive Survey. Computers & Security(2025), 104317

    work page 2025

Show all 64 references
  1. [8]

    Ravi Chauhan, Ulya Sabeel, Alireza Izaddoost, and Shahram Shah Heydari. 2021. Polymorphic adversarial cyberattacks using WGAN.Journal of Cybersecurity and Privacy1, 4 (2021), 767–792

    2021

  2. [9]

    Lin Chin-Yew. 2004. Rouge: A package for automatic evaluation of summaries. InProceedings of the Workshop on Text Summarization Branches Out, 2004

    2004

  3. [10]

    Asaf Cidon, Lior Gavish, Itay Bleier, Nadia Korshun, Marco Schweighauser, and Alexey Tsitkin. 2019. High precision detection of business email compromise. In 28th USENIX Security Symposium (USENIX Security 19). 1291–1307

    2019

  4. [11]

    Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators.arXiv preprint arXiv:2003.10555(2020)

    2020 arXiv

  5. [12]

    Cormack and Thomas R

    Gordon V. Cormack and Thomas R. Lynam. 2007.TREC 2007 Public Corpus. Retrieved March, 2025 from https://plg.uwaterloo.ca/~gvcormac/treccorpus07/

    2007

  6. [13]

    2008.CEAS 2008 Live Spam Challenge Laboratory corpus

    CEAS 2008 Public Corpus. 2008.CEAS 2008 Live Spam Challenge Laboratory corpus. Retrieved March, 2025 from https://plg.uwaterloo.ca/~gvcormac/ceascorpus/

    2008

  7. [14]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  8. [15]

    Morten W Fagerland, Stian Lydersen, and Petter Laake. 2013. The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional.BMC medical research methodology13 (2013), 1–8

    2013

  9. [16]

    Rudolf Flesch. 1979. How to write plain English.University of Canterbury. A vailable at http://www. mang. canterbury. ac. nz/writing_guide/writing/flesch. shtml.[Retrieved 5 February 2016](1979)

    1979

  10. [17]

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples.arXiv preprint arXiv:1412.6572(2014)

    2014 arXiv

  11. [18]

    right to explanation

    Bryce Goodman and Seth Flaxman. 2017. European Union regulations on algo- rithmic decision-making and a “right to explanation”.AI magazine38, 3 (2017), 50–57

    2017

  12. [19]

    Qazi Emad ul Haq, Muhammad Hamza Faheem, and Iftikhar Ahmad. 2024. Detect- ing Phishing URLs Based on a Deep Learning Approach to Prevent Cyber-Attacks. Applied Sciences14, 22 (2024), 10086

    2024

  13. [20]

    Julian Hazell. 2023. Spear phishing with large language models.arXiv preprint arXiv:2305.06972(2023)

    2023 arXiv

  14. [21]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. De- berta: Decoding-enhanced bert with disentangled attention.arXiv preprint arXiv:2006.03654(2020)

    2020 arXiv

  15. [22]

    Ling Huang, Anthony D Joseph, Blaine Nelson, Benjamin IP Rubinstein, and J Doug Tygar. 2011. Adversarial machine learning. InProceedings of the 4th ACM workshop on Security and artificial intelligence. 43–58

    2011

  16. [23]

    Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. 1977. Perplexity—a measure of the difficulty of speech recognition tasks.The Journal of the Acoustical Society of America62, S1 (1977), S63–S63

    1977

  17. [24]

    Taeri Kim, Noseong Park, Jiwon Hong, and Sang-Wook Kim. 2022. Phishing url detection: A network-based approach robust to evasion. InProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 1769–1782

    2022

  18. [25]

    Takashi Koide, Naoki Fukushi, Hiroki Nakano, and Daiki Chiba. 2024. Chatspamdetector: Leveraging large language models for effective phishing email detection.arXiv preprint arXiv:2402.18093(2024)

    2024 arXiv

  19. [26]

    Xue Li, Dongmei Zhang, and Bin Wu. 2020. Detection method of phishing email based on persuasion principle. In2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Vol. 1. IEEE, 571–574

    2020

  20. [27]

    Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning.arXiv preprint arXiv:1509.02971(2015)

    2015 arXiv

  21. [28]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 (2019)

    2019 arXiv

  22. [29]

    Zhijun Liu, Weili Lin, Na Li, and David Lee. 2005. Detecting and filtering instant messaging spam-a global and personalized approach. In1st IEEE ICNP Workshop on Secure Network Protocols, 2005.(NPSec).IEEE, 19–24

    2005

  23. [30]

    Theodore Tangie Longtchi, Rosana Montañez Rodriguez, Laith Al-Shawaf, Adham Atyabi, and Shouhuai Xu. 2024. Internet-based social engineering psychology, attacks, and defenses: A survey.Proc. IEEE(2024)

    2024

  24. [31]

    Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments.Advances in neural information processing systems30 (2017)

    2017

  25. [32]

    Justin Ma, Lawrence K Saul, Stefan Savage, and Geoffrey M Voelker. 2009. Be- yond blacklists: learning to detect malicious web sites from suspicious URLs. InProceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. 1245–1254

    2009

  26. [33]

    Quinn McNemar. 1947. Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages.Psychometrika12, 2 (1947), 153–157. doi:10.1007/BF02295996

    1947 doi

  27. [34]

    Vangelis Metsis, Ion Androutsopoulos, and Georgios Paliouras. 2006. Spam filtering with naive bayes-which naive bayes?. InCEAS, Vol. 17. Mountain View, CA, 28–69

    2006

  28. [35]

    Michael P. Fay. 2010. Two-sided Exact Tests and Matching Confidence Intervals for Discrete Data.R Journal2, 1 (2010), 53–58. https://journal.r-project.org/

    2010

  29. [36]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. nature518, 7540 (2015), 529–533

    2015

  30. [37]

    2005.The online phishing corpus

    Jose Nazario. 2005.The online phishing corpus. Retrieved March, 2025 from https://monkey.org/~jose/phishing/

    2005

  31. [38]

    Denish Omondi Otieno, Akbar Siami Namin, and Keith S Jones. 2023. The application of the bert transformer model for phishing email classification. In2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 1303–1310

    2023

  32. [39]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (...

    2022

  33. [40]

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology. 1–22. CCS ’...

    2023

  34. [41]

    European Parliament and Council of the European Union. 2024. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending Regulations (EU) 2018/1724 an...

    2024

  35. [42]

    2006.SpamAssassin public mail corpus

    Apache SpamAssassin Project. 2006.SpamAssassin public mail corpus. Retrieved March, 2025 from https://spamassassin.apache.org/old/publiccorpus/

    2006

  36. [44]

    Tingrui Qiao, Caroline Walker, Chris W Cunningham, and Yun Sing Koh. [n. d.]. Thematic-LM: a LLM-based Multi-agent System for Large-scale Thematic Analy- sis. InTHE WEB CONFERENCE 2025

    2025

  37. [45]

    2024.R: A Language and Environment for Statistical Computing

    R Core Team. 2024.R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project. org/ Functionbinom.testin packagestats

    2024

  38. [46]

    2008.CLAIR collection of fraud email

    Dragomir Radev. 2008.CLAIR collection of fraud email. Retrieved March, 2025 from http://aclweb.org/aclwiki ADCR2008T001

    2008

  39. [47]

    Fariza Rashid, Nishavi Ranaweera, Ben Doyle, and Suranga Seneviratne. 2025. LLMs are one-shot URL classifiers and explainers.Computer Networks258 (2025), 111004

    2025

  40. [48]

    Why should I trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " Why should I trust you?" Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135–1144

    2016

  41. [49]

    Frank Rosner, Alexander Hinneburg, Michael Röder, Martin Nettling, and Andreas Both. 2014. Evaluating topic coherence measures.arXiv preprint arXiv:1403.6397 (2014)

    2014 arXiv

  42. [51]

    Suranjana Samanta and Sameep Mehta. 2017. Towards crafting text adversarial samples.arXiv preprint arXiv:1707.02812(2017)

    2017 arXiv

  43. [52]

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Dis- tilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108(2019)

    2019 arXiv

  44. [53]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools.Advances in Neural Information Processing Systems36 (2023), 68539–68551

    2023

  45. [54]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  46. [55]

    Mohsen Sharifi and Seyed Hossein Siadati. 2008. A phishing sites blacklist generator. In2008 IEEE/ACS international conference on computer systems and applications. IEEE, 840–843

    2008

  47. [56]

    2008.Multiagent systems: Algorithmic, game-theoretic, and logical foundations

    Yoav Shoham and Kevin Leyton-Brown. 2008.Multiagent systems: Algorithmic, game-theoretic, and logical foundations. Cambridge University Press

    2008

  48. [57]

    1998.Reinforcement learning: An intro- duction

    Richard S Sutton, Andrew G Barto, et al. 1998.Reinforcement learning: An intro- duction. Vol. 1. MIT press Cambridge

    1998

  49. [58]

    2024.2024 Data Breach Investigations Report

    Verizon. 2024.2024 Data Breach Investigations Report. Technical Report. Verizon

    2024

  50. [59]

    Rakesh Verma and Nabil Hossain. 2013. Semantic feature selection for text with application to phishing email detection. Ininternational conference on information security and cryptology. Springer, 455–468

    2013

  51. [60]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems35 (2022), 24824–24837

    2022

  52. [61]

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al . 2023. Autogen: En- abling next-gen llm applications via multi-agent conversation.arXiv preprint arXiv:2308.08155(2023)

    2023 arXiv

  53. [62]

    Lei Xu, Alfredo Cuesta-Infante, Laure Berti-Equille, and Kalyan Veeramacha- neni. 2021. R&R: Metric-guided adversarial sentence generation.arXiv preprint arXiv:2104.08453(2021)

    2021 arXiv

  54. [63]

    Monkey Support Notification

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding.Advances in neural information processing systems32 (2019). MultiPhishGuard: An LLM-based Multi-Agent Syst...

    2019

  55. [2017]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  56. [2024]

    In2024 IEEE Symposium on Security and Privacy (SP)

    From chatbots to phishbots?: Phishing scam generation in commercial large language models. In2024 IEEE Symposium on Security and Privacy (SP). IEEE, 36–54

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.