Pith. sign in

REVIEW 5 major objections 5 minor 132 references

Generative AI for Internet of Things Security: Challenges and Opportunities

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Generative AI, especially large language models, is already a working part of IoT security: 33 surveyed systems plus three case studies show them finding vulnerabilities, fuzzing devices, detecting intrusions, and generating defensive…

desk verdict A useful but overreaching survey: the MITRE mapping and honest case studies are worth a look, but the central claim that GenAI already directly improves IoT security is not backed by the evidence. read the letter →

arxiv 2502.08886 v1 pith:74AECFS6 submitted 2025-02-13 cs.CR cs.AI

Yan Lin Aung , Ivan Christian , Ye Dong , Xiaodong Ye , Sudipta Chattopadhyay , Jianying Zhou This is my paper · ORCID
classification cs.CRcs.AI
keywords GenerativeAIforCyberSecurityLargeLanguageModelsInternetofThingsMITREATT&CKICSMitigationsVulnerabilityScanningIntrusionDetectionFuzzingSurvey
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a survey that aims to establish a concrete claim: generative AI, and large language models in particular, is already a usable component of Internet of Things security, not just a future promise. The authors compiled 33 recent systems, from LLM-based intrusion detectors and fuzzers to automated penetration testers and policy generators, and mapped each onto the ICS Mitigations framework from the MITRE ATT&CK catalog, a defensive-technique taxonomy for industrial control systems. Each tool is rated on six capabilities, and three case studies test an incident-response generator, a security agent, and a retrieval-augmented chatbot in IoT settings. A reader comes away with a structured picture of where GenAI already strengthens IoT defenses and which security tasks remain untouched.

What carries the argument

The load-bearing object is the ICS Mitigations framework from the MITRE ATT&CK catalog, a taxonomy of defensive techniques for industrial control systems, which the paper adopts on the assertion that IoT systems can be compromised in similar, if not identical, ways. The argument is carried by the mapping: each of the 33 surveyed tools is placed under one of nine mitigation techniques, including application developer guidance, exploit protection, network intrusion prevention, software update, threat intelligence, user training, and vulnerability scanning, and then rated on six self-defined capabilities: external threat detection, internal anomaly detection, response automation, research maturity, development potential, and impact on security. The three case studies put that mapping to work by applying it to IoT-specific prompts and execution environments.

What would settle it

Compare the attack techniques in a broad corpus of real IoT incidents, such as botnet campaigns, smart-home compromises, and medical-device attacks, against the ICS mitigation catalog used in this paper; if a substantial share of those attacks falls outside the catalog, the paper's research-gap analysis misses relevant defenses.

Watch

Extended reading notes

Core claim

The paper's central claim is that 'current GenAI implementations address IoT security improvements': large language models are not hypothetical future tools but already function across the IoT stack as vulnerability finders, fuzzers, intrusion detectors, penetration testers, policy writers, and training-scenario generators. The survey organizes 33 works under the ICS Mitigations framework from the MITRE ATT&CK catalog, a defensive-technique taxonomy for industrial control systems, on the stated assumption that IoT systems have similar, if not all, methods to compromise them as ICS. Each tool is rated on six capabilities: external threat detection, internal anomaly detection, response automation, research maturity, development potential, and impact on security. The three case studies show the same pattern as the literature: a prompt-modified incident-response generator produced organized but generic plans, an agent-based security tool could execute administrative commands and report attack techniques, and a retrieval-augmented chatbot answered IoT security questions with role-appropriate detail. The paper concludes that these implementations are robust enough to directly improve IoT security and that the unresolved areas, such as access management, network-traffic encryption, OS configuration, supply chain management, and program-input validation, are open research problems rather than dead ends.

Load-bearing premise

The load-bearing premise is that IoT devices can be compromised in essentially the same ways as industrial control systems, so the industrial-control-system mitigation catalog is the right meter for what GenAI should be doing for IoT security.

Editorial extensions

If this is right

  • According to the paper, LLM-based tools are ready to serve now as vulnerability scanners, fuzzers, and network intrusion detectors for IoT systems.
  • Fine-tuning on IoT-specific datasets is the paper's main lever for converting generic security LLMs into IoT-ready tools.
  • The survey's gap list names concrete open targets: access management, network-traffic encryption, OS configuration, supply chain verification, and program-input validation.
  • Among the surveyed works, several already report finding new vulnerabilities and CVEs in real IoT devices, so the claimed practical impact is not purely speculative.
  • The six-capability rating scheme gives future researchers a common template for comparing new GenAI security tools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Reader inference: the AttackGen case study itself shows that an unmodified pretrained LLM produces generic incident plans, which tempers the survey's overall optimism and suggests retrieval or fine-tuning is required before such tools are deployment-ready.
  • Reader inference: the ICS-to-IoT equivalence is assumed rather than demonstrated, so the gap analysis would need re-grounding in an IoT-specific taxonomy if real IoT incidents turn out to use techniques outside the ICS catalog.
  • Reader inference: a natural next experiment is to build an LLM agent for one untouched gap, such as access management, and compare its decisions against a conventional policy engine on a smart-home testbed.
  • Reader inference: because the six capability ratings are assigned by the authors without a documented second-opinion process, an independent re-rating of the same 33 tools would reveal how subjective the mapping is.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This survey paper reviews 33 recent works on generative AI (GenAI) and large language models (LLMs) applied to IoT security. The authors organize the surveyed works using the MITRE ATT&CK ICS Mitigations framework, introducing six capability ratings (External Threat Detection, Internal Anomaly Detection, Response Automation, Research Maturity, Development Potential, Impact on Security) in Table 1 and Table A.2. They also present three case studies: AttackGen (an LLM-based incident response plan generator), NVISOsecurity (an LLM-based security agent), and ChatIoT (an LLM/RAG-based IoT security assistant). The paper's central claim, stated in Section 7, is that current GenAI implementations directly improve IoT security across hardware, software, and network layers, and that the surveyed works together lay a foundation for future research.

Significance. The paper assembles a useful, up-to-date collection of GenAI security tools and explicitly attempts to connect them to a recognized security framework (MITRE ATT&CK ICS Mitigations). The inclusion of concrete case studies, especially the AttackGen evaluation with human expert scoring and the honest reporting that GPT-4o could not generate a tailored IoT incident plan without domain-specific training (Section 5.1), is a strength. The survey also identifies open research problems (Section 6) that could guide future work. However, the paper's central empirical claim is not supported by the evidence it presents: a large fraction of the surveyed works are not evaluated in IoT settings, and the authors' own case study demonstrates that transfer from general cyber security to IoT is non-trivial. The capability ratings in Table 1 and Table A.2 are subjective and internally inconsistent, and the survey methodology (Section 3) lacks the reproducibility expected of a systematic review. If the claims were appropriately weakened from 'directly improve' to 'show promise requiring adaptation and further validation,' the survey would be a credible starting point for researchers.

major comments (5)
  1. [Section 7] The central conclusion that 'current GenAI implementations address IoT security improvements' and 'directly improve IoT security' is an overstatement relative to the evidence in the paper. Many of the 33 surveyed works are included with explicit conditional language: LLMSecGuard 'could be used to assist' IoT development, LLift 'could be implemented for IoT systems running embedded Linux,' BERTIDS 'could be adapted to IoT,' AutoAttacker would need fine-tuning on CICIoT2023 to 'create a specialized LLM,' and ChatAFL 'has potential as a security testing tool if fine-tuned with an IoT dataset.' The paper's own AttackGen case study (Section 5.1) reports that even with added IoT context, 'the pre-trained GPT-4o could not generate a tailored incident plan for an IoT context.' These statements support a claim about potential and promising directions, not a claim about current direct improvement of IoT security. Section 7 should be revised to reflect the actual state of evidence, and the abstract's framing ('could improve' vs. 'improves') should be made consistent.
  2. [Section 4, paragraph 1] The load-bearing assumption that 'IoT systems have similar, if not all, methods to compromise them as ICS' is asserted without citation or empirical justification. The entire taxonomy, the capability ratings in Table 1/Table A.2, and the research-gap analysis in Section 6 are organized through the MITRE ATT&CK ICS Mitigations framework. If this equivalence is not defended, the structural foundation of the survey is unsupported. The authors should either provide evidence or citations for the IoT-ICS equivalence, or explicitly frame the ICS mapping as an exploratory heuristic and temper the conclusions drawn from it.
  3. [Table 1 and Table A.2] The six capability ratings (ETD, IAD, RA, RM, DP, IS) are uncalibrated subjective judgments. The paper provides no rubric for what distinguishes a full circle from a half circle or an empty circle, no inter-rater reliability assessment, and no independent audit. More seriously, several justifications in Table A.2 are internally inconsistent with the main text or appear to be incorrect. For example, Table A.2 row 10 (BERTIDS) says 'Does not detect internal anomalies as it only analyzes network logs,' but Section 4.5 describes BERTIDS as identifying and classifying anomalies in network traffic; Table A.2 row 26 (ChatAFL) credits 'stealth testing via mobile devices,' which is a feature described for FIAL (row 27); and row 27 (FIAL) says 'Automates penetration testing' although FIAL is described in Section 4.9 as a fuzzing tool. These inconsistencies undermine the reliability of the ratings and any conclusions drawn from them, including the identification of research gaps.
  4. [Section 5.3] The ChatIoT case study evaluates the authors' own patented system (ref [79]) using LLM-as-judges only, with no human validation, no external baseline, and no inter-rater or calibration analysis. The evaluation criteria (reliability, relevance, technicality, user-friendliness) are scored by an LLM based on a prompt, and the paper provides no evidence that these scores are meaningful. Since ChatIoT is presented as one of three case studies supporting the survey's central claim, this creates a self-referential validation problem. The authors should either add human evaluation and comparison with a baseline, or clearly label the case study as an illustrative demonstration rather than evidence of effectiveness.
  5. [Section 3 (Survey Methodology)] The literature selection process is not systematic or reproducible. Section 3.1 lists keyword strategies (e.g., OWASP IoT Top 10, MITRE ATT&CK ICS), and Section 3.2 states that papers from selected venues and arXiv were included, but there are no inclusion/exclusion criteria, no PRISMA-style screening counts, no date range, and no protocol for resolving disagreements. The statement that 'we sought to include works that have made significant contributions' is a subjective criterion. This lack of transparency matters because the paper's conclusions depend on the representativeness of the 33-work corpus, which the authors themselves acknowledge includes works that 'may not directly address the issue in the context of IoT.' A clearer description of the search, screening, and relevance-assessment process is needed.
minor comments (5)
  1. [Table 1] The table numbering skips row 28 (it jumps from 27 to 29), and the acronym definitions include a typo: 'IS: Implact on Security' should be 'Impact on Security.'
  2. [Section 3.1] The term 'OW ASP' appears twice and should be 'OWASP.'
  3. [Section 1.1] There are missing citation punctuation marks, e.g., 'significantly improving operational efficiency Kimani et al. [4]' should be 'improving operational efficiency [4].' Similar issues occur in Section 2.1.4.
  4. [Figure 1] Figure 1 is visually cluttered, with overlapping boxes and repeated entries (e.g., HuntGPT appears under both 'Application Developer Guidance' and 'Network Intrusion Prevention'). A hierarchical table or a cleaner layered diagram would improve readability.
  5. [Section 4.5] The description of Guastalla et al. mentions 'more than 90% accuracy' without specifying the dataset split, the number of few-shot examples, or the baseline methods, making the result difficult to interpret.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the survey's conclusions aggregate external works; only a minor self-referential ChatIoT evaluation appears, and it is not load-bearing.

full rationale

This is a survey rather than a derivation chain, and its central claim is an aggregation of 33 external works plus three case studies. The MITRE ATT&CK ICS mapping and the six-capability ratings are analytical categorizations, not equations in which an output is defined as an input. The AttackGen case study is genuinely evidential and even reports a negative result: 'the pre-trained GPT-4o could not generate a tailored incident plan for an IoT context,' which counts against the paper's own optimistic framing. The only self-referential element is the ChatIoT case study, which evaluates the authors' own patent (ref [79]) using LLM-as-judges rather than an external baseline or human validation. That is a methodological weakness and a minor self-citation, but it is not a reduction-by-construction: the LLM judge is not identical to the system under test, no fitted parameter is renamed as a prediction, and the Section 7 conclusion does not depend on ChatIoT alone, since several surveyed works are directly evaluated in IoT settings. No uniqueness theorem is imported from the authors' prior work, and no known result is renamed as a discovery. Therefore the appropriate finding is no significant circularity, with a small score increase only for the minor non-load-bearing self-citation and self-evaluation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numerical parameters are fitted, and no new entities are introduced. The six capability categories are analytical axes, not entities. The central analysis rests on assumptions about the ICS-IoT mapping, the validity of subjective ratings, the reliability of LLM judges, and the representativeness of the selected works.

assumptions (4)
  • domain assumption IoT systems have similar, if not all, methods to compromise them as ICS.
    Section 4 states this directly to justify using MITRE ATT&CK ICS Mitigations to evaluate IoT security applications.
  • ad hoc to paper The six capability ratings (ETD, IAD, RA, RM, DP, IS) are meaningful and can be consistently assigned from a paper's abstract and claims.
    Section 4 defines these capabilities, but no validation, inter-rater reliability, or independent audit is provided.
  • ad hoc to paper LLM-as-judges produce reliable scores for answer reliability, relevance, technicality, and user-friendliness without human calibration.
    Section 5.3 uses an LLM to judge ChatIoT answers; no human evaluation is reported.
  • domain assumption The 33 selected works are representative of the state of the art in GenAI for IoT security.
    Section 3 describes a structured search but does not report screening counts, inclusion criteria, or PRISMA-style transparency.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generative AI for Internet of Things Security: Challenges and Opportunities." pith.science (2026). https://pith.science/paper/74AECFS6

@misc{pith2026250208886,
  author       = {Pith},
  title        = {Pith review of: Generative AI for Internet of Things Security: Challenges and Opportunities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/74AECFS6}},
  note         = {Machine review of arXiv:2502.08886}
}
read the original abstract

As Generative AI (GenAI) continues to gain prominence and utility across various sectors, their integration into the realm of Internet of Things (IoT) security evolves rapidly. This work delves into an examination of the state-of-the-art literature and practical applications on how GenAI could improve and be applied in the security landscape of IoT. Our investigation aims to map the current state of GenAI implementation within IoT security, exploring their potential to fortify security measures further. Through the compilation, synthesis, and analysis of the latest advancements in GenAI technologies applied to IoT, this paper not only introduces fresh insights into the field, but also lays the groundwork for future research directions. It explains the prevailing challenges within IoT security, discusses the effectiveness of GenAI in addressing these issues, and identifies significant research gaps through MITRE Mitigations. Accompanied with three case studies, we provide a comprehensive overview of the progress and future prospects of GenAI applications in IoT security. This study serves as a foundational resource to improve IoT security through the innovative application of GenAI, thus contributing to the broader discourse on IoT security and technology integration.

Figures

Figures reproduced from arXiv: 2502.08886 by the authors.

Figure 1
Figure 1. Mapping GenAI Applications for Cyber Security to MITRE ATT&CK ICS Mitigations Framework [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

132 extracted references · 58 canonical work pages

  1. [79]

    Y . Dong, Y . L. Aung, S. Chattopadhyay, J. Zhou, ChatIoT: Large Lan- guage Model-Enabled Internet of Things Security Assistant”, Singapore Patent 10202402752Q, filed Sep 4, 2024. URL:https://digitalhub. ipos.gov.sg/FAMN/process/IP4SG/MN_Index#patentcontent

    2024

  2. [1]

    Sedjelmaci, F

    H. Sedjelmaci, F. Guenab, S.-M. Senouci, H. Moustafa, J. Liu, S. Han, Cyber security based on artificial intelligence for cyber-physical systems, IEEE Network 34 (2020) 6–7. doi:10.1109/MNET.2020.9105926

    work page arXiv 2020

  3. [2]

    Gupta, C

    M. Gupta, C. Akiri, K. Aryal, E. Parker, L. Praharaj, From ChatGPT to ThreatGPT: Impact of generative AI in cybersecurity and privacy, IEEE Access 11 (2023) 80218–80245. doi:10.1109/ACCESS.2023.3300381

    work page arXiv 2023

  4. [3]

    Hassanin, N

    M. Hassanin, N. Moustafa, A comprehensive overview of large language models (LLMs) for cyber defences: Opportunities and directions (2024). URL: https://arxiv.org/abs/2405.14487. arXiv:2405.14487

    arXiv 2024

  5. [4]

    Kimani, V

    K. Kimani, V . Oduol, K. Langat, Cyber security challenges for IoT-based smart grid networks, International journal of critical infrastructure pro- tection 25 (2019) 36–49

    2019

  6. [5]

    Alwahedi, A

    F. Alwahedi, A. Aldhaheri, M. A. Ferrag, A. Battah, N. Tihanyi, Machine learning techniques for IoT security: Current research and future vision with generative AI and large language models, Internet of Things and Cyber-Physical Systems (2024)

    2024

  7. [6]

    K. T. Chui, B. B. Gupta, J. Liu, V . Arya, N. Nedjah, A. Almomani, P. Chaurasia, A survey of internet of things and cyber-physical systems: standards, algorithms, applications, security, challenges, and future direc- tions, Information 14 (2023) 388

    2023

  8. [7]

    Hassija, V

    V . Hassija, V . Chamola, V . Saxena, D. Jain, P. Goyal, B. Sikdar, A survey on IoT security: application areas, security threats, and solution architec- tures, IEEE Access 7 (2019) 82721–82743

    2019

Show all 132 references
  1. [8]

    Hussain, R

    F. Hussain, R. Hussain, S. A. Hassan, E. Hossain, Machine learning in IoT security: Current solutions and future challenges, IEEE Communications Surveys & Tutorials 22 (2020) 1686–1721. doi:10.1109/COMST.2020. 2986444

    2020 doi

  2. [9]

    Sultan, M

    A. Sultan, M. A. Mushtaq, M. Abubakar, IoT security issues via blockchain: A review paper, in: Proceedings of the 2019 Inter- national Conference on Blockchain Technology, ICBCT 2019, As- sociation for Computing Machinery, New York, NY , USA, 2019, p. 60–65. URL: https://doi.o...

    2019

  3. [10]

    Jo, The promise and peril of generative AI, Nature 614 (2023) 214– 216

    A. Jo, The promise and peril of generative AI, Nature 614 (2023) 214– 216

    2023

  4. [11]

    arXiv:2303.08774

    OpenAI, GPT-4 technical report (2024). arXiv:2303.08774

    2024 arXiv

  5. [12]

    T. Wu, S. He, J. Liu, S. Sun, K. Liu, Q.-L. Han, Y . Tang, A brief overview of ChatGPT: The history, status quo and potential future development, IEEE/CAA Journal of Automatica Sinica 10 (2023) 1122–1136. doi: 10. 1109/JAS.2023.123618

    2023

  6. [13]

    Daws, US disrupts botnet used by russia-linked APT28 threat group, www.iottechnews.com / (2024)

    R. Daws, US disrupts botnet used by russia-linked APT28 threat group, www.iottechnews.com / (2024). URL: https://www.iottechnews.com/news/2024/feb/16/ us-disrupts-botnet-russia-linked-apt28-threat-group/

    2024

  7. [14]

    C. S. Eze, L. Shamir, Analysis and prevention of AI-based phishing email attacks, Electronics 13 (2024) 1839

    2024

  8. [15]

    Mitra, S

    A. Mitra, S. P. Mohanty, E. Kougianos, The world of generative AI: Deepfakes and large language models, arXiv preprint arXiv:2402.04373 (2024)

    2024 arXiv

  9. [16]

    Zeadally, E

    S. Zeadally, E. Adi, Z. Baig, I. A. Khan, Harnessing artificial intelligence capabilities to improve cybersecurity, Ieee Access 8 (2020) 23817–23837

    2020

  10. [17]

    Alzubaidi, J

    L. Alzubaidi, J. Zhang, A. J. Humaidi, A. Al-Dujaili, Y . Duan, O. Al- Shamma, J. Santamar´ıa, M. A. Fadhel, M. Al-Amidie, L. Farhan, Review of deep learning: concepts, CNN architectures, challenges, applications, future directions, Journal of Big Data 8 (2021) 53

    2021

  11. [18]

    Zhang, S

    L. Zhang, S. Wang, B. Liu, Deep learning for sentiment analysis: A survey, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8 (2018) e1253

    2018

  12. [19]

    G. Pang, C. Shen, L. Cao, A. V . D. Hengel, Deep learning for anomaly detection: A review, ACM computing surveys (CSUR) 54 (2021) 1–38

    2021

  13. [20]

    Ghosh, O

    S. Ghosh, O. Vinyals, B. Strope, S. Roy, T. Dean, L. Heck, Contex- tual LSTM (CLSTM) models for large scale NLP tasks, arXiv preprint arXiv:1602.06291 (2016)

    2016 arXiv

  14. [21]

    C. Park, J. Lee, Y . Kim, J.-G. Park, H. Kim, D. Hong, An enhanced AI-based network intrusion detection system using generative adversarial networks, IEEE Internet of Things Journal 10 (2022) 2330–2345

    2022

  15. [22]

    Yinka-Banjo, O.-A

    C. Yinka-Banjo, O.-A. Ugot, A review of generative adversarial networks and its application in cybersecurity, Artificial Intelligence Review 53 (2020) 1721–1736

    2020

  16. [23]

    Kaloudi, J

    N. Kaloudi, J. Li, The AI-based cyber threat landscape: A survey, ACM Computing Surveys (CSUR) 53 (2020) 1–34

    2020

  17. [24]

    McIntosh, T

    T. McIntosh, T. Liu, T. Susnjak, H. Alavizadeh, A. Ng, R. Nowrozy, P. Watters, Harnessing GPT-4 for generation of cybersecurity GRC poli- cies: A focus on ransomware attack mitigation, Computers & security 134 (2023) 103424

    2023

  18. [25]

    Renaud, M

    K. Renaud, M. Warkentin, G. Westerman, From ChatGPT to HackGPT: Meeting the cybersecurity threat of generative AI, MIT Sloan Manage- ment Review, 2023

    2023

  19. [26]

    Y . M. Pa Pa, S. Tanizaki, T. Kou, M. Van Eeten, K. Yoshioka, T. Mat- sumoto, An attacker’s dream? exploring the capabilities of ChatGPT for developing malware, in: Proceedings of the 16th Cyber Security Experi- mentation and Test Workshop, 2023, pp. 10–18

    2023

  20. [27]

    Greshake, S

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, M. Fritz, Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection, in: Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 2023, pp. 79–90

    2023

  21. [28]

    Gupta, C

    M. Gupta, C. Akiri, K. Aryal, E. Parker, L. Praharaj, From ChatGPT to ThreatGPT: Impact of generative AI in cybersecurity and privacy, IEEE Access (2023)

    2023

  22. [29]

    Bethany, A

    M. Bethany, A. Galiopoulos, E. Bethany, M. B. Karkevandi, N. Vish- wamitra, P. Najafirad, Large language model lateral spear phishing: A comparative study in large-scale organizational settings, arXiv preprint arXiv:2401.09727 (2024)

    2024 arXiv

  23. [30]

    Hilario, S

    E. Hilario, S. Azam, J. Sundaram, K. Imran Mohammed, B. Shanmugam, Generative AI for pentesting: the good, the bad, the ugly, International Journal of Information Security (2024) 1–23

    2024

  24. [31]

    G. Deng, Y . Liu, V . Mayoral-Vilches, P. Liu, Y . Li, Y . Xu, T. Zhang, Y . Liu, M. Pinzger, S. Rass, PentestGPT: An LLM-empowered automatic penetration testing tool (2023). arXiv:2308.06782

    2023 arXiv

  25. [32]

    Ahmadi, Open AI and its impact on fraud detection in financial in- dustry, Sina, A.(2023)

    S. Ahmadi, Open AI and its impact on fraud detection in financial in- dustry, Sina, A.(2023). Open AI and its Impact on Fraud Detection in Financial Industry. Journal of Knowledge Learning and Science Technol- ogy ISSN (2023) 2959–6386

    2023

  26. [33]

    Neupane, I

    S. Neupane, I. A. Fernandez, S. Mittal, S. Rahimi, Impacts and risk of generative AI technology on cyber defense, arXiv preprint arXiv:2306.13033 (2023)

    2023 arXiv

  27. [34]

    Kucharavy, Z

    A. Kucharavy, Z. Schillaci, L. Mar ´echal, M. W ¨ursch, L. Dolamic, R. Sabonnadiere, D. P. David, A. Mermoud, V . Lenders, Fundamentals of generative large language models and perspectives in cyber-defense, arXiv preprint arXiv:2303.12132 (2023)

    2023 arXiv

  28. [35]

    S. Sai, U. Yashvardhan, V . Chamola, B. Sikdar, Generative AI for cyber security: Analyzing the potential of ChatGPT, DALL-E and other models for enhancing the security space, IEEE Access (2024)

    2024

  29. [36]

    URL: https://attack.mitre.org/ mitigations/ics/

    MITRE, Mitigations - ICS — MITRE ATT&CK, Mitigations - ICS — MITRE ATT&CK (2024). URL: https://attack.mitre.org/ mitigations/ics/

    2024

  30. [37]

    Kavian, M

    A. Kavian, M. M. P. Kallehbasti, S. Kazemi, E. Firouzi, M. Gha- fari, LLM security guard for code, 2024. URL: https://api. semanticscholar.org/CorpusID:269502004

    2024

  31. [38]

    H. Li, Y . Hao, Y . Zhai, Z. Qian, Enhancing static analysis for practical bug detection: An LLM-integrated approach, Proc. ACM Program. Lang. 8 (2024). URL: https://doi.org/10.1145/3649828. doi:10.1145/ 3649828

    2024 doi

  32. [39]

    H. Li, Y . Hao, Y . Zhai, Z. Qian, Enhancing Static Analysis for Practical Bug Detection: An LLM-Integrated Approach (Arti- fact), 2024. URL: https://doi.org/10.5281/zenodo.10780591. doi:10.5281/zenodo.10780591

    2024 doi

  33. [40]

    T. Ali, P. Kostakos, HuntGPT: Integrating machine learning-based anomaly detection and explainable AI with large language models (LLMs) (2023). arXiv:2309.16021

    2023 arXiv

  34. [42]

    Tavallaee, E

    M. Tavallaee, E. Bagheri, W. Lu, A. A. Ghorbani, A detailed analysis of the KDD CUP 99 data set, in: 2009 IEEE Symposium on Computa- tional Intelligence for Security and Defense Applications, 2009, pp. 1–6. doi:10.1109/CISDA.2009.5356528

    2009

  35. [43]

    E. C. P. Neto, S. Dadkhah, R. Ferreira, A. Zohourian, R. Lu, A. A. Ghor- bani, CICIoT2023: A real-time dataset and benchmark for large-scale attacks in IoT environment, Sensors 23 (2023). URL: https://www. mdpi.com/1424-8220/23/13/5941. doi:10.3390/s23135941

    2023 doi

  36. [44]

    H. Cui, Y . Du, Q. Yang, Y . Shao, S. C. Liew, LLMind: Orches- trating AI and IoT with LLM for complex task execution (2024). arXiv:2312.09007

    2024 arXiv

  37. [45]

    H. Wang, Z. Wang, P. Liu, A hybrid LLM workflow can help iden- tify user privilege related variables in programs of any size (2024). arXiv:2403.15723

    2024 arXiv

  38. [46]

    D. Raman, NVISOsecurity /cyber-security-llm-agents: A collection of agents that use large language models (LLMs) to perform tasks common on our day to day jobs in cyber security., GitHub (2024). URL: https: //github.com/NVISOsecurity/cyber-security-llm-agents

    2024

  39. [47]

    Q. Wu, G. Bansal, J. Zhang, Y . Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, A. H. Awadallah, R. W. White, D. Burger, C. Wang, Au- toGen: Enabling next-gen LLM applications via multi-agent conversation (2023). arXiv:2308.08155

    2023 arXiv

  40. [48]

    Kaheh, D

    M. Kaheh, D. K. Kholgh, P. Kostakos, Cyber sentinel: Exploring con- versational agents in streamlining security tasks with GPT-4 (2023). arXiv:2309.16422

    2023 arXiv

  41. [49]

    Zhong, M

    Y . Zhong, M. Qi, R. Wang, Y . Qiu, Y . Zhang, H. Ma, VIoTGPT: Learn- ing to schedule vision tools towards intelligent video internet of things (2023). arXiv:2312.00401

    2023 arXiv

  42. [50]

    D. Saha, S. Tarek, K. Yahyaei, S. K. Saha, J. Zhou, M. Tehranipoor, F. Farahmandi, LLM for SoC security: A paradigm shift (2023). arXiv:2310.06046

    2023 arXiv

  43. [51]

    O. G. Lira, A. Marroquin, M. A. To, Harnessing the advanced capa- bilities of LLM for adaptive intrusion detection systems, in: L. Barolli (Ed.), Advanced Information Networking and Applications, Springer Na- ture Switzerland, Cham, 2024, pp. 453–464

    2024

  44. [52]

    Guastalla, Y

    M. Guastalla, Y . Li, A. Hekmati, B. Krishnamachari, Application of large language models to DDoS attack detection, in: Y . Chen, C.-W. Lin, B. Chen, Q. Zhu (Eds.), Security and Privacy in Cyber-Physical Sys- tems and Smart Vehicles, Springer Nature Switzerland, Cham, 2024, pp. 83–99

    2024

  45. [53]

    M. A. Ferrag, M. Ndhlovu, N. Tihanyi, L. C. Cordeiro, M. Debbah, 16 T. Lestable, N. S. Thandi, Revolutionizing cyber threat detection with large language models: A privacy-preserving BERT-based lightweight model for IoT /IIoT devices, IEEE Access 12 (2024) 23733–23750. doi:10....

    2024

  46. [54]

    Y . Li, Z. Xiang, N. D. Bastian, D. Song, B. Li, IDS-agent: An LLM agent for explainable intrusion detection in IoT networks, in: NeurIPS 2024 Workshop on Open-World Agents, 2024. URL:https://openreview. net/forum?id=iiK0pRyLkw

    2024

  47. [55]

    N. T. Islam, J. Khoury, A. Seong, M. B. Karkevandi, G. D. L. T. Parra, E. Bou-Harb, P. Najafirad, LLM-powered code vulnerabil- ity repair with reinforcement learning and semantic reward (2024). arXiv:2401.03374

    2024 arXiv

  48. [56]

    Bhandari, A

    G. Bhandari, A. Naseer, L. Moonen, CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Software, in: Proceed- ings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE ’21), ACM, 2021, p. 1...

    2021

  49. [57]

    J. Wang, Z. Huang, H. Liu, N. Yang, Y . Xiao, DefectHunter: A novel LLM-driven boosted-conformer-based code vulnerability detection mechanism (2023). arXiv:2309.15324

    2023 arXiv

  50. [58]

    Y . Zhou, S. Liu, J. Siow, X. Du, Y . Liu, Devign: E ffective vulnerability identification by learning comprehensive program semantics via graph neural networks, in: Advances in Neural Information Processing Sys- tems, 2019, pp. 10197–10207

    2019

  51. [59]

    Jin, Pongo-70b dataset (2023)

    W. Jin, Pongo-70b dataset (2023). URL: https://huggingface.co/ wj2003/Pongo-70B

    2023

  52. [60]

    URL: https://nvd

    NVD, CVE-2022-45788 detail, NVD (2023). URL: https://nvd. nist.gov/vuln/detail/CVE-2022-45788

    2023

  53. [61]

    Adams, AttackGen (2024)

    M. Adams, AttackGen (2024). URL: https://github.com/ mrwadams/attackgen

    2024

  54. [62]

    Yamin, E

    M. Yamin, E. Hashmi, M. Ullah, B. Katt, Applications of LLMs for generating cyber security exercise scenarios (2024). doi: 10.21203/rs. 3.rs-3970015/v1

    2024 doi

  55. [63]

    Y . Sun, D. Wu, Y . Xue, H. Liu, W. Ma, L. Zhang, M. Shi, Y . Liu, LLM4Vuln: A unified evaluation framework for decoupling and enhanc- ing LLMs’ vulnerability reasoning (2024). arXiv:2401.16185

    2024 arXiv

  56. [64]

    J. Xu, J. W. Stokes, G. McDonald, X. Bai, D. Marshall, S. Wang, A. Swaminathan, Z. Li, Autoattacker: A large language model guided system to implement automatic cyber-attacks (2024). arXiv:2403.01038

    2024 arXiv

  57. [65]

    T ´oth, T

    R. T ´oth, T. Bisztray, L. Erdodi, LLMs in web-development: Evaluat- ing LLM-generated PHP code unveiling vulnerabilities and limitations (2024). arXiv:2404.14459

    2024 arXiv

  58. [66]

    Oliinyk, M

    Y . Oliinyk, M. Scott, R. Tsang, C. Fang, H. Homayoun, et al., Fuzzing BusyBox: Leveraging LLM and crash reuse for embedded bug un- earthing, arXiv preprint arXiv:2403.03897 (2024)

    2024 arXiv

  59. [67]

    Happe, A

    A. Happe, A. Kaplan, J. Cito, LLMs as hackers: Autonomous linux priv- ilege escalation attacks (2024). arXiv:2310.11409

    2024

  60. [68]

    J. Wang, L. Yu, X. Luo, LLMIF: Augmented large language model for fuzzing IoT devices, in: 2024 IEEE Symposium on Security and Privacy (SP), IEEE Computer Society, Los Alamitos, CA, USA, 2024, pp. 196–196. URL: https://doi.ieeecomputersociety.org/10. 1109/SP54263.2024.00182. d...

    2024

  61. [69]

    R. Meng, M. Mirchev, M. B ¨ohme, A. Roychoudhury, Large language model guided protocol fuzzing, in: Proceedings of the 31st Annual Net- work and Distributed System Security Symposium (NDSS), 2024

    2024

  62. [70]

    W. Ma, B. Cui, Fuzzing IoT devices via android app interfaces with large language model, in: L. Barolli (Ed.), Advances in Internet, Data & Web Technologies, Springer Nature Switzerland, Cham, 2024, pp. 87–99

    2024

  63. [71]

    R. Fang, R. Bindu, A. Gupta, D. Kang, LLM agents can autonomously exploit one-day vulnerabilities (2024). arXiv:2404.08144

    2024 arXiv

  64. [72]

    X. Ma, L. Luo, Q. Zeng, From one thousand pages of spec- ification to unveiling hidden bugs: Large language model assisted fuzzing of matter IoT devices, in: 33rd USENIX Security Symposium (USENIX Security 24), USENIX Association, Philadel- phia, PA, 2024, pp. 4783–4800. URL: ...

    2024

  65. [73]

    Piggott, S

    B. Piggott, S. Patil, G. Feng, I. Odat, R. Mukherjee, B. Dharmalingam, A. Liu, Net-GPT: A llm-empowered man-in-the-middle chatbot for un- manned aerial vehicle, in: 2023 IEEE /ACM Symposium on Edge Com- puting (SEC), 2023, pp. 287–293. doi:10.1145/3583740.3626809

    2023

  66. [74]

    Happe, J

    A. Happe, J. Cito, Getting pwn’d by AI: Penetration testing with large lan- guage models, in: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE ’23, ACM, 2023. URL: http://dx.doi.org/ 1...

    2023

  67. [75]

    Yang, IoT software vulnerability detection techniques through large language model, in: International Conference on Formal Engineering Methods, Springer, 2023, pp

    Y . Yang, IoT software vulnerability detection techniques through large language model, in: International Conference on Formal Engineering Methods, Springer, 2023, pp. 285–290

    2023

  68. [76]

    Prensky, Digital game-based learning, Comput

    M. Prensky, Digital game-based learning, Comput. Entertain. 1 (2003) 21. URL: https://doi.org/10.1145/950566.950596. doi:10.1145/950566.950596

    2003

  69. [77]

    V .-T. Pham, M. B¨ohme, A. Roychoudhury, AFLNET: A greybox fuzzer for network protocols, in: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), 2020, pp. 460–465. doi:10.1109/ICST46399.2020.00062

    2020

  70. [78]

    S. Qin, F. Hu, Z. Ma, B. Zhao, T. Yin, C. Zhang, NSFuzz: Towards e ffi- cient and state-aware network service fuzzing, ACM Trans. Softw. Eng. Methodol. 32 (2023). URL: https://doi.org/10.1145/3580598. doi:10.1145/3580598

    2023 doi

  71. [80]

    URL: https://attack.mitre.org/ groups/G0035/

    MITRE, Group G0035 — MITRE ATT&CK, Dragonfly, TEMP.Isotope, DYMALLOY , Berserk Bear, TG-4192, Crouching Yeti, IRON LIB- ERTY , Energetic Bear, Ghost Blizzard, BROMINE, Group G0035 — MITRE ATTCK (2024). URL: https://attack.mitre.org/ groups/G0035/

    2024

  72. [81]

    B. Li, K. Mellou, B. Zhang, J. Pathuri, I. Menache, Large language models for supply chain optimization, arXiv preprint arXiv:2307.03875 (2023)

    2023 arXiv

  73. [82]

    Munley, A

    C. Munley, A. Jarmusch, S. Chandrasekaran, LLM4VV: Developing llm- driven testsuite for compiler validation, Future Generation Computer Sys- tems (2024). Appendix A. Evaluation of State-of-the-Art Works We discussed each state-of-the-art work according to the six capabilities ...

    2024

  74. [83]

    Assess the company’s ability to detect and respond to a supply chain compromise

  75. [85]

    Test the procedures in place for mitigating and recovering from a supply chain compromise

  76. [86]

    Identify areas for improvement in the company’s incident response plan. Scenario Details Company Profile • Industry: Energy /Utilities • Size: Medium (51-200 employees) Threat Actor Group • Name: Dragonfly • Tactics, Techniques, and Procedures (TTPs): Utilizing supply chain co...

  77. [88]

    Review the company’s incident response plan, focusing on supply chain compromises

  78. [89]

    Phase 2: Detection Objectives: • Test the company’s ability to detect anomalies activities associated with a supply chain compromise

    Check the availability of forensic tools, communication channels, and backup systems. Phase 2: Detection Objectives: • Test the company’s ability to detect anomalies activities associated with a supply chain compromise. Simulated Event: • A third-party vendor supplying softwar...

  79. [90]

    Deploy the compromised software update to a test environment

  80. [91]

    Monitor network tra ffic and system logs for signs of the compromise

  81. [92]

    Phase 3: Containment Objectives: • Evaluate the company’s capability to contain the threat and prevent further damage

    Use indicators of compromise (IoCs) such as unusual network connections, unexpected file changes, and system behavior anomalies. Phase 3: Containment Objectives: • Evaluate the company’s capability to contain the threat and prevent further damage. Simulated Event: • The malici...

  82. [93]

    Isolate the a ffected systems from the network

  83. [95]

    Phase 4: Eradication Objectives: • Test the company’s ability to remove the threat from a ffected systems

    Notify relevant stakeholders of the incident. Phase 4: Eradication Objectives: • Test the company’s ability to remove the threat from a ffected systems. Actions:

  84. [96]

    Perform a thorough analysis of the compromised systems to identify all malicious components

  85. [97]

    Remove the malicious payload and any associated malware

  86. [98]

    Phase 5: Recovery Objectives: • Assess the company’s procedures for restoring normal operations

    Apply patches and updates to close the vulnerability exploited by the attacker. Phase 5: Recovery Objectives: • Assess the company’s procedures for restoring normal operations. Actions:

  87. [99]

    Restore a ffected systems from clean backups

  88. [100]

    Monitor the systems to ensure that no residual threats remain

  89. [101]

    Phase 6: Lessons Learned Objectives: • Review the incident response process to identify strengths and areas for improvement

    Gradually reconnect the systems to the network. Phase 6: Lessons Learned Objectives: • Review the incident response process to identify strengths and areas for improvement. Actions:

  90. [102]

    Conduct a debriefing session with all incident response team members

  91. [103]

    Document the incident and response actions taken

  92. [104]

    Evaluation Criteria

    Update the incident response plan based on the lessons learned. Evaluation Criteria

  93. [105]

    • Accuracy in identifying IoCs

    Detection: • Time taken to detect the compromise. • Accuracy in identifying IoCs

  94. [106]

    • Communication and coordination among team members

    Containment: • Speed and e ffectiveness of containment measures. • Communication and coordination among team members

  95. [107]

    • Effectiveness of patching and updating procedures

    Eradication: • Thoroughness in removing all malicious components. • Effectiveness of patching and updating procedures

  96. [108]

    • Monitoring for residual threats

    Recovery: • Efficiency in restoring normal operations. • Monitoring for residual threats

  97. [109]

    • Implementation of improvements to the incident response plan

    Lessons Learned: • Quality of the post-incident analysis. • Implementation of improvements to the incident response plan. Conclusion This incident response testing scenario provides a comprehensive framework to evaluate the company’s readiness against a supply chain compromise...

  98. [110]

    Assess the company’s ability to detect and respond to a supply chain compromise involving an IoT-powered PLC

  99. [111]

    Evaluate the e ffectiveness of communication and coordination among incident response team members

  100. [112]

    Test the procedures for monitoring and recovering from a supply chain compromise a ffecting critical IoT devices

  101. [113]

    Identify areas for improvement in the company’s incident response plan. Scenario Details Company Profile • Industry: Energy / Utilities • Size: Medium (51-200 employees) Critical Asset: IoT powered PLC connected to the internet Threat Actor Group • Name: Dragonfly • Tactics, T...

  102. [114]

    Conduct a briefing session with the incident response team

  103. [115]

    Review the company’s incident response plan, focusing on supply chain compromise scenarios, particularly those involving IoT devices

  104. [116]

    Ensure proper monitoring mechanisms are in place for IoT devices, especially the PLC

    Check the availability of forensic tools, communication channels, and backup systems. Ensure proper monitoring mechanisms are in place for IoT devices, especially the PLC. Phase 2: Detection Objectives: • Test the company’s ability to detect anomalous activities associated wit...

  105. [117]

    Deploy the compromised firmware update to a test environment with an IoT-powered PLC

  106. [118]

    Monitor network tra ffic, system logs, and PLC activity for signs of the compromise

  107. [119]

    Phase 3: Containment Objectives: • Evaluate the company’s capability to contain the threat and prevent further damage, particularly to critical IoT devices

    Identify indicators of compromise (IoCs) such as unusual network connections, unexpected firmware changes, and anomalous PLC behavior. Phase 3: Containment Objectives: • Evaluate the company’s capability to contain the threat and prevent further damage, particularly to critica...

  108. [120]

    Isolate the a ffected PLC and any other compromised systems from the network

  109. [121]

    Block outgoing connections to the identified C2 server

  110. [122]

    Phase 4: Eradication Objectives: • Test the company’s ability to remove the threat from a ffected systems, including the critical IoT powered PLC

    Notify relevant stakeholders of the incident, emphasizing the critical nature of the compromised IoT device. Phase 4: Eradication Objectives: • Test the company’s ability to remove the threat from a ffected systems, including the critical IoT powered PLC. Actions:

  111. [123]

    Perform a thorough analysis of the compromised PLC and other a ffected systems to identify all malicious components

  112. [124]

    Remove the malicious payload and any associated malware from the PLC

  113. [125]

    Phase 5: Recovery Objectives: • Assess the company’s procedures for restoring normal operations, focusing on the critical IoT-powered PLC

    Apply patches and updates to close the vulnerability exploited by the attacker, ensuring the integrity of the firmware on the PLC. Phase 5: Recovery Objectives: • Assess the company’s procedures for restoring normal operations, focusing on the critical IoT-powered PLC. Actions:

  114. [126]

    Restore the PLC from a clean backup, ensuring its firmware is uncompromised

  115. [127]

    Monitor the PLC and other systems to ensure that no residual threats remain

  116. [128]

    Phase 6: Lessons Learned Objectives: • Review the incident response process to identify strengths and areas for improvement, with a focus on IoT devices

    Gradually reconnect the PLC and other a ffected systems to the network. Phase 6: Lessons Learned Objectives: • Review the incident response process to identify strengths and areas for improvement, with a focus on IoT devices. Actions:

  117. [129]

    Conduct a debriefing session with all incident response team members, emphasizing the response to the compromised IoT-powered PLC

  118. [130]

    Document the incident and response actions taken, particularly those involving IoT devices

  119. [131]

    Evaluation Criteria

    Update the incident response plan based on the lessons learned, including specific measures for protecting and responding to incidents involving IoT devices. Evaluation Criteria

  120. [132]

    • Accuracy in identifying IoCs, especially those related to IoT devices

    Detection: • Time taken to detect the compromise. • Accuracy in identifying IoCs, especially those related to IoT devices

  121. [133]

    • Communication and coordination among team members, with a focus on IoT device management

    Containment: • Speed and e ffectiveness of containment measures. • Communication and coordination among team members, with a focus on IoT device management

  122. [134]

    • Effectiveness of patching and updating procedures, particularly for IoT devices

    Eradication: • Thoroughness in removing all malicious components. • Effectiveness of patching and updating procedures, particularly for IoT devices

  123. [135]

    • Monitoring for residual threats, especially in IoT devices

    Recovery: • Efficiency in restoring normal operations. • Monitoring for residual threats, especially in IoT devices

  124. [136]

    • Implementation of improvements to the incident response plan, including IoT-specific considerations

    Lessons Learned: • Quality of the post-incident analysis. • Implementation of improvements to the incident response plan, including IoT-specific considerations. Conclusion This updated incident response testing scenario provides a comprehensive framework to evaluate the compan...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.