Pith. sign in

REVIEW 4 major objections 4 minor 56 references

ProfMalPlus detects malicious npm packages by feeding LLM agents annotated code slices from object-sensitive behavior graphs, reaching a 98.1% F1-score.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-02 03:09 UTC pith:QYFIM75M

load-bearing objection The 597-package field result is the real story; the 98.1% F1 is a curated-benchmark number, not a wild estimate. the 4 major comments →

arxiv 2607.13965 v1 pith:QYFIM75M submitted 2026-07-15 cs.SE cs.CR

ProfMalPlus: Agent-Coordinated Detection of Malicious NPM Packages via Static-Dynamic Analysis Synergy

Yiheng Huang , Zhijia Zhao , Bihuan Chen , Susheng Wu , Zhuotong Zhou , Yiheng Cao , Kun Hu , Xin Hu
show 1 more author
Xin Peng
This is my paper
classification cs.SE cs.CR
keywords malicious npm packagessupply-chain attackLLM agentsbehavior graphcode slicingstatic analysisdynamic analysismalware localization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that combining object-sensitive behavior graphs with coordinated LLM reasoning over annotated code slices detects malicious npm packages more accurately than existing detectors. Instead of classifying abstract graph features, it reconstructs security-relevant code with inline static, third-party, and runtime evidence, then lets judge agents reason over the code. The claim is supported by a 98.1% F1-score against five state-of-the-art detectors, a three-month deployment that found 597 previously unknown malicious packages, and line-level localization of malicious code. A sympathetic reader would care because it proposes a path from black-box detection to interpretable, evidence-driven auditing at registry scale.

Core claim

On its own terms, ProfMalPlus claims that four limitations of prior npm malware detectors—poor modeling of obfuscated code, ignoring JavaScript's object-centric features, weak static-dynamic synergy, and semantic loss from abstraction—can be addressed in one pipeline. The pipeline builds a behavior graph with object-sensitive alias and property tracking, slides backward and forward from suspicious nodes to extract source-level code slices, annotates them with evidence, and runs a loop of local judge agents, a global judge, a router, and optional third-party or dynamic enrichment until a verdict is reached. The result is an entry-level malicious/benign decision with concrete code localization

What carries the argument

The central mechanism is the behavior graph: a directed graph over statements with control-flow, control-dependency, and data-dependency edges, built by object-sensitive traversal that tracks alias relations and qualified object paths so sensitive API calls (e.g., os.hostname hidden behind function aliases or obfuscated property access) can be resolved. From suspicious anchor nodes the code slicer extracts annotated source slices; local judge agents with self-consistency verification then reason over each slice, a global judge synthesizes entry-level verdicts, and a router selects third-party enrichment (registry-derived API semantics) or dynamic augmentation (sandbox execution resolving run

Load-bearing premise

The headline numbers rest on the assumption that the curated benchmark—1,090 malicious and 3,000 benign packages, filtered to those that execute in a Linux sandbox and expose at least one suspicious anchor—fairly represents real npm packages, and that LLM verdicts on these public historical samples are not inflated by training-data memorization.

What would settle it

Run the same pipeline on npm packages published after the LLM's training cutoff, using independently confirmed takedowns as ground truth (as RQ5 does on a small scale): if the F1-score drops toward the 94.8% of the strongest baseline or below, the benchmark advantage is partly memorization. A second check: apply the detector to an unfiltered random sample of npm packages, including packages with empty behavior graphs, and see whether real-world precision remains near 83.5%.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X LinkedIn Reddit HN

If this is right

  • If the claims hold, malicious npm packages can be detected at a 98.1% F1-score with concrete code localization, making audit results actionable rather than a bare label.
  • The static-first design (about 90% of verdicts reached without sandbox execution) means registry-scale screening can reserve expensive dynamic analysis for genuinely ambiguous cases.
  • Third-party enrichment resolves most routed third-party calls at about one-sixth the runtime of sandbox execution, so documentation-driven semantics can substitute for execution.
  • The real-world result of 597 confirmed removals suggests the approach generalizes beyond benchmark samples to newly published packages postdating the model's training cutoff.
  • Ablations show self-consistency verification and dynamic augmentation are the largest contributors; removing them drops F1 by 4.49 and 3.87 points respectively.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • I infer the benchmark advantage over the strongest baseline may be partly attributable to the LLM having seen historical malicious packages in training; the RQ5 deployment is the cleaner test, and its lower false-positive rate (16.5%) is the more convincing evidence.
  • I infer the pipeline's conservative policy, while reducing false positives, will continue to miss malware that only activates at explicit user invocation or under Windows-specific conditions; extending monitoring beyond install and import time is a natural next step.
  • I infer that combining this source-level reasoning with system-call-level monitoring could resolve the binary-execution false positives the paper identifies, since opaque binaries are currently flagged by default.
  • I infer the approach transfers to other package ecosystems (PyPI, RubyGems) that share install-time and import-time attack surfaces, though object-sensitive analysis would need adapting to each language's semantics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents ProfMalPlus, a detector for malicious NPM packages that combines object-sensitive behavior graphs with a coordinated multi-agent LLM reasoning pipeline. The pipeline constructs behavior graphs from entry files, extracts security-relevant code slices, runs local judge agents with self-consistency verification, then a global judge agent. For undetermined cases, a router selects either third-party enrichment (registry-derived module/method semantics) or dynamic augmentation (sandbox execution with instrumentation), feeding evidence back into slices. A localization agent finally reports malicious snippets. The evaluation claims a 98.1% F1-score on a curated benchmark of 1,090 malicious and 3,000 benign packages, outperforming five baselines, and reports a three-month deployment that detected 597 previously unknown malicious packages that were confirmed and removed from NPM.

Significance. If the reported effectiveness held on uncurated data, the paper would be a significant contribution to software supply-chain security. The system design is detailed and reproducible: the artifact release, prompt snippets, and component-wise ablation give the reader a concrete picture of how each stage contributes. The real-world deployment with 597 confirmed removals is a meaningful field result, and the interpretability evaluation (line-level localization F1 88.9% with inter-expert agreement reported) is a strength. The use of a temporal separation argument for RQ5 is also a step in the right direction for LLM-based detection. However, the evaluation protocol has several load-bearing weaknesses that currently prevent the headline numbers from being accepted as a statement about general NPM packages.

major comments (4)
  1. [§4.1, Table 2] The benchmark underlying the headline 98.1% F1 is curated in an outcome-dependent way. Malicious packages were filtered to exclude those with no actual malicious code, non-executable obfuscated packages, and packages whose behavior is not triggered at install/import time; benign packages were retained only if they had at least one suspicious anchor. As §5.1 admits, the removed obfuscated packages "might exhibit different malicious behaviors compared to the retained set." This selection removes exactly the cases where static/dynamic synergy cannot help and where static analysis is hardest, so the 98.1% is an upper-bound for a filtered subpopulation. The paper should report performance on the full unfiltered set or explicitly reframe the claim as applying only to this curated population.
  2. [§4.2, Table 2; §5.1] The pipeline is stochastic (LLM-based), yet all reported point estimates (98.1% F1, ablation deltas) come from a single run with no confidence intervals, repeated-seed analysis, or variance reporting. The paper itself notes in §5.1 that LLM outputs are non-deterministic and that self-consistency "cannot fully eliminate" variance. The 3.5% gap over SocketAI and the 0.3% drop when removing third-party enrichment (Table 5) cannot be interpreted without knowing run-to-run variability. The authors should run the pipeline multiple times (or over multiple dataset folds) and report means, standard deviations, and CIs for the headline and ablation numbers.
  3. [§4.6, Table 7] The only uncurated evaluation, RQ5, reports 597 confirmed true positives out of 715 alarms (precision 83.5%) and, using the union-of-detectors definition for false negatives, recall 90.7%, corresponding to F1 ≈ 87% — not 98.1%. The abstract and conclusion state 98.1% without noting that the RQ5 result is substantially lower. Moreover, §5.1 acknowledges that NPM does not disclose a complete malicious list, so the union-based ground truth may underestimate false negatives, making even the 87% F1 optimistic. The paper should present the RQ5 precision/recall/F1 as the primary real-world measure and reconcile the discrepancy with the curated-benchmark headline.
  4. [§4.1, §4.6] The RQ1 dataset is drawn from public malicious-package corpora (Backstabber's Knife, MalwareBench, OSCAR, MalOSS) that may overlap with the training data of the DeepSeek-V4-Flash backbone. The temporal-separation argument in §4.6 applies only to RQ5, not to the central RQ1 comparison, so memorization could inflate the 98.1% point estimate. The paper should provide a concrete leakage test — for example, comparing performance on public versus post-training-cutoff packages, or checking whether the LLM identifies known samples without evidence — or explicitly bound the effect of possible memorization on the RQ1 results.
minor comments (4)
  1. [§4.4, Table 5] The ablated version name "w/o MSC-Detection" uses an acronym that is not expanded; for clarity it should be "w/o Malicious Shell-Command Detection" to match the description in §3.1.
  2. [§4.2, Table 3] Table 3's caption says "Stage-Wise Decision Flow" but the rows are detection runs (entry flows), not package-level totals. The caption should state this explicitly to avoid misinterpretation of the 89.9%/89.5% static-stage percentages.
  3. [§4.6, Table 7] Table 7's column headers "FP Rate" and "FN Rate" are not formally defined. Clarify the denominators (e.g., FP / total alarms, FN / (TP+FN)) either in the caption or the text.
  4. [Throughout] The manuscript contains frequent missing spaces between words (e.g., "proposeProfMalPlus", "ProfMalPlusachieves", "ProfMalPlusto"). These appear to be formatting artifacts from PDF extraction but should be cleaned in the final version.

Circularity Check

0 steps flagged

No significant circularity: benchmark caveats and self-citations do not reduce the derivation to its inputs.

full rationale

ProfMalPlus is an end-to-end detection pipeline rather than a mathematical derivation with fitted parameters, so the circularity tests must be applied to the argument chain. (1) The behavior-graph substrate is attributed to the authors' prior ProfMal [16] ("The modeling limitations L1, L2 and L3 are addressed by the behavior graph construction introduced in our previous work [16]"), but Sec. 3.2 re-specifies the construction in detail (DDG+, call graph, object-sensitive alias tracking), and ProfMal is re-run as a baseline in Table 2. The citation is attribution, not the sole support for the current results. (2) The benchmark is curated: Sec. 4.1 keeps only malicious packages executable in the Linux sandbox and only benign packages exposing at least one suspicious anchor. This is a selection/external-validity threat, not circularity: the F1 value is not defined by the curation rule, and the verdicts are produced by code reasoning rather than recovered from the inclusion criteria. Sec. 5.1 explicitly admits "These removed packages might exhibit different malicious behaviors compared to the retained set" and that RQ5 false negatives are computed against the union of detectors, which "may underestimate the absolute false negatives." These are honest limitations, not circular reductions. (3) The strongest independent evidence is RQ5: 597 newly published packages "confirmed as malicious by NPM and subsequently removed," with packages postdating the LLM training cutoff. This result does not reduce to the benchmark or to self-citation. (4) No equation in Sec. 3 sets a predicted quantity equal to a fitted input; the sensitive API list is an explicit hand-built rule set from prior studies, and the agent prompts are decision heuristics. I therefore find no significant circularity; the moderate concerns about memorization and dataset curation belong under correctness risk and external validity, not under the circularity rubric.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claim rests on a hand-curated sensitive-API taxonomy, a conservative LLM decision policy, dataset inclusion criteria, and the install/import scope. There are no fitted numerical constants in a mathematical derivation sense, but the detection threshold is effectively encoded in prompts, sensitivity labels, and the filtering rules applied to the benchmark.

free parameters (3)
  • Sensitive API list with ARD flags = hand-curated, following prior work [14,15,49,53,55]
    The anchor set and code slices are built from this list; if it mislabels benign patterns as sensitive or misses important APIs, the F1 and localization results change directly.
  • Self-consistency sample count = 3
    The local judge is invoked three times per slice; this number is chosen by hand without a sensitivity analysis.
  • Evaluation dataset inclusion rules = 1,090 malicious / 3,000 benign after exclusions
    Manual filtering and execution-based exclusion set the denominator for the 98.1% F1; it is a design choice, not a fitted constant, but it shapes the headline result.
axioms (5)
  • domain assumption Malicious npm behavior is overwhelmingly triggered during installation and import time.
    Scopes the entire detector; runtime-only attacks are out of scope and counted as false negatives by design (Sec. 1, Sec. 5.2).
  • ad hoc to paper LLM agents instructed with the provided prompts produce sufficiently accurate security verdicts after self-consistency verification.
    The reasoning module assumes prompt-engineered LLM judgement is grounded in the evidence; Table 4 shows the results are backbone-sensitive, and the paper acknowledges nondeterminism.
  • ad hoc to paper Removing benign packages with empty behavior graphs and excluding non-executable obfuscated packages does not bias the performance estimate.
    Sec. 4.1 exclusion criteria change the evaluation population; if these packages are harder or easier than the retained set, the F1 is optimistic or pessimistic.
  • domain assumption The predefined sensitive API list and ARD flags capture the security-relevant behaviors that matter.
    Sensitive, conditional, and third-party anchors are derived from this list (Table 1); omissions directly create false negatives.
  • domain assumption The object-sensitive idMap/propMap analysis is sound, and third-party tools (Joern, Jelly, NodeProf) are accurate enough for this task.
    The behavior graph and dynamic augmentation rely on these tools; the paper notes their inaccuracies may propagate (Sec. 5.2).

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ProfMalPlus: Agent-Coordinated Detection of Malicious NPM Packages via Static-Dynamic Analysis Synergy." pith.science (2026). https://pith.science/paper/QYFIM75M

@misc{pith2026260713965,
  author       = {Pith},
  title        = {Pith review of: ProfMalPlus: Agent-Coordinated Detection of Malicious NPM Packages via Static-Dynamic Analysis Synergy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QYFIM75M}},
  note         = {Machine review of arXiv:2607.13965}
}
Share X LinkedIn Reddit HN
read the original abstract

Open source software is vulnerable to supply-chain attacks through transitive dependencies, especially malicious code injected into NPM packages. Existing detectors often inadequately model obfuscated behavior, overlook JavaScript's object-centric features, poorly coordinate static and dynamic analysis, and lose semantic information during behavior abstraction. We propose ProfMalPlus, a malicious NPM package detector combining object-sensitive behavior graphs with coordinated LLM reasoning over annotated code slices. It identifies installation commands and entry files, then constructs graphs capturing sensitive APIs, third-party calls, and unresolved calls. From these graphs, ProfMalPlus extracts security-relevant slices and adds inline static analysis evidence. Local judge agents independently assess each slice. Self-consistency consolidates repeated judgements to reduce LLM variance, while a global judge synthesizes their reports into an entry-level verdict. For undetermined cases, a router selects either third-party enrichment, which adds registry derived module and method semantics, or dynamic augmentation, which executes the package in a sandbox to resolve runtime dependent behavior. The enriched evidence is fed back for reassessment. Finally, a localization agent reports malicious code snippets with explanations. ProfMalPlus achieves a 98.1% F1-score, outperforming state-of-the-art detectors by 3.5% to 52.6%. It also identified 597 previously unknown malicious packages, all confirmed and removed from NPM.

Figures

Figures reproduced from arXiv: 2607.13965 by Bihuan Chen, Kun Hu, Susheng Wu, Xin Hu, Xin Peng, Yiheng Cao, Yiheng Huang, Zhijia Zhao, Zhuotong Zhou.

Figure 1
Figure 1. Figure 1: Code Snippet of bitsoex_react-design-system_14.1.4 • We conducted experiments to demonstrate the effectiveness and practical usefulness of ProfMalPlus. ProfMalPlus achieved the highest F1-score among five state-of-the-art detectors, and detected 597 previously unknown malicious packages during three months of real-world monitoring. • We released the code of ProfMalPlus at our website [43]. 2 MOTIVATING EXA… view at source ↗
Figure 2
Figure 2. Figure 2: Code Snippet of automation.samples_0.1.15 1 const os = require("os") 2 const { exec } = require("child_process"); 3 const https = require("https"); 4 function printPlatform() { 5 const platform = os.platform(); 6 console.log(platform); 7 } 8 9 function printMountedDisks() { 10 const platform = os.platform(); 11 const cmd = platform === "win32" ? 12 "wmic logicaldisk 11 get name" : "df -h"; 13 exec(cmd, (er… view at source ↗
Figure 3
Figure 3. Figure 3: Code Snippet of echo-tool-1.0.1 also reflects JavaScript’s object-centric features, where in this example, functions are treated as objects and passed around. This tactic weakens detectors [15, 37, 49, 53] that rely on direct identifier resolution, illustrating L2. This example emphasizes the need for object-sensitive analysis to detect sensitive API calls and thus detect malicious behavior [PITH_FULL_IMA… view at source ↗
Figure 4
Figure 4. Figure 4: Approach Overview of ProfMalPlus These examples show that static and dynamic program analysis are necessary for modeling program behaviors, including obfuscated calls, object-centric aliases, and runtime traces. However, behavior modeling alone is insufficient for reliable semantic reasoning. Existing detectors often abstract source code into intermediate representations, including behavior graphs. While t… view at source ↗
Figure 5
Figure 5. Figure 5: Prompt Snippet for Malicious Shell Command Detection [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Behavior Graph of automation.samples_0.1.15 Then, we set the formal parameter toCall as an alias of 𝑜𝑏 𝑗𝑛𝑒𝑤, and update idMap accordingly (i.e., idMap[toCall] = objnew). At the function call toCall(), we derive the full name from 𝑜𝑏 𝑗𝑛𝑒𝑤 .𝑞𝑃𝑎𝑡ℎ, which is F = 𝑜𝑏 𝑗𝑜𝑠 .𝑞𝑁𝑎𝑚𝑒 + (.).𝑐𝑜𝑛𝑐𝑎𝑡[ℎ𝑜𝑠𝑡𝑛𝑎𝑚𝑒] = “os.hostname”. As it matches an entry in the sensitive API list, we mark the node as sensitive. In the subseque… view at source ↗
Figure 7
Figure 7. Figure 7: Code Slice Generated from the Code Snippet in Fig. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Prompt Snippet for the Local Behavior Judge Agent produced, or until the available evidence is exhausted. Once a package is flagged as malicious, the localization agent (Sec. 3.7) maps the malicious behavior back to concrete files and code slices. 3.4.1 Prior Analysis Context. A single LLM judgement over one code slice is often insufficient to draw a reliable conclu￾sion, because some suspicious calls may … view at source ↗
Figure 9
Figure 9. Figure 9: Prompt Snippet for the Verification Agent calls by node identifier and call type. This compact input lets the agent focus on the local behavior while still considering how the current slice is reached and what previous stages have already concluded. We instruct the LLM to first interpret the annotation format, so that it can associate each relevant statement with its node identifier and call type. We then … view at source ↗
Figure 10
Figure 10. Figure 10: Prompt Snippet for the Global Behavior Judge Agent still undetermined, the global behavior judge examines how multiple local behaviors under the same entry may jointly form a behavior chain. It therefore reasons over the verified local judgements, their node-grounded key evidence, and the execution-order relations induced by the CFG. This step is necessary because malicious packages may deliberately distr… view at source ↗
Figure 11
Figure 11. Figure 11: Prompt Snippet for the Router Agent concise description. These identifiers are used downstream to map the verdict back to concrete files and statements during localization. If the composed behavior remains suspicious but does not provide decisive evidence, the agent returns undetermined. In this case, it promotes only the specific nodes whose unresolved semantics prevent a definitive judgement, rather tha… view at source ↗
Figure 12
Figure 12. Figure 12: Prompt Snippet for Third-Party Trustworthiness Assessment agent returns an untrusted-third-party marker together with the node identifier, and the code slicer records this marker in the corresponding slice annotation; otherwise, the agent proceeds to the subsequent semantic inference. 3.5.2 Documentation Candidate Discovery. For a trusted module, the agent collects documentary evidence from the package.js… view at source ↗
Figure 13
Figure 13. Figure 13: Prompt Snippet for Evidence-Constrained Semantic Inference Finally, the agent returns the two-layer description together, and the code slicer locates the call statement by node identifier and appends the semantics as an inline annotation on the corresponding slice. 3.6 Behavior Graph Dynamic Augmentor The dynamic augmentor is invoked on demand by the router agent (Sec. 3.4.4) as the dynamic_augmentation a… view at source ↗
Figure 14
Figure 14. Figure 14: Router-Triggered Augmented BG of bitsoex_react-design-system_14.1.4 Prompt Snippet Task: You are a JavaScript cybersecurity analyst performing malicious code localization for a package that the pipeline has already judged malicious. Extract the source snippets that constitute the malicious behavior, copied verbatim from the provided code slices. Do not re-judge whether the package is malicious. Input: The… view at source ↗
Figure 15
Figure 15. Figure 15: Prompt Snippet for the Localization Agent 3.7 Localization Agent After the global behavior judge agent determines that an entry is malicious, ProfMalPlus invokes the localization agent to produce human-readable evidence. The goal of this agent is not to re-judge the entry, but to map the final malicious verdict back to the concrete source code that implements the behavior. We provide the agent with the fi… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 4 linked inside Pith

  1. [1]

    Yiheng Cao, Susheng Wu, Ruisi Wang, Bihuan Chen, Yiheng Huang, Chenhao Lu, Zhuotong Zhou, and Xin Peng. 2025. Recurring Vulnerability Detection: How Far Are We?. InProceedings of the 34th ACM SIGSOFT International Symposium on Software Testing and Analysis

    2025

  2. [2]

    2026.Qwen-3.6-Flash

    Alibaba Cloud. 2026.Qwen-3.6-Flash. Retrieved June 1, 2026 from https://www.alibabacloud.com/help/en/model-studio/vision-model

    2026

  3. [3]

    2022.GuardDog

    DataDog. 2022.GuardDog. Retrieved Jun 1, 2026 from https://github.com/DataDog/guarddog/tree/v3

    2022

  4. [4]

    2026.DeepSeek-V4-Flash

    deepseek ai. 2026.DeepSeek-V4-Flash. Retrieved June 1, 2026 from https://api-docs.deepseek.com/zh-cn/news/news260424?spm=a2c6h.13046898. publish-article.8.a25f6ffaiKyB6N

    2026

  5. [5]

    2013.Docker: Accelerated Container Application Development

    Docker. 2013.Docker: Accelerated Container Application Development. https://www.docker.com/

    2013

  6. [6]

    Ruian Duan, Omar Alrawi, Ranjita Pai Kasturi, Ryan Elder, Brendan Saltaformaggio, and Wenke Lee. 2020. Towards measuring supply chain attacks on package managers for interpreted languages. InProceedings of the 28th Annual Network and Distributed System Security Symposium

    2020

  7. [7]

    Xingan Gao, Xiaobing Sun, Sicong Cao, Kaifeng Huang, Di Wu, Xiaolei Liu, Xingwei Lin, and Yang Xiang. 2025. MALGUARD: towards real-time, accurate, and actionable detection of malicious packages in PyPI ecosystem. InProceedings of the 34th USENIX Conference on Security Symposium

    2025

  8. [8]

    Kalil Garrett, Gabriel Ferreira, Limin Jia, Joshua Sunshine, and Christian Kästner. 2019. Detecting suspicious package updates. InProceedings of the 41st International Conference on Software Engineering: New Ideas and Emerging Results. 13–16

    2019

  9. [9]

    Danielle Gonzalez, Thomas Zimmermann, Patrice Godefroid, and Max Schäfer. 2021. Anomalicious: Automated detection of anomalous and potentially malicious commits on github. InProceedings of the 43rd International Conference on Software Engineering: Software Engineering in Practice. 258–267

    2021

  10. [10]

    2018.A post-mortem of the malicious event-stream backdoor

    Danny Grander. 2018.A post-mortem of the malicious event-stream backdoor. Retrieved Jun 1, 2026 from https://snyk.io/blog/a-post-mortem-of-the- malicious-event-stream-backdoor/

    2018

  11. [11]

    Wenbo Guo, Shiwen Song, Jiaxun Guo, Zhengzi Xu, Chengwei Liu, Haoran Ou, Mengmeng Ge, and Yang Liu. 2026. Bridging expert reasoning and llm detection: A knowledge-driven framework for malicious packages. InProceedings of the 35th ACM International World Wide Web Conference. 3554–3565

    2026

  12. [12]

    Wenbo Guo, Zhengzi Xu, Chengwei Liu, Cheng Huang, Yong Fang, and Yang Liu. 2023. An Empirical Study of Malicious Code In PyPI Ecosystem. InProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering. 166–177

    2023

  13. [13]

    Sajal Halder, Michael Bewong, Arash Mahboubi, Yinhao Jiang, Md Rafiqul Islam, Md Zahid Islam, Ryan HL Ip, Muhammad Ejaz Ahmed, Gowri Sankar Ramachandran, and Muhammad Ali Babar. 2024. Malicious Package Detection using Metadata Information. InProceedings of the ACM on Web Conference. 1779–1789

    2024

  14. [14]

    Cheng Huang, Nannan Wang, Ziyan Wang, Siqi Sun, Lingzi Li, Junren Chen, Qianchong Zhao, Jiaxuan Han, Zhen Yang, and Lei Shi. 2024. DONAPI: Malicious NPM Packages Detector using Behavior Sequence Knowledge Mapping. InProceedings of the 33rd USENIX Security Symposium

    2024

  15. [15]

    Yiheng Huang, Ruisi Wang, Wen Zheng, Zhuotong Zhou, Susheng Wu, Shulin Ke, Bihuan Chen, Shan Gao, and Xin Peng. 2024. SpiderScan: Practical Detection of Malicious NPM Packages Based on Graph-Based Behavior Modeling and Matching. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 1146–1158

    2024

  16. [16]

    Yiheng Huang, Wen Zheng, Susheng Wu, Bihuan Chen, You Lu, Zhuotong Zhou, Yiheng Cao, Xiaoyu Li, and Xin Peng. 2025. ProfMal: Detecting Malicious NPM Packages by the Synergy between Static and Dynamic Analysis. InProceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering

    2025

  17. [17]

    Dhanushka Jayasuriya, Valerio Terragni, Jens Dietrich, Samuel Ou, and Kelly Blincoe. 2023. Understanding breaking changes in the wild. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1433–1444

    2023

  18. [18]

    2019.Joern - The Bug Hunter’s Workbench

    Joern. 2019.Joern - The Bug Hunter’s Workbench. Retrieved Jun 1, 2026 from https://joern.io/

    2019

  19. [19]

    Piergiorgio Ladisa, Henrik Plate, Matias Martinez, and Olivier Barais. 2023. Sok: Taxonomy of attacks on open-source software supply chains. In Proceedings of the 44th IEEE Symposium on Security and Privacy. 1509–1526

    2023

  20. [20]

    Piergiorgio Ladisa, Serena Elisa Ponta, Nicola Ronzoni, Matias Martinez, and Olivier Barais. 2023. On the Feasibility of Cross-Language Detection of Malicious Packages in npm and PyPI. InProceedings of the 39th Annual Computer Security Applications Conference. 71–82

    2023

  21. [21]

    Piergiorgio Ladisa, Merve Sahin, Serena Elisa Ponta, Marco Rosa, Matias Martinez, and Olivier Barais. 2023. The Hitchhiker’s Guide to Malicious Third-Party Dependencies. InProceedings of the 2023 Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses. 65–74

    2023

  22. [22]

    Ningke Li, Shenao Wang, Mingxi Feng, Kailong Wang, Meizhen Wang, and Haoyu Wang. 2023. MalWuKong: Towards Fast, Accurate, and Multilingual Detection of Malicious Code Poisoning in OSS Supply Chains. In2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1993–2005. Manuscript submitted to ACM 40 Huang et al

    2023

  23. [23]

    Wentao Liang, Xiang Ling, Chen Zhao, Jingzheng Wu, Tianyue Luo, and Yanjun Wu. 2025. Detecting Malicious Packages in PyPI and npm by Clustering Installation Scripts.IEEE Transactions on Software Engineering(2025)

    2025

  24. [24]

    2020.OSS Detect Backdoor

    Microsoft. 2020.OSS Detect Backdoor. Retrieved Jun 1, 2026 from https://github.com/microsoft/OSSGadget/wiki/OSS-Detect-Backdoor

    2020

  25. [25]

    2020.JavaScript/TypeScript static analyzer for call graph construction, library usage pattern matching, and vulnerability exposure analysis

    Anders Møller. 2020.JavaScript/TypeScript static analyzer for call graph construction, library usage pattern matching, and vulnerability exposure analysis. Retrieved Jun 1, 2026 from https://github.com/cs-au-dk/jelly

    2020

  26. [26]

    Thanh-Cong Nguyen, Duc-Ly Vu, and Narayan C Debnath. 2024. Classifying Benign and Malicious Open-Source Packages using Machine Learning based on Dynamic Features.International Journal of Computers and Their Applications(2024), 293

    2024

  27. [27]

    2014.Node.js documentation

    Node.js. 2014.Node.js documentation. Retrieved Jun 1, 2026 from https://nodejs.org/api/

    2014

  28. [28]

    2014.Node Package Manager (NPM)

    NPM. 2014.Node Package Manager (NPM). Retrieved Jun 1, 2026 from https://www.npmjs.com/

    2014

  29. [29]

    Marc Ohm, Felix Boes, Christian Bungartz, and Michael Meier. 2022. On the feasibility of supervised machine learning for the detection of malicious software packages. InProceedings of the 17th International Conference on A vailability, Reliability and Security. 1–10

    2022

  30. [30]

    2022.Towards Detection of Malicious Software Packages Through Code Reuse by Malevolent Actors

    Marc Ohm, Lukas Kempf, Felix Boes, and Michael Meier. 2022.Towards Detection of Malicious Software Packages Through Code Reuse by Malevolent Actors. Gesellschaft für Informatik, Bonn

    2022

  31. [31]

    Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. 2020. Backstabber’s knife collection: A review of open source software supply chain attacks. InProceedings of the 17th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. 23–43

    2020

  32. [32]

    Marc Ohm and Charlene Stuke. 2023. SoK: Practical Detection of Software Supply Chain Attacks. InProceedings of the 18th International Conference on A vailability, Reliability and Security. 1–11

    2023

  33. [33]

    2024.Rate limits

    OpenAI. 2024.Rate limits. Retrieved Jun 1, 2026 from https://platform.openai.com/docs/guides/rate-limits

    2024

  34. [34]

    2026.GPT-5.4-mini

    OpenAI. 2026.GPT-5.4-mini. Retrieved June 1, 2026 from https://openai.com/index/introducing-gpt-5-4-mini-and-nano/

    2026

  35. [35]

    Timo Pohl, Marc Ohm, Felix Boes, and Michael Meier. 2024. You Can Run But You Can’t Hide: Runtime Protection Against Malicious Package Updates For Node.js. InSicherheit, Schutz und Zuverlässigkeit: Konferenzband der 12. Jahrestagung des Fachbereichs Sicherheit der Gesellschaft für Informatik e.V. (GI), Sicherheit 2024, Worms, Germany, April 9-11, 2024. 231–241

    2024

  36. [36]

    Simone Scalco, Ranindya Paramitha, Duc-Ly Vu, and Fabio Massacci. 2022. On the feasibility of detecting injections in malicious npm packages. In Proceedings of the 17th International Conference on A vailability, Reliability and Security. 1–8

    2022

  37. [37]

    Adriana Sejfia and Max Schäfer. 2022. Practical automated detection of malicious npm packages. InProceedings of the 44th International Conference on Software Engineering. 1681–1692

    2022

  38. [38]

    Mikhail Shcherbakov, Musard Balliu, and Cristian-Alexandru Staicu. 2023. Silent spring: Prototype pollution leads to remote code execution in Node. js. InProceedings of the 32th USENIX Security Symposium. 5521–5538

    2023

  39. [39]

    Raphael J Sofaer, Yaniv David, Mingqing Kang, Jianjia Yu, Yinzhi Cao, Junfeng Yang, and Jason Nieh. 2024. RogueOne: Detecting Rogue Updates via Differential Data-flow Analysis Using Trust Domains. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering. 1–13

    2024

  40. [40]

    2024.2024 State of the Software Supply Chain

    Sonatype. 2024.2024 State of the Software Supply Chain. Retrieved Jun 1, 2026 from https://www.sonatype.com/state-of-the-software-supply- chain/introduction

    arXiv 2024

  41. [41]

    2019.Instrumentation framework for Node.js compliant to ECMAScript 2020 based on GraalVM.Retrieved Jun 1, 2026 from https: //github.com/Haiyang-Sun/nodeprof.js/

    Haiyang Sun. 2019.Instrumentation framework for Node.js compliant to ECMAScript 2020 based on GraalVM.Retrieved Jun 1, 2026 from https: //github.com/Haiyang-Sun/nodeprof.js/

    2019

  42. [42]

    Matthew Taylor, Ruturaj Vaidya, Drew Davidson, Lorenzo De Carli, and Vaibhav Rastogi. 2020. Defending against package typosquatting. In Proceedings of the 14th International Conference on Network and System Security. 112–131. [43]ProfMalPlus. 2026.ProfMalPlus. Retrieved June 20, 2026 from https://github.com/yiheng98/ProfMalPlus

    2020

  43. [44]

    Dmitrijs Trizna. 2021. Shell language processing: Unix command parsing for machine learning.arXiv preprint arXiv:2107.02438(2021)

    Pith/arXiv arXiv 2021

  44. [45]

    2023.Typosquatting campaign delivers r77 rootkit via npm

    Lucija Valentić. 2023.Typosquatting campaign delivers r77 rootkit via npm. Retrieved Jun 1, 2026 from https://www.reversinglabs.com/blog/r77- rootkit-typosquatting-npm-threat-research

    2023

  45. [46]

    Jian Wang, Zhen Li, Jixiang Qu, Deqing Zou, Shouhuai Xu, Ziteng Xu, Zhenwei Wang, and Hai Jin. 2025. Malpacdetector: An llm-based malicious npm package detector.IEEE transactions on information forensics and security(2025)

    2025

  46. [47]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. InProceedings of the 11th International Conference on Learning Representations

    2023

  47. [48]

    Elizabeth Wyss, Alexander Wittman, Drew Davidson, and Lorenzo De Carli. 2022. Wolf at the door: Preventing install-time attacks in npm with latch. InProceedings of the ACM on Asia Conference on Computer and Communications Security. 1139–1153

    2022

  48. [49]

    Zeliang Yu, Ming Wen, Xiaochen Guo, and Hai Jin. 2024. Maltracker: A fine-grained npm malware tracker copiloted by llm-enhanced dataset. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1759–1771

    2024

  49. [50]

    Nusrat Zahan, Philipp Burckhardt, Mikola Lysenko, Feross Aboukhadijeh, and Laurie Williams. 2024. MalwareBench: Malware samples are not enough. InProceedings of the 21st International Conference on Mining Software Repositories. 728–732

    2024

  50. [51]

    Williams

    Nusrat Zahan, Philipp Burckhardt, Mikola Lysenko, Feross Aboukhadijeh, and Laurie A. Williams. 2025. Leveraging Large Language Models to Detect npm Malicious Packages. InProceedings of the 47th International Conference on Software Engineering

    2025

  51. [52]

    Nusrat Zahan, Thomas Zimmermann, Patrice Godefroid, Brendan Murphy, Chandra Maddila, and Laurie Williams. 2022. What are weak links in the npm supply chain?. InProceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice. 331–340

    2022

  52. [53]

    Junan Zhang, Kaifeng Huang, Yiheng Huang, Bihuan Chen, Ruisi Wang, Chong Wang, and Xin Peng. 2024. Killing Two Birds with One Stone: Malicious Package Detection in NPM and PyPI using a Single Model of Malicious Behavior Sequence.ACM Transactions on Software Engineering Manuscript submitted to ACM ProfMalPlus: Agent-Coordinated Detection of Malicious NPM P...

    2024

  53. [54]

    Ying Zhang, Xiaoyan Zhou, Hui Wen, Wenjia Niu, Jiqiang Liu, Haining Wang, and Qiang Li. 2024. Tactics, Techniques, and Procedures (TTPs) in Interpreted Malware: A Zero-Shot Generation with Large Language Models.arXiv preprint arXiv:2407.08532(2024)

    Pith/arXiv arXiv 2024

  54. [55]

    Xinyi Zheng, Chen Wei, Shenao Wang, Yanjie Zhao, Peiming Gao, Yuanchao Zhang, Kailong Wang, and Haoyu Wang. 2024. Towards robust detection of open source software supply chain poisoning attacks in industry environments. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 1990–2001

    2024

  55. [56]

    Xiaoyan Zhou, Feiran Liang, Zhaojie Xie, Yang Lan, Wenjia Niu, Jiqiang Liu, Haining Wang, and Qiang Li. 2024. A Large-scale Fine-grained Analysis of Packages in Open-Source Software Ecosystems.arXiv preprint arXiv:2404.11467(2024)

    Pith/arXiv arXiv 2024

  56. [57]

    Xiaoyan Zhou, Ying Zhang, Wenjia Niu, Jiqiang Liu, Haining Wang, and Qiang Li. 2024. OSS Malicious Package Analysis in the Wild.arXiv preprint arXiv:2404.04991(2024). Manuscript submitted to ACM

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.