Security study based on the Chatgptplugin system: ldentifying Security Vulnerabilities

Ruomai Ren

REVIEW 5 major objections 5 minor 43 references

Security study based on the Chatgptplugin system: ldentifying Security Vulnerabilities

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ChatGPT plugin store exposes user data through public manifests and weak API authentication

desk verdict A sincere but unpolished measurement report whose central numbers don't hold up and whose main findings are already in a cited peer-reviewed paper. read the letter →

arxiv 2507.21128 v2 pith:NTQCQB52 submitted 2025-07-21 cs.CR cs.SE

Ruomai Ren This is my paper

classification cs.CRcs.SE

keywords ChatGPTpluginsecuritystorevulnerabilitiesmanifestleakageAPIauthorizationbypassOAuthscopeanalysistokenmetadataconsistency

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the ChatGPT plugin ecosystem has systemic security vulnerabilities that can expose user data to outsiders. It reports that, across 1,033 plugins, 373 manifest files were publicly reachable, 141 plugins returned valid data to requests carrying no token at all, and 24 plugins that required tokens still accepted invalid ones. It also finds that OAuth permissions often cover global, read-write, and identity scopes far beyond what a user would expect, and that many plugins have inconsistent or misleading names and legal information. If these findings hold, the plugin store, not just the underlying language model, becomes a practical attack surface for unauthorized data access.

What carries the argument

The central object is the three-layer consistency and security verification framework, a testing pipeline that the paper applies to every plugin it can reach. The Manifest Analysis Layer reconstructs candidate URLs from domain names and legal-info links to find files at the platform-required path /.well-known/ai-plugin.json. The API Request Analysis Layer sends simulated requests, with valid tokens, invalid tokens, or no token, to test whether endpoints distinguish legitimate platform traffic from outsiders. The Consistency and Integrity Analysis Layer compares the name_for_human, name_for_model, description, and legal_info_url fields across the store and the API to catch mismatches that can mislead users or misdirect model calls.

What would settle it

A concrete check would be to take a random sample of the 373 leaked manifests, record whether any contain secrets such as bearer tokens or API keys, and then send requests to the 141 no-token endpoints to see whether the returned data includes user-specific or sensitive information; if the manifests contain no secrets and the open endpoints return only nonsensitive public data, the paper's central claim about data exposure would not be supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that the ChatGPT plugin store has eight security and consistency issues: manifest file leakage, API request non-token interaction, API request authorization bypass, token leakage, misleading operation, malicious spoofing, privilege and model abuse, and misleading legal information. Evidence comes from a three-layer scan of the store: 373 of 1,033 plugins had manifest files accessible on public URLs, 141 plugins answered unauthenticated requests and returned valid data, and 24 plugins with token requirements still accepted invalid tokens. The OAuth scope analysis shows permissions such as global access, read-write, and identity/email access, with at least 12.7% of OAuth plugins requesting global access. After the issues were reported to the platform, a re-test found reductions of roughly 23-40% across file leakage, inconsistent data, and authentication failures, but the remaining counts still show the weaknesses persist.

Load-bearing premise

The load-bearing premise is that a manifest file being publicly reachable at the platform-required path is itself a security vulnerability, even though the ChatGPT platform requires plugins to host that file publicly so the store can discover them.

Editorial extensions

If this is right

An attacker who knows a plugin's domain can retrieve its manifest, learn its API endpoints, and construct requests that never pass through the ChatGPT platform.
The 141 plugins that answer unauthenticated requests mean any data those endpoints serve is reachable by anyone on the internet, so the store's intended user flow is not actually enforced.
The 24 plugins that accept invalid tokens show that a token check alone is not reliable access control, so developers and the platform need to enforce server-side validation.
The OAuth scope findings imply that a single user authorization can grant global, read-write, or identity permissions, which compounds the risk if any of those tokens leaks.
The post-reporting reductions suggest that disclosure and store redesign can mitigate some issues, but the remaining counts indicate the underlying weaknesses are not fully resolved.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's counts treat public manifests as leaks, yet the platform requires plugins to host manifests publicly; the real harm likely comes from secrets embedded in those manifests, and a follow-up study could quantify how many of the 373 manifests actually contain bearer tokens or API keys.
The 141 no-token endpoints may overstate risk if they are intentionally public APIs returning nonsensitive data; the paper never assesses the sensitivity of the data returned, so the severity of those endpoints remains unknown.
The before-after comparison is observational, not controlled; the reductions could be due to the store redesign, the disclosure, unrelated developer changes, or a combination, so the paper does not establish a causal link between reporting and improvement.
The consistency findings point to a broader design issue: when the model-facing name and the human-facing name diverge, the LLM can invoke the wrong plugin, which is a class of failure adjacent to prompt injection that the paper does not explore.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is not a new result and the numbers don't survive contact with the paper's own text. The abstract says some findings are already in the peer-reviewed ASE'24 paper [1], and the central 'manifest leakage' count treats the platform-required public manifest at /.well-known/ai-plugin.json as a vulnerability even though Section 2.2.2 says OpenAI requires exactly that path for discovery. That breaks the headline claim.

What is worth keeping: the authors did real crawling and probing of the pre-GPTs plugin store. The MixerBox case (17 plugins sharing one manifest and name_for_model) is a concrete, believable consistency problem. The OAuth scope breakdown in Section 4.2.5, whatever its methodology, shows that many plugins asked for broad read/write or offline-access scopes, which is a genuine user-facing concern. And the paper is honest in its limitations section about HTTP error ambiguity and single-request testing.

The soft spots are not minor. Table 4.1 and the text disagree on the dataset size (1033 vs 373 vs 368), the no-token percentage appears as 82.1% while the counts say 69.7%, and the percentages in Table 4.3 don't sum. The before/after comparison in Chapter 5 lacks any control: OpenAI changed the whole store UI and plugin model at the same time, so attributing the drop to the authors' report is not justified. No code or dataset is shipped, so the counts cannot be checked. Public manifests can be dangerous if they carry bearer tokens, but the paper never reports how many of the 373 accessible manifests actually contained secrets, which is the one number that would support the claim.

Who should read it: someone doing a literature review of ChatGPT plugin security might skim it, but they should cite [1] for the established findings and treat this preprint as an unpublished appendix. It is not ready for peer review as is; the internal inconsistencies alone would make refereeing wasteful. If the authors re-ran the analysis counting secret-bearing manifests and released the data, it could be a different paper.

Referee Report

5 major / 5 minor

Summary. The paper reports an empirical security study of the ChatGPT plugin store. The authors crawled plugin metadata, located manifest files at /.well-known/ai-plugin.json paths, probed plugin APIs with and without tokens, and inspected metadata consistency. They claim to identify eight security and consistency issues, including manifest file leakage, no-token API access, authorization bypass, token leakage, misleading operation, malicious spoofing, privilege and model abuse, and misleading legal information. They also present a before/after comparison around the GPTs store transition and after reporting issues to OpenAI, claiming that exposures dropped at all levels.

Significance. The topic is timely and the paper addresses an understudied part of the LLM ecosystem: the security of the plugin/extension layer rather than the model itself. The three-layer framework and the two case studies (MixerBox shared manifests in §4.3.3, OAuth scope analysis in §4.2.5) are potentially useful material, and the authors explicitly acknowledge some measurement limitations in §6.3. However, the paper's central quantitative claims are supported by internally inconsistent counts and by a leakage definition that does not account for the platform's design requirement that manifests be publicly hosted. The before/after comparison in §5.3 has no control and is not even defined on consistent numbers. The authors do not provide a reproducible dataset, scripts, or machine-checked derivations, so the empirical foundation cannot be independently verified from the manuscript. As written, the paper does not establish the claimed vulnerabilities.

major comments (5)

[§4.1.1, Table 4.1, §3.1.1.3] The empirical counts are internally inconsistent. Table 4.1 lists 373 accessible manifests, 104 hidden redirects, 12 OpenAI addresses, 6 Google Doc links, 19 GitHub links, and 518 native unreachable URLs, while §4.1.1 separately reports 37 non-valid domains (19+12+6) and 104 hidden redirects. The total dataset size is given as 1033 in §3.1.1.3, but the file-leakage count is reported as 368 in Table 5.1 and as 373 in §4.1.1, §4.2.2, and §4.2.4 discussions. The percentage statements are also incompatible: 104 hidden redirects is called 10.0%, 19 GitHub addresses 1.8%, and 6 Google Doc addresses 0.5%, which imply different denominators (approximately 1040, 1056, and 1200, respectively). The paper must reconcile these numbers and state a single denominator for every reported percentage. Without this, Tables 4.1, 4.2, 4.3, and 5.1 cannot be interpreted.
[§2.2.2, §3.1.1.1, Table 4.1] The central premise that a publicly accessible manifest is a security vulnerability is not established. The paper itself states in §2.2.2 that OpenAI's official documentation requires each plugin to host its manifest at /.well-known/ai-plugin.json so the platform can discover and invoke the plugin. Public accessibility at that exact path is therefore the intended configuration, not an unauthorized exposure. The methodology in §3.1.1.1 says that 'Unauthorised access attempts or public access by non-platforms should be identified and denied,' but no evidence is provided that the 373 accessible manifests were intended to be private or that they exposed content beyond what the platform requires. What would make manifest accessibility a vulnerability is the presence of secrets such as long-lived bearer tokens, private endpoints, or other sensitive data; §4.2.4 mentions such tokens but gives no count of how many of the 373 accessible manifests actually contained them. The manifest-leakage claim needs a reanalysis that separates 'public by design' from 'public and dangerous'.
[§4.2.4, Table 4.3] The token-type statistics are numerically impossible. The text reports 239 plugins 'without authentication' (82.1%), 70 plugins using OAuth (24.4%), and 34 plugins using Bearer Token (11.8%); these percentages sum to 118.3%, and the counts sum to 343, which does not equal the stated totals of 373 or 368. The adjacent case counts in the same section are 141 (no-token success), 27 (token-verification success), and 5 (authentication-requirement success), which do not correspond to the 239/70/34 breakdown. Additionally, Table 4.3 reports 173 successful and 159 failed requests for a total of 332, while §4.2.2 states 173 successes and 172 failures. The paper must provide a consistent classification of token types, a clear denominator, and counts that sum correctly; otherwise the 'no-token interaction' and 'authorization bypass' claims are not supported.
[§4.2.1, Table 4.2] The case definitions in the API request analysis are self-contradictory and mislabeled. Case 2 is described as 'token is required but invalid, you can access the API directly' and is assigned O=0 (failure), while Case 3 is described as 'Token is required, and the token is invalid, you can access the API directly' and is assigned O=1 (success); the two verbal descriptions are essentially identical but the outcomes differ. Case 1, 'token required and token is valid' with a successful request, is labeled 'High risk of data leakage,' yet a successful request with a valid token is the normal authenticated behavior, not an unauthorized access. This labeling contradicts the evaluation criterion in §3.1.2.2, which is about unauthenticated requests receiving the same results as authenticated ones. The analysis needs to distinguish among (a) a request from the ChatGPT platform with a valid token, (b) an external request with a token copied from a leaked manifest, and (c) an external request with no token.
[§5.3, Table 5.1] The claim that the decrease in exposures was caused by the authors' report is not supported. The before/after comparison in Table 5.1 has no control group, no significance test, no confidence intervals, and no discussion of confounding factors, most importantly the transition from the plugin store to the GPTs Explorer during the same period. The first assessment's file-leakage count is given as 368 in Table 5.1 but as 373 elsewhere in the paper, so the comparison is not defined on the same numbers. To support the causal claim, the authors would need matched per-plugin outcomes across the two periods, with reasons for each plugin's absence or presence in both measurements, and an account of platform-side changes independent of their report.

minor comments (5)

[Title and throughout] The title contains a typo ('ldentifying' should be 'Identifying'), and the paper uses inconsistent capitalization for ChatGPT ('Chatgpt', 'CHATGPT', 'chatgpt') throughout; the manuscript should be copy-edited.
[§4.2.3] The failure-cause counts of 58 (lack authorization), 66 (client errors), and 49 (rate limiting) sum to 173, which matches the total failures in one sentence but not the 159 failures reported in Table 4.3; the discrepancy should be reconciled and explained.
[§3.1.1.2, §3.1.2.3, §3.1.3.3] The mathematical formulations are largely tautological: defining E as the set of files whose access is unauthorized and P as S\E adds no measurable content, and the statement about denial of all unauthorized requests is definitional rather than an empirical claim. These sections should either be removed or replaced with operational definitions of 'unauthorized,' 'valid,' and 'secure' that can be checked against the data.
[§5.3] The phrase 'Almost at the same time, we reported the above leak to OpenAI' gives no dates, no report identifier, and no description of the feedback channel; the timeline of data collection, reporting, and the GPTs store migration should be stated precisely.
[References] Reference [24] and [25] are the same paper, and the abstract's claim that findings were 'incorporated into academic publications [1]' is self-referential without explaining what was incorporated; this should be clarified or removed.

Circularity Check

1 steps flagged · score 4.0 of 10

The 'Manifest File Leakage' finding is largely a relabeling of the platform-required public /.well-known/ai-plugin.json path; the other empirical findings retain independent content.

self definitional [Sections 2.2.2, 2.3.1, 3.1.1.1-3.1.1.3, 4.1.1, Table 4.1]
"According to the official OPENAI documentation, each plugin should contain a json file hosted on the API's domain to connect to the plugin platform. The plugin platform looks for this file under the path ”/.well-known/ai-plugin.json” or ”/.well-known” ... Accessible Files: Through an exhaustive exploration of manifest file paths, we found 373 valid manifest files. These manifest files can display the relevant OpenAI manifest documents ... Manifest File Leakage: Manifest files are exposed on public URLs, resulting in the leakage of model, API, and legal information."

The formal definition in §3.1.1.2 says E={s∈S|access to s is unauthorized}, but the operational test only checks reachability at /.well-known/ai-plugin.json, the exact path OpenAI requires for plugin discovery. Any plugin following the platform's hosting requirement therefore satisfies the measured 'accessible files' condition. Labeling this 'Manifest File Leakage' makes the conclusion (leakage) true by construction from the selection criterion (public URL exposure), rather than from evidence that access was unauthorized or that sensitive material was disclosed. The 373 count in Table 4.1 thus conflates a required design feature with a security vulnerability.

full rationale

Apart from the manifest-leakage label, the paper is an empirical crawl rather than a derivation: there are no fitted parameters, no forward model, and no prediction whose output is forced by an equation. The API-layer findings (141 no-token successes, 24 invalid-token successes, OAuth scope distribution) and the consistency-layer findings (69 metadata inconsistencies, MixerBox shared manifolds) are independent measurements from live endpoints and are not equivalent to the paper's inputs. The abstract's statement that some findings were 'incorporated into academic publications [1]' is a self-citation, but it is not load-bearing: [1] is not used to justify any central conclusion, and the overlap does not by itself make the measurements circular. The §5.3 claim that security improvements followed the authors' report to OpenAI is a self-referential causality attribution without a control group, which is a validity limitation rather than a circular derivation. Weighing the one definitional conflation against the substantial independent empirical content gives a partial-circularity score of 4.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities. The study relies on domain assumptions about the meaning of public manifests and unauthenticated responses, plus a causal assumption about the effect of reporting.

assumptions (4)

domain assumption Manifest files are required to be publicly accessible at /.well-known/ai-plugin.json, and public accessibility is itself a security vulnerability.
The paper treats public manifest accessibility as 'leakage' (Section 2.3.1, Section 4.1.1) even though OpenAI's documentation says the manifest must be hosted at that public path for the platform to discover the plugin (Section 2.2.2).
domain assumption API responses to unauthenticated or invalid-token requests are vulnerabilities without considering whether the data is sensitive or intended to be public.
The evaluation criteria in Section 3.1.2.2 state that any response to an unauthorized request is a risk, but the paper does not assess the sensitivity of the returned data.
domain assumption The crawled set of 1033 plugins is complete and representative of the store.
The paper collects all plugins visible in the store at the time, but does not discuss coverage, changes over time, or the selection bias introduced by the store's pagination and search.
ad hoc to paper The reduction in exposed data after reporting is caused by the report.
Section 5.3 compares first assessment and revisit numbers and attributes the change to reporting to OpenAI, without a control group or alternative explanations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Security study based on the Chatgptplugin system: ldentifying Security Vulnerabilities." pith.science (2026). https://pith.science/paper/NTQCQB52

@misc{pith2026250721128,
  author       = {Pith},
  title        = {Pith review of: Security study based on the Chatgptplugin system: ldentifying Security Vulnerabilities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NTQCQB52}},
  note         = {Machine review of arXiv:2507.21128}
}

read the original abstract

Plugin systems are a class of external programmes that provide users with a wide range of functionality, and while they enhance the user experience, their security is always a challenge. Especially due to the diversity and complexity of developers, many plugin systems lack adequate regulation. As ChatGPT has become a popular large-scale language modelling platform, its plugin system is also gradually developing, and the open platform provides creators with the opportunity to upload plugins covering a wide range of application scenarios. However, current research and discussions mostly focus on the security issues of the ChatGPT model itself, while ignoring the possible security risks posed by the plugin system. This study aims to analyse the security of plugins in the ChatGPT plugin shop, reveal its major security vulnerabilities, and propose corresponding improvements.

Figures

Figures reproduced from arXiv: 2507.21128 by the authors.

**Figure 2.3.** Notion [PITH_FULL_IMAGE:figures/full_fig_p011_2_3.png] view at source ↗

**Figure 2.5.** Workflow of ChatGPT plugin 13 [PITH_FULL_IMAGE:figures/full_fig_p014_2_5.png] view at source ↗

**Figure 2.6.** Manifest Sample in Openai[2] According to the official OPENAI documentation, each plugin should contain a json file hosted on the API’s domain to connect to the plugin platform. The plugin platform looks for this file under the path ”/.well-known/ai-plugin.json” or ”/.well-known”. This JSON file has a set of well-defined minimum fields[2][37]. In this manifest file, the plugin developer is asked to provide at least … view at source ↗

Figures from the paper (11 more)

**Figure 2.7.** Figure 2.7: Bearer token example [PITH_FULL_IMAGE:figures/full_fig_p016_2_7.png]

**Figure 2.8.** Figure 2.8: OAuth token example • OAuth: Another widely used authentication method, allowing users to sign in through third-party services such as Google or Facebook. OAuth provides a 15 [PITH_FULL_IMAGE:figures/full_fig_p016_2_8.png]

**Figure 2.9.** Figure 2.9: Openapi Sample[2] 2.2.3.1 The top-level structure of the API file: openapi, info, servers, paths and components, where the openapi field specifies the OpenAPI version to ensure the standardisation of the file format, the info field provides basic information about th…

**Figure 3.1.** Figure 3.1: Security Checking Model 3.1.1 Manifest Analysis Layer 3.1.1.1 The theory of Manifest Analysis Layer Manifest file exposure, this dimension we mainly aim to is to identify potential information leakage, especially the leakage of configuration file manifest file to de…

**Figure 3.2.** Figure 3.2: Steps for detecting leaks of manifest file [PITH_FULL_IMAGE:figures/full_fig_p025_3_2.png]

**Figure 4.1.** Figure 4.1: Limitations Due to Missing or Invalid External Website Links [PITH_FULL_IMAGE:figures/full_fig_p032_4_1.png]

**Figure 4.2.** Figure 4.2: Plugin Token Authentication 4.2.5 Case Study 1: Severity Analysis of Unauthorized Requests in OAuth-Authorized Plugins In order to more effectively assess the security impact of external API leakage, this section will conduct a case study. Based on Layer 2, we will …

**Figure 4.3.** Figure 4.3: OAuth Token Permissions Distribution Detailed Subcategories [PITH_FULL_IMAGE:figures/full_fig_p043_4_3.png]

**Figure 4.4.** Figure 4.4: MixerBox plugins list [PITH_FULL_IMAGE:figures/full_fig_p046_4_4.png]

**Figure 4.5.** Figure 4.5: MixerBox shared manifest file In this case study, we found that 17 plugins under the ‘MixerBox’ brand share the same description and manifest files, and this sharing strategy not only brings problems 45 [PITH_FULL_IMAGE:figures/full_fig_p046_4_5.png]

**Figure 5.1.** Figure 5.1: New GPTs Explorer UI of references, which solves the malicious competition and recommendation to a certain extent. At the same time, by improving the relevance, the store can more accurately push the tools to the users who really need them. However, the recommendatio…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 35 canonical work pages

[1]

Exploring chatgpt app ecosystem: Distribution, deployment and security,

C. Yan, R. Ren, M. H. Meng, L. Wan, T. Y. Ooi, and G. Bai, “Exploring chatgpt app ecosystem: Distribution, deployment and security,” inProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 1370–1382. [Online]. Available: https://doi.org/10...

arXiv 2024
[2]

(2023) Getting start

OPENAI. (2023) Getting start. [Online]. Available: https://platform.openai.com/ docs/plugins/getting-started

work page 2023
[3]

Hardening the security analysis of browser extensions,

B. Eriksson, P. Picazo-Sanchez, and A. Sabelfeld, “Hardening the security analysis of browser extensions,” in37TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING, ACM. NEW YORK: Assoc Computing Machinery, 2022, pp. 1694–1703

work page 2022
[4]

(2023) Chat plugin

OPENAI. (2023) Chat plugin. [Online]. Available: https://platform.openai.com/ docs/plugins/introduction/chat-plugins-beta

work page 2023
[5]

Exploit- ing programmatic behavior of llms: Dual-use through standard security attacks,

D. Kang, X. Li, I. Stoica, C. Guestrin, M. Zaharia, and T. Hashimoto, “Exploit- ing programmatic behavior of llms: Dual-use through standard security attacks,” arXiv.org, 2023

work page 2023
[6]

Multi-step jailbreaking privacy attacks on chatgpt,

H. Li, D. Guo, W. Fan, M. Xu, J. Huang, F. Meng, and Y. Song, “Multi-step jailbreaking privacy attacks on chatgpt,”arXiv (Cornell University), 2023

work page 2023
[7]

Tracking GPTs Third Party Service: Automation, Analysis, and Insights

C. Yan, L. Wan, B. Guan, F. Yu, G. Bai, and J. S. Dong, “Tracking gpts third party service: Automation, analysis, and insights,” 2025. [Online]. Available: https://arxiv.org/abs/2506.17315

work page Pith review arXiv 2025
[9]

Investigating documented privacy changes in android os,

C. Yan, M. H. Meng, F. Xie, and G. Bai, “Investigating documented privacy changes in android os,”Proc. ACM Softw. Eng., vol. 1, no. FSE, Jul. 2024. [Online]. Available: https://doi.org/10.1145/3660826

work page doi:10.1145/3660826 2024

Show all 43 references

[10]

Are your requests your true needs? checking excessive data collection in vpa app,

F. Xie, C. Yan, M. H. Meng, S. Teng, Y. Zhang, and G. Bai, “Are your requests your true needs? checking excessive data collection in vpa app,” inProceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24. New York, NY, USA: Association for...

2024
[11]

On the quality of privacy pol- icy documents of virtual personal assistant applications,

C. Yan, F. Xie, M. H. Meng, Y. Zhang, and G. Bai, “On the quality of privacy pol- icy documents of virtual personal assistant applications,”Proceedings on Privacy Enhancing Technologies, 2024

2024
[12]

How chatgpt assists novices in human activity recog- nition,

K. Kazama and M. Shuzo, “How chatgpt assists novices in human activity recog- nition,” inCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’24). ACM, 2024, pp. 575–579

2024
[13]

HOT chatgpt: The promise of chatgpt in detecting and discriminating hateful, offensive, and toxic comments on social media,

L. Li, L. Fan, S. Atreja, and L. Hemphill, “HOT chatgpt: The promise of chatgpt in detecting and discriminating hateful, offensive, and toxic comments on social media,”ACM Transactions on the Web, vol. 18, no. 2, pp. 1–36, 2024

2024
[14]

Chatbr: Automated assess- ment and improvement of bug report quality using chatgpt,

L. Bo, W. Ji, X. Sun, T. Zhang, X. Wu, and Y. Wei, “Chatbr: Automated assess- ment and improvement of bug report quality using chatgpt,” inProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering (ASE ’24). ACM, 2024, pp. 1472–1483

2024
[15]

Prompt-enhanced software vulnerability detection using chatgpt,

C. Zhang, H. Liu, J. Zeng, K. Yang, Y. Li, and H. Li, “Prompt-enhanced software vulnerability detection using chatgpt,” inProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion ’24). ACM, 2024, pp. 276–277

2024
[16]

Effectiveness of chatgpt for static analysis: How far are we?

M. M. Mohajer, R. Aleithan, N. S. Harzevili, M. Wei, A. B. Belle, H. V. Pham, and S. Wang, “Effectiveness of chatgpt for static analysis: How far are we?” inProceed- ings of the 1st ACM International Conference on AI-Powered Software (AIware 2024). ACM, 2024, pp. 151–160

2024
[17]

From chatgpt to threatgpt: Impact of generative ai in cybersecurity and privacy,

M. Gupta, C. Akiri, K. Aryal, E. Parker, and L. Praharaj, “From chatgpt to threatgpt: Impact of generative ai in cybersecurity and privacy,”IEEE access, vol. 11, pp. 80 218–80 245, 2023. 56

2023
[18]

Chatgpt hallucinates when attributing answers,

G. Zuccon, B. Koopman, and R. Shaik, “Chatgpt hallucinates when attributing answers,” inProceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP ’23). ACM, 2023, pp. 46–51

2023
[19]

Chatgpt for cybersecurity: prac- tical applications, challenges, and future directions,

M. Al-Hawawreh, A. Aljuhani, and Y. Jararweh, “Chatgpt for cybersecurity: prac- tical applications, challenges, and future directions,”Cluster computing, vol. 26, no. 6, pp. 3421–3436, 2023

2023
[20]

A new era of cybersecurity with ai: Predictions for 2024,

“A new era of cybersecurity with ai: Predictions for 2024,”CIO, 2024

2024
[21]

On the use of chatgpt for code review: Do developers like reviews by chatgpt?

M. Watanabe, Y. Kashiwa, B. Lin, T. Hirao, K. Yamaguchi, and H. Iida, “On the use of chatgpt for code review: Do developers like reviews by chatgpt?” in Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering (EASE ’24), 2024, pp. ...

2024
[22]

Detecting code smells using chatgpt: Initial insights,

L. L. Silva, J. R. da Silva, J. E. Montandon, M. Andrade, and M. T. Valente, “Detecting code smells using chatgpt: Initial insights,” inProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM ’24), 2024, pp. 400–406. [On...

2024
[23]

Refining chatgpt-generated code: Characterizing and mitigating code quality issues,

Y. Liu, T. Le-Cong, R. Widyasari, C. Tantithamthavorn, L. Li, X.-B. D. Le, and D. Lo, “Refining chatgpt-generated code: Characterizing and mitigating code quality issues,”ACM Transactions on Software Engineering and Methodology, vol. 33, no. 5, pp. 1–26, 2024. [Online]. Availa...

2024 doi
[25]

Ai model modulation with logits redistribution,

——, “Ai model modulation with logits redistribution,” inProceedings of the ACM on Web Conference 2025, ser. WWW ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 4699–4709. [Online]. Available: https://doi.org/10.1145/3696410.3714737

2025
[26]

Data hiding with deep learning: A survey unifying digital watermarking and steganography,

Z. Wang, O. Byrnes, H. Wang, R. Sun, C. Ma, H. Chen, Q. Wu, and M. Xue, “Data hiding with deep learning: A survey unifying digital watermarking and steganography,” 2023. [Online]. Available: https://arxiv.org/abs/2107.09287 57

2023 arXiv
[27]

Investigating documented privacy changes in android os,

C. Yan, M. H. Meng, F. Xie, and G. Bai, “Investigating documented privacy changes in android os,” inProceedings of the ACM on Software Engineering, vol. 1, no. FSE, 2024, pp. 2701–2724

2024
[28]

Wild extensions: Discov- ering and analyzing unlisted chrome extensions,

R. Perdisci, C. Maurice, G. Giacinto, and M. Almgren, “Wild extensions: Discov- ering and analyzing unlisted chrome extensions,” inDETECTION OF INTRU- SIONS AND MAL WARE, AND VULNERABILITY ASSESSMENT (DIMV A 2019), ser. Lecture Notes in Computer Science. Switzerland: Springer ...

2019
[29]

Many chrome browser extensions do sneaky things,

J. Kirk, “Many chrome browser extensions do sneaky things,”Networks Asia, 2014

2014
[30]

Malicious browser extensions: A grow- ing threat: A case study on google chrome: Ongoing work in progress,

G. Varshney, S. Bagade, and S. Sinha, “Malicious browser extensions: A grow- ing threat: A case study on google chrome: Ongoing work in progress,” in2018 International Conference on Information Networking (ICOIN). IEEE, 2018, pp. 188–193

2018
[31]

Experimental security analysis of sensitive data access by browser extensions,

A. Nayak, R. Khandelwal, E. Fernandes, and K. Fawaz, “Experimental security analysis of sensitive data access by browser extensions,” inProceedings of the ACM Web Conference 2024 (WWW ’24), 2024, pp. 1283–1294. [Online]. Available: https://doi.org/10.1145/3589334.3645683

2024
[32]

Extended tracking powers: Measuring the privacy diffusion enabled by browser extensions,

O. Starov and N. Nikiforakis, “Extended tracking powers: Measuring the privacy diffusion enabled by browser extensions,” inProceedings of the 26th International Conference on World Wide Web (WWW ’17), 2017, pp. 1481–1490. [Online]. Available: https://doi.org/10.1145/3038912.3052596

2017
[33]

What is in the chrome web store?

S. Hsu, M. Tran, and A. Fass, “What is in the chrome web store?” inProceedings of the 19th ACM Asia Conference on Computer and Communications Security (ASIA CCS ’24). New York, NY, USA: ACM, July 2024, pp. 785–798. [Online]. Available: https://doi.org/10.1145/3634737.3637636

2024
[34]

Notion Labs,Notion API Documentation, 2023, accessed: 2023-10-30

I. Notion Labs,Notion API Documentation, 2023, accessed: 2023-10-30. [Online]. Available: https://developers.notion.com/

2023
[35]

M. D. N. (MDN),Mozilla Firefox Extension Developer Documentation, 2023, accessed: 2023-10-30. [Online]. Available: https://developer.mozilla.org/en-US/ docs/Mozilla/Add-ons/WebExtensions

2023
[36]

[Online]

Google,Google Chrome Extension Developer Documentation, 2023, accessed: 2023-10-30. [Online]. Available: https://developer.chrome.com/docs/extensions/ 58

2023
[37]

(2023) Plugins authentication

OPENAI. (2023) Plugins authentication. [Online]. Available: https://platform. openai.com/docs/plugins/authentication

2023
[38]

Malware detection in web browser plugins using api calls with permissions,

M. S. Hossen, R. Islam, M. N. U. R. Chowdhury, A. Haque, and Q. E. A. Ratul, “Malware detection in web browser plugins using api calls with permissions,”In- ternational journal of advanced networking and applications, vol. 14, no. 6, pp. 5672–5677, 2023

2023
[39]

(2023) Plugin review process

OPENAI. (2023) Plugin review process. [Online]. Available: https://platform. openai.com/docs/plugins/review/plugin-review-process

2023
[40]

(2023) Plugins and actions terms

——. (2023) Plugins and actions terms. [Online]. Available: https://openai.com/ policies/plugin-terms

2023
[41]

[Online]

Beautiful Soup,Beautiful Soup Documentation, 2020. [Online]. Available: https://www.crummy.com/software/BeautifulSoup/bs4/doc/

2020
[42]

Selenium webdriver,

S. Stewart and S. Developers, “Selenium webdriver,” https://www.selenium.dev, 2024, version 4. [Online]. Available: https://www.selenium.dev

2024
[43]

Webmea: A google chrome extension for web security and privacy measurement studies,

M. Ren, J. Josey, and C. Yue, “Webmea: A google chrome extension for web security and privacy measurement studies,” inLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2023, vol. 14299, pp. 305–318

2023
[44]

Fakex: A framework for detecting fake reviews of browser extensions,

E. Olsson, B. Eriksson, P. Picazo-Sanchez, L. Andersson, and A. Sabelfeld, “Fakex: A framework for detecting fake reviews of browser extensions,” inProceedings of the 19th ACM Asia Conference on Computer and Communications Security (ASIA CCS ’24), 2024, pp. 769–784. [Online]. ...

2024
[45]

Collaborative verification of information flow for a high-assurance app store,

M. D. Ernst, R. Just, S. Millstein, W. Dietl, S. Pernsteiner, F. Roesner, K. Koscher, P. B. Barros, R. Bhoraskar, S. Han, P. Vines, and E. X. Wu, “Collaborative verification of information flow for a high-assurance app store,” inProceedings of the 2014 ACM SIGSAC Conference on...

2014

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Exploring chatgpt app ecosystem: Distribution, deployment and security,

C. Yan, R. Ren, M. H. Meng, L. Wan, T. Y. Ooi, and G. Bai, “Exploring chatgpt app ecosystem: Distribution, deployment and security,” inProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 1370–1382. [Online]. Available: https://doi.org/10...

arXiv 2024

[2] [2]

(2023) Getting start

OPENAI. (2023) Getting start. [Online]. Available: https://platform.openai.com/ docs/plugins/getting-started

work page 2023

[3] [3]

Hardening the security analysis of browser extensions,

B. Eriksson, P. Picazo-Sanchez, and A. Sabelfeld, “Hardening the security analysis of browser extensions,” in37TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING, ACM. NEW YORK: Assoc Computing Machinery, 2022, pp. 1694–1703

work page 2022

[4] [4]

(2023) Chat plugin

OPENAI. (2023) Chat plugin. [Online]. Available: https://platform.openai.com/ docs/plugins/introduction/chat-plugins-beta

work page 2023

[5] [5]

Exploit- ing programmatic behavior of llms: Dual-use through standard security attacks,

D. Kang, X. Li, I. Stoica, C. Guestrin, M. Zaharia, and T. Hashimoto, “Exploit- ing programmatic behavior of llms: Dual-use through standard security attacks,” arXiv.org, 2023

work page 2023

[6] [6]

Multi-step jailbreaking privacy attacks on chatgpt,

H. Li, D. Guo, W. Fan, M. Xu, J. Huang, F. Meng, and Y. Song, “Multi-step jailbreaking privacy attacks on chatgpt,”arXiv (Cornell University), 2023

work page 2023

[7] [7]

Tracking GPTs Third Party Service: Automation, Analysis, and Insights

C. Yan, L. Wan, B. Guan, F. Yu, G. Bai, and J. S. Dong, “Tracking gpts third party service: Automation, analysis, and insights,” 2025. [Online]. Available: https://arxiv.org/abs/2506.17315

work page Pith review arXiv 2025

[8] [9]

Investigating documented privacy changes in android os,

C. Yan, M. H. Meng, F. Xie, and G. Bai, “Investigating documented privacy changes in android os,”Proc. ACM Softw. Eng., vol. 1, no. FSE, Jul. 2024. [Online]. Available: https://doi.org/10.1145/3660826

work page doi:10.1145/3660826 2024