arxiv: 2602.08072 · v3 · submitted 2026-02-08 · 💻 cs.CR · cs.SE

Recognition: no theorem link

IssueGuard: Real-Time Secret Leak Prevention Tool for GitHub Issue Reports

Md Nafiu Rahman , Sadif Ahmed , Zahin Wahab , Gias Uddin , Rifat Shahriyar

Authors on Pith no claims yet

Pith reviewed 2026-05-16 05:48 UTC · model grok-4.3

classification 💻 cs.CR cs.SE

keywords secret detectionGitHub issuesCodeBERTreal-time preventionChrome extensionAPI key leakagefalse positive reductionissue report security

0 comments

The pith

A browser extension detects and blocks secret leaks in GitHub issue reports before submission by pairing regex extraction with contextual CodeBERT classification.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents IssueGuard as a Chrome extension that scans issue text in real time to catch accidental exposure of API keys, credentials, and other secrets. It first uses regex rules to pull candidate strings from unstructured content such as logs and code snippets, then applies a fine-tuned CodeBERT model to decide which candidates are genuine secrets rather than false positives. The authors report that this hybrid method reaches an F1-score of 92.70 percent on their benchmark dataset and integrates directly into the GitHub editor to show visual warnings. A reader would care because current platforms offer no built-in safeguard against this common source of leaks, leaving developers to rely on manual review or after-the-fact scanners.

Core claim

IssueGuard analyzes text as users type in GitHub and GitLab issue editors, extracts candidate secrets with regex patterns, classifies them contextually with a fine-tuned CodeBERT model to separate real secrets from false positives, and displays immediate visual warnings so that users can remove sensitive data before posting; the system achieves an F1-score of 92.70 percent on benchmark data and outperforms regex-only scanners.

What carries the argument

Regex-based candidate extraction followed by fine-tuned CodeBERT contextual classification that distinguishes actual secrets from common false positives in unstructured issue text.

If this is right

Developers receive immediate visual feedback inside the editor and can edit out secrets before submission.
The hybrid approach reduces the false-positive rate that makes pure regex scanners impractical for everyday use.
The same pipeline can be extended to other collaborative platforms that accept free-form text containing code or configuration data.
Public release of the extension and source code allows teams to deploy or adapt the detector without building it from scratch.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Adapting the classifier to additional platforms such as internal ticketing systems or documentation wikis would require only retraining on domain-specific examples.
Pairing IssueGuard with post-submission scanning tools could create a defense-in-depth layer that catches leaks missed at edit time.
Logging the types of secrets most often flagged could help organizations refine their secret-management policies without exposing the actual values.

Load-bearing premise

The CodeBERT model trained on the benchmark dataset will keep high accuracy when applied to the varied, noisy, and domain-specific text that appears in real GitHub issue reports.

What would settle it

A measurable drop in precision or recall when IssueGuard is run on a fresh collection of actual GitHub issues that contain known secrets mixed with typical logs and code snippets.

Figures

Figures reproduced from arXiv: 2602.08072 by Gias Uddin, Md Nafiu Rahman, Rifat Shahriyar, Sadif Ahmed, Zahin Wahab.

**Figure 1.** Figure 1: Workflow of IssueGuard and evaluated its theoretical accuracy offline, IssueGuard focuses on the practical engineering required to operationalize that model. In addition to the browser extension, IssueGuard also supports GitHub/Gitlab CLI, enabling similar pre-submission checks in nonbrowser workflows. The main contributions of this work are: (1) We introduce IssueGuard, a tool that integrates a transform… view at source ↗

**Figure 2.** Figure 2: Demonstration IssueGuard. As the user types, the extension sends text to the backend. Real secrets are highlighted in red, while regex-based positives such as placeholders are ignored. text to be returned, further improving responsiveness during iterative editing. We evaluate the system’s performance by deploying the backend on a machine with an AMD Ryzen 3700G CPU, 16 GB RAM, and an NVIDIA RTX 3060 GPU… view at source ↗

**Figure 3.** Figure 3: Participants’ satisfaction with various aspects of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

read the original abstract

GitHub and GitLab are widely used collaborative platforms whose issue-tracking systems contain large volumes of unstructured text, including logs, code snippets, and configuration examples. This creates a significant risk of accidental secret exposure, such as API keys and credentials, yet these platforms provide no mechanism to warn users before submission. We present \textsc{IssueGuard}, a tool for real-time detection and prevention of secret leaks in issue reports. Implemented as a Chrome extension, \textsc{IssueGuard} analyzes text as users type and combines regex-based candidate extraction with a fine-tuned CodeBERT model for contextual classification. This approach effectively separates real secrets from false positives and achieves an F1-score of 92.70\% on a benchmark dataset, outperforming traditional regex-based scanners. \textsc{IssueGuard} integrates directly into the web interface and continuously analyzes the issue editor, presenting clear visual warnings to help users avoid submitting sensitive data. The source code is publicly available at \href{https://github.com/disa-lab/IssueGuard}{https://github.com/disa-lab/IssueGuard} , and a demonstration video is available at \href{https://youtu.be/kvbWA8rr9cU}{https://youtu.be/kvbWA8rr9cU} .

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

IssueGuard is a practical Chrome extension that layers CodeBERT classification on regex for GitHub issues, with public code, but its 92.7% F1 claim rests only on an untested benchmark.

read the letter

The main thing to know is that this paper delivers a working Chrome extension rather than a novel detection method. IssueGuard pulls candidate strings with regex and runs them through a fine-tuned CodeBERT model to cut false positives, then shows warnings inside the GitHub issue editor as the user types. The authors give a 92.70% F1 on a benchmark dataset, release the full source at the linked repo, and include a demo video. That combination of live integration and open code is the concrete output here.

Referee Report

1 major / 2 minor

Summary. The paper presents IssueGuard, a Chrome extension for real-time secret leak prevention in GitHub issue reports. It extracts candidate secrets via regex and classifies them with a fine-tuned CodeBERT model to reduce false positives, claiming an F1-score of 92.70% on a benchmark dataset that outperforms traditional regex scanners. The tool integrates into the GitHub web interface to display visual warnings during editing, with publicly available source code and a demonstration video.

Significance. If the reported performance generalizes, the work fills a practical gap in collaborative platforms by providing proactive, in-browser protection against accidental credential exposure in unstructured issue text. The open-source release and demo support reproducibility and potential adoption.

major comments (1)

[Evaluation] Evaluation section: the central claim that the tool 'effectively separates real secrets from false positives' in GitHub issue reports rests on a 92.70% F1-score, yet all reported metrics are confined to an unspecified benchmark dataset. No ablation, transfer test, or even qualitative examples are supplied for the noisy, multi-line, context-rich text that appears in actual GitHub issues, leaving the headline performance number unlinked to the deployed behavior.

minor comments (2)

[Abstract] Abstract: the F1-score claim is presented without any mention of benchmark dataset size, train-test split, or baseline implementation details, which hinders immediate assessment of the result.
The manuscript would benefit from a table or figure showing example true-positive and false-positive cases from the benchmark to illustrate how the CodeBERT classifier improves on regex.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address the major comment below and will revise the paper to strengthen the connection between the reported metrics and real-world GitHub issue behavior.

read point-by-point responses

Referee: [Evaluation] Evaluation section: the central claim that the tool 'effectively separates real secrets from false positives' in GitHub issue reports rests on a 92.70% F1-score, yet all reported metrics are confined to an unspecified benchmark dataset. No ablation, transfer test, or even qualitative examples are supplied for the noisy, multi-line, context-rich text that appears in actual GitHub issues, leaving the headline performance number unlinked to the deployed behavior.

Authors: We agree that the current evaluation section would benefit from stronger linkage to actual GitHub issue text. The benchmark dataset consists of labeled examples drawn from public GitHub issues (including multi-line logs and code snippets), but we acknowledge that the manuscript does not explicitly describe its construction or provide qualitative examples from live issues. In the revised manuscript we will: (1) add a detailed description of the benchmark dataset and its collection process, (2) include an ablation study isolating the contribution of the CodeBERT classifier versus regex-only extraction, (3) add a transfer evaluation on a held-out set of real GitHub issues, and (4) provide qualitative examples showing IssueGuard's behavior on noisy, multi-line issue text. These additions will directly address the concern that headline performance is unlinked to deployed behavior. revision: yes

Circularity Check

0 steps flagged

No circularity: result is direct empirical measurement on benchmark

full rationale

The paper presents IssueGuard as a Chrome extension combining regex candidate extraction with a fine-tuned CodeBERT classifier, reporting an F1-score of 92.70% on a benchmark dataset. No equations, derivations, or fitted parameters are used to obtain the result; the performance figure is obtained by direct evaluation rather than by construction from inputs. No self-citations, uniqueness theorems, or ansatzes appear in the load-bearing claims. The central claim rests on an external benchmark measurement and does not reduce to renaming or re-deriving its own training data.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The work relies on standard pre-trained language models and regex patterns; no new free parameters, axioms, or invented entities are introduced in the abstract.

pith-pipeline@v0.9.0 · 5536 in / 1095 out tokens · 24030 ms · 2026-05-16T05:48:28.019210+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages · 3 internal anchors

[1]

[n. d.]. Gitleaks. https://github.com/gitleaks/gitleaks. Accessed: 2024-02-02. IssueGuard:Real-Time Secret Leak Prevention Tool for GitHub Issue Reports Conference’17, July 2017, Washington, DC, USA

work page 2024
[2]

Sadif Ahmed, Md Nafiu Rahman, Zahin Wahab, Gias Uddin, and Rifat Shahriyar. 2025. Secret Breach Prevention in Software Issue Reports. arXiv:2410.23657 [cs.SE] https://arxiv.org/abs/2410.23657

work page internal anchor Pith review Pith/arXiv arXiv 2025
[3]

Loubna Ben Allal, Raymond Li, Denis Kocetkov, Chenghao Mou, Christopher Akiki, Carlos Munoz Ferrandis, Niklas Muennighoff, Mayank Mishra, Alex Gu, Manan Dey, Logesh Kumar Umapathi, Carolyn Jane Anderson, Yangtian Zi, Joel Lamy Poirier, Hailey Schoelkopf, Sergey Troshin, Dmitry Abulkhanov, Manuel Romero, Michael Lappert, Francesco De Toni, Bernardo García ...

work page arXiv 2023
[4]

Setu Kumar Basak, Jamison Cox, Bradley Reaves, and Laurie Williams. 2023. A Comparative Study of Software Secrets Reporting by Secret Detection Tools. In 2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE, 1–12

work page 2023
[5]

Runhan Feng, Ziyang Yan, Shiyan Peng, and Yuanyuan Zhang. 2022. Automated detection of password leakage from public github repositories. InProceedings of the 44th International Conference on Software Engineering. 175–186

work page 2022
[6]

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020. Codebert: A pre-trained model for programming and natural languages.arXiv preprint arXiv:2002.08155 (2020)

work page internal anchor Pith review Pith/arXiv arXiv 2020
[7]

2024.State of Secrets Sprawl Report 2023

GitGuardian. 2024.State of Secrets Sprawl Report 2023. Retrieved 2024-03-12 from https://www.gitguardian.com/state-of-secrets-sprawl-report-2023

work page 2024
[8]

2023.Cryptojackers steal A WS credentials from GitHub in 5 minutes

Connor Jones. 2023.Cryptojackers steal A WS credentials from GitHub in 5 minutes. Accessed: 2024-02-02

work page 2023
[9]

Barbara A Kitchenham and Shari L Pfleeger. 2008. Personal opinion surveys. In Guide to advanced empirical software engineering. Springer, 63–92

work page 2008
[10]

2021.The secrets about exposed secrets in code

Igal Kreichman. 2021.The secrets about exposed secrets in code. Accessed: 2024- 02-02

work page 2021
[11]

Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Armel Zebaze, Ming-Ho Yee, Loge...

work page internal anchor Pith review Pith/arXiv arXiv 2023
[12]

Michael Meli, Matthew R McNiece, and Bradley Reaves. 2019. How bad can it git? characterizing secret leakage in public github repositories.. InNDSS

work page 2019
[13]

Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. 2018. Mixed Precision Training. InInternational Confer- ence on Learning Representations. https://openreview.net/forum?id=r1gs9JgRZ

work page 2018
[14]

Aakanksha Saha, Tamara Denning, Vivek Srikumar, and Sneha Kumar Kasera

work page
[15]

In2020 International Conference on COMmunication Systems & NETworkS (COMSNETS)

Secrets in source code: Reducing false positives using machine learn- ing. In2020 International Conference on COMmunication Systems & NETworkS (COMSNETS). IEEE, 168–175

work page
[16]

Vibha Singhal Sinha, Diptikalyan Saha, Pankaj Dhoolia, Rohan Padhye, and Senthil Mani. 2015. Detecting and mitigating secret-key leaks in source code repositories. In2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. IEEE, 396–400

work page 2015
[17]

2016.TruffleHog

TruffleSecurity. 2016.TruffleHog. Accessed: 2024-02-02

work page 2016