Combining GPT and Code-Based Similarity Checking for Effective Smart Contract Vulnerability Detection

Jango Zhang

REVIEW 5 major objections 6 minor 77 references

Combining GPT and Code-Based Similarity Checking for Effective Smart Contract Vulnerability Detection

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SimilarGPT pairs GPT with trusted-code comparison to double recall and cut false alarms to 12 percent.

desk verdict The SimilarGPT integration idea is worth a look, but the central empirical claims are undermined by internal inconsistencies in the reported numbers, so the paper needs a corrected evaluation before its results can be trusted. read the letter →

arxiv 2412.18225 v1 pith:35CCNNPU submitted 2024-12-24 cs.SE

Jango Zhang This is my paper · ORCID

classification cs.SE

keywords smartcontractsvulnerabilitydetectionlargelanguagemodelscodesimilaritySocraticdebatetopologicalorderingSolidityLLMauditing

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SimilarGPT is a smart-contract auditing pipeline that tries to make large language models detect vulnerabilities by giving them a reference: the closest safe function from popular third-party libraries, matched by code embedding similarity. The paper claims this retrieval step is what lifts recall, raising true positives from 20 to 38 on a 138-function test set, and that a three-role Socratic debate over the detector's findings cuts the false-positive rate from 57% to 12%. It also reports that on 13 real-world exploits the tool finds 8, against 4 for a leading LLM baseline and one each for two static and dynamic analyzers. If these numbers hold, auditors could catch a substantially larger share of copied-and-tweaked contract bugs without drowning in false alarms.

What carries the argument

The load-bearing machinery is the combination of four components: a reference codebase of 83,321 deduplicated functions from the 150 most-used third-party Solidity packages; a similarity checker using normalized Euclidean distance between 384-dimensional sentence embeddings, with a hand-set threshold of 0.65, to retrieve the closest 'safe' function; a topological ordering of the contract's call graph that fixes the order in which functions are presented to the model; and a Socratic debate among detector, critic, supporter, and judge that filters false positives. The similarity check is what turns a generic LLM audit into a comparison against a concrete correct implementation.

What would settle it

Take a random sample of the functions the GPT-assisted filter marked vulnerable in the reference codebase, have independent human auditors relabel them, and compare the model's 0.65 'similar' pairings with expert judgments; if labels or similarity judgments disagree at a nontrivial rate, the reported 20-to-38 true-positive jump loses its ground truth.

Watch

Extended reading notes

Core claim

The paper's central claim is that a smart-contract vulnerability detector becomes substantially more sensitive and more precise when the large language model is shown, alongside each function under test, the most similar function from a curated reference codebase of trusted third-party contracts, and when its verdict is then challenged by a critic, a supporter, and a judge. The reference match is computed as normalized Euclidean distance between 384-dimensional embeddings of normalized Solidity functions, with 0.65 as the threshold for treating a retrieved function as close enough to use as reference; exact clones are resolved directly, near-clones are fed into the model as comparison material, and distant functions get no augmentation. Detection order follows a topological ordering of the call graph so that called functions are audited before their callers, which the paper argues stops the model from blaming a safe function for a vulnerability in a function it calls. The evidence offered is a controlled ablation in which adding similarity checking roughly doubled true positives on the 138-function set, and a comparison in which the Socratic debate cut false positives from 57% to 12%.

Load-bearing premise

Everything hinges on the reference labels: the third-party functions must really be safe or really be vulnerable, and a 0.65 embedding-similarity score must mark the code pairs whose differences a security expert would care about.

Editorial extensions

If this is right

If the central claim is right, an LLM-based auditor can roughly double its hit rate simply by retrieving a trusted reference implementation before asking the model to judge the code.
The Socratic debate step offers a concrete way to cut LLM hallucination false positives in security settings, beyond prompt engineering.
Topological ordering of the call graph is a reusable recipe for preventing an auditing model from blaming a caller for a flaw in a function it calls.
Because the reference codebase is refreshed from current third-party packages, the approach can track newly introduced vulnerable patterns without retraining the model.
The reported results imply that code reuse is not just a supply-chain risk but an audit lever: the same cloning that spreads bugs can be used to expose them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper never reports the accuracy of the GPT-assisted labeling that marks reference functions as secure or vulnerable; we infer that measuring that label accuracy against human experts is the fastest way to know how much of the true-positive jump is real.
The 0.65 threshold is hand-set; we infer that an automatic threshold search or a learned similarity function could change the results substantially on other contract families.
The false-positive reduction was measured on one 138-function set; we infer that the Socratic benefit could shrink on datasets with more diverse vulnerability types, and stress-testing that is a natural next step.
The repeated fork exploits the paper describes suggest the same retrieval mechanism could also serve as a triage pre-filter that flags near-clones of known-vulnerable code for human auditors; the paper does not draw this conclusion.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the integration idea is sensible, but the central numbers don't add up, so the headline claims are not currently supportable.

What's new: the specific combination of GPT-4 prompting with embedding-based retrieval of third-party reference code, topological ordering of the call graph, and a Socratic multi-agent filter is not something I've seen in the cited literature. The motivation is real: smart contracts reuse a lot of library code, and comparing against known-good implementations is a reasonable way to ground an LLM. The design choices are defensible.

The soft spots are in the evaluation, and they're load-bearing. Table II's "with SimilarChecking" row sums to 143 samples when the dataset is described as 138 (67 positive, 71 negative). The without row is consistent, the with row is not. So the 20-to-38 recall gain cannot be verified from the paper as written. The reported 12% FPR doesn't match either: 12 FP out of 63 TN + 12 FP is 16%, and 12 out of 71 negatives is 16.9%. The 57% FPR for the one-stage baseline has no confusion matrix behind it. The dataset counts also contradict each other (35,705 files vs 46,918 .sol files; 357,050 functions vs 766,505 functions). The threats-to-validity section says multiple runs were averaged, but no error bars or per-run results appear. No artifact is provided. The 0.65 similarity threshold is hand-set "based on observations", and the reference labels come from a GPT-4-assisted pipeline without reported accuracy, which adds a mild circularity.

That's a lot of problems, but they are fixable. The core idea is worth testing. A corrected evaluation with reproducible artifact, per-run results, and consistent counts would be a legitimate contribution.

Who this is for: people working on LLM-assisted smart contract auditing. If you're building such a tool, the paper gives you a useful design sketch, but don't rely on its numbers.

Recommendation: send it to peer review, but with the expectation of heavy revision. It deserves referee time because the idea is novel and the field needs better empirical evidence, but as-is the empirical claims should not be accepted.

Referee Report

5 major / 6 minor

Summary. The paper presents SimilarGPT, a smart-contract vulnerability detection tool that combines GPT-4 with code-based similarity checking (CBSC). The tool vectorizes Solidity functions using all-MiniLM-L6-v2, retrieves similar code from a reference codebase of third-party libraries, and asks GPT-4 to detect vulnerabilities by comparing the target function with the reference. To manage contextual information, functions are analyzed in a topological order of call dependencies, and a Socratic debate framework (Critic, Supporter, Judge) is used to filter false positives. The evaluation claims that on a Solodit dataset of 138 samples, CBSC raises true positives from 20 to 38, and the Socratic method reduces the false-positive rate from 57% to 12%, with additional results on 13 real-world exploits compared against Slither, Mythril, and GptLens.

Significance. If the results held, the paper would make a useful practical contribution by showing that LLM-based auditing can be made both more sensitive and more precise through retrieval of reference implementations and a multi-stage debate, while the topological ordering idea addresses an acknowledged weakness of LLM context handling. The core motivation, that Ethereum smart contracts reuse third-party code heavily, is well supported by prior work. However, the paper provides no artifact, code, or dataset release, and its headline quantitative claims are contradicted by the paper's own tables and text, so the significance as currently demonstrated is limited and the empirical contribution is not verifiable.

major comments (5)

[§IV-D, Table II] The central RQ3 result is not supported by the table as reported. The 'with SimilarChecking' row gives TP=38, TN=63, FP=12, FN=30, which sums to 143 samples, while the Solodit dataset is explicitly defined in §IV-A and §IV-D as 138 samples with 67 positive and 71 negative. The row's TP+FN=68 and FP+TN=75 match neither the stated positive count (67) nor the negative count (71); only the 'without SimilarChecking' row (20+61+10+47=138) is consistent with the described dataset. Since the abstract's claim that CBSC 'more than doubles' true positives rests entirely on this table, the headline result cannot be verified from the manuscript as written.
[§IV-C and §IV-D] The false-positive-rate claims are inconsistent with Table II. The paper states that the Socratic method reduces the false-positive rate to 12%, but Table II's with-CBSC row yields 12 FP / (12 FP + 63 TN) = 16%, or 12/71 = 16.9% if computed against the stated negative set. The 57% one-stage false-positive rate is asserted in §IV-C but no confusion matrix or per-condition counts for the one-stage framework are provided anywhere in the paper, so the RQ2 improvement cannot be independently checked.
[§III-D vs. Introduction] The reported sizes of the reference codebase are contradictory. The Introduction states '35705 files and 357,050 functions in total' and '83,321 selected functions' after filtering, whereas §III-D reports 'around 46,918 .sol files,' '766,505 functions before the hash match and only 35,709 functions following the hash-based filtering.' These are irreconcilable differences in both file and function counts, and they make the data-collection pipeline described in §III-D not reproducible from the text.
[§III-D, 'Filter vulnerability code'] The construction of the reference labels introduces a circularity risk that is not quantified. The authors label vulnerable functions in the third-party codebase using 'the GPT-4-based method outlined in [16]' and then use GPT-4 as the detector in SimilarGPT. If the reference labels are wrong, the 'secure' code presented to the detector is not actually secure, and the similarity-based comparison loses its ground truth. The manuscript does not report any manual validation rate, inter-annotator agreement, or accuracy measure for this labeling step, so the reliability of the reference codebase is unknown.
[§III-B, Eq. (1) and threshold δ] The similarity threshold δ=0.65 is introduced as 'Based on our observations' with no derivation, sensitivity analysis, or validation against other thresholds or embedding models. The distance measure itself, Euclidean(e1,e2)/(||e1||+||e2||), is non-standard and the text does not state whether the embeddings are L2-normalized before computing it. Because the entire CBSC component and therefore the RQ3 recall gain depend on this hand-set threshold, the generalizability of the reported improvement is unsubstantiated.

minor comments (6)

[§I, §V] The phrase 'on Ethernet' should be 'on Ethereum' in the Introduction and in §V's related-work discussion of code-reuse rates.
[Fig. 1 caption] The caption says 'green blocks indicating GPT works and green blocks suggesting code similar analysis'; the second reference should presumably be to the blue blocks, matching the text that describes green as LLM-based and blue as data-processing components.
[§III-C] The sentence 'if LLM thinks that function B, called by function A, has a vulnerability, we can test function B first. Then, we can test function B first and then function A' contains a redundant and garbled repetition; also, the formalization f: V → V is misleading because f describes a one-to-one function, whereas the calling relation is a directed graph edge set.
[§IV-C] The text refers to Figure 5 and states that the three frameworks differ little in recall, but no figure image or numeric table is actually included in the manuscript body, so the comparison cannot be inspected; the authors should provide the data.
[References] In §III-E the text has '[46] [46]' at the end of a sentence, and §V contains the incomplete citation 'Manticore citemanticore'; both need correction.
[§IV-A] The description states that negative samples (71 non-vulnerable functions) are collected using methods 'similar to those employed to address vulnerable code from third-party libraries,' but it does not specify the origin, selection criteria, or validation process for these negative samples, which is essential for interpreting the false-positive rates.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity from GPT-4-assisted reference/test labeling and Defihack overlap in RQ1; the central Solodit claim retains external grounding.

self definitional [Section III-D (Filter vulnerability code) and Section IV-A (Datasets)]
"Subsequently, we refer to the GPT-4-based method outlined in [16] to extract relevant vulnerability knowledge from the vulnerability descriptions. We attempt to annotate the corresponding original code by labeling it as a function with a vulnerability after carefully verifying the vulnerability’s existence. ... The specific gathering methods are similar to those employed to address vulnerable code from third-party libraries."

The vulnerable-versus-secure status of the reference functions used in CBSC is assigned by a GPT-4-based extraction and annotation protocol, and the Solodit test set is described as collected with the same gathering method. Because the Detector role in SimilarGPT is also played by GPT-4, the vulnerability ground truth and the vulnerability prediction are not independent: the measured recall/precision can reduce to GPT-4 agreeing with its own annotations. The paper reports no accuracy metric for the GPT-4 labeling step and no human-verification rate, so the reference 'correct code' is not shown to be independent of the model being evaluated.
other [Section IV-A (Datasets) and Section III-D (Filter vulnerability code)]
"The first dataset comprises vulnerability data from Defihack [42], a well-known DeFi Hacks dataset, and CVE [49]. We collect 13 typical vulnerability data from the Defihack and CVE vulnerability collections. ... We employ the current vulnerability datasets for smart contract that have been collected from frequently used public datasets, including defihacks [42], slowmist [43], and github’s issues [44], according to [16]."

The same Defihack repository is used both to construct the reference vulnerability examples that SimilarGPT relies on for similarity checking and to select the 13 real-world test cases on which RQ1 is evaluated. Without an explicit statement that these 13 incidents were excluded from the reference annotations, the RQ1 result (8 of 13 detected) is not an out-of-sample prediction; it can be achieved by retrieving a reference annotation that was built from the same answer key.

full rationale

SimilarGPT is not wholly circular: its main ablation claim on the Solodit dataset (recall increase from 20 to 38 true positives) is measured against external audit reports, and the Socratic-method comparison is an internal ablation rather than a restatement of the input. The vulnerability definitions, similarity formula, and topological ordering are standard or adapted from cited prior work, not defined in terms of the target result. However, two steps weaken the independence of the derivation chain. First, the reference codebase's vulnerability labels are produced by a GPT-4-based annotation method, and the Solodit collection is described as using the same gathering method; since the detector is also GPT-4, the ground truth can be contaminated by the model under test. Second, the 13-case real-world evaluation draws from Defihack, the same dataset used to build the reference vulnerability knowledge, so RQ1 may measure retrieval of an already-encoded answer key rather than generalization. These issues are partial circularity/data-overlap rather than a formal identity between input and output, and the central Solodit result still has independent empirical content, hence a score of 4 rather than higher.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central evaluation relies on a hand-picked similarity threshold, a general-purpose embedding model, a reference codebase of assumed-secure third-party code, and a DAG assumption on call graphs. None of these are independently validated in the paper.

free parameters (2)

similarity threshold delta = 0.65
Section III-D: 'Based on our observations, 0.65 can be utilized as a similarity threshold.' This threshold determines which reference snippets are fed to the LLM and is chosen from data rather than derived.
detector temperature = 0.8
Section IV-A: the detector role is set to temperature 0.8 'to retain some creativity'. This is a hand-chosen hyperparameter that affects detection output and is not justified by any sensitivity analysis.

assumptions (3)

domain assumption Third-party library code collected from npm and GitHub can be labeled as 'secure' or 'vulnerable' with sufficient accuracy.
Section III-D: functions are labeled vulnerable via a GPT-4-based extraction from vulnerability descriptions and manual verification, but the accuracy of this labeling is never reported.
domain assumption Euclidean distance between all-MiniLM-L6-v2 embeddings of normalized Solidity code reflects vulnerability-relevant semantic similarity.
Section III-D: the paper uses a general-purpose sentence transformer to vectorize code, with no code-specific fine-tuning or validation of the similarity measure on Solidity.
domain assumption The function call graph of the analyzed contracts is acyclic.
Section III-C: topological ordering requires a DAG; the paper states that 'rings in the sequence of function calls' are left for future work, so cyclic contracts are not handled.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Combining GPT and Code-Based Similarity Checking for Effective Smart Contract Vulnerability Detection." pith.science (2026). https://pith.science/paper/35CCNNPU

@misc{pith2026241218225,
  author       = {Pith},
  title        = {Pith review of: Combining GPT and Code-Based Similarity Checking for Effective Smart Contract Vulnerability Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/35CCNNPU}},
  note         = {Machine review of arXiv:2412.18225}
}

read the original abstract

With the rapid growth of blockchain technology, smart contracts are now crucial to Decentralized Finance (DeFi) applications. Effective vulnerability detection is vital for securing these contracts against hackers and enhancing the accuracy and efficiency of security audits. In this paper, we present SimilarGPT, a unique vulnerability identification tool for smart contract, which combines Generative Pretrained Transformer (GPT) models with Code-based similarity checking methods. The main concept of the SimilarGPT tool is to measure the similarity between the code under inspection and the secure code from third-party libraries. To identify potential vulnerabilities, we connect the semantic understanding capability of large language models (LLMs) with Code-based similarity checking techniques. We propose optimizing the detection sequence using topological ordering to enhance logical coherence and reduce false positives during detection. Through analysis of code reuse patterns in smart contracts, we compile and process extensive third-party library code to establish a comprehensive reference codebase. Then, we utilize LLM to conduct an indepth analysis of similar codes to identify and explain potential vulnerabilities in the codes. The experimental findings indicate that SimilarGPT excels in detecting vulnerabilities in smart contracts, particularly in missed detections and minimizing false positives.

Figures

Figures reproduced from arXiv: 2412.18225 by the authors.

**Figure 1.** An overview of SimilarGPT, green blocks indicating GPT works and green blocks suggesting [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

**Figure 2.** The Redacted Cartel exploit [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Comparing SimilarGPT with the traditional integration models. better on recall. However, when further analyzed, we find that their frameworks do not perform well on negative samples, resulting in the highest false positive rate, reaching 57% false positive rate, for t…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

77 extracted references · 60 canonical work pages

[1]

https://ethereum.org/zh/, accessed November 4, 2024

work page 2024
[2]

https://defillama.com/, accessed November 4, 2024

work page 2024
[3]

https://defillama.com/hacks, accessed November 4, 2024

work page 2024
[4]

Demystifying ex- ploitable bugs in smart contracts,

Z. Zhang, B. Zhang, W. Xu, and Z. Lin, “Demystifying ex- ploitable bugs in smart contracts,” in2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) . IEEE, 2023, pp. 615–627

work page 2023
[5]

https://github.com/Consensys/mythril, accessed November 4, 2024

work page 2024
[6]

https://github.com/crytic/slither, accessed November 4, 2024

work page 2024
[7]

Con- fuzzius: A data dependency-aware hybrid fuzzer for smart con- tracts,

C. F. Torres, A. K. Iannillo, A. Gervais, and R. State, “Con- fuzzius: A data dependency-aware hybrid fuzzer for smart con- tracts,” in 2021 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 2021, pp. 103–119

work page 2021
[8]

https://github.com/enzymefinance/oyente, accessed November 4, 2024

work page 2024

Show all 77 references

[9]

Sereum: Pro- tecting existing smart contracts against re-entrancy attacks,

M. Rodler, W. Li, G. O. Karame, and L. Davi, “Sereum: Pro- tecting existing smart contracts against re-entrancy attacks,” arXiv preprint arXiv:1812.05934, 2018

2018 arXiv
[10]

Soltype: refinement types for arithmetic overflow in solidity,

B. Tan, B. Mariano, S. K. Lahiri, I. Dillig, and Y. Feng, “Soltype: refinement types for arithmetic overflow in solidity,” Proceedings of the ACM on Programming Languages, vol. 6, no. POPL, pp. 1–29, 2022

2022
[11]

Smart contract anddefisecuritytools:Dotheymeettheneedsofpractitioners?

S. Chaliasos, M. A. Charalambous, L. Zhou, R. Galanopoulou, A. Gervais, D. Mitropoulos, and B. Livshits, “Smart contract anddefisecuritytools:Dotheymeettheneedsofpractitioners?” inProceedingsofthe46thIEEE/ACMInternationalConference on Software Engineering, 2024, pp. 1–13

2024
[12]

Gpt-4 technical report,

J.Achiam,S.Adler,S.Agarwal,L.Ahmad,I.Akkaya,F.L.Ale- man,D.Almeida,J.Altenschmidt,S.Altman,S.Anadkat etal., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023

2023 arXiv
[13]

A survey of large language models,

W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Donget al., “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023

2023 arXiv
[17]

Propertygpt: Llm-driven formal verification of smart con- tracts through retrieval-augmented property generation,

Y. Liu, Y. Xue, D. Wu, Y. Sun, Y. Li, M. Shi, and Y. Liu, “Propertygpt: Llm-driven formal verification of smart con- tracts through retrieval-augmented property generation,”arXiv preprint arXiv:2405.02580, 2024

2024 arXiv
[19]

Code cloning in smart contracts on the ethereum platform: An extended replicationstudy,

F. Khan, I. David, D. Varro, and S. McIntosh, “Code cloning in smart contracts on the ethereum platform: An extended replicationstudy,” IEEETransactionsonSoftwareEngineering , vol. 49, no. 4, pp. 2006–2019, 2022

2006
[20]

Code cloning in smart contracts: a case study on verified contracts from the ethereum blockchain platform,

M. Kondo, G. A. Oliva, Z. M. Jiang, A. E. Hassan, and O. Mizuno, “Code cloning in smart contracts: a case study on verified contracts from the ethereum blockchain platform,” Empirical Software Engineering, vol. 25, pp. 4617–4675, 2020

2020
[21]

Demystifying the composition and code reuse in solidity smart contracts,

K. Sun, Z. Xu, C. Liu, K. Li, and Y. Liu, “Demystifying the composition and code reuse in solidity smart contracts,” inPro- ceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2023, pp. 796–807

2023
[22]

Hunting vulnerable smart contracts via graph embed- ding based bytecode matching,

J. Huang, S. Han, W. You, W. Shi, B. Liang, J. Wu, and Y. Wu, “Hunting vulnerable smart contracts via graph embed- ding based bytecode matching,”IEEE Transactions on Infor- mation Forensics and Security, vol. 16, pp. 2144–2156, 2021

2021
[23]

Eclone: Detect semantic clones in ethereum via symbolic transaction sketch,

H. Liu, Z. Yang, C. Liu, Y. Jiang, W. Zhao, and J. Sun, “Eclone: Detect semantic clones in ethereum via symbolic transaction sketch,” in Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engi...

2018
[25]

Decentralized finance: On blockchain-and smart contract-based financial markets,

F. Schär, “Decentralized finance: On blockchain-and smart contract-based financial markets,” FRB of St. Louis Review, 2021

2021
[26]

Sok: Decentralized finance (defi) attacks,

L. Zhou, X. Xiong, J. Ernstberger, S. Chaliasos, Z. Wang, Y. Wang, K. Qin, R. Wattenhofer, D. Song, and A. Gervais, “Sok: Decentralized finance (defi) attacks,” in2023 IEEE Sym- posium on Security and Privacy (SP). IEEE, 2023, pp. 2444– 2461

2023
[27]

Large language models are zero-shot reasoners,

T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,”Advances in neural information processing systems, vol. 35, pp. 22199– 22213, 2022

2022
[28]

Llm4vuln:Aunifiedevaluationframeworkfordecoupling and enhancing llms’ vulnerability reasoning,

Y. Sun, D. Wu, Y. Xue, H. Liu, W. Ma, L. Zhang, Y. Liu, and Y.Li,“Llm4vuln:Aunifiedevaluationframeworkfordecoupling and enhancing llms’ vulnerability reasoning,” arXiv preprint arXiv:2401.16185, 2024

2024 arXiv
[29]

A structured self-attentive sentence embedding,

Z. Lin, M. Feng, C. N. d. Santos, M. Yu, B. Xiang, B. Zhou, and Y. Bengio, “A structured self-attentive sentence embedding,” arXiv preprint arXiv:1703.03130, 2017

2017 arXiv
[30]

Retrieval-augmented generation for knowledge-intensive nlp tasks,

P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel et al., “Retrieval-augmented generation for knowledge-intensive nlp tasks,”AdvancesinNeuralInformationProcessingSystems , vol. 33, pp. 9459–9474, 2020

2020
[31]

Survey of hallucination in naturallanguagegeneration,

Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in naturallanguagegeneration,” ACMComputingSurveys ,vol.55, no. 12, pp. 1–38, 2023

2023
[32]

https://immunefi.com/, accessed November 4, 2024

2024
[33]

https://www.wikiwand.com/en/articles/Topological_sorting, accessed November 4, 2024

2024
[34]

https://github.com/ConsenSys/surya, accessed November 4, 2024

2024
[35]

https://www.npmjs.com/package/@openzeppelin/contracts, accessed November 4, 2024

2024
[36]

https://huggingface.co/sentence-transformers/ all-MiniLM-L6-v2, accessed November 4, 2024

2024
[37]

https://www.sbert.net/docs/sentence_transformer/ pretrained_models.html#scientific-similarity-models, accessed November 4, 2024

2024
[38]

https://www.sbert.net/docs/sentence_transformer/ pretrained_models.html, accessed November 4, 2024

2024
[39]

https://huggingface.co/sentence-transformers/ all-mpnet-base-v2, accessed November 4, 2024

2024
[40]

https://huggingface.co/sentence-transformers/ multi-qa-mpnet-base-dot-v1, accessed November 4, 2024

2024
[41]

https://huggingface.co/sentence-transformers/ multi-qa-MiniLM-L6-cos-v1, accessed November 4, 2024

2024
[42]

https://github.com/SunWeb3Sec/DeFiHackLabs?tab= readme-ov-file, accessed November 4, 2024

2024
[43]

https://hacked.slowmist.io/en/, accessed November 4, 2024

2024
[44]

https://github.com/, accessed November 4, 2024

2024
[45]

The art of socratic questioning: Recursive thinking with large language models,

J. Qi, Z. Xu, Y. Shen, M. Liu, D. Jin, Q. Wang, and L. Huang, “The art of socratic questioning: Recursive thinking with large language models,”arXiv preprint arXiv:2305.14999, 2023

2023 arXiv
[46]

Towards detecting llms hallucination via markov chain-based multi-agent debate framework,

X. Sun, J. Li, Y. Zhong, D. Zhao, and R. Yan, “Towards detecting llms hallucination via markov chain-based multi-agent debate framework,”arXiv preprint arXiv:2406.03075, 2024

2024 arXiv
[47]

Prompting large language models with the so- cratic method,

E. Y. Chang, “Prompting large language models with the so- cratic method,” in 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC). IEEE, 2023, pp. 0351–0360

2023
[48]

Knowledge-enhanced neural machine reasoning: A review,

T. Chowdhury, C. Ling, X. Zhang, X. Zhao, G. Bai, J. Pei, H. Chen, and L. Zhao, “Knowledge-enhanced neural machine reasoning: A review,”arXiv preprint arXiv:2302.02093, 2023

2023 arXiv
[49]

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=smart+ contract, accessed November 4, 2024

2024
[50]

https://solodit.xyz/, accessed November 4, 2024

2024
[51]

https://github.com/SunWeb3Sec/DeFiHackLabs/ blob/main/past/2021/README.md# 20210428-uranium---miscalculation, accessed November 4, 2024

2021
[52]

https://github.com/SunWeb3Sec/DeFiHackLabs/blob/ main/past/2021/README.md#20210915-nimbus-platform, accessed November 4, 2024

2021
[53]

https://github.com/SunWeb3Sec/DeFiHackLabs/ blob/main/past/2023/README.md# 20230416-swapos-v2---error-k-value-attack, accessed November 4, 2024

2023
[54]

Ethainter: a smart contract security analyzer for com- posite vulnerabilities,

L. Brent, N. Grech, S. Lagouvardos, B. Scholz, and Y. Smarag- dakis, “Ethainter: a smart contract security analyzer for com- posite vulnerabilities,” in Proceedings of the 41st ACM SIG- PLAN Conference on Programming Language Design and Im- plementation, 2020, pp. 454–469

2020
[55]

Vandal: A scalable se- curity analysis framework for smart contracts,

L. Brent, A. Jurisevic, M. Kong, E. Liu, F. Gauthier, V. Gramoli, R. Holz, and B. Scholz, “Vandal: A scalable se- curity analysis framework for smart contracts,”arXiv preprint arXiv:1809.03981, 2018

2018 arXiv
[56]

Zeus: analyzing safety of smart contracts

S. Kalra, S. Goel, M. Dhawan, and S. Sharma, “Zeus: analyzing safety of smart contracts.” inNdss, 2018, pp. 1–12

2018
[57]

sfuzz: An efficient adaptive fuzzer for solidity smart contracts,

T. D. Nguyen, L. H. Pham, J. Sun, Y. Lin, and Q. T. Minh, “sfuzz: An efficient adaptive fuzzer for solidity smart contracts,” in Proceedings of the ACM/IEEE 42nd International Confer- ence on Software Engineering, 2020, pp. 778–788

2020
[58]

Contractfuzzer: Fuzzing smart contracts for vulnerability detection,

B. Jiang, Y. Liu, and W. K. Chan, “Contractfuzzer: Fuzzing smart contracts for vulnerability detection,” inProceedings of the 33rd ACM/IEEE international conference on automated software engineering, 2018, pp. 259–269

2018
[59]

Finding permission bugs in smart contracts with role mining,

Y. Liu, Y. Li, S.-W. Lin, and C. Artho, “Finding permission bugs in smart contracts with role mining,” inProceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, 2022, pp. 716–727

2022
[60]

Oracle-supported dynamic exploit generation for smart contracts,

H. Wang, Y. Liu, Y. Li, S.-W. Lin, C. Artho, L. Ma, and Y. Liu, “Oracle-supported dynamic exploit generation for smart contracts,”IEEE Transactions on Dependable and Secure Com- puting, vol. 19, no. 3, pp. 1795–1809, 2020

2020
[61]

{ETHBMC}: A bounded model checker for smart contracts,

J. Frank, C. Aschermann, and T. Holz, “ {ETHBMC}: A bounded model checker for smart contracts,” in29th USENIX Security Symposium (USENIX Security 20), 2020, pp. 2757– 2774

2020
[62]

Verx: Safety verification of smart contracts,

A. Permenev, D. Dimitrov, P. Tsankov, D. Drachsler-Cohen, and M. Vechev, “Verx: Safety verification of smart contracts,” in 2020 IEEE symposium on security and privacy (SP). IEEE, 2020, pp. 1661–1677

2020
[63]

Verismart: A highly precise safety verifier for ethereum smart contracts,

S. So, M. Lee, J. Park, H. Lee, and H. Oh, “Verismart: A highly precise safety verifier for ethereum smart contracts,” in2020 IEEE Symposium on Security and Privacy (SP). IEEE, 2020, pp. 1678–1694

2020
[64]

S-gram: towards semantic-awaresecurityauditingforethereumsmartcontracts,

H. Liu, C. Liu, W. Zhao, Y. Jiang, and J. Sun, “S-gram: towards semantic-awaresecurityauditingforethereumsmartcontracts,” in Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, 2018, pp. 814–819

2018
[65]

Enabling clone detection for ethereum via smart contract birthmarks,

H. Liu, Z. Yang, Y. Jiang, W. Zhao, and J. Sun, “Enabling clone detection for ethereum via smart contract birthmarks,” in 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). IEEE, 2019, pp. 105–115

2019
[66]

Analysis of source code duplica- tion in ethreum smart contracts,

G. A. Pierro and R. Tonelli, “Analysis of source code duplica- tion in ethreum smart contracts,” in2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 2021, pp. 701–707

2021
[67]

Un- derstanding code reuse in smart contracts,

X. Chen, P. Liao, Y. Zhang, Y. Huang, and Z. Zheng, “Un- derstanding code reuse in smart contracts,” in 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 2021, pp. 470–479

2021
[68]

Checkingsmart contracts with structural code embedding,

Z.Gao,L.Jiang,X.Xia,D.Lo,andJ.Grundy,“Checkingsmart contracts with structural code embedding,”IEEE Transactions on Software Engineering, vol. 47, no. 12, pp. 2874–2891, 2020

2020
[69]

Llms cannot reliably identify and reason about security vulnerabilities (yet?): A comprehensive evaluation, framework,andbenchmarks,

S. Ullah, M. Han, S. Pujar, H. Pearce, A. Coskun, and G. Stringhini, “Llms cannot reliably identify and reason about security vulnerabilities (yet?): A comprehensive evaluation, framework,andbenchmarks,” arXivpreprintarXiv:2312.12575 , 2023

2023 arXiv
[70]

Chat- gpt for vulnerability detection, classification, and repair: How far are we?

M. Fu, C. K. Tantithamthavorn, V. Nguyen, and T. Le, “Chat- gpt for vulnerability detection, classification, and repair: How far are we?” in 2023 30th Asia-Pacific Software Engineering Conference (APSEC). IEEE, 2023, pp. 632–636

2023
[71]

Transformer-based language models for software vulnerability detection,

C. Thapa, S. I. Jang, M. E. Ahmed, S. Camtepe, J. Pieprzyk, and S. Nepal, “Transformer-based language models for software vulnerability detection,” in Proceedings of the 38th Annual Computer Security Applications Conference, 2022, pp. 481–496

2022
[72]

Do you still need a manual smart contract audit?

I.David,L.Zhou,K.Qin,D.Song,L.Cavallaro,andA.Gervais, “Do you still need a manual smart contract audit?” arXiv preprint arXiv:2306.12338, 2023

2023 arXiv
[73]

Low level source code vulnerability detectionusingadvancedbertlanguagemodel

M. Alqarni and A. Azim, “Low level source code vulnerability detectionusingadvancedbertlanguagemodel.”in CanadianAI, 2022

2022
[74]

Llbezpeky: Leveraging large language models for vulner- ability detection,

N. S. Mathews, Y. Brus, Y. Aafer, M. Nagappan, and S. McIn- tosh, “Llbezpeky: Leveraging large language models for vulner- ability detection,”arXiv preprint arXiv:2401.01269, 2024

2024 arXiv
[75]

Large language model-powered smart contract vulnerability detection: New perspectives,

S. Hu, T. Huang, F. İlhan, S. F. Tekin, and L. Liu, “Large language model-powered smart contract vulnerability detection: New perspectives,” in2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA). IEEE, 2023, p...

2023
[76]

Software vulnerability detection using large language models,

M. D. Purba, A. Ghosh, B. J. Radford, and B. Chu, “Software vulnerability detection using large language models,” in2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW). IEEE, 2023, pp. 112–119

2023
[77]

Gptscan: Detecting logic vulnerabilities in smart con- tracts by combining gpt with program analysis,

Y. Sun, D. Wu, Y. Xue, H. Liu, H. Wang, Z. Xu, X. Xie, and Y. Liu, “Gptscan: Detecting logic vulnerabilities in smart con- tracts by combining gpt with program analysis,” inProceedings of the IEEE/ACM 46th International Conference on Software Engineering, 2024, pp. 1–13

2024
[78]

The hitchhiker’s guide to program analysis: A journey with large language models,

H. Li, Y. Hao, Y. Zhai, and Z. Qian, “The hitchhiker’s guide to program analysis: A journey with large language models,”arXiv preprint arXiv:2308.00245, 2023

2023 arXiv
[79]

Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,

Y. Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang, “Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,” inProceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis, 2023, pp. 423–435

2023
[80]

Large language models are edge-case generators: Crafting unusual programs for fuzzing deep learning libraries,

Y. Deng, C. S. Xia, C. Yang, S. D. Zhang, S. Yang, and L. Zhang, “Large language models are edge-case generators: Crafting unusual programs for fuzzing deep learning libraries,” inProceedingsofthe46thIEEE/ACMInternationalConference on Software Engineering, 2024, pp. 1–13

2024
[81]

Large language model guided protocol fuzzing,

R. Meng, M. Mirchev, M. Böhme, and A. Roychoudhury, “Large language model guided protocol fuzzing,” inProceedings of the 31st Annual Network and Distributed System Security Sympo- sium (NDSS), 2024

2024
[82]

Acfix: Guiding llms with mined common rbac practices for context-aware repair of access control vulnerabilities in smart contracts,

L. Zhang, K. Li, K. Sun, D. Wu, Y. Liu, H. Tian, and Y. Liu, “Acfix: Guiding llms with mined common rbac practices for context-aware repair of access control vulnerabilities in smart contracts,” arXiv preprint arXiv:2403.06838, 2024

2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

https://ethereum.org/zh/, accessed November 4, 2024

work page 2024

[2] [2]

https://defillama.com/, accessed November 4, 2024

work page 2024

[3] [3]

https://defillama.com/hacks, accessed November 4, 2024

work page 2024

[4] [4]

Demystifying ex- ploitable bugs in smart contracts,

Z. Zhang, B. Zhang, W. Xu, and Z. Lin, “Demystifying ex- ploitable bugs in smart contracts,” in2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) . IEEE, 2023, pp. 615–627

work page 2023

[5] [5]

https://github.com/Consensys/mythril, accessed November 4, 2024

work page 2024

[6] [6]

https://github.com/crytic/slither, accessed November 4, 2024

work page 2024

[7] [7]

Con- fuzzius: A data dependency-aware hybrid fuzzer for smart con- tracts,

C. F. Torres, A. K. Iannillo, A. Gervais, and R. State, “Con- fuzzius: A data dependency-aware hybrid fuzzer for smart con- tracts,” in 2021 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 2021, pp. 103–119

work page 2021

[8] [8]

https://github.com/enzymefinance/oyente, accessed November 4, 2024

work page 2024