ETrace:Event-Driven Vulnerability Detection in Smart Contracts via LLM-Based Trace Analysis

Chenyang Peng; Haijun Wang; Hao Wu; Ming Fan; Ting Liu; Yin Wu; Yitao Zhao

REVIEW 5 major objections 5 minor 15 references

ETrace:Event-Driven Vulnerability Detection in Smart Contracts via LLM-Based Trace Analysis

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ETrace claims that event logs, decoded and interpreted by an LLM, can expose smart-contract attacks even when the contract's source code is unavailable.

desk verdict ETrace is a plausible new idea for code-free smart-contract vulnerability detection, but the preliminary experiments don't support the effectiveness claim; it deserves a New Idea track referee, not full-paper acceptance. read the letter →

arxiv 2506.15790 v3 pith:E5CRNOFZ submitted 2025-06-18 cs.CR cs.SE

Chenyang Peng , Haijun Wang , Yin Wu , Hao Wu , Ming Fan , Yitao Zhao , Ting Liu This is my paper · ORCID

classification cs.CRcs.SE

keywords smartcontractsvulnerabilitydetectioneventlargelanguagemodelstransactionlogschain-of-thoughtpromptingreentrancyblockchainsecurity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a smart contract's emitted events—the function names, addresses, and values recorded in a transaction's log—carry enough semantic information to reveal attack behavior, even when the contract's source code is not available. ETrace is a framework built on that idea: it decodes transaction-log topics with the contract ABI, asks a large language model to explain each event and then judge the whole sequence, and matches the judgment against four predefined attack patterns. The claim matters because many real-world DeFi attacks run through third-party contracts whose code is inaccessible, which defeats conventional code-based analysis. The authors validate the framework on four documented incidents and report that the LLM judgments align with the actual attack types.

What carries the argument

The central object is the fine-grained event sequence: transaction-log topics are decoded through the contract ABI into triples of function name, address, and value, then sliced by function name. The argument is carried by an LLM used as a semantic interpreter with chain-of-thought prompting: it first explains each event in plain terms, then forms a holistic judgment, and a pattern-matching stage attaches that judgment to one of four predefined attack patterns. This two-step design is what converts raw log entries into an attack verdict, and it is also what produces interpretable output, since every verdict comes with the event-level reasoning behind it.

What would settle it

Run ETrace on a held-out set of labeled attack transactions covering the four vulnerability classes, with the ABIs of the involved contracts withheld, and require the final pattern-matching verdict to match the documented incident type; if decoding fails for a material share of inputs or the verdicts match ground truth no better than chance, the central claim collapses.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that fine-grained event sequences extracted from transaction logs can be semantically interpreted to reconstruct the intent of a smart-contract transaction without reading its code. Each event is reduced to a function name, an address, and a value; a large language model, prompted with the characteristics of known attacks and asked to reason step by step, produces an event-by-event explanation followed by a comprehensive judgment in the form Summary–Pattern Analysis–Further Recommendation. A final pattern-matching stage checks that judgment against hand-specified patterns for reentrancy, integer overflow, flash-loan attacks, and denial of service. The paper reports that this pipeline identified the correct attack type in all four real-world case studies it examines.

Load-bearing premise

The pipeline depends on having an ABI for every contract involved in a transaction, because event topics are decoded through that ABI into the function names and values the LLM analyzes; without this interface metadata, the event sequence the whole method relies on cannot be built.

Editorial extensions

If this is right

Transactions emitting telltale event sequences can be flagged as reentrancy, integer overflow, flash-loan, or denial-of-service attacks even when the contract's source code is unpublished.
Security monitoring becomes possible from on-chain logs alone, so attacks that pass through third-party dependencies such as oracles, bridges, or unaudited DEX components become visible to the detector.
Every verdict is interpretable: the chain-of-thought output narrates the events and the pattern analysis explains how the attack unfolded.
Because the input is a transaction hash and recorded logs, the method can be applied retroactively to historical incidents without re-executing the contract.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the paper leaves implicit: the 'no code' promise is conditional on ABI availability, since topics are decoded with the ABI of each involved contract; extending ETrace to contracts without public ABIs would require recovering interface metadata from bytecode or registries, which is a testable next step.
The reported confusion between large values in reentrancy traces and integer overflow suggests that adding value-range thresholds or cross-event consistency checks would sharpen the LLM's judgments, and this can be measured on the same four incidents.
If the approach scales, event-trace analysis could serve as a forensic layer: given a transaction hash, an auditor would get a structured attack narrative without needing the contract code or a live chain.
Because events are data that contracts choose to emit, a deployment should cross-check event authenticity against state-transition evidence; otherwise forged or spoofed logs could mislead the LLM, a risk the paper's own cited work on phantom events raises.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: ETrace is exactly what it looks like—a four-page New Idea paper with a plausible new combination and an evaluation that cannot back up the effectiveness claim it makes. I'd give the idea a fair hearing at a workshop-style venue, but the authors need to be told the experiments as presented support 'illustrated' not 'validated.'

What's new: I agree with the reader's novelty score. Prior work uses events for execution, logging, and testing, and LLMs for code analysis; nobody has pointed LLMs at decoded event traces to infer attack intent. That is a real gap, especially for proxy contracts and third-party dependencies without source code. The pipeline—decode logs, prompt with vulnerability conditions, chain-of-thought analysis, pattern match—is clearly described, and the four real incidents (XSURGE, Beauty Chain, MEVBOT, GovernMental) give it concrete grounding.

Where it falls down: the stress-test has it right. The vulnerability conditions in Section 3.2 are hand-crafted from the same four incidents used as the entire evaluation set in Section 4.1. Table 5 shows the LLM essentially restating those conditions in hedged language ('could potentially be'). There are no negative controls, no precision/recall, no out-of-sample cases, no baseline comparison. The one admitted failure—the integer-overflow false positive during reentrancy analysis—confirms over-triggering, but the rate is never quantified. On top of that, Section 3.1 requires the ABI to decode topics, which undercuts the 'code-free' selling point when ABIs are absent; the paper doesn't acknowledge that dependency.

What's solid: the examples are real, the tables match on-chain events, and the conclusion is honest about the work being preliminary. The citation pattern looks fine—the related work cites recent event and LLM papers and doesn't pad.

Bottom line: I wouldn't cite this as evidence for the effectiveness of event-trace LLM detection, and I'd want a blinded evaluation with negatives, released prompts, and an ABI-availability discussion before treating it as more than a proposal. But as a New Idea paper, it articulates a gap and a plausible approach, and a serious referee could help the authors shape it into something testable. So: send it out, but with a clear expectation of major revision.

Referee Report

5 major / 5 minor

Summary. The paper proposes ETrace, a framework for detecting smart-contract vulnerabilities without source code access. ETrace extracts and decodes event logs from transaction traces using contract ABIs, feeds the decoded events to an LLM with chain-of-thought prompting and hand-crafted vulnerability conditions for reentrancy, integer overflow, flash-loan attacks, and denial-of-service, and then applies a pattern-matching stage to produce a final judgment. The authors claim that preliminary experiments on four real-world incidents confirm the effectiveness of the approach.

Significance. The problem is timely and the event-centric, code-free detection idea is worth exploring. The paper presents a clear pipeline and is transparent about some limitations, such as the lack of a value threshold in Section 4.2. However, the reported evaluation is too weak to support the central effectiveness claim: it uses only the four motivating incidents, all positive cases, with no negative controls, no benign transactions, no quantitative metrics, and no baselines. Because the vulnerability conditions in the prompts are hand-crafted from the same four incidents, the results are equally consistent with the LLM rephrasing cues already supplied in the prompt as with ETrace genuinely inferring attack intent. The claimed contribution is therefore not established by the current evidence.

major comments (5)

[Section 4.1 and 4.2] The evaluation is limited to exactly the four motivating incidents, one per target vulnerability, and reports only qualitative 'comprehensive judgments' in Table 5. There are no negative cases, no benign transactions, no precision/recall metrics, and no comparison against any baseline detector. The judgments in Table 5 largely restate the vulnerability conditions given in the prompts (e.g., 'The FlashLoan event followed by the Swap events with abnormal price fluctuations could potentially indicate a FlashLoanAttack event'). Therefore the results in Section 4.2 cannot substantiate the Abstract's claim that 'preliminary experiments have confirmed the effectiveness of ETrace'; correct labels on the training-like examples are also consistent with mere prompt-following.
[Section 3.2 and Section 2] The prompt-engineering stage embeds vulnerability conditions for the same four attack types that are motivated in Section 2 and used as the evaluation set in Section 4.1. For example, the reentrancy condition in Section 3.2 is derived from the XSURGE pattern of repeated calls and alternating transfers shown in Table 1. This circular setup means the evaluation cannot show generalization to unseen attacks, and it does not test whether ETrace can detect a vulnerability whose event signature was not already encoded in the prompt.
[Section 4.2] The manuscript admits that, because no value threshold was set, the LLM incorrectly identified integer overflows during reentrancy analysis when values were too high. This is an acknowledged false-positive mode, but its frequency is never quantified, no error analysis is provided, and no mitigation is evaluated. The existence of an unquantified over-triggering behavior further weakens the claim that the method is effective.
[Section 3.1] The data-preprocessing stage decodes event topics 'using the ABI of the corresponding contract address.' This is a load-bearing dependency: for contracts without available source code, the ABI may also be unavailable or must be reconstructed, and the paper does not discuss how ETrace handles such cases. The motivating scenario of code-less contracts therefore relies on an unstated assumption about metadata availability.
[Section 3.4] The pattern-matching stage is described only conceptually as 'a judgment to the pattern analysis results' and is not given an algorithmic specification or an independent evaluation. Table 5 reports only the LLM's comprehensive judgments, so the reader cannot tell what pattern matching adds beyond the LLM output, or whether it is implemented at all.

minor comments (5)

[Section 4.2 and Table 5] There is a typo in Section 4.2 ('Table .5' should be 'Table 5'), and the caption of Table 5 reads 'Compehensive Judgement' instead of 'Comprehensive Judgment'.
[References] References [5] and [13] are the same paper (Liu et al., 'Phantom events'); this duplication should be removed and the citation numbering adjusted.
[Section 3.3] The phrase 'None code requirement' should be rephrased as a 'no-code requirement' or 'without-source-code requirement' for clarity.
[Table 3] The row 'Swap None →None out0=0, out1=0' is unclear: the 'None' addresses and zero-valued outputs need explanation, as they are not self-explanatory from the raw event data.
[Introduction] The sentence 'As a result, the trading volume of DEXs continues to grow steadily' is not logically connected to the preceding sentence about blockchain logs; the transition should be clarified.

Circularity Check

1 steps flagged · score 6.0 of 10

Effectiveness claim rests on an evaluation set identical to the incidents from which the prompt conditions were hand-crafted; no independent validation is reported.

fitted input called prediction [Section 3.2 (Prompt Engineering) and Section 4.1 (Dataset); Abstract]
"the vulnerability conditions encompass the characteristics of four known attack types: reentrancy, integer overflow, flash loan attack, and DoS. ... We evaluated ETrace on four Etherscan-recorded attack events. ... our preliminary experiments have confirmed the effectiveness of ETrace."

The same four incidents that motivated the vulnerability conditions are used as the evaluation set, and the conditions are stated to be the characteristics of those four attack types. Thus the LLM is handed the answer schema before seeing the traces; its outputs like 'The FlashLoan event followed by the Swap events with abnormal price fluctuations could potentially indicate a FlashLoanAttack event' restate the prompt conditions. The experiment has no negative or benign controls and no out-of-sample attacks, so the reported 'confirmation' of effectiveness is at least partly a product of prompt construction rather than an independent measurement. This is analogous to fitting a detector to the test cases and then reporting detection on those cases as validation.

full rationale

ETrace's pipeline has independent engineering content: event decoding, prompt design, chain-of-thought analysis, and pattern matching are not formally derived from the attacks. The circularity is concentrated in the validation chain. Section 2 defines the four attack patterns from four incidents (XSURGE, Beauty Chain, MEVBOT, GovernMental); Section 3.2 injects those same four attack descriptions into the prompts; Section 4.1 evaluates on exactly those four incidents; and Table 5's outputs restate the prompt conditions. Consequently, the 'preliminary experiments confirmed the effectiveness' claim is consistent with the LLM echoing the supplied vulnerability conditions rather than independently detecting vulnerabilities. The admitted over-triggering limitation (integer-overflow misidentification during reentrancy analysis) is acknowledged but not quantified, and no precision, recall, or false-positive rate is given. The ABI requirement is a practical limitation, not a circularity. Self-citations to prior work by the same authors are not load-bearing here. Because the central effectiveness claim is not independently tested, but the method itself contains non-circular components, the score is 6 rather than higher.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim depends on hand-authored attack descriptions derived from the evaluation incidents, an implicit ABI-availability assumption, and an unspecified LLM's reliability. No numerical parameters are fit in the traditional sense, but the prompt design and threshold absence function as uncalibrated choices.

free parameters (2)

Value anomaly threshold
Section 4.2 states no threshold was set, so the LLM implicitly decides what value is abnormal; this caused false integer-overflow detections during reentrancy analysis.
LLM model and sampling configuration = Unspecified
The paper does not name the LLM, its version, temperature, or prompt details; all outputs in Table 5 depend on this unstated configuration.

assumptions (4)

domain assumption An ABI is available for every contract whose events are decoded, even when source code is unavailable.
Section 3.1 decodes event parameters using the ABI of the contract address; no mechanism is provided for obtaining ABIs of black-box contracts.
ad hoc to paper The four attack types' event signatures are distinctive and were correctly characterized from the motivating incidents.
Section 2 derives event characteristics from the same four incidents later used as the evaluation set, so the characterization is tailored to the test cases.
domain assumption LLM chain-of-thought analysis over event sequences yields accurate semantic interpretations without source code.
The framework depends entirely on LLM reliability; the paper mentions hallucination risk in Section 5 but provides no measurement.
domain assumption Event logs retrieved for the four incidents are complete and accurate.
The method assumes the explorer-provided transaction logs contain all relevant emitted events; no verification is described.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ETrace:Event-Driven Vulnerability Detection in Smart Contracts via LLM-Based Trace Analysis." pith.science (2026). https://pith.science/paper/E5CRNOFZ

@misc{pith2026250615790,
  author       = {Pith},
  title        = {Pith review of: ETrace:Event-Driven Vulnerability Detection in Smart Contracts via LLM-Based Trace Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E5CRNOFZ}},
  note         = {Machine review of arXiv:2506.15790}
}

read the original abstract

With the advance application of blockchain technology in various fields, ensuring the security and stability of smart contracts has emerged as a critical challenge. Current security analysis methodologies in vulnerability detection can be categorized into static analysis and dynamic analysis methods.However, these existing traditional vulnerability detection methods predominantly rely on analyzing original contract code, not all smart contracts provide accessible code.We present ETrace, a novel event-driven vulnerability detection framework for smart contracts, which uniquely identifies potential vulnerabilities through LLM-powered trace analysis without requiring source code access. By extracting fine-grained event sequences from transaction logs, the framework leverages Large Language Models (LLMs) as adaptive semantic interpreters to reconstruct event analysis through chain-of-thought reasoning. ETrace implements pattern-matching to establish causal links between transaction behavior patterns and known attack behaviors. Furthermore, we validate the effectiveness of ETrace through preliminary experimental results.

Figures

Figures reproduced from arXiv: 2506.15790 by the authors.

**Figure 1.** Architecture for ETrace. obtain all function names and their associated parameters. The decoded information is then segmented based on the function names, ultimately yielding fine-grained event information. We categorize event information, into three main components: function name, address, and value. The function name indicates which functions were called between contracts, while the address reveals how many cont… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 13 canonical work pages

[1]

Decentralized exchange,

S. Malamud and M. Rostek, “Decentralized exchange, ” American Economic Review, vol. 107, no. 11, pp. 3320–3362, 2017

work page 2017
[2]

Defort: Automatic detection and analysis of price manipulation attacks in defi applications,

M. Xie, M. Hu, Z. Kong, C. Zhang, Y. Feng, H. Wang, Y. Xue, H. Zhang, Y. Liu, and Y. Liu, “Defort: Automatic detection and analysis of price manipulation attacks in defi applications, ” inProceedings of the 33rd ACM SIGSOFT Interna- tional Symposium on Software Testing and Analysis , 2024, pp. 402–414

work page 2024
[3]

Tokenscout: Early detection of ethereum scam tokens via temporal graph learning,

C. Wu, J. Chen, Z. Zhao, K. He, G. Xu, Y. Wu, H. Wang, H. Li, Y. Liu, and Y. Xiang, “Tokenscout: Early detection of ethereum scam tokens via temporal graph learning, ” inProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security , 2024, pp. 956–970

work page 2024
[4]

An event driven framework for smart contract execution,

M. Kaleem, K. Kasichainula, R. Karanjai, L. Xu, Z. Gao, L. Chen, and W. Shi, “An event driven framework for smart contract execution, ” inProceedings of the 15th ACM International Conference on Distributed and Event-based Systems , 2021, pp. 78–89

work page 2021
[6]

A novel method based on log files for smart contract testing,

F. Salzano and R. Pareschi, “A novel method based on log files for smart contract testing, ” in2023 IEEE/ACM 6th International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB) . IEEE, 2023, pp. 1–4

work page 2023
[7]

Efficient Logging for Blockchain Applications

C. Klinkmüller, I. Weber, A. Ponomarev, A. B. Tran, and W. van der Aalst, “Efficient logging for blockchain applications, ”arXiv preprint arXiv:2001.10281, 2020

work page Pith review arXiv 2001
[8]

Making smart contracts smarter,

L. Luu, D.-H. Chu, H. Olickel, P. Saxena, and A. Hobor, “Making smart contracts smarter, ” inProceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 254–269

work page 2016
[9]

Using my functions should follow my checks: understanding and detecting insecure openzeppelin code in smart contracts,

H. Liu, D. Wu, Y. Sun, H. Wang, K. Li, Y. Liu, and Y. Chen, “Using my functions should follow my checks: understanding and detecting insecure openzeppelin code in smart contracts, ” in33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, 2024, pp. 3585–3601

2024

Show all 15 references

[10]

Smart contract fuzzing towards profitable vulnerabilities,

Z. Kong, C. Zhang, M. Xie, M. Hu, Y. Xue, Y. Liu, H. Wang, and Y. Liu, “Smart contract fuzzing towards profitable vulnerabilities, ”Proceedings of the ACM on Software Engineering, vol. 2, no. FSE, pp. 153–175, 2025

2025
[11]

A survey of mythril, a smart contract security analysis tool for evm bytecode,

N. Sharma and S. Sharma, “A survey of mythril, a smart contract security analysis tool for evm bytecode, ”Indian J Natural Sci , vol. 13, no. 75, pp. 39–41, 2022

2022
[12]

Sereum: Protecting existing smart contracts against re-entrancy attacks,

M. Rodler, W. Li, G. O. Karame, and L. Davi, “Sereum: Protecting existing smart contracts against re-entrancy attacks, ”arXiv preprint arXiv:1812.05934, 2018

2018 arXiv
[13]

Phantom events: Demystifying the issues of log forgery in blockchain,

Y. Liu, Y. Dong, Y. Liu, X. Luo, and Y. Li, “Phantom events: Demystifying the issues of log forgery in blockchain, ”arXiv preprint arXiv:2502.13513, 2025

2025 arXiv
[14]

Study and analysis of chat gpt and its impact on different fields of study,

D. Kalla, N. Smith, F. Samaah, and S. Kuraku, “Study and analysis of chat gpt and its impact on different fields of study, ”International journal of innovative science and research technology , vol. 8, no. 3, 2023

2023
[15]

Advscanner: Generating adversarial smart contracts to exploit reentrancy vulnerabilities using llm and static analysis,

Y. Wu, X. Xie, C. Peng, D. Liu, H. Wu, M. Fan, T. Liu, and H. Wang, “Advscanner: Generating adversarial smart contracts to exploit reentrancy vulnerabilities using llm and static analysis, ” inProceedings of the 39th IEEE/ACM International Conference on Automated Software Engi...

2024
[16]

Skyeye: Detecting imminent attacks via analyzing adversarial smart contracts,

H. Wang, Y. Hu, H. Wu, D. Liu, C. Peng, Y. Wu, M. Fan, and T. Liu, “Skyeye: Detecting imminent attacks via analyzing adversarial smart contracts, ” inPro- ceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, 2024, pp. 1570–1582. Received 5 ...

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Decentralized exchange,

S. Malamud and M. Rostek, “Decentralized exchange, ” American Economic Review, vol. 107, no. 11, pp. 3320–3362, 2017

work page 2017

[2] [2]

Defort: Automatic detection and analysis of price manipulation attacks in defi applications,

M. Xie, M. Hu, Z. Kong, C. Zhang, Y. Feng, H. Wang, Y. Xue, H. Zhang, Y. Liu, and Y. Liu, “Defort: Automatic detection and analysis of price manipulation attacks in defi applications, ” inProceedings of the 33rd ACM SIGSOFT Interna- tional Symposium on Software Testing and Analysis , 2024, pp. 402–414

work page 2024

[3] [3]

Tokenscout: Early detection of ethereum scam tokens via temporal graph learning,

C. Wu, J. Chen, Z. Zhao, K. He, G. Xu, Y. Wu, H. Wang, H. Li, Y. Liu, and Y. Xiang, “Tokenscout: Early detection of ethereum scam tokens via temporal graph learning, ” inProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security , 2024, pp. 956–970

work page 2024

[4] [4]

An event driven framework for smart contract execution,

M. Kaleem, K. Kasichainula, R. Karanjai, L. Xu, Z. Gao, L. Chen, and W. Shi, “An event driven framework for smart contract execution, ” inProceedings of the 15th ACM International Conference on Distributed and Event-based Systems , 2021, pp. 78–89

work page 2021

[5] [6]

A novel method based on log files for smart contract testing,

F. Salzano and R. Pareschi, “A novel method based on log files for smart contract testing, ” in2023 IEEE/ACM 6th International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB) . IEEE, 2023, pp. 1–4

work page 2023

[6] [7]

Efficient Logging for Blockchain Applications

C. Klinkmüller, I. Weber, A. Ponomarev, A. B. Tran, and W. van der Aalst, “Efficient logging for blockchain applications, ”arXiv preprint arXiv:2001.10281, 2020

work page Pith review arXiv 2001

[7] [8]

Making smart contracts smarter,

L. Luu, D.-H. Chu, H. Olickel, P. Saxena, and A. Hobor, “Making smart contracts smarter, ” inProceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 254–269

work page 2016

[8] [9]

Using my functions should follow my checks: understanding and detecting insecure openzeppelin code in smart contracts,

H. Liu, D. Wu, Y. Sun, H. Wang, K. Li, Y. Liu, and Y. Chen, “Using my functions should follow my checks: understanding and detecting insecure openzeppelin code in smart contracts, ” in33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, 2024, pp. 3585–3601

2024