Detecting State Manipulation Vulnerabilities in Smart Contracts Using LLM and Static Analysis

Haijun Wang; Hao Wu; Ming Fan; Shangwang Li; Ting Liu; Yin Wu; Yitao Zhao

REVIEW 3 major objections 6 minor 13 references

Detecting State Manipulation Vulnerabilities in Smart Contracts Using LLM and Static Analysis

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that LLM semantic reasoning, anchored by backward dependency and forward propagation graphs, detects price-manipulation attack contracts before they execute.

desk verdict A clearly written four-page feasibility study of a three-role LLM pipeline over static dependency and propagation graphs for DeFi price manipulation; the four-case demo supports plausibility, but the lack of controls and overreaching practicality claim need fixing. read the letter →

arxiv 2506.08561 v2 pith:JMGEZTHS submitted 2025-06-10 cs.SE

Hao Wu , Haijun Wang , Shangwang Li , Yin Wu , Ming Fan , Yitao Zhao , Ting Liu This is my paper · ORCID

classification cs.SE

keywords SmartContractLargeLanguageModelVulnerabilityDetectionDeFiPriceManipulationStaticAnalysisStateProactiveAttack

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes PriceSleuth, a pipeline that couples a large language model with static program analysis to find price-manipulation (PM) attack contracts in DeFi protocols before the attack executes. The claim is that the LLM can read the semantics of price calculation mechanisms while dependency and taint graphs supply the program structure, so detection does not need hand-written attack patterns. On four real incidents drawn from four protocol types, PriceSleuth reports identifying the manipulated price statement and the subsequent malicious transfer, with each analysis finishing within one minute. If that holds, newly deployed attack contracts can be screened in the window between deployment and exploitation, which transaction-based detectors cannot offer.

What carries the argument

The machinery is a pair of program graphs plus a three-role LLM prompt pipeline. The price calculation dependency graph (PCDG), built by backward data-dependency analysis, records how the price variable is derived and which inputs feed it. The price exploitation propagation graph (PEPG), built by taint propagation, records where the manipulated price flows. The Reasoner, Detector, and Verifier prompts each consume one code slice or graph, with zero-shot chain-of-thought reasoning, so the static analysis delimits the semantic judgement the LLM has to make at each step.

What would settle it

Run PriceSleuth on safe DeFi contracts containing flashloan-funded swaps with no attack, and check how often the Detector and Verifier still say True; if the false-alarm rate is high, the one-minute proactive alert loses its value. Alternatively, replay a held-out set of known PM attacks and count how many the pipeline misses, which would directly refute the claimed generalization.

Watch

Extended reading notes

Core claim

The central discovery is that a price-manipulation attack can be established from the contract itself by splitting the question into three LLM judgements that are each anchored by a static graph. A Reasoner LLM first extracts the price calculation mechanism from the code of the located logic function. A Detector LLM then examines the backward price calculation dependency graph (PCDG) and the accompanying code slice to decide whether the price is externally controllable. A Verifier LLM finally examines the forward taint propagation graph (PEPG) to decide whether the manipulated price reaches a transfer or other harmful sink. The paper reports that this three-stage split makes LLM reasoning work across DEX, lending, staking-based and deposit-based yield farming protocols, and that the detected manipulation and propagation relationships match the public security reports for all four incidents.

Load-bearing premise

The method assumes the LLM reliably identifies the true price-calculation statements and correctly decides whether a price is manipulable and then exploited, a judgment the paper never measures against ground truth.

Editorial extensions

If this is right

Attack contracts can be screened between deployment and first use, since the pipeline consumes bytecode and open-source DeFi code and finishes in about a minute.
The same manipulable-price pattern is detectable across DEX, lending, staking-based and deposit-based yield farming, so protocol-specific rules are not required.
The output is not just a flag: the identified price manipulation relationship and malicious propagation relationship can be checked against security reports.
Code-level analysis catches attacks that have no prior on-chain transaction pattern, extending detection beyond post-hoc transaction monitoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the paper does not pursue is running the pipeline continuously on newly deployed contracts to turn the one-minute analysis into an alert service; the evaluation does not demonstrate this.
The three-stage split could be reused for other manipulable state variables such as reward rates, reserve ratios, or collateral factors, which the paper lists only as future work.
A decisive test the paper leaves undone is measuring the Reasoner's extraction accuracy against hand-labeled price statements; without that number, the dependency and propagation graphs cannot be credited for the final detections.
Because the paper biases traversal toward flashloan callbacks, attacks funded by ordinary capital rather than flashloans may receive lower priority and need a different prioritization rule.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear Colleague,

You should know this paper is a four-page New Idea Track feasibility study, not a full evaluation. It does one thing worth noting: it splits the LLM's job into three roles—Reasoner, Detector, Verifier—and feeds each one a different static-analysis artifact (PCDG for backward dependency, PEPG for taint propagation). That structure over graph representations of price calculation is not in GPTScan or DeFiScope, so it is a genuine, if incremental, architectural contribution. The writing is clear, the pipeline is described in enough detail to reproduce, and the four incidents cover DEX, lending, and two flavors of yield farming.

The soft spot is exactly where the stress-test points: the evaluation is four positive examples from a public incident repository, all run through the same ChatGPT-4o pipeline, with no negative set, no baseline without the graphs, no ablation of the three roles, and no artifact. That means Table 1 cannot tell us whether the static graphs do the work, whether the LLM recognizes these well-known incidents from training, or whether the LLM would flag any price-read-plus-transfer pattern as manipulable. The paper's own intro overreaches when it calls this "a practical solution for proactive attack detection"—with no false-positive rate, that phrase is not supported. There are also two internal inconsistencies: Section 3.1.2 says xCCG is constructed, Section 5 says recovering xCCG is future work; and the Figure 2 labels flip between "Forward" and "Backward" dependency analysis.

That said, the paper is honest about being preliminary, and the pipeline is plausible. The central feasibility claim—that an LLM guided by static dependency and propagation graphs can detect PM attacks in four different protocol types—holds up as a demonstration. What does not hold up is the generalization from four positives to a practical detection system.

I would send this to peer review. It is a reasonable New Idea submission that would benefit from referees pushing for controls. The right outcome is probably major revision: add a negative set, an ablation, and reword the generalization claims.

Referee Report

3 major / 6 minor

Summary. The paper proposes PriceSleuth, a pipeline for proactive detection of price manipulation (PM) attack contracts in DeFi. It decompiles bytecode with Gigahorse, constructs an inter-contract call graph, uses a ChatGPT-4o Reasoner with zero-shot prompting to locate price calculation mechanisms (PCMs), builds backward price-calculation dependency graphs (PCDG) via Slither, uses a Detector LLM to decide whether the price is manipulable, and finally constructs price exploitation propagation graphs (PEPG) and uses a Verifier LLM to confirm malicious exploitation. The authors report preliminary results on four real incidents from DeFiHackLabs (BTB, HFLH, WarpStaking, WOOFi), all detected within one minute, with manipulation and propagation relationships matching incident reports. Future work and limitations are discussed in Section 5.

Significance. If the central claim were fully supported, PriceSleuth would be a useful blueprint for combining LLM semantic reasoning with static dependency and taint analysis for proactive DeFi attack detection. The paper has concrete strengths: it anchors the pipeline to real incidents, uses independent static analyzers (Gigahorse and Slither), provides explicit prompt templates, and has no fitted parameters or circular derivation. However, the current evidence is limited to four positive examples with no negative set, no precision or recall measurement, no repeated runs, no baseline or ablation, and no released artifact. The paper should therefore be read as a feasibility demonstration; the general and practical claims in Sections 1 and 4.2 go beyond what Table 1 can support.

major comments (3)

[Section 4.2, Table 1] The entire experimental evidence consists of four positive samples selected from the public DeFiHackLabs repository. There is no negative or benign set, no precision or recall measurement, and no repeated runs. Since these four incidents are public and well known, the claim in Section 4.2 that PriceSleuth is capable of detecting PM attacks in different types of DeFi protocols and is a practical solution for proactive attack detection is not supported by the data: the results only show that the pipeline can flag four known attack contracts. At minimum, the evaluation needs a set of benign contracts that also contain flashloan calls, getReserves() reads, and transfer logic, together with precision and recall numbers and run-to-run stability.
[Sections 3.2.2, 3.3, Table 1] No control condition separates the static-analysis components (PCDG and PEPG) from the LLM's prior knowledge. In every case the same ChatGPT-4o model sees both the code and the graphs, so the reported True decisions could arise from the LLM recognizing the incident or its code pattern rather than from the proposed integration. An ablation that supplies the LLM with (i) raw code only and (ii) code with the dependency and propagation graphs removed, plus a static-analysis-only baseline, is necessary to attribute the result to PriceSleuth's mechanism.
[Sections 3.1.3 and 3.2.2] The intermediate LLM outputs that the pipeline depends on, namely PCM extraction, dependency-based manipulation decisions, and propagation verification, are never validated against ground truth. The Reasoner prompt asks for a confidence score helping to select answers, but no data are reported on the accuracy or stability of these extractions. If the Reasoner misses or invents a price calculation statement, the subsequent Slither-based dependency and propagation analysis cannot compensate, so without a ground-truth comparison for PCM extraction and repeated runs under varied sampling, the one-minute detection result cannot be expected to generalize.

minor comments (6)

[Section 2, Reference [4]] The text says "Augest" instead of "August," and Reference [4] is titled "EDG Finance" while the body text consistently uses "EGD Finance." These should be corrected.
[Figures 1 and 2] Both figures contain repeated and unrelated pipeline diagrams (e.g., "Rug Pull Detection," "SRCG," "TFBG," "RGCN Learning," "UAGNN Learning") that are not part of PriceSleuth and appear to be leftover material from another work. These fragments should be removed because they obscure the actual architecture.
[Figure 2 vs. Sections 3.2.1 and 3.3] Figure 2 labels conflict with the text: the figure shows "Forward Data Dependency Analysis" where Section 3.2.1 describes backward data dependency analysis, and it shows "Backward Taint Data Propagation Analysis" where Section 3.3 describes forward propagation slices. The figure and text must be reconciled.
[Section 3.1.3, Figure 3] The prompt is described as zero-shot chain-of-thought, but it also instructs the model to "simulate answer my question in the background five times and give me the most frequently appearing answers." This is self-consistency or majority voting rather than standard chain-of-thought, and should be described accurately.
[Section 4.2] The statement that PriceSleuth "successfully completed the analysis within one minute" is reported without a measurement methodology. Please specify whether the time is per contract, whether it includes LLM API latency, and what hardware and network conditions were used.
[Section 5 and Reproducibility] Section 5 says the method is limited to function-level analysis and plans to recover xCCG, but Section 3.1.2 already states that PriceSleuth constructs the xCCG; this contradiction should be clarified. Additionally, no artifact or complete prompt data is provided, which limits reproducibility; a repository with prompts, graph formats, and evaluation scripts would strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the pipeline's outputs are not defined by its inputs, and the self-citations are ancillary.

full rationale

PriceSleuth's derivation chain is self-contained and externally anchored: PCM extraction is performed by the Reasoner LLM on code snippets; the PCDG is built by Slither backward dependency analysis; the PEPG is built by Slither taint propagation; and the final alert is the Verifier LLM's True/False decision. None of these steps is defined in terms of the output, and no parameter is fitted to the four evaluation incidents. The paper's claim that 'PriceSleuth successfully detected the four PM attacks' is an empirical report on external incidents from DeFiHackLabs, not a construction. The only self-citations ([6], [9]) are to published prior work used for a probabilistic callee-matching heuristic and as motivational support for the deployment-to-attack interval; these are not invoked to forbid alternatives or to justify the central vulnerability-detection claim. Concerns that ChatGPT-4o may have memorized the four public incidents, and the absence of negative controls, are evaluation-validity threats rather than circularity; they do not make the pipeline's output equivalent to its input by construction. There is also an internal inconsistency between the claimed xCCG construction in Section 3.1.2 and the Future Plans statement that xCCG recovery remains to be done, but this is a consistency and limitation issue, not a circular step.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No numeric free parameters are introduced; the LLM confidence scores are part of the prompt protocol rather than fitted quantities used to derive results. The listed assumptions are untested domain premises on which the whole pipeline rests. No new physical or conceptual entities are postulated; PCDG, PEPG, and xCCG are internal graph representations of existing code, not entities with independent falsifiable handles.

assumptions (5)

domain assumption Price manipulation attacks are rooted in manipulable price calculation mechanisms that are identifiable from contract source and dependency graphs.
Used throughout Sections 2 and 3 to justify that PCM extraction plus dependency analysis is the right detection target; no evidence is given that all PM attacks have this form.
domain assumption ChatGPT-4o with zero-shot chain-of-thought prompts extracts the correct price calculation statements and makes correct manipulation judgments.
Invoked in Sections 3.1.3, 3.2.2, and 3.3; no evaluation of LLM output accuracy, hallucination rate, or stability is provided.
domain assumption Flashloan callback signatures in the inter-contract call graph identify likely attack paths, so prioritizing them improves detection.
Section 3.1.2 assigns traversal priority based on this hypothesis; no ablation tests it.
domain assumption Slither's backward data dependency and taint propagation on open-source DeFi code are sound enough to build PCDG and PEPG.
Sections 3.2.1 and 3.3 rely on these graphs as faithful summaries of all relevant influences on and uses of the price variable.
domain assumption The four selected incidents from DeFiHackLabs are representative of price manipulation attacks in DEX, lending, staking-based, and deposit-based protocols.
Section 4.1 selects one incident per category; the generalization claim in Section 4.2 depends on this representativeness.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Detecting State Manipulation Vulnerabilities in Smart Contracts Using LLM and Static Analysis." pith.science (2026). https://pith.science/paper/JMGEZTHS

@misc{pith2026250608561,
  author       = {Pith},
  title        = {Pith review of: Detecting State Manipulation Vulnerabilities in Smart Contracts Using LLM and Static Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JMGEZTHS}},
  note         = {Machine review of arXiv:2506.08561}
}

read the original abstract

An increasing number of DeFi protocols are gaining popularity, facilitating transactions among multiple anonymous users. State Manipulation is one of the notorious attacks in DeFi smart contracts, with price variable being the most commonly exploited state variable-attackers manipulate token prices to gain illicit profits. In this paper, we propose PriceSleuth, a novel method that leverages the Large Language Model (LLM) and static analysis to detect Price Manipulation (PM) attacks proactively. PriceSleuth firstly identifies core logic function related to price calculation in DeFi contracts. Then it guides LLM to locate the price calculation code statements. Secondly, PriceSleuth performs backward dependency analysis of price variables, instructing LLM in detecting potential price manipulation. Finally, PriceSleuth utilizes propagation analysis of price variables to assist LLM in detecting whether these variables are maliciously exploited. We presented preliminary experimental results to substantiate the effectiveness of PriceSleuth . And we outline future research directions for PriceSleuth.

Figures

Figures reproduced from arXiv: 2506.08561 by the authors.

**Figure 2.** shows the overview design of PriceSleuth, consisting of three main modules. Initially, given the bytecode of a smart contract, PriceSleuth leverages the open-source decompiler [3] to decompile the contract bytecode and extract behavior information. Then it constructs the inter-contract call graph (xCCG). By leveraging the xCCG and explorer (e.g.,Etherscan), PriceSleuth locates the logic function, which is used by t… view at source ↗

**Figure 4.** , to guide the LLM in analyzing the code logic and identifying price manipulation. Similar to Section 3.1.3, we first define the role of the model. Then, we outline the task using chain-of-thought (CoT) patterns to instruct the LLM in deducing the process of potential price manipulation and provide key explanations. Specifically, the LLM follows a step-by-step reasoning process. It first examines the definition of… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 7 canonical work pages

[1]

DeFiHackLabs. 2025. https://github.com/SunWeb3Sec/DeFiHackLabs/tree/main. Accessed: March, 2025

work page 2025
[2]

Josselin Feist, Gustavo Grieco, and Alex Groce. 2019. Slither: a static analysis framework for smart contracts. In2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB). IEEE, 8–15

2019
[3]

Neville Grech, Lexi Brent, Bernhard Scholz, and Yannis Smaragdakis. 2019. Giga- horse: thorough, declarative decompilation of smart contracts. In2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 1176–1186

2019
[4]

EDG Finance Attack Incident. 2025. https://x.com/BlockSecTeam/status/ 1556483435388350464. Accessed: March, 2025

work page 2025
[5]

Queping Kong, Jiachi Chen, Yanlin Wang, Zigui Jiang, and Zibin Zheng. 2023. Defitainter: Detecting price manipulation vulnerabilities in defi protocols. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1144–1156

work page 2023
[6]

Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Haijun Wang, Zhengzi Xu, Xiaofei Xie, and Yang Liu. 2024. Gptscan: Detecting logic vulnerabilities in smart contracts by combining gpt with program analysis. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13

2024
[7]

OpenAi Fine tuning Guideline. 2024. https://platform.openai.com/docs/guides/ fine-tuning. Accessed: March, 2025

work page 2024
[8]

Dabao Wang, Bang Wu, Xingliang Yuan, Lei Wu, Yajin Zhou, and Helei Cui. 2024. Defiguard: A price manipulation detection service in defi using graph neural networks.IEEE Transactions on Services Computing(2024)

2024

Show all 13 references

[9]

Haijun Wang, Yurui Hu, Hao Wu, Dijun Liu, Chenyang Peng, Yin Wu, Ming Fan, and Ting Liu. 2024. Skyeye: Detecting Imminent Attacks via Analyzing Adversarial Smart Contracts. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 1570–1582

2024
[10]

Siwei Wu, Zhou Yu, Dabao Wang, Yajin Zhou, Lei Wu, Haoyu Wang, and Xingliang Yuan. 2024. DeFiRanger: Detecting DeFi Price Manipulation Attacks. IEEE Transactions on Dependable and Secure Computing21, 4 (2024), 4147–4161

2024
[11]

Bosi Zhang, Ningyu He, Xiaohui Hu, Kai Ma, and Haoyu Wang. 2025. Following Devils’ Footprint: Towards Real-time Detection of Price Manipulation Attacks. arXiv preprint arXiv:2502.03718(2025)

2025 arXiv
[12]

Zhuo Zhang, Brian Zhang, Wen Xu, and Zhiqiang Lin. 2023. Demystifying ex- ploitable bugs in smart contracts. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 615–627

2023
[13]

Juantao Zhong, Daoyuan Wu, Ye Liu, Maoyi Xie, Yang Liu, Yi Li, and Ning Liu. 2025. DeFiScope: Detecting Various DeFi Price Manipulations with LLM Reasoning.arXiv preprint arXiv:2502.11521(2025)

2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

DeFiHackLabs. 2025. https://github.com/SunWeb3Sec/DeFiHackLabs/tree/main. Accessed: March, 2025

work page 2025

[2] [2]

Josselin Feist, Gustavo Grieco, and Alex Groce. 2019. Slither: a static analysis framework for smart contracts. In2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB). IEEE, 8–15

2019

[3] [3]

Neville Grech, Lexi Brent, Bernhard Scholz, and Yannis Smaragdakis. 2019. Giga- horse: thorough, declarative decompilation of smart contracts. In2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 1176–1186

2019

[4] [4]

EDG Finance Attack Incident. 2025. https://x.com/BlockSecTeam/status/ 1556483435388350464. Accessed: March, 2025

work page 2025

[5] [5]

Queping Kong, Jiachi Chen, Yanlin Wang, Zigui Jiang, and Zibin Zheng. 2023. Defitainter: Detecting price manipulation vulnerabilities in defi protocols. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1144–1156

work page 2023

[6] [6]

Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Haijun Wang, Zhengzi Xu, Xiaofei Xie, and Yang Liu. 2024. Gptscan: Detecting logic vulnerabilities in smart contracts by combining gpt with program analysis. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13

2024

[7] [7]

OpenAi Fine tuning Guideline. 2024. https://platform.openai.com/docs/guides/ fine-tuning. Accessed: March, 2025

work page 2024

[8] [8]

Dabao Wang, Bang Wu, Xingliang Yuan, Lei Wu, Yajin Zhou, and Helei Cui. 2024. Defiguard: A price manipulation detection service in defi using graph neural networks.IEEE Transactions on Services Computing(2024)

2024