REVIEW 5 major objections 5 minor 27 references
BLOCKS: Blockchain-supported Cross-Silo Knowledge Sharing for Efficient LLM Services
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read BLOCKS claims that blockchain can turn isolated private knowledge into trustworthy LLM context by sharing distilled prompts instead of raw data.
desk verdict Real implementation and a sensible cache, but the security proof does not hold: the central trustworthiness claim rests on an unvalidated assumption. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism has three parts. First, a smart-contract reputation system defined by equations (2)–(5) tracks interdependent reputations of LLM services, prompts, knowledge providers, and validators, using consistency (CS) and confidence (CF) metrics, with an exponential moving-average update. Second, Proof of Impact (PoI) consensus lets the next block be proposed by the knowledge provider with the highest impact $C = R(\beta A_p + (1-\beta) A_v)$, where $R$ is reputation and $A_p, A_v$ are access counts for prompt generation and validation; rewards are distributed proportionally to reputation, so equation (12) drives malicious providers' rewards to zero. Third, PROCache computes a priority score $\text{Priority} = (\text{Frequency} \times \text{Cost} / \text{Size})^{R_t - R_b}$ that promotes prompts based on both access utility and reputation, defending the cache against adversarial pollution. Supporting components include the Query Generation Module (QGM) for turning queries into retrieval rationales and a hash-bucket key-value store layered over the Cosmos IA VL-Tree for compact on-chain storage.
What would settle it
Run BLOCKS against an adversarial benchmark of indirect prompt injection attacks specifically crafted to pass low-cost validator checks; if a malicious prompt's reputation stays high, or a provider whose prompts are later shown harmful continues to receive rewards, then Assumption 1 fails and the security guarantees in equations (10) and (12) collapse.
Extended reading notes
Core claim
On the paper's own terms, the result is a complete four-role blockchain workflow—knowledge providers, LLM servers, validators, and external storage—that turns distributed private knowledge into retrievable, verifiable prompts. The discovery claim is that the combination of reputation formulas, threshold validation against prompt injection, and Proof of Impact consensus aligns incentives such that malicious providers' reputations and rewards converge to zero while honest providers' reputations stabilize at the true quality level of their contributions. The paper reports that this design, implemented on a Cosmos-based chain, achieves efficient LLM knowledge sharing in a blockchain environment, with PROCache reducing ledger storage by approximately 80% and maintaining higher in-cache reputation than LFU or LRU-2 baselines. To the best of the authors' knowledge, this is the first blockchain-based external knowledge framework tailored specifically for LLMs.
Load-bearing premise
The whole reward and security story depends on honest validators actually being able to tell malicious prompts from honest ones, so that malicious content reliably receives lower scores.
Editorial extensions
If this is right
- Knowledge silos can participate without publishing raw data, because only distilled prompts, hashes, and reputation records go on-chain.
- Under the Byzantine bound $f < n/3$, malicious actors using self-promotion, collusion, or slandering see their reputations and rewards converge to zero, making honest contribution the stable strategy.
- PROCache reduces blockchain ledger storage by about 80% in the reported setup while keeping in-cache reputation higher than LFU and LRU-2 baselines, addressing a central QoS bottleneck.
- LLM services gain a direct API path through QGM plus PROCache retrieval, allowing external knowledge to be incorporated without requiring silos to expose their underlying datasets.
Reading between the lines
- If Assumption 1 holds, the same reputation-plus-PoI machinery could be reused for other markets of untrusted machine-generated content, such as shared datasets, summaries, or code snippets, not only LLM prompts.
- A natural extension the paper leaves implicit is measuring end-to-end LLM answer quality, for example hallucination rates with and without BLOCKS on a fixed multi-silo benchmark, rather than only reputation and cache metrics.
- The PROCache priority formula could be extracted as a standalone defense against adversarial cache pollution in any retrieval system, independent of its blockchain setting.
- The paper's security analysis assumes honest validators can reliably distinguish malicious prompts; an empirical stress test against state-of-the-art indirect prompt injection attacks would show how far this assumption extends in practice.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes BLOCKS, a Cosmos/Tendermint-based framework for cross-silo knowledge sharing tailored to LLM services. Knowledge is distilled into prompts on-chain; a smart-contract reputation mechanism with cross-validation, a Proof-of-Impact (PoI) consensus, a PROCache priority cache, and a hash-bucket key-value store are introduced to handle incentives, quality of service, and security. The experiments use 11 nodes and 253 questions from WikiQA, TruthfulQA, and MathQA, reporting reputation decay of malicious nodes, cache delay/reputation comparisons against LFU and LRU-k, and an approximately 80% reduction in blockchain storage.
Significance. If the security and incentive properties were rigorously established, BLOCKS would address a real problem: coordinating untrusted knowledge silos for LLM retrieval. The paper ships a concrete implementation on Cosmos SDK with a public code link, and the measured storage reduction and PROCache comparison are concrete empirical contributions. However, the central trustworthiness claim is not established: the security analysis rests on an unvalidated assumption and on convergence claims that do not follow from the equations as written. The evaluation is too small and too scripted to substitute for that missing analysis.
major comments (5)
- [Section IV-B, Eqs. (2)-(5), (10)] The claim that malicious reputations converge to zero is not derivable from the stated update rules. Eqs. (2) and (5) write R_l and R_v on both sides; if these are interpreted as fixed-point equations, their nontrivial fixed point is 1 - Mean(CS), which can be positive under the stated Byzantine bound (for example, with n=11 and f=3, a minority malicious validator can have CS around 0.8, yielding a limiting reputation of about 0.2). Eq. (10) needs a time-indexed derivation that explicitly shows how Mean(CS) for malicious nodes and the threshold mechanism drive reputation to zero; otherwise the PoI reward exclusion in Eq. (12) is unsupported.
- [Section III-A and Section V-B, Assumption 1] Assumption 1 is load-bearing but never validated. Section III-A delegates attack detection to a low-cost model plus a threshold, but the paper reports no discrimination accuracy for that model. Section V-B simulates predetermined malicious strategies and plots the reputation trajectories; because those trajectories are produced by the same reputation equations being tested, they do not demonstrate that honest validators can reliably distinguish malicious from honest prompts out-of-sample. Please report the validation model's detection accuracy (e.g., precision/recall or ROC) on held-out malicious prompts, or soften the security claims accordingly.
- [Section III-A, Eq. (1), and Eq. (7)] Several equations cannot be checked because symbols are undefined or inconsistently used. In Eq. (1), m is defined as the number of validations but does not appear, while n appears both as the number of LLM services and as the denominator size. In Eq. (7), R_b is introduced as a 'reputation threshold' but its value and update rule are not given, and the priority expression has no clear range when R_t < R_b. Please define every symbol where it first appears and align the notation across Eqs. (1)-(8).
- [Section IV-A, Eq. (8)] The reward formula U_p = R_b × R_p / Σ R_p reuses R_b, which was previously a cache reputation threshold, and does not define its value or relationship to provider reputation. This appears to conflate two different quantities. Please clarify whether R_b is a minimum reputation eligibility threshold or a typo for another variable, and state how the constant is chosen.
- [Section V-B] The empirical evaluation is too small to support the strength of the claims. The setup uses 11 nodes and 253 questions, and Figs. 4-5 report a single simulation trajectory with confidence intervals across agents rather than across repeated runs with different random seeds. The security experiments would be more convincing with multiple seeds, statistical error bars, and a direct comparison of the validation model's detection performance against the simulated attack strategies.
minor comments (5)
- [Section IV heading] The heading 'PUTEVERYTHINGTOGETHER' appears to be a typo; it should read 'PUT EVERYTHING TOGETHER.'
- [Reference [27]] The URL for MathQA is misspelled as 'aclantholoy.org'; it should be 'aclanthology.org.'
- [Fig. 7] Figures 6 and 7 would benefit from explicit axis labels and units; in Fig. 7, 'Number of index' should be 'Number of stored prompt indices.'
- [Section III-A, Eq. (3)] The notation in Eq. (3) uses V_i, R_i, Acc, and R_l without defining the index ranges or the exact meaning of 'Acc' (presumably accuracy); please define these terms.
- [Abstract and Section V] The abstract describes 'extensive experiments,' but the evaluation uses subsets of three datasets and 253 questions; please calibrate the wording to the actual scale.
Circularity Check
Security validation is self-referential: Eq. (10)'s reputation decay is built into Assumption 1 and the update rules (2)-(5), so the simulation 'validates' the mechanism by running the mechanism itself; no independent detection accuracy is reported.
-
self definitional
[Section IV-B.1, Eq. (10); Section III-A, Eqs. (2)-(5); Section V-B, Reputation Security]
"As shown in Eq. (2) and Eq. (5), both malicious users who provide distorted feedback and validators who perform incorrect validation suffer a decrease in reputation over time, ultimately reaching zero as iteration t→ ∞: lim_{t→∞} R_t_l = 0, lim_{t→∞} R_t_v = 0. ... The findings indicate that the reputation of malicious nodes decays to zero across all adversarial strategies, while honest nodes maintain stable and significantly higher reputations, validating the effectiveness of our reputation mechanism."
Eqs. (2) and (5) are the reputation update rules; Assumption 1 already states that malicious prompts are identified and receive lower scores, and Section III-A adds a threshold that 'directly recognize[s] this node as a malicious node and apply[ies] penalty.' Iterating the same update with low scores or a penalty is exactly the decay reported in Eq. (10). The experiment then simulates this identical update rule and scripted malicious behavior and presents the resulting decay as 'validating' the mechanism. The reputation-security result is therefore contained in the construction of the reputation score and the assumed detection capability, not derived from an independent measurement of detection accuracy or out-of-sample adversarial behavior.
-
self definitional
[Section IV-B.2, Eq. (12)]
"Since R t p converges to V ∗ p and malicious providers experience reputation degradation, it follows that: lim_{t→∞} W t_{km} = 0, for all malicious providers."
Eq. (12) is not an independent security result; it inherits Eq. (10). The premise 'malicious providers experience reputation degradation' is the same reputation-update construction, so the reward-exclusion conclusion is already present in the mechanism's definition. Moreover, if interpreted strictly, convergence to V*_p alone does not imply zero rewards; the conclusion only holds if the reputation mechanism has already been assumed to drive malicious reputations to zero—which is exactly the result being proved.
full rationale
The paper's load-bearing security claim is that malicious nodes' reputations decay to zero and malicious providers earn zero rewards. On inspection, this claim does not come from an independent prediction: it is produced by iterating the paper's own update rules (2)-(5) under Assumption 1 and the threshold penalty, and the experimental section validates the mechanism by simulating the same equations with scripted malicious agents. That is a partial self-definitional reduction: the security outcome is an algebraic consequence of the assumed detector and the update rule, not a separately tested or measured result. Separately, Eq. (10) is not actually implied by Eqs. (2)-(5) for f<n/3 when CS is bounded away from 1; this correctness gap reinforces the self-referential nature of the validation. The paper also self-cites its prior blockchain work ([7]-[9]) and a co-authored reputation-sharing paper ([24]), but those citations are motivational, not load-bearing, so they do not raise the score further. The cache, storage, and QoS evaluations are compared against external baselines (LFU, LRU-2) and are not circular. Overall, the circularity is real but partial, hence a score of 6 rather than 8 or 10.
Assumptions & free parameters
free parameters (5)
- alpha (EMA coefficient)
- beta (PoI reward balance)
- k (cache promotion threshold)
- R_b (reputation threshold)
- similarity threshold for validation
assumptions (4)
- domain assumption Assumption 1: Honest validators can identify malicious prompts and score them lower than honest prompts.
- domain assumption Assumption 2: Malicious validators do not gain an excessive advantage in validation over honest validators; formalized by inequality (9).
- domain assumption Byzantine adversary bound f < n/3.
- domain assumption Honest participants' claims are expected to be consistent.
invented entities (2)
-
PROCache
-
Proof of Impact (PoI) consensus
Cite this review
Pith. "Pith review of BLOCKS: Blockchain-supported Cross-Silo Knowledge Sharing for Efficient LLM Services." pith.science (2026). https://pith.science/paper/6T63MNCH
@misc{pith2026250621033,
author = {Pith},
title = {Pith review of: BLOCKS: Blockchain-supported Cross-Silo Knowledge Sharing for Efficient LLM Services},
year = {2026},
howpublished = {\url{https://pith.science/paper/6T63MNCH}},
note = {Machine review of arXiv:2506.21033}
}
read the original abstract
The hallucination problem of Large Language Models (LLMs) has increasingly drawn attention. Augmenting LLMs with external knowledge is a promising solution to address this issue. However, due to privacy and security concerns, a vast amount of downstream task-related knowledge remains dispersed and isolated across various "silos," making it difficult to access. To bridge this knowledge gap, we propose a blockchain-based external knowledge framework that coordinates multiple knowledge silos to provide reliable foundational knowledge for large model retrieval while ensuring data security. Technically, we distill knowledge from local data into prompts and execute transactions and records on the blockchain. Additionally, we introduce a reputation mechanism and cross-validation to ensure knowledge quality and provide incentives for participation. Furthermore, we design a query generation framework that provides a direct API interface for large model retrieval. To evaluate the performance of our proposed framework, we conducted extensive experiments on various knowledge sources. The results demonstrate that the proposed framework achieves efficient LLM service knowledge sharing in blockchain environments.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
A review of current trends, techniques, and challenges in large language models (llms),
R. Patil and V . Gudivada, “A review of current trends, techniques, and challenges in large language models (llms),”Appl. Sci., vol. 14, no. 5, p. 2074, 2024
work page 2024
-
[2]
S. Zhao, Y . Yang, Z. Wang, Z. He, L. K. Qiu, and L. Qiu, “Retrieval augmented generation (rag) and beyond: A comprehensive survey on how to make your llms use external data more wisely,”arXiv preprint arXiv:2409.14924, 2024
arXiv 2024
-
[3]
X. Li, R. Zhao, Y . K. Chia, B. Ding, S. Joty, S. Poria, and L. Bing, “Chain-of-knowledge: Grounding large language models via dynamic knowledge adapting over heterogeneous sources,” inICLR, 2024
work page 2024
-
[4]
Injecting domain-specific knowledge into large language models: A comprehensive survey,
Z. Song, B. Yan, Y . Liu, M. Fang, M. Li, R. Yan, and X. Chen, “Injecting domain-specific knowledge into large language models: A comprehensive survey,”arXiv preprint arXiv:2502.10708, 2025
arXiv 2025
-
[5]
Autowebglm: A large language model-based web navigating agent,
H. Lai, X. Liu, I. L. Iong, S. Yao, Y . Chen, P. Shen, H. Yu, H. Zhang, X. Zhang, Y . Donget al., “Autowebglm: A large language model-based web navigating agent,” inProc. 30th ACM SIGKDD, 2024, pp. 5295– 5306
work page 2024
-
[6]
Benchmarking and defending against indirect prompt injection attacks on large language models,
J. Yi, Y . Xie, B. Zhu, K. Hines, E. Kiciman, G. Sun, X. Xie, and F. Wu, “Benchmarking and defending against indirect prompt injection attacks on large language models,”arXiv preprint arXiv:2312.14197, 2023
arXiv 2023
-
[7]
S. Yuan, Q. Zhou, J. Li, S. Guo, H. Chen, C. Wu, and Y . Yang, “Adaptive incentive and resource allocation for blockchain-supported edge video streaming systems: A cooperative learning approach,”IEEE Trans. Mobile Comput., vol. 24, no. 2, pp. 539–556, Feb. 2025
work page 2025
-
[8]
S. Yuan, J. Li, and C. Wu, “Jora: Blockchain-based efficient joint computing offloading and resource allocation for edge video streaming systems,”Journal of Systems Architecture, vol. 133, p. 102740, 2022
work page 2022
Show all 27 references
-
[9]
Jira: Joint incentive design and resource allocation for edge-based real-time video streaming systems,
S. Yuan, J. Li, H. Chen, Z. Han, C. Wu, and Y . Zhang, “Jira: Joint incentive design and resource allocation for edge-based real-time video streaming systems,”IEEE Trans. Wireless Commun., vol. 22, no. 5, pp. 2901–2916, May 2023
2023
-
[10]
Blockchain-empowered trustworthy data sharing: Fundamentals, applications, and challenges,
T. L. Nguyen, L. Nguyen, T. Hoang, D. Bandara, Q. Wang, Q. Lu, X. Xu, L. Zhu, and S. Chen, “Blockchain-empowered trustworthy data sharing: Fundamentals, applications, and challenges,”ACM Comput. Surv., vol. 57, no. 8, pp. 1–36, 2025
2025
-
[11]
Coraldb: A collaborative database for data sharing based on permissioned blockchain,
W. Li, W. Tian, Z. Yan, Z. Li, J. Gao, F. Wu, J. Liu, W. Chen, and J. Ren, “Coraldb: A collaborative database for data sharing based on permissioned blockchain,”IEEE Trans. Mob. Comput., vol. 23, no. 9, pp. 8886–8901, 2024
2024
-
[12]
Preserving edge knowledge sharing among iot services: A blockchain-based approach,
G. Li, M. Dong, L. T. Yang, K. Ota, J. Wu, and J. Li, “Preserving edge knowledge sharing among iot services: A blockchain-based approach,” IEEE Trans. Emerg. Top. Comput. Intell., vol. 4, no. 5, pp. 653–665, 2020
2020
-
[13]
An edge intelligent blockchain-based reputation system for iiot data ecosystem,
S. Khezr, A. Yassine, R. Benlamri, and M. S. Hossain, “An edge intelligent blockchain-based reputation system for iiot data ecosystem,” IEEE Trans. Ind. Inform., vol. 18, no. 11, pp. 8346–8355, 2022
2022
-
[14]
Secure and efficient data sharing for iot based on blockchain and reputation mechanism,
W. Yang, C. Hou, Z. Zhang, X. Wang, and S. Chen, “Secure and efficient data sharing for iot based on blockchain and reputation mechanism,” IEEE Internet Things J., 2024
2024
-
[15]
Cosmos whitepaper,
J. Kwon and E. Buchman, “Cosmos whitepaper,”A Netw. Distrib. Ledgers, vol. 27, pp. 1–32, 2019
2019
-
[16]
Bring your own knowledge: A survey of methods for llm knowledge expansion,
M. Wang, A. Stoll, L. Lange, H. Adel, H. Sch ¨utze, and J. Str ¨otgen, “Bring your own knowledge: A survey of methods for llm knowledge expansion,”arXiv preprint arXiv:2502.12598, 2025
2025 arXiv
-
[17]
Llm technologies and information search,
L. Liu, J. Meng, and Y . Yang, “Llm technologies and information search,”J. Econ. Technol., vol. 2, pp. 269–277, 2024
2024
-
[18]
Bertscore: Evaluating text generation with bert,
T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,”arXiv preprint arXiv:1904.09675, 2019
1904 arXiv
-
[19]
Truthfulqa: Measuring how models mimic human falsehoods,
S. Lin, J. Hilton, and O. Evans, “Truthfulqa: Measuring how models mimic human falsehoods,” 2022. [Online]. Available: https: //arxiv.org/abs/2109.07958
2022 arXiv
-
[20]
Wikiqa: A challenge dataset for open- domain question answering,
Y . Yang, W.-t. Yih, and C. Meek, “Wikiqa: A challenge dataset for open- domain question answering,” inEMNLP. Lisbon, Portugal: Association for Computational Linguistics, Sep. 2015, pp. 2013–2018
2015
-
[21]
Slimchain: scaling blockchain transactions through off-chain storage and parallel processing,
C. Xu, C. Zhang, J. Xu, and J. Pei, “Slimchain: scaling blockchain transactions through off-chain storage and parallel processing,”Proc. VLDB Endow., vol. 14, no. 11, p. 2314–2326, Jul. 2021
2021
-
[22]
A survey on replacement strate- gies in cache memory for embedded systems,
P. Panda, G. Patil, and B. Raveendran, “A survey on replacement strate- gies in cache memory for embedded systems,”2016 IEEE DISCOVER, pp. 12–17, 2016
2016
-
[23]
LlamaIndex,
J. Liu, “LlamaIndex,” 11 2022. [Online]. Available: https://github.com/ jerryjliu/llama index
2022
-
[24]
Reputation-based distributed knowledge sharing system in blockchain,
S. Hu, L. Hou, G. Chen, J. Weng, and J. Li, “Reputation-based distributed knowledge sharing system in blockchain,” inMobiQuitous. New York, NY , USA: ACM, 2018, pp. 476–481
2018
-
[25]
A survey on blockchain interoperability: Past, present, and future trends,
R. Belchior, A. Vasconcelos, S. Guerreiro, and M. Correia, “A survey on blockchain interoperability: Past, present, and future trends,”Acm Computing Surveys (CSUR), vol. 54, no. 8, pp. 1–41, 2021
2021
-
[26]
The latest gossip on bft consensus,
E. Buchman, J. Kwon, and Z. Milosevic, “The latest gossip on bft consensus,” 2019. [Online]. Available: https://arxiv.org/abs/1807.04938
2019 arXiv
-
[27]
Mathqa: Towards interpretable math word problem solving with operation-based formalisms,
A. Amini, S. Gabriel, S. Lin, R. Koncel-Kedziorski, Y . Choi, and H. Hajishirzi, “Mathqa: Towards interpretable math word problem solving with operation-based formalisms,” inNAACL-HLT. Association for Computational Linguistics, Jun. 2019, pp. 2357–2367. [Online]. Available: ht...
2019
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.