REVIEW 4 major objections 5 minor 1 cited by
Large Language Model Federated Learning with Blockchain and Unlearning for Cross-Organizational Collaboration
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that a hybrid public/private blockchain federated-learning framework lets organizations co-train large language models without sharing raw data, and that a LoRA-based unlearning loop removes specific data contributions…
desk verdict A plausible systems combo that badly overclaims: the unlearning experiments never measure retain-set utility and the reported accuracies contradict the central promise. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument turns on three working parts. The hybrid ledger uses a public blockchain as an immutable transaction log for model updates and unlearning submissions, and private data collections with endorsement policies restrict sensitive computations to authorized organizations. The LoRA forget adapter is a lightweight low-rank adaptation attached to the model: forward and backward passes on the forget set $D_{\text{forget}}$ run for $E_u$ epochs, and the adapter's adjusted gradients produce a small parameter delta that stands for the unlearning and is what gets verified and uploaded. The multi-agent Q-learning layer gives each organization a Q-table over participation states and actions so resource contributions are chosen to maximize long-term reward. Smart contracts tie these together by checking JWT-based identities, aggregating private-chain models at set epochs, and applying validation criteria before any update or unlearning result is committed to the chain.
What would settle it
Run the unlearning loop on a dataset split into forget and retained portions, then measure accuracy on the retained portion immediately afterward. If retained accuracy falls toward the reported forget-set levels (0.95–1.25% on IMDB and roughly 8% on Twitter, both near chance for these tasks), the claim that selective unlearning preserves overall performance is falsified. A follow-up probe that asks whether the model still behaves differently on the supposed-to-be-forgotten examples than on never-seen examples would settle whether the target data is actually gone.
Extended reading notes
Core claim
The paper's central claim is that trust, privacy, and regulatory compliance can be solved together rather than traded off in cross-organizational LLM training. Its hybrid blockchain architecture keeps a public, immutable record of model updates and transactions so that contributions are transparent, while private blockchain channels keep sensitive computations and data hidden from outsiders. Each organization acts as a Q-learning agent that decides when and how much to contribute, aligning individual incentives with the collective model. When an organization requests data withdrawal, the framework avoids full retraining: it runs a short LoRA-parameterized forgetting loop on the data to be removed, extracts the resulting parameter changes, verifies them against a validation set in a smart contract, and records the unlearning transaction on the public chain. The experiments argue that this achieves forgetting comparable to retraining from scratch at much lower cost, with the hybrid ledger adding roughly six seconds of consensus overhead and about 48 seconds of one-time setup.
Load-bearing premise
The load-bearing premise is that a few rounds of gradient updates on the data to be erased, routed through a small low-rank adapter, remove exactly that data's influence while leaving the model's behavior on all other data intact; the paper reports no accuracy measurement on the data that should be kept, so this premise is assumed rather than shown.
Editorial extensions
If this is right
- Organizations with competing interests can co-train a shared LLM while keeping raw data in-house: the public chain provides tamper-evident records of every update, and the private chains restrict sensitive computations to authorized peers.
- A data-removal request can be handled without a full retraining run; the LoRA forgetting loop produces a parameter delta that is verified and committed on-chain, which is the paper's mechanism for GDPR-style erasure requests.
- The hybrid ledger adds only modest overhead, about 6 seconds of consensus time and 48 seconds of one-time setup, so the transparency and compliance benefits do not require sacrificing training throughput.
- Because LoRA hyperparameters (rank, alpha, dropout) control how much accuracy falls on the forget set, the framework can be tuned to trade unlearning strength against retained utility.
- Each organization's participation strategy is chosen by Q-learning against its own utility, aligning individual incentives with the collective goal while the ledger keeps everyone accountable.
Reading between the lines
- The reported metric is accuracy on the forget set after unlearning, which shows that something was forgotten but not that only the target data was forgotten; a retained-data accuracy column and a membership probe on the forget set would be needed to confirm selective erasure.
- The same LoRA-forgetting loop could be attached to other parameter-efficient fine-tuning methods, and the smart-contract verification step could be strengthened with zero-knowledge proofs so validation does not expose the validation data.
- Because the reported forget-set accuracies on IMDB, a binary task, are near chance, the decisive test is whether retained-data utility survives; adding a retain-set evaluation would turn this from a claim about forgetting into a claim about selective forgetting.
- The multi-agent Q-learning layer is largely independent of the blockchain and the unlearning loop, so its contribution to trust and incentive alignment could be ablated separately; the paper does not isolate it.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a hybrid blockchain-based federated learning framework for large language models, combining public and private blockchains, multi-agent Q-learning, and a LoRA-based unlearning mechanism. The stated goal is to enable transparent model-update sharing on a public chain while protecting sensitive computations in private chains, and to support selective data removal without compromising overall model performance. The manuscript presents eight algorithms describing registration, model upload, private-chain training, unlearning, verification, and aggregation, followed by experiments on IMDB and Twitter datasets that report initial and final accuracies for several LoRA configurations, a retrain-from-scratch comparison, and a blockchain overhead analysis. The central claim about unlearning is not supported by the reported results: the final accuracies collapse to near-zero values, and no retain-set or held-out utility metric is reported.
Significance. If the central claims were validated, the framework could be a useful blueprint for privacy-aware, trust-enhanced collaborative LLM training in regulated domains such as healthcare and education. The paper provides a fairly complete architectural description with numbered algorithms and a time-cost analysis of the hybrid blockchain, which are useful system-building blocks. However, the core novelty—LoRA-based unlearning—is specified only as a high-level gradient-adjustment loop with no formal objective, and the experimental evidence directly contradicts the claim that unlearning does not compromise overall performance. No code, machine-checked proofs, or parameter-free derivations are provided. As submitted, the significance of the contribution is substantially weakened because the key performance claim is unverified and, under the natural reading of the reported tables, refuted.
major comments (4)
- [Section VII-B, Tables I and II] The reported final accuracies of 0.95%–1.25% on IMDB and 7.93%–9.04% on Twitter, against initial accuracies of 74%–98%, are catastrophic utility loss if 'Final Accuracy' means accuracy on the general test set after unlearning. This directly contradicts the abstract's claim that unlearning enables 'selective removal of specific data contributions without compromising the model's overall performance.' If 'Final Accuracy' instead means accuracy on the forget set, then the tables only demonstrate forgetting and provide no evidence about the model's behavior on the retained data or held-out data. The paper never defines which interpretation is intended and never reports retain-set accuracy, so the central claim is empirically unsupported.
- [Algorithm 6, Section V-G] The unlearning objective is underspecified. Lines 5–8 direct the reader to compute gradients 'emphasizing' the forget set and to adjust the LoRA adapter, but they do not specify whether the loss is minimized or maximized, nor do they include any retain-set regularization term. Consequently, there is no mechanism in the algorithm construction that guarantees preservation of performance on D \ D_forget. The subsequent verification step in Algorithm 7, line 6, evaluates only an unspecified 'training loss and accuracy' on D_val with no defined acceptance criteria and no membership-inference test, so it cannot establish that the forget data has been selectively removed while other data remains intact.
- [Section VII-B, Table III] The comparison with 'Retrain from Scratch' is not informative. The baseline final accuracies are themselves near zero (0.85% on IMDB, 7.84% on Twitter), and the table reports only a single final-accuracy scalar, not retain-set or held-out accuracy. Moreover, the initial accuracies for the baseline differ from the corresponding initial accuracies of the proposed method (e.g., 95.60% versus 97.10% on IMDB), so the comparison is not on an equal footing. These issues make it impossible to conclude that the proposed unlearning approach achieves performance comparable to retraining.
- [Section VII-A, 'Experimental Comparisons'] The paper dismisses all existing baselines by stating that 'there are no direct counterparts in the current literature for comparison,' yet the related-work section discusses multiple unlearning methods for LLMs, including gradient-ascent-based approaches and selective knowledge negation. Without any comparison to these established unlearning baselines, and without the retain-set utility metric that those baselines report, the conclusion that the proposed method exhibits 'significant advantages in terms of computational efficiency, versatility, and adaptability' is not supported by the presented evidence.
minor comments (5)
- [Section V-J] The case studies in Section V-J contain inconsistent algorithm references: the text cites Algorithm 5 for private-chain training and Algorithm 4 for the unlearning process, but the actual unlearning algorithm is Algorithm 6 and the private-chain training algorithm is Algorithm 4. This mismatching makes the case-study walkthrough difficult to follow.
- [Abstract and Section I] The text contains several grammatical and typographical errors, such as 'the way computers understand and process human language, but using them effectively across different organizations remains still difficult' and 'the public data may consumed up while private data from organizations are highly needed.' These errors should be corrected throughout.
- [Algorithm 1] In Algorithm 1, a JSON Web Token is defined as a public–secret key pair ('jwt ← Pk, Sk'), which conflates key generation with token creation; a JWT should be a signed token issued after authentication, not the key pair itself. This should be clarified.
- [Section VI-B] The security analysis claims that the framework uses 'threshold signatures and zero-knowledge proofs,' but no cryptographic protocol or implementation detail is provided anywhere in the paper, and these mechanisms are not referenced in any algorithm. The security claims therefore outrun the presented system design.
- [Figures 2–7 and Table IV] The captions of Figures 2–7 do not define the axes or explain what 'Accuracy' refers to, and the text does not specify whether the plotted values are forget-set accuracy, retain-set accuracy, or overall test accuracy. Additionally, the 'Setup Time' bullet in Section VII-B reports 48 seconds for hybrid blockchain setup, whereas Table IV lists 84 seconds at t=0 for the hybrid method; these numbers need to be reconciled.
Circularity Check
No significant circularity: the framework is an implemented design whose claims rest on constructive algorithms and reported experiments, not on fitted parameters or load-bearing self-citations.
full rationale
This paper is primarily an engineering and systems proposal: the hybrid public/private blockchain federated learning pipeline (Algorithms 1-8) is described constructively, and the central unlearning claim is presented as an experimental result rather than as a derived theorem. No equation in the paper is defined in terms of the quantity it is supposed to establish, and no parameter is fitted to a target outcome and then reported as an independent prediction. The LoRA hyperparameters (rank, alpha, dropout) are varied as implementation choices and their effects on accuracy are reported in Tables I and II; this is hyperparameter tuning, not the construction of a prediction from fitted values. The self-citations [2], [6], and [10] are background references for federated learning, LLM security, and blockchain-enabled federated learning respectively; none is used as a load-bearing justification for the proposed mechanism, and no uniqueness or impossibility theorem is imported from the authors' prior work. The most serious weakness in the paper is empirical rather than circular: the retain-set utility after unlearning is never directly measured, and if the reported final accuracies of 0.95-1.25% on IMDB are interpreted as retained-set accuracy, they would contradict the claim of preserving overall performance. That is a completeness or correctness concern, not circularity, because the paper does not define successful unlearning as its own reported metric. Under the stated criteria requiring a specific equation-level reduction or a fitted parameter renamed as a prediction, no circular step can be exhibited. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (6)
- LoRA rank r =
2, 8, 16, 32
- LoRA alpha =
1, 2, 4, 8, 16
- LoRA dropout =
0.1 to 0.5
- Unlearning epochs Eu, learning rate eta, LoRA parameter lambda =
not specified in paper
- Q-learning hyperparameters and reward design =
not specified
- Private and global blockchain epochs =
not specified
assumptions (4)
- domain assumption Federated learning without raw data sharing protects privacy.
- domain assumption Blockchain immutability, consensus, threshold signatures and zero-knowledge proofs guarantee the claimed security properties.
- ad hoc to paper Gradient-based updates on the forget set remove data influence while preserving retain-set performance.
- domain assumption Q-learning converges to a useful participation policy in this multi-agent blockchain setting.
Cite this review
Pith. "Pith review of Large Language Model Federated Learning with Blockchain and Unlearning for Cross-Organizational Collaboration." pith.science (2026). https://pith.science/paper/BVY66OO4
@misc{pith2026241213551,
author = {Pith},
title = {Pith review of: Large Language Model Federated Learning with Blockchain and Unlearning for Cross-Organizational Collaboration},
year = {2026},
howpublished = {\url{https://pith.science/paper/BVY66OO4}},
note = {Machine review of arXiv:2412.13551}
}
read the original abstract
Large language models (LLMs) have transformed the way computers understand and process human language, but using them effectively across different organizations remains still difficult. When organizations work together to improve LLMs, they face several main challenges. First, organizations hesitate to share their valuable data with others. Second, competition between organizations creates trust problems during collaboration. Third, new privacy laws require organizations to be able to delete specific data when requested, which is especially difficult when multiple organizations are learning from shared data. Traditional federated learning approaches do not address these interconnected challenges, particularly in scenarios where participants cannot fully trust each other or the central aggregator. To overcome these limitations, we propose a hybrid blockchain-based federated learning framework that uniquely combines public and private blockchain architectures with multi-agent reinforcement learning. Our framework enables transparent sharing of model update through the public blockchain while protecting sensitive computations in private chains. Each organization operates as an intelligent agent, using Q-learning to optimize its participation strategy and resource allocation, thus aligning individual incentives with collective goals. Notably, we introduce an efficient unlearning mechanism based on Low-Rank Adaptation (LoRA) that enables selective removal of specific data contributions without compromising the model's overall performance. Through extensive experimentation on real-world datasets, we demonstrate that our framework effectively balances privacy protection, trust establishment, and regulatory compliance while maintaining high model performance.
Figures
Forward citations
Cited by 1 Pith paper
-
A Weighted Byzantine Fault Tolerance Consensus Driven Trusted Multiple Large Language Models Network
A weighted Byzantine fault tolerance consensus for multi-LLM networks is proposed, but its security proof assumes equal weights while the protocol lets the leader set weights, and its quality evaluation is self-referential.
Reference graph
Works this paper leans on
-
[1]
M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, M. B. Shaikh, N. Akhtar, J. Wu, S. Mirjalili et al. , “Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects,” Authorea Preprints, 2023
work page 2023
-
[2]
Fedrecovery: Differentially private machine unlearning for federated learning frame- works,
L. Zhang, T. Zhu, H. Zhang, P. Xiong, and W. Zhou, “Fedrecovery: Differentially private machine unlearning for federated learning frame- works,” IEEE Transactions on Information Forensics and Security, 2023
2023
-
[3]
A comprehensive survey on edge data integrity verification: Fundamentals and future trends,
Y . Zhao, Y . Qu, Y . Xiang, M. P. Uddin, D. Peng, and L. Gao, “A comprehensive survey on edge data integrity verification: Fundamentals and future trends,” ACM Comput. Surv. , vol. 57, no. 1, Oct. 2024. [Online]. Available: https://doi.org/10.1145/3680277
-
[4]
Federatedtrust: A solution for trustworthy federated learning,
P. M. S. S ´anchez, A. H. Celdr ´an, N. Xie, G. Bovet, G. M. P ´erez, and B. Stiller, “Federatedtrust: A solution for trustworthy federated learning,” Future Generation Computer Systems , vol. 152, pp. 83–98, 2024
work page 2024
-
[5]
Deep learning based attack detection for cyber-physical system cybersecurity: A survey,
J. Zhang, L. Pan, Q.-L. Han, C. Chen, S. Wen, and Y . Xiang, “Deep learning based attack detection for cyber-physical system cybersecurity: A survey,” IEEE/CAA Journal of Automatica Sinica , vol. 9, no. 3, pp. 377–391, 2021
work page 2021
-
[6]
The security of using large language models - a survey with emphasis on chatgpt,
W. Zhou, X. Zhu, Q.-L. Han, L. Li, X. Chen, S. Wen, and Y . Xiang, “The security of using large language models - a survey with emphasis on chatgpt,” IEEE/CAA Journal of Automatica Sinica , 2025
work page 2025
-
[7]
General data protection regulation (gdpr),
G. D. P. Regulation, “General data protection regulation (gdpr),” Intersoft Consulting, Accessed in October , vol. 24, no. 1, 2018
work page 2018
-
[8]
Lora: Low-rank adaptation of large language models,
E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021
arXiv 2021
Show all 33 references
-
[9]
Blockchain-enabled federated learning: A survey,
Y . Qu, M. P. Uddin, C. Gan, Y . Xiang, L. Gao, and J. Yearwood, “Blockchain-enabled federated learning: A survey,” ACM Comput. Surv., vol. 55, no. 4, Nov. 2022. [Online]. Available: https: //doi.org/10.1145/3524104
2022 doi
-
[10]
Fedtwin: Blockchain- enabled adaptive asynchronous federated learning for digital twin net- works,
Y . Qu, L. Gao, Y . Xiang, S. Shen, and S. Yu, “Fedtwin: Blockchain- enabled adaptive asynchronous federated learning for digital twin net- works,” IEEE Network, vol. 36, no. 6, pp. 183–190, 2022
2022
-
[11]
Llm-based federated recommendation,
J. Zhao, W. Wang, C. Xu, Z. Ren, S.-K. Ng, and T.-S. Chua, “Llm-based federated recommendation,” arXiv preprint arXiv:2402.09959 , 2024
2024 arXiv
-
[12]
Fedbiot: Llm local fine-tuning in federated learning without full model,
F. Wu, Z. Li, Y . Li, B. Ding, and J. Gao, “Fedbiot: Llm local fine-tuning in federated learning without full model,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2024, pp. 3345–3355
2024
-
[13]
Federatedscope-llm: A comprehensive package for fine-tuning large language models in federated learning,
W. Kuang, B. Qian, Z. Li, D. Chen, D. Gao, X. Pan, Y . Xie, Y . Li, B. Ding, and J. Zhou, “Federatedscope-llm: A comprehensive package for fine-tuning large language models in federated learning,” in Proceed- ings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Da...
2024
-
[14]
Openfedllm: Training large language models on decentralized private data via federated learning,
R. Ye, W. Wang, J. Chai, D. Li, Z. Li, Y . Xu, Y . Du, Y . Wang, and S. Chen, “Openfedllm: Training large language models on decentralized private data via federated learning,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2024, pp. ...
2024
-
[15]
Towards safer large language models through machine unlearning,
Z. Liu, G. Dou, Z. Tan, Y . Tian, and M. Jiang, “Towards safer large language models through machine unlearning,” arXiv preprint arXiv:2402.10058, 2024
2024 arXiv
-
[16]
Unlearn what you want to forget: Efficient unlearning for llms,
J. Chen and D. Yang, “Unlearn what you want to forget: Efficient unlearning for llms,” arXiv preprint arXiv:2310.20150 , 2023
2023 arXiv
-
[17]
Large language model unlearning,
Y . Yao, X. Xu, and Y . Liu, “Large language model unlearning,” arXiv preprint arXiv:2310.10683, 2023
2023 arXiv
-
[18]
Tofu: A task of fictitious unlearning for llms,
P. Maini, Z. Feng, A. Schwarzschild, Z. C. Lipton, and J. Z. Kolter, “Tofu: A task of fictitious unlearning for llms,” arXiv preprint arXiv:2401.06121, 2024
2024 arXiv
-
[19]
Who’s harry potter? approximate un- learning in llms,
R. Eldan and M. Russinovich, “Who’s harry potter? approximate un- learning in llms,” arXiv preprint arXiv:2310.02238 , 2023
2023 arXiv
-
[20]
Bc4llm: Trusted artificial intel- ligence when blockchain meets large language models,
H. Luo, J. Luo, and A. V . Vasilakos, “Bc4llm: Trusted artificial intel- ligence when blockchain meets large language models,” arXiv preprint arXiv:2310.06278, 2023
2023 arXiv
-
[21]
Dynamic large language models on blockchains,
Y . Gong, “Dynamic large language models on blockchains,” arXiv preprint arXiv:2307.10549, 2023
2023 arXiv
-
[22]
Blockchain-enabled trustworthy federated unlearning,
Y . Lin, Z. Gao, H. Du, J. Ren, Z. Xie, and D. Niyato, “Blockchain-enabled trustworthy federated unlearning,” arXiv preprint arXiv:2401.15917, 2024
2024 arXiv
-
[23]
Assessing how large language models can be integrated with or used for blockchain technology: Overview and illustrative case study,
J. G. M. Mboma, O. T. Tshipata, W. V . Kambale, and K. Kyamakya, “Assessing how large language models can be integrated with or used for blockchain technology: Overview and illustrative case study,” in 2023 27th International Conference on Circuits, Systems, Communications and...
2023
-
[24]
Blockchain-based proof- of-authenticity frameworks for explainable ai,
D. Malhotra, P. Saini, and A. K. Singh, “Blockchain-based proof- of-authenticity frameworks for explainable ai,” Multimedia Tools and Applications, vol. 83, no. 13, pp. 37 889–37 911, 2024
2024
-
[25]
Privacyeafl: Privacy- enhanced aggregation for federated learning in mobile crowdsensing,
M. Zhang, S. Chen, J. Shen, and W. Susilo, “Privacyeafl: Privacy- enhanced aggregation for federated learning in mobile crowdsensing,” IEEE Transactions on Information Forensics and Security , 2023
2023
-
[26]
Relac: Revocable and lightweight access control with blockchain for smart consumer electronics,
J. Zong, C. Wang, J. Shen, C. Su, and W. Wang, “Relac: Revocable and lightweight access control with blockchain for smart consumer electronics,” IEEE Transactions on Consumer Electronics, vol. 70, no. 1, pp. 3994–4004, 2023
2023
-
[27]
Blockchain-based data integrity verification scheme in aiot cloud–edge computing environment,
Y . Li, J. Shen, S. Ji, and Y .-H. Lai, “Blockchain-based data integrity verification scheme in aiot cloud–edge computing environment,” IEEE Transactions on Engineering Management , 2023
2023
-
[28]
Wooldridge, An introduction to multiagent systems
M. Wooldridge, An introduction to multiagent systems . John Wiley & Sons, 2009
2009
-
[29]
Q-learning,
C. J. Watkins and P. Dayan, “Q-learning,” Machine learning, vol. 8, no. 3-4, pp. 279–292, 1992
1992
-
[30]
Fedloc: Federated learning framework for data-driven cooperative localization and location data processing,
F. Yin, Z. Lin, Q. Kong, Y . Xu, D. Li, S. Theodoridis, and S. R. Cui, “Fedloc: Federated learning framework for data-driven cooperative localization and location data processing,” IEEE Open Journal of Signal Processing, vol. 1, pp. 187–215, 2020
2020
-
[31]
Blockchain mutability: Challenges and proposed solutions,
E. Politou, F. Casino, E. Alepis, and C. Patsakis, “Blockchain mutability: Challenges and proposed solutions,” IEEE Transactions on Emerging Topics in Computing, vol. 9, no. 4, pp. 1972–1986, 2019
1972
-
[32]
Distributed hybrid double-spending attack prevention mechanism for proof-of-work and proof-of-stake blockchain consensuses,
N. A. Akbar, A. Muneer, N. ElHakim, and S. M. Fati, “Distributed hybrid double-spending attack prevention mechanism for proof-of-work and proof-of-stake blockchain consensuses,” Future Internet , vol. 13, no. 11, p. 285, 2021. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGU...
2021
-
[33]
zk-authfeed: Protecting data feed to smart contracts with authenticated zero knowledge proof,
Z. Wan, Y . Zhou, and K. Ren, “zk-authfeed: Protecting data feed to smart contracts with authenticated zero knowledge proof,” IEEE Transactions on Dependable and Secure Computing , vol. 20, no. 2, pp. 1335–1347, 2022
2022
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.