REVIEW 5 major objections 6 minor 1 cited by
Safe-FedLLM: Delving into the Safety of Federated Large Language Models
T0 review · 5 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read A lightweight probe on LoRA weight deltas can identify malicious clients during federated fine-tuning of large language models and restore most of the safety that data-poisoning would otherwise destroy.
desk verdict A clean, cheap probe defense for poisoned LoRA updates in FedLLM that holds up in a matched, shared-seed setting — worth refereeing, not yet a general solution. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
LoRA-Probe: a logistic-regression classifier trained offline on L2-normalized concatenated first-layer LoRA B-matrix deltas (ΔB = B_t − B_0), kept fixed during federated training and evaluated on each client's uploaded deltas to output a maliciousness probability. Around it, the Safety Defense Module combines: Step-Level Bayesian accumulation of per-step decisions with time decay; Client-Level sigmoid-calibrated smoothing of the probe score over rounds; Shadow-Level an independent shadow LoRA branch that only generates probe signals and is decoupled from the global model, thus stable under distribution drift; plus security-gated round skipping and sample-size-weighted secure aggregation.
What would settle it
Run the authors' setup but let each client draw its own LoRA initialization seed while the server's probe is trained with the fixed seed used in the paper; if detection precision and final safety metrics fall to the undefended level, the claim that LoRA deltas are reliable endogenous safety signals is refuted.
Extended reading notes
Core claim
The central claim is that LoRA weights from different types of clients exhibit distinguishable intrinsic properties, enabling them to serve as effective endogenous safety signals. Concretely: if a server trains offline a linear classifier on labeled benign and malicious LoRA updates (the difference between a client's first-layer B matrices at local step t and their initialization), that fixed probe can detect malicious federated updates at inference time. Feeding the probe's scores into step-, client-, and shadow-level weighting and gating aggregation restores safety close to the all-benign baseline across two LLM backbones and up to 50% malicious clients, with only about 3% training-time ov
Load-bearing premise
The defense collapses if the offline probe is trained on LoRA deltas whose distribution does not match live federated training—the paper's own limitation is that the LoRA initialization seed must be identical between client-side training and probe training, along with matching preprocessing, optimizer, learning rate, batch size, and local steps.
Editorial extensions
If this is right
- Even a 20% share of malicious clients seriously damages a federated LLM's safety; without defense, safety metrics fall by large margins.
- Classical robust aggregation rules (geometry-based and trust-based) mostly fail on high-dimensional LoRA updates, while probe-based weighting succeeds.
- A single fixed linear probe on first-layer LoRA deltas detects malicious updates with near-perfect true-positive rates and near-zero false-positive rates in the shadow-level variant.
- Security-weighted aggregation preserves utility and adds only marginal training-time overhead.
- The defense remains effective as the malicious-client ratio rises from 20% to 50%, and transfers across two LLM backbones.
Reading between the lines
- The paper's L2 normalization means the probe reads update direction, not magnitude; an attack that scales up benign-looking updates to dominate aggregation may be neutralized by weighting, but an attack that mimics benign update direction would evade the probe—worth testing.
- The poor cross-backbone transferability the authors report suggests the separability is tied to the particular parameterization; a testable extension is to train the probe with whitening or feature alignment and see whether one probe can serve several backbones.
- Because the probe is fixed after offline training, an adaptive attacker who observes the probe could craft updates that lie in the benign region; retraining the probe periodically or using the shadow branch as a moving reference may close that gap.
- The shadow-level result (near-perfect detection with zero false positives) hints that decoupling the detection signal from the trained model is the key design choice; applying the same idea to other parameter-efficient fine-tuning methods could transfer the defense.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the security of federated fine-tuning of LLMs with LoRA. It reports that FedLLM is highly vulnerable to malicious clients (with as few as 20% malicious clients degrading safety) and that, in their experiments, benign and malicious LoRA updates are linearly separable. Based on this, the authors propose Safe-FedLLM, a defense consisting of a fixed logistic-regression 'LoRA-Probe' trained on first-layer LoRA B-matrix deltas, plus three defense levels (Step-Level, Client-Level, Shadow-Level) that use probe outputs to compute per-client security weights, with optional round skipping and security-weighted aggregation. Experiments on Llama-3.1-8B and Qwen2.5-7B across four data combinations claim large safety improvements over FedAvg and classic robust aggregation baselines, with negligible training-time overhead and stable performance up to 50% malicious clients.
Significance. If the central claim holds—that LoRA update deltas contain a stable, seed-invariant 'intrinsic' signal separating benign from malicious updates—then a lightweight, server-side probe could become a practical defense for FedLLM with low overhead. The paper is one of the first to address security specifically for PEFT-based FedLLM and ships an open-source implementation, which is a strength. However, the significance is currently conditional: the evidence is entirely in-distribution, the attack model is passive data poisoning rather than an adaptive adversary, and the framework's own stated assumption of a shared LoRA initialization seed is a non-trivial restriction on the threat model. The claimed 100% TPR/0% FPR detection in Table 7 should be treated as an upper bound from a matched-configuration evaluation, not as evidence of a general property.
major comments (5)
- [§3.3, Appendix B, Limitations] The paper's central claim—that benign and malicious LoRA updates exhibit 'distinguishable intrinsic properties'—is only demonstrated under a single shared LoRA initialization seed. Appendix B states that initial LoRA parameters are fixed for all clients, and the Limitations explicitly admit that the framework assumes the client-side LoRA seed matches the probe-training seed. The probe is a fixed linear classifier trained in the coordinate system of that seed; changing the seed changes the random projection underlying the B-matrix deltas. The paper provides no experiment varying the seed between probe training and deployment, nor across clients. Since all three defense modules (Eqs. 5-10) consume probe outputs, a probe-accuracy collapse under seed mismatch would nullify the entire defense. This is a load-bearing limitation, not a minor detail. Please add an experiment where the probe is t
- [§5, Table 7] The evaluation is entirely in-distribution: the probe is trained on deltas generated from the same attack protocol (BeaverTails/MaliciousGen) with the same local training configuration used at deployment, and the test set is held out from the same distribution. The perfect detection results (100% TPR, 0% FPR for Shadow-Level across most settings) are thus an upper bound under matched conditions. The paper does not evaluate against even simple obfuscations a malicious client could apply, such as scaling the local update, adding noise, changing the number of local steps, or using different malicious data. An adversary aware of the probe could potentially craft updates that appear benign to a fixed linear classifier. Please include at least a perturbed/adaptive attack evaluation (e.g., scale factor, noise injection, or a known-probe evasion attempt).
- [§4.2, Eq. (10)] The early-stage defense stabilization freezes security factors after round Rf=20. For a client that is first sampled in a round r>Rf, its security weight w_i is not defined by Eq. (10); the paper does not specify what value should be used. In a realistic FL deployment with more clients than can be sampled in 20 rounds, this is a practical correctness gap. It also means that a client that turns malicious after round 20 retains its early benign weight, creating a blind window for late-onset or late-joining attackers. Please clarify the initialization and update rule for clients not seen by round Rf, and discuss the security implications.
- [§5.1, Implementation Details / all tables] All tables report point estimates from what appears to be a single run; no error bars, standard deviations, or multiple-seed repetitions are provided. Given the small client-sample sizes (3 clients per round), the differences between some conditions (e.g., MT-1 scores in Tables 2-3) are within plausible noise. The absence of any statistical uncertainty is particularly problematic for the headline 100% detection results. Please report mean±std over at least 3-5 independent seeds for the main results, and state total client counts and client sampling procedure.
- [§4.2, Eq. (10) and Figure 4] Figure 4 shows that Step-Level and Client-Level classification precision declines steadily with rounds, which is why the authors freeze security factors after round 20. This means that after round 20 the defense is not updating its beliefs about clients at all. If a client's behavior changes later (e.g., it begins sending poisoned updates after the freeze), the defense will continue assigning its old weight. The paper does not analyze this post-freeze attack window or provide any fallback mechanism. A concrete test: simulate a malicious client that only begins poisoning after round 20 and report the safety metrics.
minor comments (6)
- [Eq. (5)] The Bayesian parameters α_i and β_i are not defined with initial values. If they start at zero, the ratio α/(α+β) is undefined in the first round. Please specify initialization (e.g., α_i=β_i=1 or a prior).
- [Appendix C, Eq. (13)] The two-stage sigmoid g(s) is discontinuous at s=0.8: the left limit is approximately 0.491 while the right limit is approximately 0.634. A discontinuity can cause abrupt changes in security factors for small score perturbations around 0.8. Please confirm this is intended or smooth the function.
- [§5.1, Baselines] The paper uses FedLLM-Attack (Ye et al., 2024a) to generate attack data but does not compare against the defense proposed in that same work. Adding that FedLLM-specific defense as a baseline would strengthen the comparison.
- [§4.1, Eq. (2)] The feature vector uses only the first transformer layer. While Figure 2 shows separability for this choice, there is no ablation study showing that first-layer features are optimal or that using more layers would change results. A short ablation would support the 'endogenous signal' claim.
- [§5.2, Table 4] Table 4 reports that Shadow-Level doubles GPU memory to 41.24GB. This is a substantial resource increase that may make the framework impractical on many GPUs. The claim of 'negligible wall-clock time' should be accompanied by peak-memory reasoning or a note on hardware requirements.
- [General] The paper contains several typos: 'incuding' in §5.1, 'the separability of LoRA weights' in the Conclusions, and the phrase 'we find that the separability of LoRA weights' is grammatically incomplete. Also, the total number of clients in the federated setup is never stated; only 'sampling 3 clients per round' is given.
Circularity Check
No significant circularity: the probe is a supervised classifier, but final safety gains are validated on external benchmarks.
full rationale
The claimed derivation chain is not circular. LoRA-Probe is a supervised logistic-regression classifier (Eqs. 1–3) fitted offline to labeled benign/malicious LoRA deltas; this is a fitted model, not a first-principles prediction, and the paper presents it as such. The central empirical claim—that benign and malicious updates are separable in LoRA space—is supported by an unsupervised LDA+PCA visualization (Figure 2) and by held-out classification metrics (Table 7), which are standard validation rather than construction. The defense's final safety results are measured on external benchmarks (AdvBench Rule/MD-Judge/RM, MT-Bench) that were not used to train the probe, so the observed safety improvements are independent of the probe's training labels. The acknowledged limitations (shared LoRA seed, backbone transferability, long-horizon drift) constrain generalization but do not make any equation equivalent to its inputs. Self-citations (Tian et al. 2023; Chen et al. 2025; Zeng et al. 2025) are contextual and not load-bearing.
Assumptions & free parameters
free parameters (7)
- tau_cls =
0.8
- gamma =
0.95
- eta =
7
- tau_skip =
0.2
- Rf =
20
- k =
10
- feature_layer_index =
1 (first transformer layer)
assumptions (5)
- domain assumption Malicious clients are simulated as clients that fine-tune on harmful instruction datasets (BeaverTails/MaliciousGen) and upload genuine LoRA updates.
- domain assumption Benign and malicious LoRA B-matrix deltas are linearly separable in the L2-normalized first-layer feature space.
- domain assumption The server can train a representative probe offline using labeled benign/malicious samples that match the deployment's data preprocessing, optimizer, learning rate, batch size, local steps, and LoRA initialization seed.
- domain assumption A fixed probe remains reliable across rounds, with early freezing of security factors sufficient to handle distribution drift.
- domain assumption The shadow LoRA branch with fixed learning rate remains distributionally stable while the global model evolves.
Cite this review
Pith. "Pith review of Safe-FedLLM: Delving into the Safety of Federated Large Language Models." pith.science (2026). https://pith.science/paper/DIAUVKKU
@misc{pith2026260107177,
author = {Pith},
title = {Pith review of: Safe-FedLLM: Delving into the Safety of Federated Large Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/DIAUVKKU}},
note = {Machine review of arXiv:2601.07177}
}
read the original abstract
Federated learning (FL) addresses privacy and data-silo issues in the training of large language models (LLMs). Most prior work focuses on improving the efficiency of federated learning for LLMs (FedLLM). However, security in open federated environments, particularly defenses against malicious clients, remains underexplored. To investigate the security of FedLLM, we conduct a preliminary study to analyze potential attack surfaces and defensive characteristics from the perspective of LoRA updates. We find two key properties of FedLLM: 1) LLMs are vulnerable to attacks from malicious clients in FL, and 2) LoRA updates exhibit distinct behavioral patterns that can be effectively distinguished by lightweight classifiers. Based on these properties, we propose Safe-FedLLM, a probe-based defense framework for FedLLM, which constructs defenses across three levels: Step-Level, Client-Level, and Shadow-Level. The core concept of Safe-FedLLM is to perform probe-based discrimination on each client's local LoRA updates, treating them as high-dimensional behavioral features and using a lightweight classifier to determine whether they are malicious. Extensive experiments demonstrate that Safe-FedLLM effectively improves FedLLM's robustness against malicious clients while maintaining competitive performance on benign data. Notably, our method effectively suppresses the impact of malicious data without significantly affecting training speed, and remains effective even under high malicious client ratios.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 1 Pith paper
-
Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda
This research agenda argues that cloud-native architectures, microservices, autoscaling, and emerging trends like serverless inference and federated learning are required to make large language models efficient and scalable.
Reference graph
Works this paper leans on
-
[3]
In IEEE Transactions on Signal Processing
Robust aggregation for federated learning. In IEEE Transactions on Signal Processing. Jiaxing Qi, Zhongzhi Luan, Shaohan Huang, Carol Fung, Hailong Yang, and Depei Qian. 2024. FD- LoRA: Personalized federated learning of large lan- guage model via dual LoRA tuning.arXiv preprint arXiv:2406.07925. Qwen Team. 2024. Qwen2.5 technical report.arXiv preprint ar...
arXiv 2024
-
[2021]
InProceedings of the 28th Annual Network and Distributed System Security Symposium (NDSS)
Fltrust: Byzantine-robust federated learning via trust bootstrapping. InProceedings of the 28th Annual Network and Distributed System Security Symposium (NDSS). Jiawei Chen, Xiao Yang, Zhengwei Fang, Yu Tian, Yin- peng Dong, Zhaoxia Yin, and Hang Su. 2025. Auto- breach: Universal and adaptive jailbreaking with ef- ficient wordplay-guided optimization via ...
arXiv 2025
-
[2022]
InAdvances in Neural Information Processing Systems 35: Annual Confer- ence on Neural Information Processing Systems 2022 (NeurIPS 2022)
Training language models to follow instruc- tions with human feedback. InAdvances in Neural Information Processing Systems 35: Annual Confer- ence on Neural Information Processing Systems 2022 (NeurIPS 2022). Krishna Pillutla, Sham Kakade, and Zaid Harchaoui
2022
-
[2023]
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng
Towards building the federated gpt: Federated instruction tuning.arXiv preprint arXiv:2305.05644. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatgpt interaction logs in the wild. InThe Twelfth In- ternational Conference on Learning Representations (ICLR). Lianmin Zheng, Wei-Lin Chiang, Ying Sheng,...
arXiv 2024
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.