REVIEW 5 major objections 5 minor 15 references
Trajectory-Aware Information Matching for Multi-Step Gradient Inversion in Federated Learning
T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that NL-SME, a quadratic Bezier surrogate trajectory with a learned control point and per-parameter scaling, makes multi-step FedAvg updates reconstructable, cutting gradient-matching loss to as low as 0.0009 and raising…
desk verdict The Bézier trajectory isn't doing the work in this paper: the d-vector scaling is, and the headline metric is computed on scaled gradients, so the main empirical claim needs a fairer test before it should be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the quadratic Bezier surrogate trajectory $\hat{w}(t,P_1)=(1-t)^2 w_0+2(1-t)tP_1+t^2 w_T$ for $t\in[0,1]$, with a learnable control point $P_1$ in parameter space. Setting $P_1=(w_0+w_T)/2$ recovers linear interpolation, so the model strictly generalizes SME, and the control point encodes the curvature of the SGD path. NL-SME optimizes dummy data $\tilde{D}$, the curve parameter $t$, and $P_1$ together, using a cosine-similarity loss between the surrogate gradient and the observed update $w_0-w_T$, total-variation smoothing, control-point regularization toward the linear midpoint, per-parameter scaling factors $d_i\in[0.1,10]$ that reweight each gradient coordinate, and optionally a classification-consistency loss.
What would settle it
Run NL-SME and SME with identical per-parameter scaling and classification-loss components; if their reconstruction quality converges, the Bezier trajectory itself is not the cause of the gains, and if SME still lags, the curve is load-bearing.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a nonlinear parametric surrogate, specifically a quadratic Bezier curve $\hat{w}(t,P_1)=(1-t)^2 w_0+2(1-t)t P_1+t^2 w_T$, can approximate the hidden sequence of SGD states inside a FedAvg round well enough for gradient inversion to recover private images. The learned control point $P_1$ gives the surrogate extra dimensions of freedom compared with the single interpolation parameter $\alpha$ of SME, and the per-parameter scaling factors $d_i$ reweight gradient coordinates during matching. The paper reports order-of-magnitude reductions in similarity loss and large PSNR/SSIM gains over IG, SME, and FEDLEAK, and shows the attack stays computationally cheap: memory cost independent of local step count and runtime comparable to SME. The conclusion drawn is that observable multi-step FedAvg updates still carry reconstructable signals, contradicting the implicit security assumption behind linear surrogate models.
Load-bearing premise
The headline result assumes the improvement comes from the curved surrogate trajectory and not from extra free per-weight scaling factors that only NL-SME has, plus an optional classification loss that baselines are not given.
Editorial extensions
If this is right
- Multi-step FedAvg updates remain invertible: NL-SME recovers readable images even when clients run 20-50 local epochs with 50 local samples on FEMNIST and CIFAR-100.
- Security audits that assume straight-line surrogate trajectories understate the real privacy risk of federated learning.
- The attack's memory cost is independent of the number of local steps, so it remains feasible as federated training scales.
- Adaptive optimizers such as AdamW can push NL-SME's PSNR below the visual corruption threshold, which points to a defense direction the paper documents.
Reading between the lines
- The paper leaves open whether the per-parameter scaling factors $d_i$ or the Bezier geometry do the heavy lifting; a control experiment giving SME the same $d_i$ vector and the same classification-consistency loss would separate the two.
- The same nonlinear-surrogate idea should transfer to other multi-step aggregation rules where the server observes only the cumulative parameter change, such as FedProx or SCAFFOLD; whether it helps will depend on how curved those client trajectories are.
- A direct test on a quadratic convex objective, where SGD trajectories are straight, would show whether NL-SME's advantage vanishes; if it does not, the gain comes from added optimization capacity rather than trajectory shape.
- Defenses built around linear surrogate attacks need re-evaluation under the nonlinear surrogate, since the attack's threat model differs from what linear-assumption evaluations assumed.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes NL-SME, a gradient-inversion attack for multi-step FedAvg settings. NL-SME replaces the linear interpolation surrogate of SME with a learnable quadratic Bézier curve parameterized by a control point P1 and curve parameter t, and adds per-parameter scaling factors d_i plus regularization. The paper claims that NL-SME achieves order-of-magnitude reductions in cosine-similarity loss and large PSNR/SSIM gains over FEDLEAK, IG, and SME on FEMNIST and CIFAR-100 (Table 1), while remaining computationally comparable to SME. The authors also provide robustness experiments under more complex training settings and an ablation study (Table 5).
Significance. If the central claim is established, the work is relevant to the federated-learning privacy community because it suggests that observable multi-step FedAvg updates still contain reconstructable signals and that nonlinear trajectory modeling can improve surrogate-model attacks. The paper also ships code, includes a broad comparison, and its ablation structure is a useful step. However, the significance currently hinges on the fairness of the comparison and on the attribution of the gains to the nonlinear trajectory model; as reported, the ablation actually suggests that the per-parameter scaling, not the Bézier path, drives most of the improvement. This needs to be resolved before the empirical claims can be accepted.
major comments (5)
- [§4.2, Eq. (13), Eq. (10)] The headline comparison in Table 1 is not apples-to-apples. Equation (13) defines the matched gradient as an elementwise product with per-parameter scaling factors d_i ∈ [0.1, 10], and Eq. (10) computes the cosine-similarity loss on this scaled gradient. This adds one free parameter per model weight that can rotate the gradient direction toward w0 − wT independently of the surrogate trajectory. The baselines IG, SME, and FEDLEAK do not appear to have this extra capacity, so the Lsim values reported for NL-SME are not measuring the same quantity as the baselines. To support the central claim, the authors should either give the baselines the same scaling capacity or report Lsim computed with d_i = 1 for NL-SME.
- [§5.4, Table 5] The ablation study undermines the paper's attribution of the gains to nonlinear trajectory modeling. Table 5 shows that Variant A (nonlinear path without the regularization/scaling mechanism) is worse than SME (Lsim 0.104 vs. 0.056), while Variant B (scaling/regularization without the nonlinear path) improves Lsim to 0.026. This indicates that the per-parameter scaling, not the quadratic Bézier surrogate, is the main driver of the reported improvements. The text acknowledges this partially but still concludes that 'nonlinear trajectory modeling provides geometric modeling capabilities'; the experiment as reported does not support that conclusion, and the paper needs either a different ablation design or a revised claim.
- [§5.1, §4.2] Many experimental details that are load-bearing for reproducibility are omitted. The manuscript does not state the model architecture, the number of attack iterations, the learning rates for D, t, and P1, the values of λ_TV, λ_P, λ_d, and λ_cls, or whether the classification consistency loss L_cls was active in the Table 1 numbers. Since L_cls directly uses ground-truth labels and can improve PSNR/SSIM independently of trajectory matching, the reader cannot determine whether the headline results are due to the proposed trajectory model or to an auxiliary loss. These details should be provided for all settings.
- [§3.3, Eq. (7); §4.3] The theoretical claims are asserted rather than demonstrated. Equation (7) states that the linear surrogate's gradient-direction bias has a lower bound ε_bias > 0, but no proof or precise assumptions are given; for an arbitrary non-convex loss it is not obvious that such a uniform positive lower bound exists. Similarly, §4.3 claims that 'under appropriate regularization parameter settings, control point regularization terms ensure bounded Hessian matrix condition numbers, guaranteeing local convergence,' but the conditions are not specified. Because the introduction explicitly claims a theoretical demonstration of the limitation of linear assumptions, these statements need to be either proved or clearly labeled as conjectures.
- [§5.3, Table 4] Table 4 contains a result that is not consistent with the text's robustness claims. At R=20, NL-SME reports PSNR 8.05 and SSIM 0.180, which are worse than SME's 9.27 and 0.201. The text states that 'NL-SME exhibits stable high performance' and that its advantages persist under increasing training complexity, but this row shows a reversal. The authors should discuss this failure case and moderate the robustness claims accordingly.
minor comments (5)
- [Table 1] Several table entries appear to be typographically corrupted, e.g., '0.78850' and '0.74120' should presumably be '0.7885' and '0.741'.
- [Eq. (7)] The norm notation in Eq. (7) is garbled in the manuscript; the double vertical bars around the difference are missing or misplaced.
- [§4.2, §5.2] The paper should state explicitly, for every experiment, whether L_cls is included. The current phrasing 'can be selectively added' leaves the reader unsure which table numbers use it.
- [Figure 3] The convergence plot reports a single setting or a representative run; it would be clearer to state this in the caption and to report variance across random seeds.
- [§5.3] The paper uses R=1, 10, 20 to denote training-complexity levels but does not define the construction precisely until the later text; a short description of how R is implemented (e.g., which optimizer changes and which perturbations) should appear before Table 3.
Circularity Check
Headline L_sim gains are partly forced by the per-parameter d_i scalings used inside the optimized cosine loss; PSNR/SSIM remain external evidence.
-
fitted input called prediction
[Section 4.2, Eq. (10) and Eq. (13); Section 5.4, Table 5]
"Finally, to further improve attack accuracy, we extend the dvec scaling mechanism. ... \tilde{\nabla}_{\hat w}\ell(\hat w(t,P_1),\tilde D)=\{d_i\cdot\nabla_{w_i}\ell(\hat w(t,P_1),\tilde D)\}_{i=1}^{|w|} where d_i\in[0.1,10.0] is the scaling factor for the i-th parameter. ... L_{\cos}(\tilde D,t,P_1)=1-\frac{\langle w_0-w_T,\tilde{\nabla}_{\hat w}\ell(\hat w(t,P_1),\tilde D)\rangle}{\|w_0-w_T\|\cdot\|\tilde{\nabla}_{\hat w}\ell(\hat w(t,P_1),\tilde D)\|}"
The reported L_sim is not an independent measure of trajectory fidelity: it is exactly the cosine loss in Eq. (10), evaluated on the d-scaled gradient defined in Eq. (13). The d_i are free optimization variables, one per model weight, bounded in [0.1,10], and they are optimized against the same observed w_0-w_T that defines the target of L_cos. A near-zero L_sim therefore chiefly certifies that the per-coordinate scalings were fit, not that the Bézier surrogate recovered the true client trajectory. The paper's own Table 5 makes this concrete: keeping the dvec/PR mechanism but removing the nonlinear path gives L_sim=0.026, while keeping the nonlinear path without PR gives L_sim=0.104, worse than SME's 0.056.
full rationale
No load-bearing self-citation chain is present: the method builds on external prior work (DLG, IG, SME, FEDLEAK, Garipov et al.), and the authors do not rely on their own prior uniqueness theorems. The quadratic Bézier surrogate is a genuine extension of SME, initialized to reduce to linear interpolation, so its expressiveness claim is not circular. The main circularity is narrower: the headline 'order-of-magnitude improvements in cosine similarity loss' is measured on the same d-scaled cosine objective that the method optimizes, with one free scaling factor per parameter. The ablation confirms that the scaling mechanism, not the nonlinear path, is the dominant driver of the improved L_sim. Because PSNR and SSIM are externally defined image-quality metrics and do provide independent evidence of reconstruction quality, the paper is only partially circular rather than wholly so. The score of 5 reflects that the L_sim claim reduces by construction, while the reconstruction-quality claim retains independent content.
Assumptions & free parameters
free parameters (5)
- Control point P1 =
optimized (|w|-dimensional), initialized at (w0+wT)/2
- Curve parameter t =
optimized scalar, initialized 0.5
- Per-parameter scaling factors d_i =
optimized, each in [0.1, 10.0], initialized 1.0
- Regularization coefficients lambda_TV, lambda_P, lambda_d (and lambda_cls if used) =
not reported
- Learning rates eta, eta_t, eta_P1 =
not reported
assumptions (4)
- domain assumption A quadratic Bezier curve between w0 and wT can approximate the true multi-step SGD trajectory well enough for gradient matching (Garipov et al. mode connectivity).
- domain assumption Gradient matching at the surrogate point can recover the private data, i.e., the update direction w0 - wT carries sufficient information about the training data.
- ad hoc to paper Under Lipschitz continuity, the linear surrogate's gradient direction bias has a positive lower bound epsilon_bias > 0.
- ad hoc to paper With appropriate regularization, the Hessian condition number is bounded, guaranteeing local convergence.
Cite this review
Pith. "Pith review of Trajectory-Aware Information Matching for Multi-Step Gradient Inversion in Federated Learning." pith.science (2026). https://pith.science/paper/3JSHNBIF
@misc{pith2026250922082,
author = {Pith},
title = {Pith review of: Trajectory-Aware Information Matching for Multi-Step Gradient Inversion in Federated Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/3JSHNBIF}},
note = {Machine review of arXiv:2509.22082}
}
read the original abstract
Federated learning enables distributed information sharing and collaborative model training without exposing raw client data. However, shared gradients or model updates may still contain sensitive information, making federated learning vulnerable to gradient inversion attacks. Most existing gradient inversion attacks rely on simplified update observations, such as single-step gradients or endpoint-based matching. In practical FL, however, FedAvg produces an accumulated trajectory-dependent update after multiple local steps, rather than a gradient computed at a single model state.To address this issue, we propose NL-SME, a trajectory-aware information matching method for multi-step gradient inversion. NL-SME constructs a learnable nonlinear surrogate trajectory to approximate hidden local states and integrates trajectory-level information with calibrated gradient matching. For perturbed updates, NL-SME can further use an observed-update reliability-aware strategy to reduce the influence of unreliable components. Extensive experiments under diverse multi-step FedAvg settings show that NL-SME outperforms state-of-the-art gradient inversion baselines in reconstruction quality and update-matching accuracy. Additional evaluations on natural and medical images, as well as under fused-update observations and representative defense strategies, further suggest that observable multi-step updates may still retain reconstruction signals. These results reveal potential privacy leakage risks in federated information sharing. Code is available at https://anonymous.4open.science/r/NL-SME-main/README.md.
Figures
Reference graph
Works this paper leans on
-
[2]
Dimitar I Dimitrov, Mislav Balunovic, Nikola Konstantinov, and Martin Vechev
URLhttps: //ijcai-preprints.s3.us-west-1.amazonaws.com/2025/7683.pdf. Dimitar I Dimitrov, Mislav Balunovic, Nikola Konstantinov, and Martin Vechev. Data leakage in federated averaging.Transactions on Machine Learning Research,
work page 2025
-
[4]
Boosting Gradient Leakage Attacks: Data Reconstruction in Realistic FL Settings
URLhttp:// arxiv.org/abs/2506.08435. Rida T Farouki. The bernstein polynomial basis: A centennial retrospective.Computer Aided Geometric Design, 29(6):379–419,
-
[6]
doi: 10.3390/jsan11010014. URLhttps://doi. org/10.3390/jsan11010014. Thomas Jeter, Thanh Nguyen, Rakan Alharbi, and Jae T. Seo. Securing federated learning against active reconstruction attacks.ACM Transactions on Privacy and Security, 28(4):1–28,
-
[7]
URLhttps://dl.acm.org/doi/10.1145/3762639
doi: 10.1145/3762639. URLhttps://dl.acm.org/doi/10.1145/3762639. Keller Jordan, Hanie Sedghi, Olga Saukh, Rahim Entezari, and Behnam Neyshabur. REPAIR: RENormalizing permuted activations for interpolation repair. InProceedings of the Eleventh International Conference on Learning Representations, ICLR’23, Kigali, Rwanda,
-
[10]
URLhttps://doi.org/10.1145/3637528.3671573
1145/3637528.3671573. URLhttps://doi.org/10.1145/3637528.3671573. Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions.IEEE Signal Processing Magazine, 37(3):50–60,
-
[11]
URL https://doi.org/10.1038/s41746-024-01226-1
doi: 10.1038/s41746-024-01226-1. URL https://doi.org/10.1038/s41746-024-01226-1. Yew Lok Poy, Zheng Yong Loke, Sandeep Darmaraju, Chin Heng Goh, and Boon Heng Kwan. Enhanced particle swarm optimisation for multi-robot path planning with b´ezier curve smoothing. Robotics, 13(10):141,
-
[12]
doi: 10.3390/robotics13100141. URLhttps://www.mdpi. com/2218-6581/13/10/141. Jiaqi Ren, Pengyu Chen, and Rui Wang. Revisiting mode connectivity in neural networks with bezier surface. InProceedings of the 13th International Conference on Learning Represen- tations (ICLR). OpenReview.net,
-
[13]
Wenqi Wei, Ling Liu, Margaret Loper, Ka-Ho Chow, Mehmet Emre Gursoy, Stacey Truex, and Yanzhao Wu
URLhttps://openreview.net/forum?id= 1NevL7zdHS. Wenqi Wei, Ling Liu, Margaret Loper, Ka-Ho Chow, Mehmet Emre Gursoy, Stacey Truex, and Yanzhao Wu. A framework for evaluating gradient leakage attacks in federated learning.arXiv preprint arXiv:2004.10397,
arXiv 2004
Show all 15 references
-
[14]
URLhttps://doi.org/10.1007/ s10462-025-11248-0
doi: 10.1007/s10462-025-11248-0. URLhttps://doi.org/10.1007/ s10462-025-11248-0. 12 Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M Alvarez, Jan Kautz, and Pavlo Molchanov. See through gradients: Image batch recovery via gradinversion. InProceedings of the IEEE/CVF Conference on...
-
[16]
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen
URLhttps: //arxiv.org/abs/2310.01190. Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. iDLG: Improved deep leakage from gradients. arXiv:2001.02610,
2001 arXiv
-
[18]
Ligeng Zhu, Zhijian Liu, and Song Han
URLhttp://arxiv.org/abs/2306.00127. Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch´e-Buc, E. Fox, and R. Garnett (eds.),Advances in Neural Information Processing Systems, volume
-
[2012]
Deep ensembles: A loss landscape per- spective.arXiv preprint arXiv:1912.02757,
Stanislav Fort, Surya Ganguli, and Stanislaw Jastrzebski. Deep ensembles: A loss landscape per- spective.arXiv preprint arXiv:1912.02757,
1912 arXiv
-
[2023]
URLhttps://arxiv.org/abs/2211.08403
Open- Review.net. URLhttps://arxiv.org/abs/2211.08403. Peter Kairouz, H Brendan McMahan, Brendan Avent, Aur ´elien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Ad- vances and open problems in federated l...
-
[2024]
URLhttps://doi.org/10.1145/3659099
doi: 10.1145/3659099. URLhttps://doi.org/10.1145/3659099. Lingjuan Lyu, Han Yu, and Qiang Yang. Threats to federated learning: A survey.arXiv preprint arXiv:2003.02133,
2003 arXiv
-
[2025]
URLhttps://doi.org/10.1016/j.robot.2025
doi: 10.1016/j.robot.2025.104969. URLhttps://doi.org/10.1016/j.robot.2025. 104969. Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Koneˇcn`y, H Brendan McMa- han, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings.arXiv preprint a...
2025
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.