Pith. sign in

REVIEW 5 major objections 4 minor 35 references

FedEve: On Bridging the Client Drift and Period Drift for Cross-device Federated Learning

T0 review · 5 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Kalman fusion makes two federated drifts cancel each other

desk verdict Useful empirical framing of period drift and a sensible server-side update, but the variance-reduction theorem and convergence proof rely on assumptions that break under non-iid client sampling. read the letter →

arxiv 2508.14539 v1 pith:F7SGG4IO submitted 2025-08-20 cs.LG cs.DC

Tao Shen , Zexi Li , Didi Zhu , Ziyu Zhao , Chao Wu , Fei Wu This is my paper
classification cs.LGcs.DC
keywords federatedlearningcross-deviceclientdriftperiodKalmanfilternon-iiddatapartialparticipationvariancereduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper identifies a neglected failure mode in cross-device federated learning: when only a small subset of clients participates each round, the optimization objective shifts round to round, an effect it calls period drift. It argues that period drift, not the well-studied client drift from multiple local updates, becomes the dominant source of instability as data heterogeneity grows. The central claim is that these two drifts can partially compensate each other if the server treats its own momentum-based update as a noisy prediction and the clients' aggregate update as a noisy observation, then fuses them with a Kalman filter. The paper proves that the fused update has lower variance than either drift alone and reports consistent accuracy gains over FedAvg, FedProx, SCAFFOLD, and FedOpt on non-iid cross-device benchmarks. If correct, the result means a practical server-side change with no extra client cost or hyperparameters can tame the hardest part of cross-device heterogeneity.

What carries the argument

The carry instrument is the predict-observe framework instantiated as a Kalman filter over model updates, where Nesterov momentum on the server acts as the prediction with period drift as its noise, and the average of client updates acts as the observation with client drift as its noise; the central identity is the Kalman gain ratio G_kal = σ²_{t+1}/(σ²_{t+1} + σ²_{Rt}) that weights the two sources and the variance recursion σ²_{t+1} = (1 - G_kal)σ²_{t+1} that propagates the reduced fused variance.

What would settle it

Compute the empirical covariance between the period-drift estimate (momentum minus aggregate update) and client-drift estimate (individual local updates minus aggregate) across rounds; if the covariance is consistently nonzero, the independent-noise assumption fails and the optimal Kalman gain formula needs adjustment.

Watch

Extended reading notes

Core claim

The paper's central claim is that period drift—the distribution shift of the randomly selected client subset relative to the full client population—scales with data heterogeneity and, in cross-device settings with heavy non-iid data, hurts convergence more than client drift. To counter it, FedEve casts server momentum as a prediction of the update step (corrupted by period drift) and the clients' averaged local updates as an observation (corrupted by client drift), then combines them with a Kalman filter. Theorem 2.5 states that the fused model has variance σ²_fused = (σ²_pred σ²_Rt)/(σ²_pred + σ²_Rt), strictly smaller than both the prediction variance and the client-drift variance, with the

Load-bearing premise

The Kalman gain formula assumes period drift and client drift are independent noises, but both stem from the same random client subset, so they are likely correlated.

Editorial extensions

If this is right

  • Cross-device federated learning can reduce the variance of model updates without extra client communication or client-side storage, since all extra computation lives on the server.
  • The larger the period drift (smaller α, fewer sampled clients), the more FedEve's Kalman gain moves weight toward the client observation, giving an automatic, hyperparameter-free adaptation to heterogeneity.
  • FedAvg is recovered as the special case where the server makes no prediction (G_kal = 1), so the method is a strict generalization of the standard baseline.
  • The convergence theorem F.3 gives the standard O(1/T) non-convex rate with an irreducible sampling-error term scaled by G²_kal·σ²_S(1 - S/N), which vanishes when all clients participate.
  • The same predict-observe logic can be applied to any server-side optimizer (adaptivity, lookahead, etc.) that can be written as a prediction function over update steps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper treats period drift and client drift as independent noises in the Kalman derivation, but both are generated from the same randomized client subset S_t each round, so they are likely correlated; a correlated-noise Kalman update (with an off-diagonal term) would be a natural testable extension.
  • FedEve's momentum update is a Nesterov-type momentum whose gain is learned online from drift variance estimates; this suggests a bridge to adaptive momentum schemes, where the gain schedule is replaced by an estimated signal-to-noise ratio.
  • Because the variance of period drift is estimated as the squared difference between momentum and the current client aggregate, the algorithm implicitly assumes the momentum tracks the global objective; in settings with biased client selection (non-random sampling), the estimate could be systematically skewed, and a bias-corrected estimate would be worth testing.
  • The empirical claim that period drift dominates client drift at high heterogeneity is testable beyond the paper's four ablation settings: one could vary the number of sampled clients S at fixed heterogeneity to directly verify the predicted relation period drift ∝ H/S.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper identifies a new source of error in cross-device federated learning, called 'period drift', which arises because the subset of clients participating in each round has a data distribution that deviates from the overall population. It distinguishes period drift from the better-known client drift and proposes FedEve, a server-side Kalman-filter method that fuses a momentum-based prediction (corrupted by period drift) with the client-average observation (corrupted by client drift). The central theoretical claim (Theorem 2.5) is that the fused update has variance smaller than each individual drift, with the fusion weights determined by the drift variances. The paper reports experiments on FEMNIST, CIFAR-100, and MovieLens-1M showing accuracy/AUC improvements over FedAvg, FedAvgM, FedProx, SCAFFOLD, and FedOpt.

Significance. If the theoretical result were sound, period drift would be a useful conceptual decomposition and FedEve would be an attractive, communication-free server-side modification. The empirical evaluation is broad and the paper makes a genuine attempt to separate the two drift sources experimentally. However, the central variance-reduction theorem is not established: it rests on an independence claim that is false under the paper's own definitions, and the Gaussian-product derivation contains a basic mathematical error. The concept of period drift may still be of interest, but the paper's principal contribution, as framed in the abstract and contributions list, is not currently supported.

major comments (5)
  1. [Section 2.3, Lemma 2.4; Appendix D.3] Lemma 2.4 is not proven and is false under the paper's definitions. Under Definition 2.1, Q_t depends on the random subset S_t through the selected clients' objective, and R_t depends on the same S_t through w*_{S_t} and the local optima of clients in S_t. Therefore Q_t and R_t are generally dependent even after conditioning on the data D: by the law of total covariance, Cov(Q_t,R_t|D)=E[Cov(Q_t,R_t|S_t,D)]+Cov(E[Q_t|S_t,D],E[R_t|S_t,D]), and the second term is generally nonzero. The proof in D.3 invokes the mutual independence chain w0 ⊥ Q0 ⊥ ... ⊥ Qt ⊥ R0 ⊥ ..., which is stronger than Assumption 2.3 and is not derived. The Kalman gain (15c) and the variance-reduction formula (11) both require this independence, so Theorem 2.5 is unsupported.
  2. [Appendix D.5.2, Eqs. (37)-(38)] The derivation of the fused normal distribution states that the product A×B of two normal random variables is normal with the displayed moments. This is false for the product of independent Gaussian random variables. What would be needed is the product of two Gaussian densities over a common variable, and the notation and moment formula in Eq. (38) do not express that. In addition, Theorem 2.5's Eq. (11) is misprinted: the fused mean is written as 'µ1σ2_Rt + µ2 σ̂2_t+1 / σ2_Rt' without the correct denominator. These defects make the closed-form posterior (11) and the claimed variance reduction unclear and unjustified.
  3. [Section 2.4, Eq. (16)] The estimators σ²_Qt and σ²_Rt in Eq. (16) are empirical mean-squared deviations computed from M_t, Δw̃_t, and per-client updates. These quantities are functions of the same random subset S_t and model state w_t that enter the Kalman update, and no unbiasedness or consistency argument is supplied. The theorem assumes σ²_Qt and σ²_Rt are known variances that are external to the update; plugging in data-dependent estimates changes the posterior and invalidates the stated variance-reduction guarantee. The scaling factors |S_t|d and |S_t|²d are also not derived.
  4. [Appendix F, Theorem F.3; proofs F.4.2-F.4.3] The convergence proof relies on two unjustified steps. First, after deriving a recursion for E[M_t], the proof assumes 'for simplicity' that E[M_t]≈∇f(w_t); this is essentially the property to be proven. Second, Lemma F.5's proof asserts that the cross term E⟨Δw̃_t−∇f(w_t), ∇f(w_t)−M_t⟩ vanishes because Δw̃_t is 'independent of past information'; however, Δw̃_t is computed from clients starting at w_t, which is a function of past information and past noises, so that independence is not established. The O(1/T) convergence claim is therefore not proven.
  5. [Assumption 2.2; Appendix D.1-D.2] The Gaussian-noise model for drift is asserted, and the justification in D.1 appeals to a CLT for averaged quantities. The noise in FL arises from client sampling without replacement and multiple local SGD steps on non-convex objectives; the conditions needed for a Gaussian limit are not established. The empirical test in Appendix D.2 (Fig. 7) tracks only one coordinate of the parameter vector; p-values >0.05 for that single coordinate do not support a multivariate Gaussian assumption on the entire parameter drift. Since the Kalman equations and Theorem 2.5 depend critically on the Gaussian form, the theoretical evidence is conditional on a nontrivial, unverified distributional assumption.
minor comments (4)
  1. [Algorithm 1; Section 2.4] The algorithm requires initial values for M_0 and σ²_0, but these are not specified. The text says FedEve 'does not introduce additional hyperparameter tuning', yet these quantities are free parameters of the method.
  2. [Tables 1 and 3; Section 3.1] The FEMNIST 'natural, E=1' entry for FedAvg is 82.37±0.18 in Table 1 and 82.46±0.18 in Table 3; the discrepancy is not explained. Also, CIFAR-10 is mentioned in the setup (Section 3.1) but no CIFAR-10 results are reported.
  3. [Appendix D.4; Section 3.2] Appendix D.4 refers to 'Figure 4' when discussing the Kalman-gain boxplots, but the actual figure is Figure 8 in the appendix. Section 3.2 contains a dangling reference to 'Figure 3.2' instead of a figure/table number.
  4. [Algorithm 1 comment] The pseudo-code says 'Model update: executes Equations (13)-(17) in the main paper', but the model update equations are (15a)-(15f), and Eq. (16) defines variance estimates rather than an update. The equation numbering should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the variance-reduction theorem is a standard Kalman-filter identity derived from explicit Gaussian/independence assumptions, and the experimental claims are benchmarked against external baselines.

full rationale

The paper's central theoretical claim (Theorem 2.5, Eq. 11) is a standard Kalman-filter identity: if prediction and observation noises are independent zero-mean Gaussians with variances sigma_hat^2 and sigma_R^2, then the posterior variance is sigma_hat^2 * sigma_R^2 / (sigma_hat^2 + sigma_R^2), which is smaller than each individual variance. This is a mathematical consequence of the explicitly stated Gaussian assumptions, not a parameter fitted to data and then read back as a prediction. The variance estimates in Eq. (16) are adaptive noise-covariance estimates used inside the filter; they are not hidden targets of the theorem. No load-bearing argument is justified by a self-citation: the Gaussian-noise assumption is supported by external references (CLT, SGD-noise literature), and the empirical baselines are external. The main weakness noted in the manuscript—Assumption 2.3/Lemma 2.4 requiring independence of period drift and client drift even though both depend on the same randomly sampled client subset S_t—is a validity or robustness concern about the assumptions, not a circularity. The paper does not define Q_t or R_t in terms of the variance-reduction result it later derives, nor does it fit a parameter and then announce the same quantity as a prediction. If the independence assumption fails, Theorem 2.5 is not applicable to real FL dynamics, but the derivation itself is not circular. Therefore, the appropriate circularity finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on several ad hoc assumptions: Gaussianity of the noise, independence of the two drifts, known variances, and a homogeneity assumption in the convergence proof. The independence assumption is the most fragile because both drifts share the same client sampling randomness. The free parameters are mostly unstated initial values, and the only invented entity is a conceptual re-framing.

free parameters (3)
  • global learning rate eta_g = 1
    Set to 1 for all experiments; hand-chosen, not fitted, but still a manually assigned scalar.
  • initial variance sigma_0^2 = not specified
    The Kalman variance recursion (Eqs. 15b, 15f) requires an initial variance value, but the paper does not state how sigma_0^2 is initialized.
  • initial momentum M_0 = not specified
    Equation (15d) updates M_t, but the initial value is not given; presumably zero, but unstated.
assumptions (4)
  • ad hoc to paper Aggregated server parameters equal optimal parameters plus independent Gaussian noise (Assumption 2.2).
    The paper justifies via CLT, but in non-convex non-iid settings the aggregate model is not an unbiased Gaussian perturbation of the optimum; this is load-bearing for the Kalman derivation.
  • ad hoc to paper Period drift Q_t and client drift R_t are independent of each other and of the current model state (Assumption 2.3, Lemma 2.4).
    Both drifts depend on the same client subset S_t, creating correlation through sampling; conditional independence given D does not imply the required independence.
  • ad hoc to paper Noise variables Q_t and R_t are normally distributed with zero mean and known variances (Section 2.3).
    Gaussianity is asserted rather than derived; the empirical normality test in Appendix D.2 covers one scalar parameter, not the high-dimensional model updates.
  • domain assumption For the convergence proof, all client gradients share a common mean gradient (implicit in Lemma F.4 proof).
    The proof of Lemma F.4 assumes E[grad fi(w)] = mu for all clients, which contradicts the non-iid setting the paper studies.
invented entities (1)
  • Period drift
    purpose: Name the distribution shift between the selected clients' objective and the global objective in partial participation FL.
    This is a re-framing of the variance of client sampling (earlier studied by Cho et al. 2022, Fraboni et al. 2023), not a separately measurable entity; it is not falsifiable outside the algorithm.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedEve: On Bridging the Client Drift and Period Drift for Cross-device Federated Learning." pith.science (2026). https://pith.science/paper/F7SGG4IO

@misc{pith2026250814539,
  author       = {Pith},
  title        = {Pith review of: FedEve: On Bridging the Client Drift and Period Drift for Cross-device Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F7SGG4IO}},
  note         = {Machine review of arXiv:2508.14539}
}
read the original abstract

Federated learning (FL) is a machine learning paradigm that allows multiple clients to collaboratively train a shared model without exposing their private data. Data heterogeneity is a fundamental challenge in FL, which can result in poor convergence and performance degradation. Client drift has been recognized as one of the factors contributing to this issue resulting from the multiple local updates in FedAvg. However, in cross-device FL, a different form of drift arises due to the partial client participation, but it has not been studied well. This drift, we referred as period drift, occurs as participating clients at each communication round may exhibit distinct data distribution that deviates from that of all clients. It could be more harmful than client drift since the optimization objective shifts with every round. In this paper, we investigate the interaction between period drift and client drift, finding that period drift can have a particularly detrimental effect on cross-device FL as the degree of data heterogeneity increases. To tackle these issues, we propose a predict-observe framework and present an instantiated method, FedEve, where these two types of drift can compensate each other to mitigate their overall impact. We provide theoretical evidence that our approach can reduce the variance of model updates. Extensive experiments demonstrate that our method outperforms alternatives on non-iid data in cross-device settings.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 21 canonical work pages

  1. [1]

    Revisiting the Noise Model of Stochastic Gradient Descent

    Barak Battash and Ofir Lindenbaum. Revisiting the noise model of stochastic gradient descent. arXiv preprint arXiv:2303.02749, 2023

    work page Pith review arXiv 2023

  2. [2]

    Leaf: A benchmark for federated settings

    Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Kone c n \`y , H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018

    arXiv 2018

  3. [3]

    On bridging generic and personalized federated learning for image classification

    Hong-You Chen and Wei-Lun Chao. On bridging generic and personalized federated learning for image classification. In International Conference on Learning Representations, 2021

    work page 2021

  4. [4]

    Towards understanding biased client selection in federated learning

    Yae Jee Cho, Jianyu Wang, and Gauri Joshi. Towards understanding biased client selection in federated learning. In International Conference on Artificial Intelligence and Statistics, pages 10351--10375. PMLR, 2022

    work page 2022

  5. [5]

    A general theory for client sampling in federated learning

    Yann Fraboni, Richard Vidal, Laetitia Kameni, and Marco Lorenzi. A general theory for client sampling in federated learning. In Trustworthy Federated Learning: First International Workshop, FL 2022, Held in Conjunction with IJCAI 2022, Vienna, Austria, July 23, 2022, Revised Selected Papers, pages 46--58. Springer, 2023

    work page 2022

  6. [6]

    Towards Federated Learning on Time-Evolving Heterogeneous Data

    Yongxin Guo, Tao Lin, and Xiaoying Tang. Towards federated learning on time-evolving heterogeneous data. arXiv preprint arXiv:2112.13246, 2021

    work page Pith review arXiv 2021

  7. [7]

    The movielens datasets: History and context

    F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5 0 (4): 0 1--19, 2015

    2015

  8. [8]

    The non-iid data quagmire of decentralized machine learning

    Kevin Hsieh, Amar Phanishayee, Onur Mutlu, and Phillip Gibbons. The non-iid data quagmire of decentralized machine learning. In International Conference on Machine Learning, pages 4387--4398. PMLR, 2020

    work page 2020

Show all 35 references
  1. [9]

    Measuring the effects of non-identical data distribution for federated visual classification

    Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335, 2019

    1909 arXiv

  2. [10]

    Advances and open problems in federated learning

    Peter Kairouz, H Brendan McMahan, Brendan Avent, Aur \'e lien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning , 1...

    2021

  3. [11]

    Reddi, Sebastian U

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, and Ananda Theertha Suresh. SCAFFOLD : Stochastic Controlled Averaging for Federated Learning . arXiv:1910.06378 [cs, math, stat], 2021

    1910 arXiv

  4. [12]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  5. [13]

    Brownian motion in a field of force and the diffusion model of chemical reactions

    Hendrik Anthony Kramers. Brownian motion in a field of force and the diffusion model of chemical reactions. Physica, 7 0 (4): 0 284--304, 1940

    1940

  6. [14]

    Gradient-based learning applied to document recognition

    Yann LeCun, L \'e on Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998

    1998

  7. [15]

    Federated learning: Challenges, methods, and future directions

    Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37 0 (3): 0 50--60, 2020 a

    2020

  8. [16]

    Federated Optimization in Heterogeneous Networks

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated Optimization in Heterogeneous Networks . arXiv:1812.06127 [cs, stat], 2020 b

    2020 arXiv

  9. [17]

    Don't use large mini-batches, use local sgd

    Tao Lin, Sebastian U Stich, Kumar Kshitij Patel, and Martin Jaggi. Don't use large mini-batches, use local sgd. arXiv preprint arXiv:1808.07217, 2018

    2018 arXiv

  10. [18]

    Ensemble distillation for robust model fusion in federated learning

    Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, 33: 0 2351--2363, 2020

    2020

  11. [19]

    Eine neue herleitung des exponentialgesetzes in der wahrscheinlichkeitsrechnung

    Jarl Waldemar Lindeberg. Eine neue herleitung des exponentialgesetzes in der wahrscheinlichkeitsrechnung. Mathematische Zeitschrift, 15 0 (1): 0 211--225, 1922

    1922

  12. [20]

    Stochastic gradient descent as approximate bayesian inference

    Stephan Mandt, Matthew D Hoffman, and David M Blei. Stochastic gradient descent as approximate bayesian inference. Journal of Machine Learning Research, 18: 0 1--35, 2017

    2017

  13. [21]

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Ag \"u era y Arcas

    H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Ag \"u era y Arcas. Communication- Efficient Learning of Deep Networks from Decentralized Data . arXiv:1602.05629 [cs], 2017

    2017 arXiv

  14. [22]

    Fedfast: Going beyond average for faster training of federated recommender systems

    Khalil Muhammad, Qinqin Wang, Diarmuid O'Reilly-Morgan, Elias Tragos, Barry Smyth, Neil Hurley, James Geraci, and Aonghus Lawlor. Fedfast: Going beyond average for faster training of federated recommender systems. In Proceedings of the 26th ACM SIGKDD International Conference ...

    2020

  15. [23]

    Adaptive federated optimization

    Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Kone c n \`y , Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization. arXiv preprint arXiv:2003.00295, 2020

    2003 arXiv

  16. [24]

    A tail-index analysis of stochastic gradient noise in deep neural networks

    Umut Simsekli, Levent Sagun, and Mert Gurbuzbalaban. A tail-index analysis of stochastic gradient noise in deep neural networks. In International Conference on Machine Learning, pages 5827--5837. PMLR, 2019

    2019

  17. [25]

    Understanding generalization of federated learning via stability: Heterogeneity matters

    Zhenyu Sun, Xiaochun Niu, and Ermin Wei. Understanding generalization of federated learning via stability: Heterogeneity matters. arXiv preprint arXiv:2306.03824, 2023

    2023 arXiv

  18. [26]

    On the noisy gradient descent that generalizes as sgd

    Jingfeng Wu, Wenqing Hu, Haoyi Xiong, Jun Huan, Vladimir Braverman, and Zhanxing Zhu. On the noisy gradient descent that generalizes as sgd. In International Conference on Machine Learning, pages 10367--10376. PMLR, 2020

    2020

  19. [27]

    Understanding short-horizon bias in stochastic meta-optimization

    Yuhuai Wu, Mengye Ren, Renjie Liao, and Roger Grosse. Understanding short-horizon bias in stochastic meta-optimization. arXiv preprint arXiv:1803.02021, 2018

    2018 arXiv

  20. [28]

    A diffusion theory for deep learning dynamics: Stochastic gradient descent exponentially favors flat minima

    Zeke Xie, Issei Sato, and Masashi Sugiyama. A diffusion theory for deep learning dynamics: Stochastic gradient descent exponentially favors flat minima. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.n...

    2021

  21. [29]

    Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating

    Xin Yao, Tianchi Huang, Rui-Xiao Zhang, Ruiyu Li, and Lifeng Sun. Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating . CoRR, abs/1910.08234, 2019

    1910 arXiv

  22. [30]

    Lookahead optimizer: k steps forward, 1 step back

    Michael Zhang, James Lucas, Jimmy Ba, and Geoffrey E Hinton. Lookahead optimizer: k steps forward, 1 step back. Advances in neural information processing systems, 32, 2019

    2019

  23. [31]

    Federated Learning with Non - IID Data

    Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated Learning with Non - IID Data . arXiv:1806.00582 [cs, stat], 2018

    2018 arXiv

  24. [32]

    Deep interest network for click-through rate prediction

    Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages...

    2018

  25. [33]

    Diurnal or nocturnal? federated learning of multi-branch networks from periodically shifting distributions

    Chen Zhu, Zheng Xu, Mingqing Chen, Jakub Kone c n \`y , Andrew Hard, and Tom Goldstein. Diurnal or nocturnal? federated learning of multi-branch networks from periodically shifting distributions. In International Conference on Learning Representations, 2022

    2022

  26. [34]

    The anisotropic noise in stochastic gradient descent: Its behavior of escaping from sharp minima and regularization effects

    Zhanxing Zhu, Jingfeng Wu, Bing Yu, Lei Wu, and Jinwen Ma. The anisotropic noise in stochastic gradient descent: Its behavior of escaping from sharp minima and regularization effects. In International Conference on Machine Learning, pages 7654--7663. PMLR, 2019

    2019

  27. [35]

    Strength of minibatch noise in sgd

    Liu Ziyin, Kangqiao Liu, Takashi Mori, and Masahito Ueda. Strength of minibatch noise in sgd. arXiv preprint arXiv:2102.05375, 2021

    2021 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.