Pith. sign in

REVIEW 4 major objections 4 minor 58 references

A Hybrid Stochastic Gradient Tracking Method for Distributed Online Optimization Over Time-Varying Directed Networks

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that a hybrid variance-reduced gradient-tracking algorithm, TV-HSGT, achieves a dynamic regret bound over time-varying directed networks without assuming uniformly bounded gradients.

desk verdict A promising but unproven synthesis: hybrid variance reduction plus AB gradient tracking for time-varying directed networks, with a repairable but load-bearing gap in the main theorem's proof. read the letter →

arxiv 2508.20645 v1 pith:BIBEGFED submitted 2025-08-28 cs.LG cs.DCmath.OC

Xinli Shi , Xingxing Yuan , Longkang Zhu , Guanghui Wen This is my paper · ORCID
classification cs.LGcs.DCmath.OC MSC 68W1590C2590C35
keywords distributedonlineoptimizationdynamicregretgradienttrackingvariancereductiontime-varyingdirectednetworksrow-columnstochasticmatriceshybridgradientsstrongconvexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces TV-HSGT, a distributed online optimization algorithm for agents connected by time-varying directed networks, and proves a dynamic regret bound for it in a stochastic strongly convex setting. The central claim is that combining a hybrid stochastic-gradient estimator (a convex mixture of the current stochastic gradient and a stochastic recursive gradient) with a gradient-tracking variable lets each agent track the global descent direction without the usual uniformly-bounded-gradient assumption. The proof bounds four coupled error terms through a single linear inequality, then chooses the step size so that the constant bounding matrix has spectral radius below one. If correct, this gives the first dynamic regret bound for a hybrid variance-reduced gradient-tracking method over time-varying directed topologies, and it indicates that unbounded gradients need not preclude sublinear dynamic regret.

What carries the argument

The central object is the coupled error vector $V_t = [\mathbb{E}\|x_t-\hat{x}_t\|^2_{\phi_t},\ \mathbb{E}S^2(y_t,\pi_t),\ \mathbb{E}\|\hat{x}_t-x_t^*\|^2,\ \mathbb{E}\|z_t-\nabla F_t(x_t)\|^2]^\top$, stacking consensus error, gradient-tracking error, optimality error, and hybrid gradient-estimation error. The argument drives this vector through a time-invariant linear inequality $V_{t+1}\le M(\alpha)V_t + b_{1,t}+b_2$, where $M(\alpha)$ is constructed from uniform bounds on the network-dependent quantities (a positive lower bound $\eta$ on $\phi_t^\top\pi_t$, contraction constants $c,\tau$, and bounds $\psi,\kappa,\varphi$). The hybrid estimator $z_{i,t+1}=(1-\beta)(z_{i,t}-\nabla\hat{f}_{i,t+1}(x_{i,t},\xi_{i,t+1}))+\nabla\hat{f}_{i,t+1}(x_{i,t+1},\xi_{i,t+1})$ damps the stochastic noise term, while the row-stochastic $A_t$ and column-stochastic $B_t$ updates form the AB communication scheme that avoids Perron-vector estimation.

What would settle it

Construct a sequence of strongly connected directed graphs whose smallest positive edge weight decays to zero while preserving the row-stochastic and column-stochastic conditions, and compute the product $\phi_t^\top \pi_t$ from the recursions; if this product tends to zero, the uniform lower bound $\eta$ in (22) does not exist and the step-size condition (46) forces $\alpha \to 0$, so the claimed bound cannot hold with a fixed step size. Separately, one can verify algebraically at the (3,3) entry whether the constant matrix $M(\alpha)$ actually dominates the time-varying $M_t(\alpha)$ entrywise; since $m_{11,t}=\mu n \phi_t^\top \pi_t \le \mu n = m_{11}$, the (3,3) entry of $M_t(\alpha)-M(\alpha)$ is nonnegative whenever $\phi_t^\top \pi_t<1$, so the dominance claim requires checking the other entries.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that this setting admits the dynamic regret bound $R_d^T \le O(\|V_0\| + \sum_{t=0}^{T-1}\|b_{1,t}\| + \beta^2\sigma^2 T \|b'_2\|)$, where $V_0$ collects the initial consensus, tracking, optimality, and gradient-estimation errors, $b_{1,t}$ accumulates the temporal drift of the objective gradients and minimizers (through $q_t$ and $p_t$), and the $\beta^2\sigma^2$ term is the residual stochastic noise the hybrid estimator damps. The analysis avoids the pervasive bounded-gradient assumption using only Lipschitz stochastic gradients, bounded variance, and strong convexity. The bound follows by showing the four error components contract jointly: the step-size condition (46) makes the spectral radius of the time-invariant matrix $M(\alpha)$ smaller than one, and the per-step disturbances are summed as a geometric series.

Load-bearing premise

The proof rests on assuming the network's time-varying weight structure, through the overlap $\phi_t^\top \pi_t$ and the per-step contraction constants, admits fixed uniform bounds that hold at every time step; if edge weights can decay without a positive floor, the contraction step-size condition (46) has no positive solution and the regret bound collapses.

Editorial extensions

If this is right

  • In the static case $f_t = f$, Corollary 1 yields a linearly convergent gradient-tracking algorithm with variance reduction, whose steady-state error can be made arbitrarily small by reducing $\beta$.
  • Under sublinear decay of the temporal variations $q_t$ and $p_t$, with a decaying step size and mixing parameter, the dynamic regret becomes sublinear in $T$.
  • The algorithm requires no Perron-vector estimation or out-degree information; agents use only local rows and columns of the weight matrices, making it implementable under local information.
  • The step-size range (46) is explicit and computable from the uniform bounds, giving a concrete tuning rule for contraction.
  • The hybrid estimator reduces the stochastic-gradient variance floor from $\sigma^2$ to $\beta^2\sigma^2$ in the regret bound, directly quantifying the benefit of the variance-reduction mechanism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same four-error coupled contraction could likely be replayed with other variance-reduction estimators (e.g., SVRG-style periodic snapshots), since the proof only needs the estimator drift controlled by $\|x_{t+1}-x_t\|^2$ as in Lemma 16; the hybrid mixture is sufficient, not necessary.
  • The explicit $\beta^2\sigma^2$ floor in the bound suggests a concrete tuning rule: decay $\beta$ in step with the drift measures $q_t$ and $p_t$ (or with $1/\sqrt{T}$) to convert the variance term into a sublinear contribution; the paper states the general condition but does not give a schedule.
  • If the uniform $\eta$-floor fails on sparse or large-scale networks, one could adapt the proof to per-agent or per-step step sizes, treating the time-varying $M_t(\alpha)$ rather than a single $M(\alpha)$; the current analysis points to that generalization but does not carry it out.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes TV-HSGT, a distributed online stochastic optimization algorithm for time-varying directed networks. The algorithm combines an AB (row-stochastic/column-stochastic) communication scheme, gradient tracking, and a hybrid variance-reduction update for stochastic gradients. The main theoretical claim is a dynamic regret bound, stated in Theorem 1, that does not assume uniformly bounded gradients. The paper also gives a static-optimization corollary and reports numerical experiments on A9A and MNIST logistic regression tasks, comparing against DSGD, DSGT, and DSGT-HB.

Significance. If the main theorem is correct, this would be a useful contribution: it targets a genuinely harder setting than most prior work, combining time-varying directed graphs, stochastic gradients, variance reduction, and no bounded-gradient assumption. The proof framework—constructing a coupled four-component error recursion and then applying a spectral-radius argument—is reasonable and builds on established contraction lemmas. The paper also makes good use of existing AB/Push-Pull tools rather than inventing ad hoc assumptions. However, several concrete proof errors currently block the central claim, so the contribution is not yet established as written.

major comments (4)
  1. [§4.2, Proposition 1 and Eq. (38)] The entrywise bound M_t(α) ≤ M(α) is false at position (3,3). In M_t(α), the (3,3) entry is 1 − α m11_t with m11_t = µn φ_t^T π_t, while M(α) in Eq. (38) sets m11 = µn. Since φ_t^T π_t ≤ 1 and α > 0, we have 1 − αµn ≤ 1 − αµn φ_t^T π_t, so M(α)'s (3,3) entry is smaller than M_t(α)'s entry, not larger. For an upper-bounding matrix, m11 must be a uniform lower bound on µn φ_t^T π_t, i.e., m11 = µnη. Such an η does follow from Lemmas 5 and 6 together with Assumptions 5 and 6. Until this is corrected, the conclusion ρ(M(α)) < 1 does not imply contraction of the time-varying recursion, and Theorem 1 is unproven as written.
  2. [Theorem 1, condition (46)] The step-size condition in (46) does not ensure that Lemma 11 applies at every time. Lemma 11 requires 0 < α < 2/(n(µ + L_g) φ_t^T π_t) for each t. Since η defined in (22) is a lower bound on φ_t^T π_t, the threshold 2/(n(µ + L_g)η) is larger than the individual thresholds, and the smallest threshold over t occurs at the largest value of φ_t^T π_t, not at the lower bound η. Thus α satisfying α < 2/(n(µ + L_g)η) can violate the per-step condition from Lemma 11. A sufficient uniform condition would be α < 2/(n(µ + L_g)), using φ_t^T π_t ≤ 1, or alternatively an upper bound on max_t φ_t^T π_t.
  3. [Theorem 1 statement and final lines of proof] The driving-noise term in the final regret bound is inconsistent with the recursion established in Proposition 1. Proposition 1 defines b2 = [0, 6nτ²ψ/(1−τ)β²σ², 0, 2nβ²σ²]^T, while Theorem 1 states the bound in terms of b'_2 = [0, 6nτ²ψ/(1−τ), 0, n]^T and the proof concludes with b2 = β²σ² b'_2. The fourth component of β²σ² b'_2 is nβ²σ², not 2nβ²σ². Either Proposition 1's b2 should be corrected to have nβ²σ² in the fourth component, or b'_2's fourth component should be 2n. Without this, the stated O(β²σ²T‖b'_2‖) term does not follow from (30).
  4. [Appendix I, proof of Corollary 1] The geometric-series argument in the proof of Corollary 1 has a sign error. The displayed derivation gives lim sup_{t→∞} V_t ≤ −(I−M(α))^{-1} b. Since b ≥ 0 and M(α) has spectral radius less than 1, the matrix (I−M(α))^{-1} = Σ_{k≥0} M(α)^k is nonnegative with nonnegative entries, so (I−M(α))^{-1} b is nonnegative. A negative upper bound on the nonnegative vector V_t is impossible. The correct limiting bound is lim sup_{t→∞} V_t ≤ (I−M(α))^{-1} b. This error directly affects the stated static-case result.
minor comments (4)
  1. [§5, experimental setup] TV-HSGT is tested on time-varying directed graphs, while the baselines DSGD, DSGT, and DSGT-HB are run on fixed complete graphs with symmetric doubly stochastic weights. This confounds the algorithmic comparison with the topology difference; the authors should either run the baselines under the same AB time-varying directed scheme or clearly discuss this limitation.
  2. [Throughout] There are numerous typographical and formatting issues, including missing spaces in equations such as 'minT t=1', the caption 'or different algorithms' in Figs. 7 and 10, and inconsistent notation between z_t and z_{t+1} in the algorithm description. A careful proofread is needed.
  3. [§1, Table 1] The comparison in Table 1 lists only doubly stochastic or row-stochastic predecessors; the text would benefit from explicitly discussing how the claimed 'no Perron vector estimation' property differs from the Push-Sum based methods when the network is time-varying directed.
  4. [§4.2, Eq. (22)] The uniform bounds c, τ, ψ, κ, φ in Eq. (22) are introduced by assertion. For a fixed finite agent set, Assumptions 4–6 do imply uniform bounds, but the authors should state this explicitly and point to the finiteness of the set of admissible graphs, since the proof depends on these bounds in a load-bearing way.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dynamic regret bound is derived from standard external lemmas and an externally defined regret criterion.

full rationale

The derivation is self-contained. The dynamic regret in Definition 1 is an external criterion; the weighted average x̂_t is fixed before the analysis via the eigenvector ϕ_t of A_t and is not tuned afterward to force the bound. Lemmas 1–8 are quoted from external groups (e.g., [32,36,41]), none of which states the final regret bound; they supply only standard contraction, smoothness, and norm identities. Proposition 1 and Theorem 1 then assemble four error terms by substitution and by algebraic coefficient bounds, with the step-size condition (46) obtained from a constructed δ vector and positivity constraints, not from a parameter fitted to the desired regret expression. The regret decomposition (48) and the geometric-series summation are direct manipulations of the derived recursion. No fitted input is renamed as a prediction, and no load-bearing self-citation is used. The uniform-bound assertion in (22) and the m11 entry in (38) are potential correctness or soundness issues, but they are algebraic gap concerns, not circular reductions: the claimed conclusion is not identical to an assumption or to a fitted value.

Assumptions & free parameters 2 free parameters · 7 assumptions · 0 invented entities

The central claim rests on six domain assumptions that are standard in the distributed online optimization literature. No physically or mathematically new entity is invented. The algorithm has two user-chosen hyperparameters, α and β, but neither is fitted to data to force the theorem.

free parameters (2)
  • step size α = 0.001 in experiments; constrained by (46) in theory
    A hyperparameter required by the convergence analysis. The theory only requires it to lie in a bounded interval; the experiments set it by hand.
  • mixing parameter β = 0.01 in main experiments; swept over 0.01 to 0.5
    Controls the balance between current stochastic gradient and recursive gradient. Smaller β gives lower noise in experiments. It is chosen by the user, not fitted by the theory.
assumptions (7)
  • domain assumption The global objective f_t is µ-strongly convex (Assumption 1).
    Used in Lemma 1 and throughout the contraction analysis in Lemma 11. If strong convexity fails, the stated regret bound does not follow.
  • domain assumption Stochastic gradient estimators are L_g-Lipschitz in mean square (Assumption 2).
    Central to bounding the difference between consecutive hybrid gradients and to establishing smoothness of the objective. The whole proof depends on this condition.
  • domain assumption Stochastic gradients are unbiased and have bounded variance σ² (Assumption 3).
    Controls the noise terms in Lemmas 14 and 16. The variance term σ² appears directly in the regret bound.
  • domain assumption The time-varying graph G_t is strongly connected with self-loops, or C-strongly connected over blocks (Assumption 4).
    Needed for the existence of the stochastic vectors ϕ_t and π_t and for the contraction lemmas in Lemmas 5 through 8.
  • domain assumption The decision mixing matrix A_t is row-stochastic with smallest positive entry bounded below by a (Assumption 5).
    Required for the consensus contraction in Lemma 7 and for the definition of the weighted average x̂_t.
  • domain assumption The gradient tracking matrix B_t is column-stochastic with smallest positive entry bounded below by b (Assumption 6).
    Required for the gradient tracking contraction in Lemma 8 and for preserving the sum of tracking variables.
  • standard math Standard norm inequalities and matrix contraction lemmas from prior literature (Lemmas 1 through 8) are valid as cited.
    The proof imports these results without re-deriving them. If any cited lemma is misstated or inapplicable, the subsequent bounds break.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Hybrid Stochastic Gradient Tracking Method for Distributed Online Optimization Over Time-Varying Directed Networks." pith.science (2026). https://pith.science/paper/BIBEGFED

@misc{pith2026250820645,
  author       = {Pith},
  title        = {Pith review of: A Hybrid Stochastic Gradient Tracking Method for Distributed Online Optimization Over Time-Varying Directed Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BIBEGFED}},
  note         = {Machine review of arXiv:2508.20645}
}
read the original abstract

With the increasing scale and dynamics of data, distributed online optimization has become essential for real-time decision-making in various applications. However, existing algorithms often rely on bounded gradient assumptions and overlook the impact of stochastic gradients, especially in time-varying directed networks. This study proposes a novel Time-Varying Hybrid Stochastic Gradient Tracking algorithm named TV-HSGT, based on hybrid stochastic gradient tracking and variance reduction mechanisms. Specifically, TV-HSGT integrates row-stochastic and column-stochastic communication schemes over time-varying digraphs, eliminating the need for Perron vector estimation or out-degree information. By combining current and recursive stochastic gradients, it effectively reduces gradient variance while accurately tracking global descent directions. Theoretical analysis demonstrates that TV-HSGT can achieve improved bounds on dynamic regret without assuming gradient boundedness. Experimental results on logistic regression tasks confirm the effectiveness of TV-HSGT in dynamic and resource-constrained environments.

Figures

Figures reproduced from arXiv: 2508.20645 by the authors.

Figure 1
Figure 1. Time-averaged regret on the A9A dataset for different [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 5
Figure 5. Loss under different β values on the A9A dataset [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Accuracy under different β values on the A9A dataset Mi is the per-round batch size, and r i is the regularization coefficient. All experimental settings match those of the structured￾data experiments in Subsection 5.1. Each agent processes 100 random images per round. Figs. 7–9 show comparisons of time-averaged regret, loss, and accuracy across algo￾rithms. The results demonstrate that TV-HSGT converges fastest, si… view at source ↗
Figures from the paper (3 more)
Figure 7
Figure 7. Figure 7: Time-averaged regret on the MNIST dataset or different [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 11
Figure 11. Figure 11: Loss under different β values on the MNIST dataset [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Accuracy under different β values on the MNIST dataset By the property of the global optimal solution x ∗ t P , namely n i=1 ∇fi,t(x ∗ t ) = 0, we obtain [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 57 canonical work pages

  1. [1]

    Distributed stochastic optimization with gradient track- ing over time- varying directed networks

    Duong Thuy Anh Nguyen, Duong Tung Nguyen, and Angelia Nedic. Distributed stochastic optimization with gradient track- ing over time- varying directed networks. In 2023 57th Asilomar Conference on Signals, Systems, and Computers , pages 1605– 1609, 2023

    work page 2023

  2. [2]

    Decentralized online convex optimization with compressed communications

    Xuanyu Cao and Tamer Ba¸ sar. Decentralized online convex optimization with compressed communications. Automatica, 156:111186, 2023

    work page 2023

  3. [3]

    Vincent Poor

    Xuanyu Cao, Junshan Zhang, and H. Vincent Poor. Online stochastic optimization with time-varying distributions. IEEE Transactions on Automatic Control , 66(4):1840–1847, 2021

    work page 2021

  4. [4]

    Gtadam: Gradient tracking with adap- tive momentum for distributed online optimization.IEEE Trans- actions on Control of Network Systems , 10(3):1436–1448, 2022

    Guido Carnevale, Francesco Farina, Ivano Notarnicola, and Giuseppe Notarstefano. Gtadam: Gradient tracking with adap- tive momentum for distributed online optimization.IEEE Trans- actions on Control of Network Systems , 10(3):1436–1448, 2022

    work page 2022

  5. [5]

    Diffusion lms strategies for distributed estimation

    Federico S Cattivelli and Ali H Sayed. Diffusion lms strategies for distributed estimation. IEEE transactions on signal processing , 58(3):1035–1048, 2009

    work page 2009

  6. [6]

    Accelerated distributed stochastic nonconvex optimization over time-varying directed networks

    Yiyue Chen, Abolfazl Hashemi, and Haris Vikalo. Accelerated distributed stochastic nonconvex optimization over time-varying directed networks. IEEE Transactions on Automatic Control , 70(4):2196–2211, 2025

    work page 2025

  7. [7]

    Local differential privacy for decentralized online stochastic optimization with guaranteed op- timality and convergence speed

    Ziqin Chen and Yongqiang Wang. Local differential privacy for decentralized online stochastic optimization with guaranteed op- timality and convergence speed. IEEE Transactions on Auto- matic Control, pages 1–16, 2024

    work page 2024

  8. [8]

    Optimization and learning with information streams: Time-varying algorithms and applications

    Emiliano Dall’Anese, Andrea Simonetto, Stephen Becker, and Liam Madden. Optimization and learning with information streams: Time-varying algorithms and applications. IEEE Sig- nal Processing Magazine , 37(3):71–83, 2020

    work page 2020

Show all 58 references
  1. [9]

    Saga: A fast incremental gradient method with support for non-strongly convex composite objectives

    Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. Saga: A fast incremental gradient method with support for non-strongly convex composite objectives. In Proceedings of the 28th Inter- national Conference on Neural Information Processing Systems - Volume 1 , page 1646–1654, 2014

    2014

  2. [10]

    Improving dynamic regret in distributed online mirror descent using primal and dual informa- tion

    Nima Eshraghi and Ben Liang. Improving dynamic regret in distributed online mirror descent using primal and dual informa- tion. In Learning for Dynamics and Control Conference , pages 637–649. PMLR, 2022

    2022

  3. [11]

    Distributed stochastic gradient tracking methods with momentum acceleration for non-convex optimization

    Juan Gao, Xin-Wei Liu, Yu-Hong Dai, Yakui Huang, and Jun- hua Gu. Distributed stochastic gradient tracking methods with momentum acceleration for non-convex optimization. Computa- tional Optimization and Applications , 84(2):531–572, November 2022

    2022

  4. [12]

    Introduction to online convex optimiza- tion

    Elad Hazan et al. Introduction to online convex optimiza- tion. Foundations and Trends ® in Optimization , 2(3-4):157– 325, 2016

    2016

  5. [13]

    Matrix analysis

    Roger A Horn and Charles R Johnson. Matrix analysis . Cam- bridge university press, 2012

    2012

  6. [14]

    On- line distributed optimization via dual averaging

    Saghar Hosseini, Airlie Chapman, and Mehran Mesbahi. On- line distributed optimization via dual averaging. In 52nd IEEE Conference on Decision and Control , pages 1484–1489, 2013

    2013

  7. [15]

    Faster adaptive decentralized learning algorithms

    Feihu Huang and Jianyu Zhao. Faster adaptive decentralized learning algorithms. In Proceedings of the 41st International Conference on Machine Learning , pages 20490–20525, 2024

    2024

  8. [16]

    Accelerating stochastic gradient descent using predictive variance reduction

    Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. InProceedings of the 27th International Conference on Neural Information Processing Systems, page 315–323, 2013

    2013

  9. [17]

    Stochastic dual averaging for decentralized online optimization on time- varying communication graphs

    Soomin Lee, Angelia Nedi´ c, and Maxim Raginsky. Stochastic dual averaging for decentralized online optimization on time- varying communication graphs. IEEE Transactions on Auto- matic Control, 62(12):6407–6414, 2017

    2017

  10. [18]

    Coordinate dual averaging for decentralized online optimization with non- separable global objectives

    Soomin Lee, Angelia Nedi´ c, and Maxim Raginsky. Coordinate dual averaging for decentralized online optimization with non- separable global objectives. IEEE Transactions on Control of Network Systems , 5(1):34–44, 2018

    2018

  11. [19]

    Distributed Optimization: Advances in Theo- ries, Methods, and Applications

    Huaqing Li, Qingguo L¨ u, Zheng Wang, Xiaofeng Liao, and Tingwen Huang. Distributed Optimization: Advances in Theo- ries, Methods, and Applications . Springer, 2020

    2020

  12. [20]

    Npga: A unified algorithmic framework for decentralized constraint-coupled optimization

    Jingwang Li and Housheng Su. Npga: A unified algorithmic framework for decentralized constraint-coupled optimization. IEEE Transactions on Control of Network Systems , 11(3):1655– 1666, 2024

    2024

  13. [21]

    Online distributed stochastic gradient algorithm for nonconvex opti- mization with compressed communication

    Jueyou Li, Chaojie Li, Jing Fan, and Tingwen Huang. Online distributed stochastic gradient algorithm for nonconvex opti- mization with compressed communication. IEEE Transactions on Automatic Control , 69(2):936–951, 2024

    2024

  14. [22]

    A survey of decentralized online learning

    Xiuxian Li, Lihua Xie, and Na Li. A survey of decentralized online learning. CoRR, 2022

    2022

  15. [23]

    Distributed online convex optimization with an aggregative variable

    Xiuxian Li, Xinlei Yi, and Lihua Xie. Distributed online convex optimization with an aggregative variable. IEEE Transactions on Control of Network Systems , 9(1):438–449, 2022

    2022

  16. [24]

    Online optimization with predictions and switching costs: Fast algorithms and the fundamental limit

    Yingying Li, Guannan Qu, and Na Li. Online optimization with predictions and switching costs: Fast algorithms and the fundamental limit. IEEE Transactions on Automatic Control , 66(10):4761–4768, 2021

    2021

  17. [25]

    Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent

    Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. In Proceedings of the 31st Interna- tional Conference on Neural Inform...

    2017

  18. [26]

    A compressed gradient tracking method for decentralized optimization with linear convergence

    Yiwei Liao, Zhuorui Li, Kun Huang, and Shi Pu. A compressed gradient tracking method for decentralized optimization with linear convergence. IEEE Transactions on Automatic Control , 67(10):5622–5629, 2022

    2022

  19. [27]

    An optimal hybrid variance-reduced algorithm for stochastic composite non- convex optimization

    Deyi Liu, Lam M Nguyen, and Quoc Tran-Dinh. An optimal hybrid variance-reduced algorithm for stochastic composite non- convex optimization. arXiv preprint arXiv:2008.09055 , 2020

    2008 arXiv

  20. [28]

    Decentralized federated learning: Balancing communication and computing costs

    Wei Liu, Li Chen, and Wenyi Zhang. Decentralized federated learning: Balancing communication and computing costs. IEEE Transactions on Signal and Information Processing over Net- works, 8:131–143, 2022

    2022

  21. [29]

    Online dictionary learning for sparse coding

    Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. Online dictionary learning for sparse coding. In Proceedings of the 26th Annual International Conference on Machine Learning , page 689–696, 2009

    2009

  22. [30]

    Online optimization in dynamic environments: Improved regret rates for strongly convex problems

    Aryan Mokhtari, Shahin Shahrampour, Ali Jadbabaie, and Ale- jandro Ribeiro. Online optimization in dynamic environments: Improved regret rates for strongly convex problems. In 2016 IEEE 55th Conference on Decision and Control (CDC) , page 7195–7201, 2016

    2016

  23. [31]

    Dadam: A consensus-based distributed adaptive gradient method for online optimization

    Parvin Nazari, Davoud Ataee Tarzanagh, and George Michai- lidis. Dadam: A consensus-based distributed adaptive gradient method for online optimization. IEEE Transactions on Signal 15 Processing, 70:6065–6079, 2022

    2022

  24. [32]

    AB/Push-Pull method for distributed optimization in time-varying directed networks

    Angelia Nedi´ c, Duong Thuy Anh Nguyen, and Duong Tung Nguyen. AB/Push-Pull method for distributed optimization in time-varying directed networks. Optimization Methods and Software, pages 1–28, 2023

    2023

  25. [33]

    Fast conver- gence rates for distributed non-bayesian learning

    Angelia Nedi´ c, Alex Olshevsky, and C´ esar A Uribe. Fast conver- gence rates for distributed non-bayesian learning. IEEE Trans- actions on Automatic Control , 62(11):5538–5553, 2017

    2017

  26. [34]

    Distributed stochastic optimization with gradient track- ing over time-varying directed networks

    Duong Thuy Anh Nguyen, Duong Tung Nguyen, and Angelia Nedic. Distributed stochastic optimization with gradient track- ing over time-varying directed networks. In Proceedings of the 57th Asilomar Conference on Signals, Systems, and Computers , pages 1605–1609, 2023

    2023

  27. [35]

    Accelerated AB/Push–Pull methods for distributed op- timization over time-varying directed networks

    Duong Thuy Anh Nguyen, Duong Tung Nguyen, and Angelia Nedi´ c. Accelerated AB/Push–Pull methods for distributed op- timization over time-varying directed networks. IEEE Transac- tions on Control of Network Systems , 11(3):1395–1407, 2024

    2024

  28. [36]

    Distributed Nash equilibrium seeking over time-varying directed communication networks

    Duong Thuy Anh Nguyen, Duong Tung Nguyen, and Angelia Nedi´ c. Distributed Nash equilibrium seeking over time-varying directed communication networks. IEEE Transactions on Con- trol of Network Systems , pages 1–12, 2025

    2025

  29. [37]

    Nguyen, Jie Liu, Katya Scheinberg, and Martin Tak´ aˇ c

    Lam M. Nguyen, Jie Liu, Katya Scheinberg, and Martin Tak´ aˇ c. SARAH: A novel method for machine learning problems using stochastic recursive gradient. In Proceedings of the 34th Interna- tional Conference on Machine Learning , page 2613–2621, 2017

    2017

  30. [38]

    Distributed personalized gradient track- ing with convex parametric models

    Ivano Notarnicola, Andrea Simonetto, Francesco Farina, and Giuseppe Notarstefano. Distributed personalized gradient track- ing with convex parametric models. IEEE Transactions on Au- tomatic Control, 68(1):588–595, 2023

    2023

  31. [39]

    Distributed stochastic gradient track- ing methods

    Shi Pu and Angelia Nedi´ c. Distributed stochastic gradient track- ing methods. Mathematical Programming, 187(1):409–457, 2021

    2021

  32. [40]

    Push–pull gradient methods for distributed optimization in networks.IEEE Transactions on Automatic Control , 66(1):1–16, 2021

    Shi Pu, Wei Shi, Jinming Xu, and Angelia Nedi´ c. Push–pull gradient methods for distributed optimization in networks.IEEE Transactions on Automatic Control , 66(1):1–16, 2021

    2021

  33. [41]

    Harnessing smoothness to accelerate distributed optimization

    Guannan Qu and Na Li. Harnessing smoothness to accelerate distributed optimization. IEEE Transactions on Control of Net- work Systems , 5(3):1245–1260, 2017

    2017

  34. [42]

    Fakhteh Saadatniaki, Ran Xin, and Usman A. Khan. Decentral- ized optimization over time-varying directed graphs with row and column-stochastic matrices. IEEE Transactions on Auto- matic Control, 65(11):4769–4780, 2020

    2020

  35. [43]

    An online optimiza- tion approach for multi-agent tracking of dynamic parameters in the presence of adversarial noise

    Shahin Shahrampour and Ali Jadbabaie. An online optimiza- tion approach for multi-agent tracking of dynamic parameters in the presence of adversarial noise. In 2017 American Control Conference (ACC), pages 3306–3311, 2017

    2017

  36. [44]

    Distributed online op- timization in dynamic environments using mirror descent

    Shahin Shahrampour and Ali Jadbabaie. Distributed online op- timization in dynamic environments using mirror descent. IEEE Transactions on Automatic Control , 63(3):714–725, 2018

    2018

  37. [45]

    Distributed detection: Finite-time analysis and impact of net- work topology

    Shahin Shahrampour, Alexander Rakhlin, and Ali Jadbabaie. Distributed detection: Finite-time analysis and impact of net- work topology. IEEE Transactions on Automatic Control , 61(11):3256–3268, 2015

    2015

  38. [46]

    Online learning and online convex optimization

    Shai Shalev-Shwartz et al. Online learning and online convex optimization. Foundations and Trends ® in Machine Learning , 4(2):107–194, 2012

    2012

  39. [47]

    Optimized gradient tracking for decentralized online learning

    Shivangi Dubey Sharma and Ketan Rajawat. Optimized gradient tracking for decentralized online learning. IEEE Transactions on Signal Processing , 72:1443–1459, 2024

    2024

  40. [48]

    Pham N. H. Phan D. T. Tran-Dinh, Q. and L. M. Nguyen. A hybrid stochastic optimization framework for composite noncon- vex optimization. Mathematical Programming, 191:1005–1071, 2022

    2022

  41. [49]

    Distributed weakly convex optimization under random time-delay interfer- ence

    Mengli Wei, Wenwu Yu, Hongzhe Liu, and Qian Xu. Distributed weakly convex optimization under random time-delay interfer- ence. IEEE Transactions on Network Science and Engineering , 11(1):212–224, 2023

    2023

  42. [50]

    A hybrid variance- reduced method for decentralized stochastic non-convex opti- mization

    Ran Xin, Usman Khan, and Soummya Kar. A hybrid variance- reduced method for decentralized stochastic non-convex opti- mization. In Proceedings of the 38th International Conference on Machine Learning , pages 11459–11469, 18–24 Jul 2021

    2021

  43. [51]

    Khan, and Soummya Kar

    Ran Xin, Usman A. Khan, and Soummya Kar. Variance-reduced decentralized stochastic optimization with accelerated conver- gence. IEEE Transactions on Signal Processing , 68:6255–6271, 2020

    2020

  44. [52]

    Menghui Xiong, Daniel W. C. Ho, Baoyong Zhang, Deming Yuan, and Shengyuan Xu. Distributed online mirror descent with delayed subgradient and event-triggered communications. IEEE Transactions on Network Science and Engineering , 11(2):1702– 1715, 2024

    2024

  45. [53]

    Distributed autonomous online learning: Regrets and intrinsic privacy-preserving properties

    Feng Yan, Shreyas Sundaram, SVN Vishwanathan, and Yuan Qi. Distributed autonomous online learning: Regrets and intrinsic privacy-preserving properties. IEEE Transactions on Knowledge and Data Engineering , 25(11):2483–2493, 2012

    2012

  46. [54]

    Online distributed opti- mization algorithm with dynamic regret analysis under unbal- anced graphs

    Songquan Yao, Siyu Xie, and Tao Li. Online distributed opti- mization algorithm with dynamic regret analysis under unbal- anced graphs. Automatica, 174:112116, 2025

    2025

  47. [55]

    An adap- tive primal-dual subgradient algorithm for online distributed constrained optimization

    Deming Yuan, Daniel WC Ho, and Guo-Ping Jiang. An adap- tive primal-dual subgradient algorithm for online distributed constrained optimization. IEEE Transactions on Cybernetics , 48(11):3045–3055, 2017

    2017

  48. [56]

    Distributed adaptive subgradi- ent algorithms for online learning over time-varying networks

    Mingchuan Zhang, Bowei Hao, Quanbo Ge, Junlong Zhu, Rui- juan Zheng, and Qingtao Wu. Distributed adaptive subgradi- ent algorithms for online learning over time-varying networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems , 52(7):4518–4529, 2022

    2022

  49. [57]

    Ravier, Michael M

    Yan Zhang, Robert J. Ravier, Michael M. Zavlanos, and Vahid Tarokh. A distributed online convex optimization algorithm with improved dynamic regret. In 2019 IEEE 58th Conference on Decision and Control (CDC) , pages 2449–2454, 2019

    2019

  50. [58]

    Online convex programming and generalized infinitesimal gradient ascent

    Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th Interna- tional Conference on Machine Learning , pages 928–936, 2003. 16

    2003

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.