Stochastic Online Feedback Optimization for Networks of Non-Compliant Agents

Andrey Bernstein; Caio Kalil Lauand

REVIEW 3 major objections 3 minor 1 cited by

Stochastic Online Feedback Optimization for Networks of Non-Compliant Agents

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper extends online feedback optimization to networked systems whose agents implement setpoints with random deviations, proving a mean-square tracking-error bound for a projection-based stochastic gradient algorithm driven only by noi

desk verdict Genuine extension of OFO to non-compliant agents, but the headline asymptotic bound (4) is not supported by the paper's own proof. read the letter →

arxiv 2508.21414 v2 pith:7OQZSJAT submitted 2025-08-29 math.OC

Caio Kalil Lauand , Andrey Bernstein This is my paper · ORCID

classification math.OC MSC 90C2590C1593E35

keywords onlinefeedbackoptimizationstochasticgradientdescentnon-compliantagentsmean-squareerrorboundstime-varyingoptimalpowerflowmeasurementapproximation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Online feedback optimization (OFO) steers a networked system to an optimal operating point using real-time input-output measurements instead of a full model. This paper extends OFO to networks of non-compliant agents: given a setpoint, each agent implements a random input, so the optimization objective is an expectation over that randomness. The authors prove that a projected stochastic gradient update, whose gradient is reconstructed from noisy measurements of the actual input and output, tracks the time-varying minimizer with mean-square error bounded by terms reflecting compliance volatility, measurement error, and problem variability (Theorem 2.3). In the static, noiseless, full-information case the bound recovers the classical SGD rate; when the compliance randomness is time-invariant and the problem is static, it recovers deterministic OFO convergence. A regularized variant covers cases where the main theorem's full-rank condition fails, and a real-time optimal power flow experiment shows fewer voltage violations than the deterministic baseline.

What carries the argument

The carrying object is the surrogate gradient (5), nabla-hat-Gamma(n) = (A^circ_{n+1})^T C^T nabla g_y^{(n)}(y_hat_n) + (A^circ_{n+1})^T nabla g_x^{(n)}(x_hat_n), inserted into the projected iteration (3). Non-compliant agents are modeled by x_n = phi(u_n, Phi_{n+1}) = A_{n+1} u_n + b_{n+1}, with A_{n+1} the random compliance matrix. The proof machinery combines strong convexity and Lipschitz smoothness (A1) to make the true stochastic gradient strongly monotone in conditional mean (Lemma 2.1); assumption (A4) supplies a recovered compliance matrix A^circ whose L4 error is epsilon_m; Lemma A.2 bounds the mismatch between surrogate and true gradient; and Lemmas A.3-A.6 convert this into a rec

What would settle it

Run the algorithm with a non-diagonal random compliance matrix (or with nonzero offset b_{n+1}) and noisy input measurements x_hat_n; if the observed mean-square error violates the Theorem 2.3 bound or fails to stay bounded, assumption (A4) is falsified. Alternatively, use input-measurement noise with finite second but infinite fourth moment to test whether the L4 condition in (A4) is necessary.

Watch

Extended reading notes

Core claim

The central claim is that a projected stochastic-gradient rule can optimize a time-varying convex network problem even when the agents that apply the control inputs are non-compliant: the implemented input is x_n = A_{n+1} u_n + b_{n+1}, a random function of the requested setpoint. The algorithm (3)-(5) uses only noisy measurements x_hat_n, y_hat_n and a recovered compliance matrix A^circ_{n+1} to build a surrogate gradient, and it tracks the true moving minimizer u*_n in mean square. Under assumptions (A1)-(A5), Theorem 2.3 gives limsup E||u_N - u*_N||^2 <= alpha epsilon_a + epsilon_b + epsilon_c + b_{2.3} alpha^{3/2} sqrt(epsilon_c(alpha epsilon_a + epsilon_b)), where epsilon_a measures vo

Load-bearing premise

Everything rests on assumption (A4): that the random compliance matrix A_{n+1} can be recovered from noisy input observations and setpoints with L4 error epsilon_m; the experiments only exercise the exactly recoverable diagonal case, so if real compliance is non-diagonal, offset, or too noisy, the bound need not hold.

Editorial extensions

If this is right

Controllers can steer networks of agents that statistically disobey setpoints, provided they can measure what was actually implemented.
The mean-square bound quantifies the step-size trade-off: smaller alpha reduces noise amplification but increases tracking lag, with the cross term capturing the interaction.
When the agents' randomness is time-invariant and the problem is static, the error vanishes and deterministic OFO behavior is recovered.
When the full-rank assumption (A5) fails, regularization (7)-(8) restores an analogous bound with eta-dependent constants.
In real-time optimal power flow simulations, the stochastic algorithm keeps voltages inside a desired band more consistently than the deterministic baseline across several compliance distributions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The general theorem is only as usable as assumption (A4): outside the diagonal exact-recovery case, a practical implementation would need some other estimator for the composite gradient, a direction the paper does not develop.
The bound offers an implicit recipe for choosing the step-size alpha by minimizing the right-hand side with estimated epsilon terms; the paper does not propose such a scheduler.
The i.i.d. assumption on Phi_n could likely be relaxed to Markovian or correlated randomness, but the proof's martingale-difference structure would have to be reworked.
In power systems, the stochastic formulation naturally models solar intermittency and partial inverter response; the voltage-violation reduction seen in the 33-node test suggests expectation-aware setpoints are less fragile than deterministic ones, though only one feeder and linearized Jacobians were tested.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper does something real: it extends online feedback optimization to agents whose implemented inputs are random linear functions of the setpoint, and it gives a finite-time mean-square tracking bound under explicit assumptions. That model is a natural gap in the OFO literature, and the derivation of the finite-time bound (Theorem 2.2) is structurally sound: the lemmas build up to a contraction plus a noise/drift term, and the recursion is standard.

The soft spot is the advertised asymptotic bound. Theorem 2.3 states equation (4), with epsilon_c = gamma_bar^2/mu_f, a constant that does not vanish as alpha -> 0. But in the appendix, after the geometric-series step, the proof obtains a limsup bound with a term gamma_bar^2/(mu_f alpha). For fixed gamma_bar > 0 this diverges as alpha -> 0, while (4) stays constant. So the claimed tracking bound is not a consequence of the proof. The static case gamma_bar = 0 is fine, and the finite-time bound is untouched. But the headline qualitative claim—that you can track a moving minimizer with O(1) error as the step-size vanishes—is not established. The tracking-lag term changes the trade-off.

A second weakness is Assumption (A4): the general theorem assumes the random compliance matrix A_{n+1} can be recovered from noisy input/output observations. That's a strong premise; for general non-diagonal A or non-zero offset b it won't hold. The paper acknowledges this and only demonstrates exact recovery for the diagonal case, which is fine, but it means the general theory is conditional on a recoverability premise that is likely to be the deciding factor in practice.

The experiments show the algorithm works on toy problems and on a realistic OPF test case, but the plots have no error bars and the scaling check is purely qualitative. Minor.

Bottom line: the problem formulation and the finite-time analysis are worth a serious referee. But the stated main theorem is wrong as written, and the paper should not be accepted without fixing that. If the authors correct the bound to include the 1/alpha term, the contribution is useful to the OFO and networked-optimization community. I would not cite the headline bound in its current form, but I would send the paper to review with a request for revision.

Referee Report

3 major / 3 minor

Summary. The paper extends online feedback optimization (OFO) to a stochastic setting in which agents are non-compliant: the actual control input is a random affine function of the setpoint. The algorithm is an approximate projected stochastic gradient rule (Eq. (3)) driven by the measurement-based gradient surrogate (5). Under assumptions (A1)-(A5), the authors derive a finite-time mean-square tracking bound (Theorem 2.2), an asymptotic bound (Eq. (4), Theorem 2.3), and a regularized variant (Corollary 2.4). Numerical experiments include a toy tracking problem, a static step-size scaling study, and a real-time optimal power flow case study on the IEEE 33-node feeder.

Significance. If Theorem 2.3 were correct, the paper would make a useful contribution: it would bring stochastic non-compliance into the OFO framework, recover standard SGD rates in the static case, and provide a principled way to track a time-varying minimizer from noisy input-output measurements. The paper is also clearly written and the static-case reduction to known SGD bounds is plausible. However, the advertised asymptotic tracking bound does not follow from the proof, and a second gap affects the projection step for time-varying constraint sets. Because these issues undermine the central claim, the significance of the paper as it stands is considerably reduced.

major comments (3)

[Appendix, Proof of Theorem 2.3; Eq. (4)] The advertised bound (4) does not follow from the proof. After the geometric-series step, the proof obtains limsup_{N->∞} E||u_N-u*_N||^2 <= (1/(mu_f alpha))[q_alpha + gamma_bar^2] + ... . With q_alpha = b_{A.5}[alpha^2(xi+sqrt(xi))+alpha epsilon_m], the term gamma_bar^2/(mu_f alpha) remains. The theorem defines epsilon_c := gamma_bar^2/mu_f and claims (4), in which epsilon_c appears without a factor 1/alpha. For fixed gamma_bar>0, the proof's bound diverges as alpha->0, whereas (4) tends to epsilon_c. Thus the central claim that a time-varying minimizer can be tracked with O(1) steady-state error as alpha->0 is not established; the correct bound contains an O(1/alpha) tracking-lag term. The static case gamma_bar=0 survives, but the advertised time-varying guarantee does not. Corollary 2.4 inherits the same issue.
[Appendix, Lemma A.5; Theorem 2.2] The proof of Lemma A.5 uses the inequality ||Proj_{U^{(n)}}{u_{n-1}-alpha grad Gamma} - beta_{n-1}|| <= ||u_{n-1}-alpha grad Gamma - beta_{n-1}||, with beta_{n-1}=u*_{n-1}-alpha E[grad]. This inequality is valid only if beta_{n-1} is in U^{(n)} (or if one projects beta_{n-1} as well). The assumptions only give u*_{n-1} in U^{(n-1)}; no assumption ensures u*_{n-1} or beta_{n-1} belongs to U^{(n)}. The statement "by linearity of the projection operator" is not valid because projection is nonlinear. Without an additional assumption such as nested feasible sets or feasibility of all previous minimizers, the finite-time bound (6) is not established, even in the static case if U^{(n)} varies.
[Section 2.1, Assumption (A4)] Assumption (A4) postulates that the random compliance matrix A_{n+1} can be recovered from (x_hat_n, u_n) with L4 error epsilon_m. For the general affine model x_n = A_{n+1}u_n + b_{n+1}, this is not generally possible: even with exact x_n and known u_n, one d-dimensional observation cannot identify the d^2 entries of A_{n+1} (and b_{n+1}) when the matrix is non-diagonal or the offset is nonzero; with noisy x_hat_n recovery is even harder. The paper only demonstrates the diagonal exact-recovery case (x_hat_n=x_n, b=0, A=diag(Phi)), and the experiments use exactly this structure. Since (A4) is the bridge between measurements and the gradient surrogate (5), the claim of a general network result is overstated; the theorem should either be restricted to a class where recoverability is guaranteed or accompanied by concrete sufficient conditions.

minor comments (3)

[Section 2.2, Eq. (9)] Equation (9) has an unmatched bracket/parenthesis: the right-hand side reads ||A^T_{n+1}C^T[\nabla\tilde g_y^{(n)}(y_n)||, which should be ||A^T_{n+1}C^T[\nabla g_y^{(n)}(y_n)-\nabla g_y^{(n)}(y*_n)]|| or similar.
[Appendix, Lemma A.5] The notation H_n in the proof of Lemma A.5 is written with two terms of order alpha^2 and one term of order alpha; the sign structure is not transparent. Please expand the expansion of ||E_n - alpha M_n||^2 explicitly to show how H_n is obtained.
[Section 3.2] In the OPF experiments, the feasible sets U_i^{(n)} depend on the time-varying available power \bar{P}_i^n. Given the projection issue raised in the major comments, it would be helpful to state whether u*_{n-1} remains feasible for U^{(n)} in these experiments, or to report the projection error separately.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the mean-square tracking bound is derived from stated assumptions; prior self-citations are not load-bearing.

full rationale

The paper's central result is a conditional MSE bound for a projection-based stochastic gradient algorithm using gradient surrogate (5). The derivation chain is self-contained: Lemma 2.1 establishes Lipschitz, strong-monotonicity, and variance properties from (A1)-(A3); Lemma A.2 converts the measurement/recovery error in (A4) into a gradient-estimation bias; Lemmas A.3-A.6 assemble the recursion; Theorem 2.2 sums the recursion; Theorem 2.3 applies a geometric-series bound. None of these steps defines the output in terms of the input or fits a parameter to the predicted quantity. Assumption (A4) explicitly assumes recoverability of A_{n+1} with precision epsilon_m; that is a stated modeling assumption rather than a hidden reuse of the conclusion, and the experiments use the diagonal exact-recovery case explicitly called out in the paper. The citations to [5,8] provide the OFO algorithm template for the compliant case, but the non-compliance extension and its error bounds are derived here rather than imported. One non-circular correctness concern is that the Appendix's final limsup inequality contains a gamma_bar^2/(mu_f alpha) term that does not appear in the claimed bound (4); that is an internal-consistency issue, not a circularity.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities, forces, or dimensions. It postulates a random linear compliance model x_n = A_{n+1}u_n + b_{n+1}, which is a modeling assumption rather than an invented entity. The central results rest on the five assumptions (A1)-(A5); the most non-standard are (A4), recoverability of the compliance matrix from measurements, and (A5), full column rank, which the authors relax using regularization.

free parameters (4)

step-size alpha = 2e-3 (toy), 5e-2 (power)
Chosen by hand in experiments; the theory only constrains alpha < mu_f/(2L_f^2) or alpha < eta/(2(L_f+eta)^2), and the MSE bound scales with alpha.
regularization eta = 1e-3
Chosen by hand in the power-system experiments to satisfy the step-size condition for the regularized problem (7)-(8).
objective weights kappa_P, kappa_Q, kappa_y = 4, 1, 8
Selected by hand in the power experiments; no sensitivity analysis is reported.
proof constants b_{A.5}, b_{2.3}
Introduced as unspecified constants in the MSE bounds; without numerical values, the bound (4) cannot be evaluated quantitatively, only structurally.

assumptions (6)

domain assumption (A1) g_x^(n), g_y^(n) strongly convex with parameter mu and Lipschitz gradients with parameter L_g
Stated in Section 2.1; used for Lipschitz gradient in quadratic mean (Lemma 2.1(i)), strong convexity (Lemma 2.1(ii)), and noise bounds.
domain assumption (A2) Feasible sets U^(n) convex, compact, uniformly bounded
Stated in Section 2.1; used to bound iterates and cross terms (Lemma A.4).
domain assumption (A3) Phi_n i.i.d. with bounded 4th moments and bounded disturbance r_n
Stated in Section 2.1; makes the gradient noise a martingale difference sequence and supplies the L4 moment bounds needed for multiplicative noise.
domain assumption (A4) Measurement errors bounded in L4 and A_{n+1} recoverable from (x_hat_n, u_n) with precision epsilon_m
Stated in Section 2.1; this is the key enabling assumption for building the gradient surrogate (5) from measurements.
domain assumption (A5) C*A_bar and A_bar have full column rank
Stated in Section 2.1; needed for strong monotonicity of the gradient in conditional mean (Lemma 2.1(ii)); the authors relax it via regularization in Corollary 2.4.
standard math Projection non-expansiveness, Jensen's inequality, Cauchy-Schwarz, and dominated convergence for exchanging expectation and differentiation
Used throughout the appendix proofs (Lemmas A.2-A.6 and Corollary A.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Stochastic Online Feedback Optimization for Networks of Non-Compliant Agents." pith.science (2026). https://pith.science/paper/7OQZSJAT

@misc{pith2026250821414,
  author       = {Pith},
  title        = {Pith review of: Stochastic Online Feedback Optimization for Networks of Non-Compliant Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7OQZSJAT}},
  note         = {Machine review of arXiv:2508.21414}
}

abstract

In several applications of online optimization to networked systems such as power grids and robotic networks, information about the system model and its disturbances is not generally available. Within the optimization community, increasing interest has been devoted to the framework of online feedback optimization (OFO), which aims to address these challenges by leveraging real-time input-output measurements to empower online optimization. We extend the OFO framework to a stochastic setting, allowing the subsystems comprising the network (the $\textit{agents}$) to be $\textit{non-compliant}$. This means that the actual control input implemented by the agents is a random variable depending upon the control setpoint generated by the OFO algorithm. Mean-square error bounds are obtained for the general algorithm and the theory is illustrated in application to power systems.

Figures

Figures reproduced from arXiv: 2508.21414 by the authors.

**Figure 1.** (a) Tracking of a moving optimizer; (b) Steady state MSE for a static optimization problem. (i) Then, ∇uf (n) is Lipschitz continuous in quadratic mean in its first variable: there is Lf < ∞ such that for each n, E[∥∇u ˜f (n) (un, Φn+1)∥ 2 | Fn] ≤ L 2 f ∥u˜n∥ 2 (ii) If in addition (A5) holds, ∇uf (n) is strongly monotone in its first variable in conditional mean: there is a constant µf > 0 such that for each n, E[∇u… view at source ↗

**Figure 2.** Schematic for the 33-node test network [1]. Boxed nodes represent the controllable PV units and node 1 is the feeder head. The remaining nodes are uncontrollable loads. Setup: We consider the IEEE 33-node test feeder [1], in which the controllable nodes (agents) were populated with photovoltaic (PV) systems (see [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Voltage magnitude profiles for the 4th agent (node 29). Tab. 1 displays results from several experiments with different choices of {Φn}. It is possible to see more power curtailing resulting from the D-OFO algorithm when there is more uncertainty at the controllable nodes. That is, when there is a larger presence of unexpected loads contributing to the power injections at these nodes. Irrespective of the choice of {… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Online Feedback Optimization for Constrained Stochastic Problems with Decision-Dependent Distributions: Extended Version
math.OC 2026-06 unverdicted novelty 6.0 of 10

Develops projected primal-dual OFO algorithm for decision-dependent stochastic optimization and bounds mean-square tracking error with four interpretable terms.

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.