{"id":"85e5599a-ba8b-4cbc-b77f-65222d51ca2a","arxiv_id":"2502.01014","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"R-AdaZO changes the second-moment update of adaptive zeroth-order optimization to use the smoothed first moment instead of the raw gradient estimate, with a new variance-aware convergence analysis and faster empirical convergence.","lead":"R-AdaZO is a one-line modification of the zeroth-order optimizer ZO-AdaMM: the second moment, which controls per-parameter step sizes, is built from the smoothed first moment instead of the raw noisy gradient estimate. The paper provides a variance-aware convergence analysis and reports faster convergence in synthetic optimization, black-box adversarial attacks, and memory-efficient fine-tuning of OPT models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The main proof relies on the false independence claim in Thm 5.3 (Appx A.3, eq. (34)(b)); the cross term between m_t−E[m_t] and E[m_t]−∇F_μ(θ_{t−1}) does not vanish, so the V bound driving the claimed speedup is not established.","rationale":"The reader's weakest assumption is exactly the right one. I checked whether the step could be rescued by martingale structure: the variance-only part (31) is fine because the {ξ_τ} are martingale differences after conditioning on θ_{τ−1}; the wording 'independence' is wrong but those crossings do vanish. The problematic crossing is (34)(b), where the estimator noise is not the only source of randomness: B depends on θ_{t−1}, hence on all previous ξ_τ. Writing A = m_t − E[m_t], B = E[m_t] − ∇F_μ(θ_{t−1}), and δ_t = m_t − ∇F_μ(θ_{t−1}), one obtains E[AB] = −Var(∇F_μ(θ_{t−1})) − Cov(δ_t, ∇F_μ(θ_{t−1})), which is not identically zero and can have either sign; no sign or magnitude control is given. The exact recursion shows the missing covariance is coupled to the update length and to previous δ, so it is not a higher-order term. Because V in (22) is defined from Thm 5.3 and drives both Lemma 5.7 and the final Thm 5.9/5.10 comparison, this is the load-bearing point. I do not claim the theorem is false: for constant gradients the variance coefficient (1−β1)/(1+β1) is exactly correct, and a more careful recursive proof may recover the same constant. But as written, the central formal advantage is unsupported. The empirical results are extensive and the algorithm is a one-line change to an existing method; they do not compensate for the formal gap in a theory paper. CONDITIONAL remains the right verdict, so I would not move the reader's recommendation.","tokens_in":28357,"tokens_out":11954,"duration_ms":111184,"concrete_test":"Independently re-derive Thm 5.3 starting from the exact recursion δ_t = β1 δ_{t−1} + β1(g_{t−1}−g_t) + (1−β1)ε_t, with ε_t the martingale-difference noise in the ZO gradient. Do not use (34)(b); instead keep the term 2β1² E[δ_{t−1}·(g_{t−1}−g_t)] and bound it via Lemma 5.2 and the update length. If the resulting coefficient of Σ² in the stationary variance exceeds (1−β1)/(1+β1), or if an extra covariance term of order L η Σ survives, then the V bound in (22) and the claimed speedup in Thm 5.9 need revision. A complementary numerical check: simulate a scalar quadratic with β1=0.9, β2=0.99, η=1e−3, Gaussian noise, average E|m_t−∇F_μ(θ_{t−1})|² over 1e5 runs, and compare with (17); an RHS violation at any t would falsify the theorem as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorem 5.9's advertised advantage over ZO-AdaMM is the factor V² = ‖v0‖ + (1−β1)Σ² + bias versus V̂² = ‖v0‖ + (1+β1)Σ². This factor is inherited from Thm 5.3 via Thm 5.4 and Lemma 5.7. The proof of Thm 5.3 in Appx A.3 decomposes δ_t = m_t − ∇F_μ(θ_{t−1}) as (m_t − E[m_t]) + (E[m_t] − ∇F_μ(θ_{t−1})) and asserts in eq. (34)(b) that the cross term vanishes 'with respect to {ξ_τ}'. This assertion is false. Writing g_t = ∇F_μ(θ_{t−1}) and δ_t = m_t − g_t, one has A = (g_t−E g_t)+(δ_t−E δ_t) and B = −(g_t−E g_t)+E δ_t, so E[AB] = −Var(g_t) − Cov(g_t, δ_t), which is not zero in general; δ_t depends on the same history as θ_{t−1}. Concretely, δ_t = β1 δ_{t−1} + β1(g_{t−1}−g_t) + (1−β1)ε_t with E[ε_t|F_{t−1}]=0, so Cov(g_t, δ_t) is generically O(Σ²) or O(L η ‖δ‖), not negligible. The conditional-mean centering that does make cross terms vanish is at h_t = β1 m_{t−1} + (1−β1)g_t, not at E[m_t]; using h_t changes the variance coefficient from (1−β1)/(1+β1)Σ² to a recursion involving β1² E‖δ_{t−1}‖² and an uncontrolled cross term 2β1² E[δ_{t−1}·(g_{t−1}−g_t)]. Since no bound on that cross term is supplied, inequality (17) — and hence (19), (22), (23), and the constant-factor speedup in Thm 5.9 — is not established by the given proof.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces R-AdaZO, a modified adaptive zeroth-order optimizer that changes the second-moment update of ZO-AdaMM from using squared noisy gradient estimates to using squared first-moment estimates (Algorithm 2). The authors claim that this one-line change (i) provides the first analysis of variance reduction from the first moment in ZO optimization (Theorem 5.3), (ii) refines the second moment to better approximate a variance-free ideal (Theorem 5.4 vs. Corollary 5.5), and (iii) yields a faster convergence bound for non-convex stochastic ZO problems (Theorem 5.9 vs. Theorem 5.10), with a constant-factor improvement that depends on the variance term. The paper validates the algorithm empirically on synthetic functions, black-box adversarial attacks, and memory-efficient LLM fine-tuning, where R-AdaZO shows consistent speedups over ZO-AdaMM and ZO-RMSProp.","tokens_in":28964,"tokens_out":8897,"duration_ms":81539,"significance":"If the theoretical claims were fully validated, this would be a useful and practical contribution: the algorithm is a simple modification of an existing method, preserves memory and computation costs, and the experiments show substantial empirical gains across diverse tasks. The paper also attempts to provide a variance-aware convergence framework for adaptive ZO methods, which could be of independent interest. However, the formal claims rest on a key proof step that is not justified as written, so the theoretical contribution is currently not established. The empirical results are consistent and encouraging, but the paper's central advertised speedup is unsupported without a corrected proof.","major_comments":[{"comment":"The decomposition in the proof of Theorem 5.3 assumes that E[(m_t,i - E[m_t,i])(E[m_t,i] - ∇F_μ(θ_{t-1}))] = 0 'with respect to {ξ_τ}'. This is not true. The term m_t,i - E[m_t,i] is a weighted sum of centered gradient estimators (1-β1)Σ_{τ=1}^t β1^{t-τ}(∇̂_i f(θ_{τ-1},ξ_τ) - ∇F_μ(θ_{τ-1})), while E[m_t,i] - ∇F_μ(θ_{t-1}) depends on the same history through θ_{τ-1} for τ ≤ t. Although E[∇̂_i f(θ_{τ-1},ξ_τ) - ∇F_μ(θ_{τ-1}) | θ_{τ-1}] = 0, the past centered terms are not conditionally mean-zero given the future bias term; they are only mean-zero conditional on the filtration up to their own time. Since θ_{t-1} and the bias term are functions of all earlier ξ's, the cross term is generically nonzero. Concretely, writing δ_t = m_t - ∇F_μ(θ_{t-1}), the cross term contains contributions of order Σ² and LηE[||δ||], as the skeptic's derivation shows. Because inequality (17) is the foundation for (19), (22), and Lemma 5.7, the constant-factor speedup claimed in Theorem 5.9 is not established by the given proof. The authors either need to bound the cross term rigorously or restructure the argument with a different centering that makes the cross term vanish by construction, with explicit control of the resulting additional terms.","section":"Appendix A.3, Eq. (34)(b)"},{"comment":"The same unjustified independence assumption is reused when bounding E[√(β2||v_t|| + ζ)] in Lemma 5.7. In the expansion leading to (40), the cross term 2(m_t,i - E[m_t,i])(E[m_t,i] - ∇F_μ(θ_{t-1})) is retained, but in taking expectations and moving to (41) it is effectively discarded without a bound. Since Lemma 5.7 provides the bound on Term A in Lemma 5.6, and Theorem 5.9 directly invokes Lemma 5.7, the final convergence result inherits the unproven cross-term cancellation. A valid proof must either show this term is nonnegative or provide an explicit upper bound that is then propagated through the square-root step. As written, the bound on V in (22) may be an underestimate, and the advertised advantage over ZO-AdaMM may disappear once the cross term is accounted for.","section":"Appendix A.7, Eqs. (40)-(41)"},{"comment":"The paper claims that because the upper bound in (19) has a smaller variance term than the upper bound in (20), the refined second moment estimate 'better approaches' the variance-free ideal (18). An upper bound on E[v_t,i] does not by itself imply that the actual sequence v_t,i is closer to the ideal; a tighter upper bound could be an artifact of a looser analysis or of the bias term in orange. To support the claimed geometric interpretation, the authors should compare the estimators directly (e.g., via a bound on E|v_t,i - ideal_{t,i}| or a high-probability statement) rather than comparing upper bounds on their expectations. This issue is interpretive rather than load-bearing for Theorem 5.9, but it is used to motivate the algorithm and should be corrected.","section":"Section 5.2, Remark after Corollary 5.5 and Theorem 5.4"}],"minor_comments":[{"comment":"The phrase 'the first analysis to the variance reduction' should read 'the first analysis of the variance reduction.' Similar grammatical issues appear in the contribution list, e.g., 'the improved second moment estimates with a more accurate approximation of its variance-free ideal' would be clearer as 'an improved second-moment estimate that more accurately approximates its variance-free ideal.'","section":"Abstract and Section 1"},{"comment":"The informal statement of Theorem 5.8 lists conditions 'β1 ≤ √β2, β2 ≥ 1/2, m0,i = 0, v0,i > 0' but does not mention the additional condition '2β2 ≥ 1' used in the proof (e.g., in bounding term 3 and in (57)). The formal conditions should be aligned with those actually required by the proof.","section":"Theorem 5.8 statement and Appendix A.8"},{"comment":"The notation 'ϵ²/3' is ambiguous: it could mean ε²/3 or ε^{2/3}. The proof later states that the choices are of order O(ε²), so presumably the intended meaning is ε²/3 (i.e., ε² divided by 3). Please clarify the notation to avoid confusion.","section":"Appendix A.8, equations (58) and (72)"},{"comment":"The claim that 'we employ the same initialization and hyperparameters: β1 = 0.9, β2 = 0.99 and K = 10, η = 0.001, µ = 0.005, for all methods' is not meaningful for ZO-SGD, which has no β1 or β2. Please specify which hyperparameters are shared and which are method-specific, or state that all methods use the same function-evaluation budget and step-size schedule where applicable.","section":"Section 6.1, first paragraph"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a simple and empirically effective modification of ZO-AdaMM, and the experiments are extensive. However, the central theoretical claim of a constant-factor speedup rests on an invalid independence assumption in the proof of Theorem 5.3 and its reuse in Lemma 5.7. This is a load-bearing technical error, not a mere presentation issue. I do not think the error is necessarily unfixable: the authors might salvage the theorem by adding a bound on the cross term, or by modifying the algorithm/analysis so that a valid bias-variance decomposition holds. But as written, the manuscript's main formal contribution is unsupported. I therefore recommend major revision rather than rejection, provided the authors can either repair the proof or appropriately downgrade the theoretical claims to match what can be rigorously shown. The empirical results themselves appear consistent and could support a more modest claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the algorithm is a one-line modification of ZO-AdaMM — use m_t^2 instead of g_t^2 in the second moment — and the empirical gains look real: roughly 4–5x fewer iterations in black-box attacks and LLM fine-tuning. The theory, though, does not hold up. The main proof step in Theorem 5.3 (Appendix A.3, equation 34(b)) assumes independence between m_t − E[m_t] and E[m_t] − ∇F_μ(θ_{t−1}). That is not true: both depend on the same trajectory, and the cross term is generally O(Σ^2). I checked the calculus in the stress-test note and it lands. Because Theorem 5.9 inherits the variance coefficient from this step, the advertised constant-factor speedup over ZO-AdaMM is not established.\n\nWhat is genuinely new: the bias-variance decomposition for the first moment in ZO adaptive methods, and the variance-aware convergence framework. The variance reduction insight itself has precursors in prior-guided ZO estimation, but this paper is the first to give it a formal decomposition for the m_t update. The empirical work is solid and consistent: the speedup shows up across synthetic functions, MNIST black-box attack, and OPT fine-tuning. I give credit for the simplicity and for shipping clean experiments.\n\nSoft spots, in order: (1) The independence assumption is load-bearing. If it cannot be repaired, the theorem becomes a conjecture. The authors could try centering at the conditional mean h_t = β1 m_{t−1} + (1−β1)∇F_μ(θ_{t−1}), but as the stress-test notes, that changes the variance coefficient and introduces uncontrolled cross terms. (2) The claim that m_t^2 better approximates the variance-free ideal is based only on upper bounds; that is not a two-sided error argument, so \"more accurate approximation\" is an overstatement. (3) Minor: the LLM results report training loss only, not downstream accuracy, and the adversarial attack setup is a single image (or at least not clearly more). These are minor.\n\nWho is this for? Anyone working on zeroth-order adaptive optimizers will want to know about the algorithm and the empirical result. The theoretical framework is worth reading with a skeptical eye. The paper deserves a serious referee: it would be a conditional accept with a request to fix the proof or soften the claims. If the proof cannot be fixed, the paper should be clear that the advantage is empirical.\n\nRecommendation: send to peer review; require the authors to address the independence step before publication.","headline":"The algorithm is a simple one-line variant of ZO-AdaMM with consistent empirical gains, but the main theoretical speedup is not established because the proof of Theorem 5.3 relies on a false independence assumption.","tokens_in":29431,"tokens_out":5979,"would_cite":false,"duration_ms":52166,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C56","90C15","68W40"],"pacs":[],"model":"deepseek-v4-flash","headline":"R-AdaZO shows that using the momentum, not the raw gradient, in the second-moment update provably shrinks the convergence bound for adaptive zeroth-order optimization.","keywords":["zeroth-order optimization","adaptive optimization","momentum","variance reduction","non-convex optimization","black-box adversarial attack","large language model fine-tuning","Adam"],"falsifier":"A Monte Carlo test can settle the proof's key step: run R-AdaZO on a random quadratic with $d=10^4$, $K=10$, $\\beta_1=0.9$, $\\beta_2=0.99$, and estimate the cross-covariance between $m_t-\\mathbb{E}[m_t]$ and $\\mathbb{E}[m_t]-\\nabla F_\\mu(\\theta_{t-1})$ over many independent seeds at early iterations $t$; if the cross-covariance is non-negligible, the independence assumption behind equation (34)(b) is violated and the advertised $V$ bound is not supported by the proof as written.","tokens_in":28178,"feed_emoji":"⚡","tokens_out":13513,"duration_ms":115409,"temperature":0.7,"pith_summary":"R-AdaZO argues that existing adaptive zeroth-order optimizers such as ZO-AdaMM underuse momentum, and that a one-line change—updating the second moment with the squared momentum $m_t^2$ instead of the squared noisy gradient $g_t^2$—makes the same algorithm converge provably faster. The paper establishes a variance-bias decomposition showing that the first moment reduces the variance of zeroth-order gradient estimates by a factor of $(1-\\beta_1)/(1+\\beta_1)$, and that the refined second moment more closely approximates the variance-free ideal. Its main convergence theorem gives an explicit averaged-gradient-norm bound for non-convex smooth objectives and locates the speedup in a smaller variance contribution, replacing $(1+\\beta_1)\\Sigma^2$ with $(1-\\beta_1)\\Sigma^2$ up to a small bias term. This matters because zeroth-order optimization is the workhorse for black-box attacks and memory-efficient large-language-model fine-tuning, and the fix costs no extra function evaluations or memory.","feed_headline":"Squaring momentum, not gradients, speeds up zeroth-order optimization","feed_subtitle":"A one-line change to the second-moment update provably shrinks the variance term in the bound.","key_machinery":"The load-bearing object is the refined second-moment update $v_t=\\beta_2 v_{t-1}+(1-\\beta_2)m_t^2$, where $m_t$ is the exponential moving average of spherical finite-difference gradient estimates $\\hat{\\nabla}f(\\theta,\\xi)=\\frac{d}{K}\\sum_{k=1}^{K}\\frac{f(\\theta+\\mu u_k;\\xi)-f(\\theta;\\xi)}{\\mu}u_k$. The proof operates on the randomized smoothing $F_\\mu(\\theta)=\\mathbb{E}_{u\\sim \\mathbb{B}^d}[F(\\theta+\\mu u)]$, for which this estimator is unbiased, and uses Hölder's inequality to split the averaged gradient norm into a second-moment component and a normalized-gradient component. Theorem 5.3's variance-bias decomposition is the step that lets the squared momentum inherit variance reduction: it bounds $\\mathbb{E}|m_{t,i}-\\nabla_i F_\\mu(\\theta_{t-1})|^2$ by a variance term $\\frac{1-\\beta_1}{1+\\beta_1}\\Sigma^2$ plus a bias term driven by $\\eta^2$. The comparison between $V$ and $\\hat V$ then transfers that reduction into the convergence bound.","core_discovery":"On the paper's own terms, the central formal claim is Theorem 5.9. Under bounded function values and coordinate-wise Lipschitz smoothness, with $\\beta_1 \\le \\sqrt{\\beta_2}$, $\\beta_2 \\ge 1/2$, $1-\\beta_2 \\sim O(\\epsilon^2)$, $\\eta \\sim O(\\epsilon^2)$, and $T \\sim O(\\epsilon^{-4})$, R-AdaZO guarantees $$\\frac{1}{T}\\sum_{t=0}^{T-1}\\mathbb{E}[\\|\\nabla F(\\theta_t)\\|] \\le \\frac{(1+\\beta_1)\\sqrt{d}}{\\sqrt{\\beta_1(1-\\beta_2)}}\\$epsilon^{2}$ + \\left(4\\sqrt{\\zeta}+\\sqrt{V d}\\right)\\epsilon + \\mu L\\sqrt{d},$$ where $V^2=\\|v_0\\|+(1-\\beta_1)\\Sigma^2+\\frac{\\beta_1(1+\\beta_1)^2 L^2\\eta^2 d}{(1-\\beta_1)^2(1-\\beta_2)}$ and $\\Sigma^2=8(\\sigma^2+C^2)d/(K\\mu^2)$. The analogous guarantee for ZO-AdaMM has $\\hat{V}^2=\\|v_0\\|+(1+\\beta_1)\\Sigma^2$. Comparing the two bounds is the paper's core argument: the variance contribution of the noisy gradient estimator is reduced by the factor $(1-\\beta_1)/(1+\\beta_1)$, and this is what makes R-AdaZO converge faster.","pith_inferences":["One extension the paper leaves implicit is that the same squared-moment substitution can be applied to other Adam-style optimizers ported to the zeroth-order setting, where the identical variance-reduction mechanism should apply.","The framework separates the estimator variance $\\Sigma^2$ from the update rule, so combining R-AdaZO with existing lower-variance gradient estimators (for example, surrogate or prior-guided estimates) should further shrink the bound without changing the algorithm.","A practical tuning consequence not pursued by the paper is that a schedule for $\\beta_1$ could exploit the variance reduction early while limiting the bias late, since Theorem 5.3 exposes exactly that trade-off.","If the comparison is tight, R-AdaZO is effectively a free upgrade for deployed ZO pipelines: one changed line, unchanged per-iteration cost, and a smaller theoretical error bound."],"forward_implications":["The paper supplies the first variance-aware convergence framework for adaptive ZO methods; Lemma 5.6 and Lemmas 5.7–5.8 can be reused to analyze other ZO updates, not just R-AdaZO and ZO-AdaMM.","The variance term in the convergence bound drops from $(1+\\beta_1)\\Sigma^2$ to $(1-\\beta_1)\\Sigma^2$ (plus a small bias term), which is the formal source of the claimed speedup in noise-dominated regimes.","The refined second moment in R-AdaZO is provably closer to the variance-free ideal than the ZO-AdaMM second moment, so the per-coordinate step scalings better track the geometry of the smoothed objective.","The same computational cost of $O(Kd)$ per iteration and $O(d)$ memory as ZO-AdaMM is preserved, so the theoretical improvement does not trade away resources.","Experiments report a $5.4\\times$ reduction in iterations for a black-box adversarial attack and $3.75$–$4.29\\times$ speedups for LLM fine-tuning, consistent with the constant-factor improvement."],"supporting_citations":[{"why":"Defines the ZO-AdaMM algorithm that R-AdaZO modifies and the baseline to beat.","marker":"(Chen et al., 2019)"},{"why":"Provides the other adaptive ZO baseline with the same ZO-AdaMM-style update.","marker":"(Nazari et al., 2020)"},{"why":"Supplies the randomized-smoothing function $F_\\mu$ and the bias/decomposition lemmas used throughout the proof.","marker":"(Duchi et al., 2012)"},{"why":"Gives the gradient-estimator identity used to prove Lemma 5.1 and the variance bound in Lemma 5.2.","marker":"(Flaxman et al., 2004)"},{"why":"Contributes the Adam proof techniques that Theorem 5.8 adapts, including the potential function and Lemmas A.1–A.2.","marker":"(Wang et al., 2024a)"},{"why":"Provides the RMSProp/Adam convergence framework that motivates the Hölder decomposition and the variance-aware analysis.","marker":"(Zhang et al., 2024a)"},{"why":"Is the prior result on momentum's variance reduction in first-order optimization that this paper extends to the zeroth-order setting.","marker":"(Liu et al., 2020)"},{"why":"Provides the Adam update structure whose moment estimates R-AdaZO reuses.","marker":"(Kingma & Ba, 2015)"}],"fun_headline_variants":["Refined ZO optimizer cuts variance for faster convergence","Moment variance reduction speeds up zeroth-order optimization","Refined moments cut ZO gradient noise for faster convergence","R-AdaZO: variance-reduced moments for faster ZO"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof of Theorem 5.3 assumes that the fluctuation $m_t-\\mathbb{E}[m_t]$ is independent of the bias $\\mathbb{E}[m_t]-\\nabla F_\\mu(\\theta_{t-1})$, even though both are constructed from the same sequence of random perturbations and function evaluations; if that independence fails, the variance reduction at the heart of the speedup is not established.","fun_headline_variants_meta":{"raw":{"variants":["Refined ZO optimizer cuts variance for faster convergence","Moment variance reduction speeds up zeroth-order optimization","Refined moments cut ZO gradient noise for faster convergence","R-AdaZO: variance-reduced moments for faster ZO"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001083,"raw_usage":{"total_tokens":4634,"prompt_tokens":1159,"completion_tokens":3475,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":775,"completion_tokens_details":{"reasoning_tokens":3408}},"tokens_in":775,"tokens_out":3475,"duration_ms":26232,"temperature":1.0,"reasoning_tokens":3408,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T16:53:21.498171+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A Monte Carlo test can settle the proof's key step: run R-AdaZO on a random quadratic with $d=10^4$, $K=10$, $\\beta_1=0.9$, $\\beta_2=0.99$, and estimate the cross-covariance between $m_t-\\mathbb{E}[m_t]$ and $\\mathbb{E}[m_t]-\\nabla F_\\mu(\\theta_{t-1})$ over many independent seeds at early iterations $t$; if the cross-covariance is non-negligible, the independence assumption behind equation (34)(b) is violated and the advertised $V$ bound is not supported by the proof as written.","supporting_citations":[{"cited_title":"Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization","cited_arxiv_id":null,"evidence_quote":"Defines the ZO-AdaMM algorithm that R-AdaZO modifies and the baseline to beat."},{"cited_title":"C., Bartlett, P","cited_arxiv_id":null,"evidence_quote":"Supplies the randomized-smoothing function $F_\\mu$ and the bias/decomposition lemmas used throughout the proof."},{"cited_title":"An improved analysis of stochastic gradient descent with momentum","cited_arxiv_id":null,"evidence_quote":"Is the prior result on momentum's variance reduction in first-order optimization that this paper extends to the zeroth-order setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Adam update structure whose moment estimates R-AdaZO reuses."}],"review_version":1}