{"id":"790809c5-4446-4678-996d-dac992f92e4f","arxiv_id":"2412.08951","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Stochastic gradient ascent with a diagonal empirical-Fisher per-coordinate scale learns Dirichlet process mixture posteriors on par with batch coordinate ascent on image datasets up to 108K images.","lead":"This paper trains a Dirichlet process mixture model for image clustering with stochastic gradient ascent instead of closed-form variational updates, scaling the step per parameter with an empirical Fisher information. It reports similar or better clustering accuracy than a batch baseline on datasets up to 108,000 images, at lower per-iteration cost.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'Fisher information' in SGA+F is an empirical squared-gradient preconditioner, not a Fisher metric over variational parameters; Eq. 13 therefore does not establish automatic optimal stepsize, and the central contribution reduces to an RMSprop-like heuristic.","rationale":"Good-faith reading: the paper's intended contribution is an SGA learner for DPM that uses Fisher information to set an adaptive step; the strongest claim is that SGA+F matches or beats closed-form MM at lower cost. For that claim to hold, the Fisher-based update must follow from the natural-gradient argument. I checked Eqs. 10–13: the Fisher matrix in Eq. 10 is defined through gradients of ln q(θ) with respect to the hidden variables θ, not with respect to variational parameters. In standard natural-gradient VI, the metric is E_q[∇φ ln q(θ;φ) ∇φ ln q(θ;φ)ᵀ]; this is a different object. For a fixed-variance Gaussian q, Fisher w.r.t the mean is constant, so it cannot produce the adaptive, data-dependent scaling claimed. The paper's F′ (Eq. 11) is an empirical squared-gradient average, which is an RMSprop/Adagrad-style preconditioner. Therefore the theoretical derivation is misapplied and the 'automatic optimal step' claim is not established. This is load-bearing because contribution (ii) and the abstract's central promise depend on it; if the paper were reframed as an empirical adaptive-gradient heuristic with controlled comparisons, the central claim would need to change accordingly. I agree with the reader's weakest_assumption. The concrete test of recomputing the natural gradient for q(μ_k) is decisive and inexpensive. Additional issues—the feature-extractor mismatch in Table 3 and the fixed η=0.1 in Section 5.2—reinforce but are not needed for the primary concern. Given no code and no formal verification, the current manuscript does not support the theoretical or empirical headline.","tokens_in":13769,"tokens_out":6436,"duration_ms":67147,"concrete_test":"Compute the true natural-gradient update for this DPM variational family: set q(μ_k)=N(μ_k; m_k, σ_k²) with fixed σ_k and variational parameter m_k; then the Fisher information w.r.t. m_k is E_q[(∂ ln q/∂m_k)²] = 1/σ_k². Substitute this into Eq. 13 and compare with the SGA+F update of Eqs. 9–11. If the resulting preconditioner differs from the empirical minibatch average in Eq. 11—it will, since Eq. 11 is data-dependent—then Eq. 13 does not justify the SGA+F stepsize, settling the concern. If the two updates coincide, the concern does not land.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim requires that Fθ in Eq. 10 is the Fisher metric of the distribution being optimized, so that Eq. 13 follows from the KL-constrained step in Eq. 12. But θ here are the hidden variables (cluster means μ and stick weights v), and Eq. 9 updates the point estimates E[θ] along ∇θ ln q(θ). The empirical Fθ′ in Eq. 11 is just the minibatch average of squared gradients ∇θ ln q(θ), not E_q[(∇φ ln q(θ;φ))²] with φ the variational parameters. For a Gaussian variational posterior with fixed variance, the true Fisher information w.r.t. its mean is a constant, whereas Eq. 11 is data-dependent; if q is a point-mass MAP estimate, the Fisher metric is not even well-defined. Hence Eqs. 12–13 do not apply, and at best Eq. 9 is a diagonal adaptive-gradient heuristic (Adagrad/RMSprop-like). The abstract's 'automatic optimal stepsize' and the paper's contribution (ii) therefore rest on an unsupported derivation. A separate confound—Table 3 reports SGA+F with VGG16/Place205 while Section 5.1.4 states VGG16/ImageNet—further weakens the empirical comparison, but the Fisher misapplication is primary.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes stochastic gradient ascent (SGA) updates for variational inference in Dirichlet process mixture models, with two variants: SGA+M (momentum) and SGA+F (Fisher-information-scaled adaptive stepsize). The authors claim that SGA learning is faster than closed-form coordinate ascent (the MM baseline) without sacrificing NMI or accuracy, that the Fisher information automatically provides an optimal stepsize, and that the approach scales to large datasets (Caltech256, SUN397) with deep ConvNet features. The paper reports experiments on six object/scene datasets, including model selection, CPU time, NMI, and accuracy, and compares with several Bayesian nonparametric baselines from the literature.","tokens_in":14059,"tokens_out":7578,"duration_ms":73426,"significance":"If the central claims were established, the paper would offer a practical closed-form-free DPM inference algorithm with adaptive stepsize, applicable to high-dimensional features and datasets with up to 397 classes. The authors provide pseudocode and a convergence diagnostic based on the variational lower bound. However, the theoretical justification for the Fisher-information stepsize is not valid as written, and the experimental protocol contains confounds that prevent the reported improvements from being attributed to the proposed method. With the derivation corrected and the method reframed as a heuristic adaptive-gradient preconditioner, an empirical contribution might remain, but the current manuscript does not support the 'automatic optimal stepsize' claim or the comparative performance claims.","major_comments":[{"comment":"The natural-gradient derivation is misapplied. The Fisher information matrix in natural gradient is the expected outer product of the gradient of log q(theta; phi) with respect to the variational parameters phi. Here q is the variational posterior over hidden variables theta = (mu, v), and Eq. (10) defines F_theta as the expectation of squared gradients with respect to theta itself. For a Gaussian q(mu) with fixed variance, the Fisher information with respect to the mean is a constant, whereas Eq. (11) is data-dependent; if q is a point-mass MAP estimate, the Fisher metric is not even well-defined. Consequently, Eq. (13) does not follow from the KL-constrained optimization in Eq. (12), and the abstract's 'automatic optimal stepsize' is not established. At best, Eq. (9) is a diagonal adaptive-gradient preconditioner reminiscent of RMSprop, not a natural-gradient update.","section":"Section 3.3, Eqs. (10)-(13)"},{"comment":"The claim that 'all SGAs outperforms MM on all 5 evaluation criteria' is contradicted by Table 2. For Caltech10, the SGA+M model estimate is 24 versus MM's 13.3 (ground truth 10); for Caltech101, SGA+M is 139.7 versus MM's 126.3; for Scene15, SGA+F is 18 versus MM's 17.7. Thus not even the SGA+F variant is uniformly closer to ground truth across all datasets, and the SGA+M variant is often worse than MM. The summary in the text overstates the empirical evidence.","section":"Section 5.2, Table 2"},{"comment":"The comparison against the MM baseline is not a fair test of closed-form coordinate ascent. For the larger datasets, MM is restricted to a fixed random subset of about 5K images ('there is a large portion of dataset that MM will never see'), while the SGA methods see the entire dataset through random minibatches. The reported CPU-time and accuracy gains therefore conflate algorithmic advantage with access to more data. In addition, the feature extractor is inconsistent: Section 5.1.4 states VGG16 pretrained on ImageNet, while Section 7.2 states that the proposed method uses VGG16 pretrained on Place205. Table 3 also compares methods with different feature types (SIFT, Fisher vector, AlexNet FC7, VGG16), so the NMI/ACC differences cannot be attributed to the DPM learner.","section":"Sections 5.2 and 7.2, Table 3"},{"comment":"The paper's title promises theory, but no convergence theorem is provided for Algorithm 1. Eq. (7) cites a Barzilai-Borwein stepsize for stochastic gradient descent, but that rule is never used in the proposed algorithm. Section 4.3 only checks pointwise concavity of coordinate-wise log-posteriors, which does not establish convergence of the alternating stochastic updates. The paper should either provide a formal convergence analysis or remove the theoretical claim from the title and abstract.","section":"Sections 3.1 and 4.3"}],"minor_comments":[{"comment":"There are numerous typos and grammatical errors, including 'As as intermediate approach', 'do not sacrifice performance for speed', 'motiviation', 'datset', 'trunction level', and 'reflex'. The manuscript needs thorough proofreading.","section":"Abstract and Section 1"},{"comment":"Section 5.2 states that all SGA learners use a fixed learning rate eta = 0.1, while Algorithm 1 line 3(i) gives the initialization range 1e-1 <= eta <= 1e-3. Clarify how eta is chosen and whether any sensitivity analysis was performed.","section":"Section 5.2 and Algorithm 1"},{"comment":"The text references 'Table 6' when discussing LDPO-A-FC, but the manuscript contains only Table 3; correct the cross-reference.","section":"Section 7.5"},{"comment":"The equations for Accuracy and NMI are typeset incorrectly ('Uinf o', 'Hrefers'), making the definitions hard to read; rewrite them cleanly.","section":"Section 5.1.2"},{"comment":"There is a typo 'eqn (121)' in the discussion of Eq. (12), and the statement that F_theta is a scalar under diagonal covariance should be clarified: the diagonal entries of the Fisher matrix are used, not a scalar Fisher information.","section":"Section 3.3"},{"comment":"No error bars or variance estimates are reported for NMI/ACC despite the statement that experiments were averaged over at least 5 runs; adding standard deviations would improve interpretability.","section":"Tables 2 and 3"},{"comment":"The pruning threshold T_HR and the re-ordering of E[v_k] are described heuristically; please specify how T_HR was set for each dataset and whether the re-ordering affects model selection.","section":"Section 4.5"}],"recommendation":"reject","confidential_remarks":"The comparison to MM [28] is natural, but [28] is the first author's earlier work and the paper does not discuss the relationship beyond empirical wins. The main reason for the rejection is technical: the Fisher-information derivation is invalid, and the empirical protocol does not isolate the proposed method's contribution. If the authors revise, they should reframe SGA+F as an RMSprop-like heuristic, provide a proper convergence analysis, and rerun the baselines under matched feature and data-access conditions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is the short version: the method works as a cheap adaptive-gradient heuristic for DPM variational inference, but the Fisher-information framing in Section 3.3 is a mix-up, and the paper should not be published in its current form. That said, the heuristic is new to DPM, the experiments are extensive, and the paper deserves a serious referee, not a desk reject.\n\nWhat is actually new: applying per-coordinate adaptive scaling (momentum, then squared-gradient normalization) to the variational log-posteriors of a Dirichlet process mixture is a reasonable extension of SGA-VI, and the special case of a diagonal empirical Fisher is exactly what makes it cheap. The authors run it on six datasets, including SUN397 at 108K images, and report CPU time, model selection, and convergence curves. That is useful empirical work. The comparison to the MM baseline, despite some loose ends, is honest in spirit – they show that a stochastic learner can match a batch learner at lower cost.\n\nThe problems: the central derivation does not hold. Fθ in Eq. (10) is the empirical second moment of ∇θ ln q(θ), not the Fisher metric of the variational family with respect to its parameters. For a Gaussian q with fixed variance, the Fisher metric w.r.t. the mean is constant; Eq. (11) is data-dependent, so Eqs. (12–13) do not apply. That makes the 'automatic optimal stepsize' claim unsupported. The fixed learning rate η=0.1 contradicts the claim of automatic adaptation. And Table 3's comparisons mix feature extractors (SIFT, Fisher vectors, AlexNet vs. VGG16/Place205), so the headline numbers are not apples-to-apples. The stress-test note is right on both points.\n\nI also note the paper does not include code, which is a reproducibility concern, and there are minor typos throughout. But these are fixable. The heuristic itself is not nonsense; the theory section just overreaches.\n\nMy recommendation: send to peer review with a clear request to (a) remove or completely rewrite the natural-gradient claim, (b) present the method as an adaptive-gradient heuristic, and (c) run matched-feature comparisons. That is a heavy revision, but the empirical kernel is worth the effort. For a reading group, the paper is a good case study in how a plausible theoretical story can be subtly wrong.","headline":"A useful adaptive-gradient heuristic for DPM variational inference, but the Fisher-information justification does not hold up; deserves revision, not desk rejection.","tokens_in":14609,"tokens_out":3524,"would_cite":false,"duration_ms":35269,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Fisher-information step size makes stochastic Bayesian clustering as accurate as closed-form inference.","keywords":["Dirichlet process mixture","stochastic gradient ascent","Fisher information","adaptive stepsize","variational inference","Bayesian nonparametrics","image clustering","model selection"],"falsifier":"Run SGA+F on a synthetic Dirichlet process Gaussian mixture with a known posterior, record the effective stepsize $F_\\theta^{-1}\\eta$ for each coordinate, and compare it with the inverse diagonal of the true posterior covariance. If the ratio between the two varies strongly across coordinates, or diverges when the likelihood is flat, the empirical-Fisher metric is not the natural-gradient metric and the adaptive-step claim is refuted.","tokens_in":13541,"feed_emoji":"📊","tokens_out":6362,"duration_ms":64387,"temperature":0.7,"pith_summary":"This paper tries to establish that a Dirichlet process mixture—a Bayesian clustering model that also estimates the number of clusters—can be learned with stochastic gradient ascent rather than the usual closed-form coordinate ascent, without paying a performance penalty. The proposed update scales the gradient of the variational log-posterior by an empirical Fisher information term, which the authors argue is the correct direction for steepest ascent along the curvature of the posterior and gives an automatic adaptive stepsize. In experiments across six image datasets, the Fisher-scaled learner matches or beats the closed-form maximization–maximization baseline in normalized mutual information and accuracy, estimates a model closer to the ground-truth number of classes, and runs faster; on SUN397, with about 108,000 images and 397 classes, it is about 30% faster than the baseline. If correct, this removes the closed-form bottleneck that has limited scalable variational inference for Bayesian nonparametrics.","feed_headline":"Fisher steps let stochastic gradients match Bayesian clustering","feed_subtitle":"Adaptive Fisher steps keep accuracy on par with closed-form inference while speeding up six image datasets.","key_machinery":"The carrying object is an empirical Fisher-information scaling of the stochastic gradient. For each variational parameter $\\theta$ (cluster mean $\\mu_k$ or stick-breaking weight $v_k$), the update replaces the plain gradient step by one divided by $F'_\\theta = \\frac{1}{M}\\sum_{n=1}^M (\\nabla_\\theta \\ln q(\\theta))^2$, the mean square of the minibatch gradient; with the diagonal-covariance assumption the Fisher term is a scalar per dimension, so the cost is just a per-coordinate division. This scaling does the work of adapting the stepsize to the local curvature: the derivation from Eq. (12)–(13) treats $F_\\theta$ as the metric in a KL-divergence constraint, which is what makes the scaled gradient an approximate natural-gradient step rather than an arbitrary damping factor.","core_discovery":"On the paper's own terms, the central claim is that Fisher information can be injected into stochastic gradient ascent for variational inference in Dirichlet process mixtures, turning a slow constant-stepsize learner into a fast adaptive one. Concretely, the variational expectations for the cluster means and stick-breaking weights are updated by $E[\\theta] = E[\\theta]' + F_\\theta^{-1} \\eta \\nabla_\\theta \\ln q(\\theta)$, where $F_\\theta = E[(\\nabla_\\theta \\ln q(\\theta))^2]$ is the Fisher information of the variational log-posterior, computed empirically over each minibatch with negligible overhead since the gradient is already available. The authors derive this scaling from the KL-divergence constraint that makes $F_\\theta$ the Riemannian metric for steepest ascent, and they position it as the first use of Fisher information in stochastic-gradient-ascent variational inference for a Dirichlet process mixture. Their experiments compare against the closed-form maximization–maximization learner and report on-par or better NMI and accuracy on all six datasets, better model selection on several, and lower CPU time, including on Caltech256 and SUN397 with VGG16 features.","pith_inferences":["An extension the authors do not pursue: the same empirical-Fisher scaling applies to any exponential-family variational posterior with computable gradients, so topic models and other Bayesian nonparametrics are natural next targets.","A test the paper does not run: on a synthetic Gaussian DPM with known posterior covariance, SGA+F's effective per-coordinate stepsize should track the inverse posterior variance; if it does not, the natural-gradient interpretation fails.","Because $F_\\theta$ is recomputed from each random minibatch, the adaptive stepsize inherits minibatch noise; the class-balanced minibatches used in all experiments are probably a silent condition for the reported stability."],"forward_implications":["The closed-form requirement for scalable variational inference in Dirichlet process mixtures is removed: any model where $\\nabla_\\theta \\ln q(\\theta)$ can be computed, even without an analytic expectation, becomes learnable by SGA+F.","Adaptive stepsize via Fisher information should converge faster and with less fluctuation than constant-stepsize stochastic gradient ascent on the same objective, as the convergence plots in the paper indicate.","The method inherits the scalability of minibatch training, so a Dirichlet process mixture can be applied to datasets at the scale of SUN397 (108K images, 397 classes) that batch closed-form learners cannot fully see.","For the six datasets tested, SGA+F should deliver NMI and accuracy at least as good as the closed-form MM baseline while using less CPU time.","High-dimensional deep features (4096-dimensional VGG16) are usable because the diagonal-covariance assumption keeps per-iteration cost linear in the feature dimension."],"supporting_citations":[{"why":"Supplies the Dirichlet process mixture model and the closed-form variational inference baseline that SGA+F must match.","marker":"[7]"},{"why":"The stochastic variational inference baseline; the paper contrasts its fixed diminishing stepsize with the adaptive Fisher stepsize.","marker":"[8]"},{"why":"Establishes stochastic gradient descent/ascent as approximate Bayesian inference and defines the SGA-VI line of work this paper extends.","marker":"[9]"},{"why":"Provides the Barzilai-Borwein stepsize condition used in Section 3.1 to motivate adaptive stepsize in stochastic gradient ascent.","marker":"[16]"},{"why":"The maximization-maximization (MM) closed-form learner that is the main comparison baseline in Experiment 1.","marker":"[28]"},{"why":"Justifies natural gradient learning along the curvature of the variational log-posterior, the motivation for the Fisher scaling.","marker":"[31]"},{"why":"Gives the KL-divergence constraint derivation showing that the optimal update is proportional to $F^{-1}\\nabla$, the basis of Eq. (12)-(13).","marker":"[32]"},{"why":"Supplies the normalized mutual information and accuracy evaluation code used in the experiments.","marker":"[35]"}],"fun_headline_variants":["Adaptive Fisher steps boost speed of Bayesian mixture inference","Fisher information tunes step sizes for fast Dirichlet process learning","Stochastic gradients with Fisher steps match closed-form accuracy","Fisher-scaled step sizes accelerate Bayesian clustering without compromise","Fisher-informed step sizes for efficient Dirichlet mixture variational inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the empirical second moment of the gradient, $F_\\theta = E[(\\nabla_\\theta \\ln q(\\theta))^2]$, is the right curvature metric for choosing the stepsize; if this scalar scaling does not capture the true curvature of the variational posterior, the adaptive-stepsize claim collapses and the method becomes a heuristic preconditioned gradient.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive Fisher steps boost speed of Bayesian mixture inference","Fisher information tunes step sizes for fast Dirichlet process learning","Stochastic gradients with Fisher steps match closed-form accuracy","Fisher-scaled step sizes accelerate Bayesian clustering without compromise","Fisher-informed step sizes for efficient Dirichlet mixture variational inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000749,"raw_usage":{"total_tokens":3356,"prompt_tokens":987,"completion_tokens":2369,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":2290}},"tokens_in":603,"tokens_out":2369,"duration_ms":19955,"temperature":1.0,"reasoning_tokens":2290,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:21:32.575633+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SGA+F on a synthetic Dirichlet process Gaussian mixture with a known posterior, record the effective stepsize $F_\\theta^{-1}\\eta$ for each coordinate, and compare it with the inverse diagonal of the true posterior covariance. If the ratio between the two varies strongly across coordinates, or diverges when the likelihood is flat, the empirical-Fisher metric is not the natural-gradient metric and the adaptive-step claim is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Dirichlet process mixture model and the closed-form variational inference baseline that SGA+F must match."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The stochastic variational inference baseline; the paper contrasts its fixed diminishing stepsize with the adaptive Fisher stepsize."},{"cited_title":"Mandt, M","cited_arxiv_id":null,"evidence_quote":"Establishes stochastic gradient descent/ascent as approximate Bayesian inference and defines the SGA-VI line of work this paper extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Barzilai-Borwein stepsize condition used in Section 3.1 to motivate adaptive stepsize in stochastic gradient ascent."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The maximization-maximization (MM) closed-form learner that is the main comparison baseline in Experiment 1."},{"cited_title":"Honkela, M","cited_arxiv_id":null,"evidence_quote":"Justifies natural gradient learning along the curvature of the variational log-posterior, the motivation for the Fisher scaling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the normalized mutual information and accuracy evaluation code used in the experiments."}],"review_version":1}