REVIEW 4 major objections 4 minor 1 cited by
Deep Clustering via Probabilistic Ratio-Cut Optimization
T0 review · 4 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read A probabilistic relaxation of the graph ratio-cut, optimized by stochastic gradient descent, achieves lower ratio-cut and higher clustering accuracy than spectral relaxation across MNIST, Fashion-MNIST, and CIFAR-10.
desk verdict Novel probabilistic ratio-cut relaxation with a real practical method, but the 'unbiased gradient' claim is unsupported and likely false; revise the theory and tighten the experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the probabilistic assignment matrix $P \in [0,1]^{n \times k}$ with rows on the simplex, interpreted as Bernoulli parameters for each vertex's membership in each cluster; the ratio assignment for a cluster is $f^{(\ell)} = a^{(\ell)}/\sqrt{|\hat C_\ell|}$. The derivation rests on an identity (Lemma 4.1) giving $E[(f_i - f_j)^2] = (p_i + p_j - 2p_i p_j) E[1/(1+|\hat C_{\perp(i,j)}|)]$, and on Lemma 4.2, which converts the reciprocal-size expectation into the integral $\int_0^1 \prod_{m\neq i,j} (1 - p_m t) dt$. Lemma 4.4 bounds that integral by Jensen's inequality and produces the matrix surrogate $L_{rc}(W,P) = \mathrm{Tr}(P^{-1}(1-P)^T W P)$. The final mechanism is an online training loop with a moving average $\bar p_t$ and a stop-gradient operator, which the authors state gives an unbiased gradient of $L_{rc}$, plus a KL-divergence regularizer that prevents all probability mass from collapsing into one cluster.
What would settle it
Take a graph small enough to enumerate every cluster assignment (say $n = 12$), compute the exact expected ratio-cut, run PRCut's moving-average stop-gradient update for a step, and compare the direction it moves to the exact gradient of the true expectation; a mismatch on any instance would refute the unbiased-gradient claim.
Extended reading notes
Core claim
The central claim is that the expected ratio-cut of a random clustering has a tractable upper bound whose minimization is a faithful proxy for solving the original problem. For assignment probabilities $P$, the paper derives $L_{rc}(W,P) = \mathrm{Tr}(P^{-1}(1-P)^T W P)$ as a surrogate objective (Equation 7), obtained by bounding an integral identity for the expected size-normalized cut. PRCut minimizes this surrogate plus a KL regularization term by backpropagating a stop-gradient estimate of $\partial L_{rc}/\partial P$ through a softmax network. The authors report that this online procedure lowers the actual ratio-cut below the spectral relaxation on MNIST, Fashion-MNIST, and CIFAR-10, and that with label-based similarities it clusters as accurately as a supervised MLP classifier.
Load-bearing premise
The central premise is that the moving-average stop-gradient rule yields an unbiased gradient of the surrogate $L_{rc}$, and that minimizing $L_{rc}$ also lowers the true expected ratio-cut; the paper asserts both without proof, and the first is generally false because $1/E[Z]$ is not $E[1/Z]$.
Editorial extensions
If this is right
- PRCut removes the eigendecomposition bottleneck: the entire training loop only needs batches of size $b$ and $O(b^2)$ memory, while still optimizing a ratio-cut objective.
- With true-label similarities, PRCut's clustering accuracy tracks the supervised classifier on MNIST (0.987 vs 0.980), Fashion-MNIST (0.887 vs 0.885), and CIFAR-10 (0.571 vs 0.582), showing high fidelity between clustering and the similarity measure.
- As a drop-in clustering module for self-supervised representations, PRCut evaluates representation quality and beats spectral clustering on all three datasets, and reaches the best reported Fashion-MNIST clustering among the compared deep methods (0.791 ACC).
- Minimizing $L_{rc}$ directly targets the global ratio-cut rather than pairwise-only contrastive objectives, so the clusters reflect global similarity structure rather than local neighborhoods.
- The surrogate objective can be computed for sparse $k$-NN graphs at $O((m/n) k b^2)$ per batch, making deep ratio-cut optimization practical for large graphs.
Reading between the lines
- Because $L_{rc}$ is a smooth function of $P$, it could serve directly as a training signal for the representation encoder itself, turning clustering and representation learning into a single end-to-end objective rather than a two-stage pipeline.
- The unbiased-gradient issue is generic: any objective containing $1/E[Z]$ inherits bias from moving-average denominators, so a proof technique here would transfer to normalized-cut and other size-normalized graph objectives.
- In the balanced-cluster offline setting, the Hessian of $L_{rc}$ is proportional to $-W$, so a Sinkhorn-Knopp-type projection could yield a deterministic offline mode without spectral decomposition, which the authors only mention as future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PRCut, a deep clustering method that models binary cluster assignments as random variables and optimizes an upper bound on the expected ratio-cut. The authors derive an exact integral expression for the expected ratio-cut, an upper bound via a Poisson-binomial argument, and a stochastic optimization procedure for learning the assignment probabilities with a neural network. They report experiments on MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100, comparing against spectral clustering and several deep clustering baselines, and they also evaluate PRCut as a tool for assessing self-supervised representations.
Significance. If the theoretical claims were fully sound, this would be a valuable contribution: it offers a parametric, online alternative to spectral clustering that avoids eigendecomposition, and the probabilistic derivation of the expected ratio-cut is an elegant and original starting point. The empirical study is reasonably broad, the code is released, and the use of the method to score representation quality is a useful idea. However, the central theoretical guarantee — an unbiased gradient estimator for the online objective — is not established and is mathematically suspect, and the displayed gradient formula in Equation (8) does not match the derivative of the stated objective. These issues compromise the paper's main claim and require substantial correction.
major comments (4)
- [Section 4.2 and Algorithm 1] The claim that the stop-gradient moving-average estimator yields an unbiased gradient is not proved and is mathematically suspect. The objective Lrc contains the factor 1/p̄ with p̄ = (1/n)∑ p_i(θ), a parameter-dependent quantity. Algorithm 1 replaces p̄ by a moving average p̄_ma and stops the gradient, so the true chain-rule term arising from differentiating through the denominator is discarded. Moreover, because p̄_ma is itself random, Jensen's inequality gives E[1/p̄_ma] ≥ 1/E[p̄_ma] whenever p̄_ma has positive variance, so the reciprocal is generally a biased estimate. Appendix B.2 only differentiates the pointwise summand and does not address differentiation through the average p̄. The abstract's 'unbiased estimate of its gradient' is therefore unsupported; the claim must be either proven under explicit assumptions or withdrawn.
- [Equation (8) and Appendix B.2] Equation (8) does not appear to be the correct derivative of Lrc as defined in Equation (7). For L = (1/p)∑_{a,b} W_ab(p_a+p_b−2p_ap_b) with p = (1/n)∑_m p_m, the derivative with respect to p_i is (1/p^2)[2p∑_j W_ij(1−2p_j) − S/n], where S = ∑_{a,b} W_ab(p_a+p_b−2p_ap_b). Equation (8) instead contains p∑_j W_ij(1−2p_j) (missing the factor 2) and, in the denominator term, only ∑_j W_ij(p_i+p_j−2p_ip_j) rather than the full S. As written, the update in Algorithm 1 is not the gradient of Lrc, so it is unclear what objective the online procedure actually minimizes. The authors should re-derive the gradient and correct Equation (8) and Appendix B.2.
- [Appendix A.6, proof of Lemma 4.5] The proof contains an equality that is only an upper bound. After integrating (n/(n−2))^{n−2}(1−p̄t)^{n−2}, the exact value is (n/(n−2))^{n−2}[1−(1−p̄)^{n−1}]/((n−1)p̄), not (n/(n−2))^{n−2}/((n−1)p̄). The subsequent equality to (e^2−o(n))/(np̄) is also asymptotic rather than exact. These steps can likely be repaired while preserving the inequality, but the proof as printed is inaccurate and should be corrected.
- [Section 4.1 and Lemma 4.5] The paper proves only an upper bound on the expected ratio-cut; it does not analyze how close the minimizers of Lrc are to the minimizers of the true expected ratio-cut, nor how the KL regularization in Section 4.3 affects this relationship. The claim that PRCut 'optimizes the graph ratio-cut' is therefore supported only empirically, not by the theory. The authors should state this limitation explicitly and, if possible, provide a quantitative gap analysis or at least an experimental sanity check on the tightness of the bound.
minor comments (4)
- [Equation (7)] The equality between the double-sum expression and Tr(P^{-1}(1_{n,k}-P)^T W P) is off by a factor of 2 for symmetric W; please verify and clarify the intended scaling, since the gradient and the learning rate depend on this factor.
- [Section 5, experimental setup] The paper reports 'bach size' instead of 'batch size', and there is a spacing issue in 'F ashion-MNIST'. More substantively, the number of random seeds or repeated runs for PRCut is not reported; spectral clustering is explicitly reported as the best run, so a fair comparison should state the variance or number of runs for PRCut.
- [Algorithm 1 and Section 4.2] The notation β_t = β/t is introduced in the moving-average update, but the hyperparameter table lists only a single β value; please clarify how β_t is computed in practice (e.g., whether t is reset each epoch or is the global step count).
- [Section 4.4] The description of the k-nearest-neighbor graph says the kernel is set to 1 when one vertex is among the other's k nearest neighbors, but the experiments list k = 100 while the graph size k is also used as the number of clusters elsewhere; please disambiguate the two uses of k.
Circularity Check
No significant circularity: the ratio-cut derivation and benchmarks are external; the moving-average gradient claim is a rigor gap, not a circular reduction.
full rationale
The central derivation is not circular. Theorem 4.1 computes the expected ratio-cut from the stated probabilistic assignment model; Lemmas 4.2–4.5 bound it using convexity and Poisson-binomial identities; Eq. (7) is an algebraic rewriting of that bound after dropping constants. None of these steps defines the objective in terms of the reported benchmark quantities. The experimental comparisons are external: spectral clustering, VaDE, VMM, Turtle, and the self-supervised representations are not produced by PRCut's fitted parameters, and the reported ratio-cut is computed by Eq. (3) from the raw Laplacian, not from the optimized bound. The self-citation to Mohan and Monteleoni (2017) is background motivation and is not load-bearing. The one load-bearing assumption that is not proved is the Section 4.2 claim that the moving-average estimator gives an unbiased gradient: Eq. (8) contains 1/p-bar and 1/p-bar-squared, and p-bar_ma is a random moving average of the network's own batch outputs, so E[1/p-bar_ma] is not equal to 1/E[p-bar_ma], and Appendix B.2 differentiates the pointwise loss rather than the actual stochastic estimator. This is a correctness and rigor gap in the online-learning guarantee, not a circular reduction: no derived quantity is equivalent by construction to the fitted p-bar_ma, and the benchmark claims do not reduce to the estimator. I therefore find no significant circularity.
Assumptions & free parameters
free parameters (3)
- KL regularization weight γ =
100.0
- moving average parameter β =
0.8
- k-NN graph size (PRCut) =
100
assumptions (5)
- domain assumption The random assignment variables a_i^(ℓ) are independent across vertices given P.
- domain assumption The kernel/similarity matrix W is symmetric, non-negative, and an accurate reflection of the true cluster structure.
- ad hoc to paper The stop-gradient moving-average estimator produces an unbiased gradient.
- ad hoc to paper Minimizing the upper bound Lrc with KL regularization yields low expected ratio-cut.
- standard math Jensen's inequality, Fubini's theorem, and Gauss-Legendre quadrature
Cite this review
Pith. "Pith review of Deep Clustering via Probabilistic Ratio-Cut Optimization." pith.science (2026). https://pith.science/paper/EZ7LHV67
@misc{pith2026250203405,
author = {Pith},
title = {Pith review of: Deep Clustering via Probabilistic Ratio-Cut Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/EZ7LHV67}},
note = {Machine review of arXiv:2502.03405}
}
read the original abstract
We propose a novel approach for optimizing the graph ratio-cut by modeling the binary assignments as random variables. We provide an upper bound on the expected ratio-cut, as well as an unbiased estimate of its gradient, to learn the parameters of the assignment variables in an online setting. The clustering resulting from our probabilistic approach (PRCut) outperforms the Rayleigh quotient relaxation of the combinatorial problem, its online learning extensions, and several widely used methods. We demonstrate that the PRCut clustering closely aligns with the similarity measure and can perform as well as a supervised classifier when label-based similarities are provided. This novel approach can leverage out-of-the-box self-supervised representations to achieve competitive performance and serve as an evaluation method for the quality of these representations.
Forward citations
Cited by 1 Pith paper
-
Dimensionally Reduced Open-World Clustering: DROWCULA
A no-labels pipeline built from known parts (DINOv2, normalization, UMAP/t-SNE, K-means) reports strong clustering and novel-class accuracies on four benchmarks.
Reference graph
Works this paper leans on
-
[1]
R. Alam and S. Bora. On sensitivity of eigenvalues and eigendecompositions of matrices. Linear Algebra and its Applications, 396: 0 273--301, 2005. ISSN 0024-3795
work page 2005
-
[2]
Iterative bregman projections for regularized transportation problems, 2014
Jean-David Benamou, Guillaume Carlier, Marco Cuturi, Luca Nenna, and Gabriel Peyré. Iterative bregman projections for regularized transportation problems, 2014
work page 2014
-
[3]
Sean X. Chen and Jun S. Liu. Statistical applications of the poisson-binomial and conditional bernoulli distributions. Statistica Sinica, 7 0 (4): 0 875--892, 1997. ISSN 10170405, 19968507
work page 1997
-
[4]
A simple framework for contrastive learning of visual representations, 2020
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations, 2020
2020
-
[5]
solo-learn: A library of self-supervised methods for visual representation learning
Victor Guilherme Turrisi da Costa, Enrico Fini, Moin Nabi, Nicu Sebe, and Elisa Ricci. solo-learn: A library of self-supervised methods for visual representation learning. Journal of Machine Learning Research, 23 0 (56): 0 1--6, 2022. URL http://jmlr.org/papers/v23/21-1155.html
work page 2022
-
[6]
Dhillon, Yuqiang Guan, and Brian Kulis
Inderjit S. Dhillon, Yuqiang Guan, and Brian Kulis. Kernel k-means: Spectral clustering and normalized cuts. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '04, pp.\ 551–556, New York, NY, USA, 2004. Association for Computing Machinery. ISBN 1581138881
work page 2004
-
[7]
Estepa, Ignacio Sarasua, Bhalaji Nagarajan, and Petia Radeva
Imanol G. Estepa, Ignacio Sarasua, Bhalaji Nagarajan, and Petia Radeva. All4one: Symbiotic neighbour contrastive learning via self-attention and redundancy reduction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 16243--16253, October 2023
work page 2023
-
[8]
Absalom E. Ezugwu, Abiodun M. Ikotun, Olaide O. Oyelade, Laith Abualigah, Jeffery O. Agushaka, Christopher I. Eke, and Andronicus A. Akinyelu. A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects. Engineering Applications of Artificial Intelligence, 110: 0 1047...
work page 2022
Show all 35 references
-
[9]
Let go of your labels with unsupervised transfer, 2024
Artyom Gadetsky, Yulun Jiang, and Maria Brbic. Let go of your labels with unsupervised transfer, 2024
2024
-
[10]
S. Guha, N. Mishra, R. Motwani, and L. O'Callaghan. Clustering data streams. In Proceedings 41st Annual Symposium on Foundations of Computer Science, pp.\ 359--366, 2000
2000
-
[11]
Hagen and Andrew B
Lars W. Hagen and Andrew B. Kahng. New spectral methods for ratio cut partitioning and clustering. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., 11: 0 1074--1085, 1991
1991
-
[12]
Bulirsch J
R. Bulirsch J. Stoer. Introduction to Numerical Analysis, volume 1. Springer New York, NY, 2013
2013
-
[13]
A decoder-free variational deep embedding for unsupervised clustering
Qiang Ji, Yanfeng Sun, Junbin Gao, Yongli Hu, and Baocai Yin. A decoder-free variational deep embedding for unsupervised clustering. IEEE Transactions on Neural Networks and Learning Systems, 33: 0 5681--5693, 2021
2021
-
[14]
Variational deep embedding: An unsupervised and generative approach to clustering, 2017
Zhuxi Jiang, Yin Zheng, Huachun Tan, Bangsheng Tang, and Hanning Zhou. Variational deep embedding: An unsupervised and generative approach to clustering, 2017
2017
-
[15]
Learning multiple layers of features from tiny images, 2009
Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009
2009
-
[16]
Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation
Tejas D Kulkarni, Karthik Narasimhan, Ardavan Saeedi, and Josh Tenenbaum. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Pr...
2016
-
[17]
Lecun, L
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998. doi:10.1109/5.726791
1998 doi
-
[18]
Contrastive clustering
Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive clustering. CoRR, abs/2009.09687, 2020. URL https://arxiv.org/abs/2009.09687
2009 arXiv
-
[19]
S. P. Lloyd. Least square quantization in pcm. Bell Telephone Laboratories Paper, 28 0 (2): 0 129--137, March 1957. ISSN 0018-9448. doi:10.1109/TIT.1982.1056489
1957
-
[20]
Machado, Marc G
Marlos C. Machado, Marc G. Bellemare, and Michael Bowling. A L aplacian framework for option discovery in reinforcement learning. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine ...
2017
-
[21]
Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu, Gerald Tesauro, and Murray Campbell
Marlos C. Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu, Gerald Tesauro, and Murray Campbell. Eigenoption discovery through the deep successor representation. CoRR, abs/1710.11089, 2017 b
2017 arXiv
-
[22]
Exploiting sparsity to improve the accuracy of nyström-based large-scale spectral clustering
Mahesh Mohan and Claire Monteleoni. Exploiting sparsity to improve the accuracy of nyström-based large-scale spectral clustering. In 2017 International Joint Conference on Neural Networks (IJCNN), pp.\ 9--16, 2017
2017
-
[23]
Algorithms for the assignment and transportation problems
James Munkres. Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics, 5 0 (1): 0 32--38, 1957. ISSN 03684245. URL http://www.jstor.org/stable/2098689
1957
-
[24]
On spectral clustering: Analysis and an algorithm
Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. In T. Dietterich, S. Becker, and Z. Ghahramani (eds.), Advances in Neural Information Processing Systems, volume 14. MIT Press, 2001
2001
-
[25]
Dinov2: Learning robust visual features without supervision, 2024
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabba...
2024 arXiv
-
[26]
K. B. Petersen and M. S. Pedersen. The matrix cookbook, nov 2012. URL http://www2.compute.dtu.dk/pubdb/pubs/3274-full.html. Version 20121115
2012
-
[27]
David Pfau, Stig Petersen, Ashish Agarwal, David G. T. Barrett, and Kimberly L. Stachenfeld. Spectral inference networks: Unifying deep and spectral learning, 2020
2020
-
[28]
Learning transferable visual models from natural language supervision, 2021
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https://ar...
2021 arXiv
-
[29]
Spectralnet: Spectral clustering using deep neural networks
Uri Shaham, Kelly Stanton, Henry Li, Ronen Basri, Boaz Nadler, and Yuval Kluger. Spectralnet: Spectral clustering using deep neural networks. In International Conference on Learning Representations, 2018
2018
-
[30]
Fast and accurate k-means for large datasets
Michael Shindler and Alex Wong. Fast and accurate k-means for large datasets. NIPS, 24, 01 2011
2011
-
[31]
Andrew Stirn and David A. Knowles. The vampprior mixture model, 2024. URL https://arxiv.org/abs/2402.04412
2024 arXiv
-
[32]
A tutorial on spectral clustering
Ulrike von Luxburg. A tutorial on spectral clustering. CoRR, abs/0711.0189, 2007
2007 arXiv
-
[33]
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. CoRR, abs/1708.07747, 2017. URL http://arxiv.org/abs/1708.07747
2017 arXiv
-
[34]
Streaming spectral clustering
Shinjae Yoo, Hao Huang, and Shiva Prasad Kasiviswanathan. Streaming spectral clustering. In 2016 IEEE 32nd International Conference on Data Engineering (ICDE), pp.\ 637--648, 2016
2016
-
[35]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.