REVIEW 4 major objections 4 minor 107 references
A Tutorial on Discriminative Clustering and Mutual Information
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Contrastive clustering does not maximize a learned mutual information: with the augmentation distribution fixed, $I(x;\mathrm{Aug}(x))$ is constant, and the object being trained is an energy-based variational model of…
desk verdict A useful historical tutorial on discriminative clustering with a genuinely clarifying reinterpretation of contrastive learning, currently held back by two easily fixable algebraic errors and an overstrong optimality claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the energy-based variational conditional $q(x|z) = p_{\mathrm{data}}(x) e^{E(x,z)}/Z(z)$, with $Z(z) = \mathbb{E}_{p_{\mathrm{data}}(x)}[e^{E(x,z)}]$, inserted into the standard variational lower bound on mutual information. It turns the intractable conditional $p(x|z)$ into a tractable approximation whose normalizer is estimated by Monte Carlo; the resulting InfoNCE bound is the objective that contrastive losses implement. The second key move is observing that $z = \mathrm{Aug}(x)$ with a fixed augmentation distribution makes $I(x;\mathrm{Aug}(x))$ constant, leaving the critic $E_\theta$ as the only trainable quantity, so the model's actual task is density estimation of $p(x|\mathrm{Aug}(x))$.
What would settle it
Train a contrastive model with a fixed augmentation on data whose true conditional $p(x|\mathrm{Aug}(x))$ is known analytically, then estimate the normalized $q_\theta(x|\mathrm{Aug}(x))$ implied by the trained critic through Eq. (53) and compare it with the true conditional; a systematic mismatch at large critic capacity would break the claim that contrastive learning is density estimation of $p(x|\mathrm{Aug}(x))$.
Extended reading notes
Core claim
The paper's central discovery is an identity about what contrastive clustering actually optimizes. Using the energy-based variational family $q(x|z) = p_{\mathrm{data}}(x) e^{E(x,z)}/Z(z)$ with $Z(z) = \mathbb{E}_{p_{\mathrm{data}}}[e^{E(x,z)}]$, the InfoNCE lower bound becomes a Monte Carlo estimate of $\log(e^{E}/\sum e^{E})$ plus $\log n$, and the entropy term cancels. The authors observe that in contrastive settings $z = \mathrm{Aug}(x)$, so the mutual information $I(x;\mathrm{Aug}(x))$ depends only on the fixed augmentation distribution and is constant with respect to the model parameters. Consequently, maximizing the lower bound cannot be maximizing mutual information; it drives $q(x|\mathrm{Aug}(x))$ toward $p(x|\mathrm{Aug}(x))$. The paper states this as 'contrastive learning is about learning the distribution $p(x|\mathrm{aug}(x))$ using the approximate $q(x|\mathrm{aug}(x))$,' and it draws the historical line from early mutual-information clustering through regularized information maximization, the information bottleneck, deep clustering, and contrastive methods.
Load-bearing premise
The load-bearing premise is that the network used to score similarities is flexible enough to represent the true conditional distribution of an original sample given its augmented version; the paper offers no capacity or tightness analysis, so the identification holds exactly only for an optimal critic.
Editorial extensions
If this is right
- Contrastive clustering should be described as density estimation of $p(x|\mathrm{Aug}(x))$ via an energy-based model, not as mutual information maximization; claims of maximizing $I(x;\mathrm{Aug}(x))$ refer to a constant.
- The clustering hypothesis in modern methods lives in the choice of augmentation distribution and of the critic family, so those choices carry the inductive bias that decision boundaries carried in earlier methods.
- Performance differences between deep clustering methods are more plausibly attributed to architecture, regularizations, and augmentation design than to the mutual information objective itself.
- Model selection for contrastive clustering should use internal scores based on neighbourhood and invariance structure, rather than generic distance-based metrics such as silhouette or Davies-Bouldin, which encode a different clustering hypothesis.
Reading between the lines
- A testable extension the authors leave implicit: if the trained object really is an energy-based $q(x|\mathrm{Aug}(x))$, then sampling from this model (e.g. by Markov-chain or score-based methods) should produce plausible data conditioned on the augmentation; this would separate the density-estimation interpretation from a purely discriminative one.
- The same constant-mutual-information argument applies to self-supervised contrastive learning generally, not just clustering: any temperature-scaled contrastive loss with fixed view generation is better analyzed as learning a conditional view distribution than as maximizing information between views.
- The reinterpretation suggests a concrete design direction: replace the Monte Carlo estimate of the normalizer $Z(z)$ with a learned or analytic partition function, potentially improving contrastive learning beyond what InfoNCE-style normalization allows.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This tutorial reviews discriminative clustering and the role of mutual information as a training objective, from early work by Bridle et al. and Krause et al. through deep clustering and contrastive learning. It introduces generative versus discriminative modelling, explains mutual-information estimation and its limitations, discusses internal model-selection metrics, and presents reproducible experiments with the GemClus package. The paper's central interpretive claim, developed in Sections 5.3.2 and 8, is that contrastive clustering does not maximize a learned mutual information: because the augmentation strategy is fixed, I(x;Aug(x)) is constant, and the actual optimized object is an energy-based variational model q_theta(x|z) proportional to p_data(x) exp(E(x,z)), so that "contrastive learning is about learning the distribution p(x|aug(x)) using the approximate q(x|aug(x))".
Significance. If the central claim is adequately supported, this tutorial fills a genuine gap: it connects historical MI-based discriminative clustering to modern contrastive methods, and it offers a provocative reinterpretation of contrastive losses as energy-based posterior estimation rather than MI maximization. The paper is generally clearly written, and it ships reproducible code (GemClus and a companion notebook) and an honest admission that the contrastive example in Section 7.6 is deliberately rigged. However, the interpretive claim is currently asserted from an optimal-critic argument without a tightness or expressivity analysis, and several display equations in the derivation are incorrect or inconsistent with each other. The correctable nature of these issues makes major revision appropriate.
major comments (4)
- [5.3.2, Eq. (62)] Equation (62) is introduced as "rewriting Eq. (59)" for contrastive learning, but the denominator has the indices reversed. Eq. (59) has denominator sum_j exp(E(xxx_j, z_i)); with z_i = Aug(xxx_i) this is sum_j exp(E(xxx_j, Aug(xxx_i))). Eq. (62) instead writes sum_j exp(E(xxx_i, Aug(xxx_j))). These are different lower bounds: the first corresponds to the variational family q(x|z) used in Eq. (53), while the second corresponds to a family q(z|x). Only the first orientation supports the paper's conclusion that contrastive learning learns q(x|Aug(x)). The code in Listing 5 uses softmax(similarities, dim=0), which implements the first orientation, so Eq. (62) should be corrected to match either the derivation or the code.
- [5.3.1, Eqs. (56)-(58)] The multi-sample derivation leading to the InfoNCE lower bound is not correct as written. Eq. (56) is true only if the variable on the right is the representation zzz_1 of the first sample and the other samples are independent of (xxx_1, zzz_1). Eq. (57) then changes the object: I(xxx_1,...,xxx_n; zzz) = (1/n) sum_i I(xxx_i; zzz_i) is not a standard identity unless zzz denotes the collection (zzz_1,...,zzz_n), in which case the sum has no 1/n factor (equivalently, by i.i.d. each term equals I(xxx_1; zzz_1)). The final lower bound Eq. (59) is correct, but the intermediate steps should be rewritten with explicit random variables so that readers can follow the derivation.
- [5.3.2 and Section 8] The central claim that contrastive learning "is about learning the distribution p(x|aug(x)) using the approximate q(x|aug(x))" relies on the optimal-critic identity E(xxx,zzz) = log p(zzz|xxx) in Eq. (60). This identity yields q = p(x|z) only when the critic family contains that log-density and the energy-based family in Eq. (53) is rich enough to represent the posterior. For a fixed finite-capacity critic, the objective is only a lower bound on the constant I(xxx;Aug(xxx)); no tightness, expressivity, or consistency argument is given. As a result, Section 8's statement that "the true underlying model is an energy-based model" overstates what the derivation shows. The paper's admission in Section 7.6 that the example is rigged is commendable, but it also means that the experiment provides no independent support for the variational claim. Please add an explicit discussion of the idealized conditions under which the interpretation is exact and soften the conclusions accordingly.
- [5.1, Eq. (43)] Equation (43) has a sign error: the mutual-information decomposition is I(xxx;y) = H(y) - H(y|xxx), not H(y|xxx) - H(y). The surrounding sentence that a good clustering model is "fair but firm" corresponds to the correct sign, since fairness is H(y) and firmness is -H(y|xxx). This is a tutorial, so the equation should be fixed to avoid propagating the error.
minor comments (4)
- [2.2.1, Eq. (8)] The sigmoid is written as 1/(1 - e^{-a}), which is undefined at a = 0 and is not a sigmoid; it should be 1/(1 + e^{-a}).
- [3.2, Eq. (26)] The expression after "we ignore the constant" is the negative conditional entropy -E[sum_k p_theta(y=k|x) log p_theta(y=k|x)], not the cross-entropy; the factor multiplying log p_theta(y=k|x) should be p_data(y=k|x) (or the empirical indicator) to match Eq. (28).
- [2.1, Eq. (21)] The KL notation has mismatched parentheses: it should read D_KL(q1(z) || q2(z)) without the extra parenthesis after q1(z).
- [Throughout] Several typos should be cleaned up: "Econstrastive" in Eqs. (62)-(63), "we we do not have access" in Section 3.1, "the case the case" in the Introduction, and "ernegy-based" in Section 5.4.
Circularity Check
No circularity: the contrastive-learning reinterpretation follows from standard variational lower bounds; self-citations are illustrative only.
full rationale
The paper's central interpretive claim in Sec. 5.3.2 — that contrastive clustering trains an energy-based variational model q_theta(x|Aug(x)) approximating p(x|Aug(x)) — is obtained by substituting the energy-based family q(x|z)=p_data(x)e^{E(x,z)}/Z(z) (Eq. 53) into the Barber-Agakov lower bound (Eqs. 50-52) and identifying z with Aug(x). This is a standard mathematical reduction, not a fitted parameter renamed as a prediction; I(x;Aug(x)) is indeed constant under a fixed augmentation distribution, and the lower bound's optimal critic E*=log p(z|x) makes q equal to p(x|z) in the ideal case. The paper's worked example (Sec. 7.6) is transparent that the augmentation choice is rigged, which is a limitation of the demonstration rather than circularity. Self-citations to the authors' GEMINI work, GemClus package, and Corollary 1 of Ohl et al. (2023a) support illustrative claims and software examples, but the central variational argument cites Barber and Agakov (2003) and Poole et al. (2019) and does not depend on those self-citations. The absence of a tightness/capacity analysis for finite critic families is a rigor gap, not a circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption Samples are i.i.d. from an unknown data distribution p_data(x).
- domain assumption The data distribution p_data(x) is left unmodeled in discriminative clustering; only sampling is assumed possible (Section 2.3).
- standard math The variational proposal q(x|z) belongs to the energy-based family q(x|z) = p_data(x) exp(E(x,z)) / Z(z) (Eq. 53).
- standard math Mutual information is invariant to adding an independent variable, and the multi-sample lower bound (Eq. 59) is a valid lower bound.
Cite this review
Pith. "Pith review of A Tutorial on Discriminative Clustering and Mutual Information." pith.science (2026). https://pith.science/paper/54TW4GNF
@misc{pith2026250504484,
author = {Pith},
title = {Pith review of: A Tutorial on Discriminative Clustering and Mutual Information},
year = {2026},
howpublished = {\url{https://pith.science/paper/54TW4GNF}},
note = {Machine review of arXiv:2505.04484}
}
read the original abstract
To cluster data is to separate samples into distinctive groups that should ideally have some cohesive properties. Today, numerous clustering algorithms exist, and their differences lie essentially in what can be perceived as ``cohesive properties''. Therefore, hypotheses on the nature of clusters must be set: they can be either generative or discriminative. As the last decade witnessed the impressive growth of deep clustering methods that involve neural networks to handle high-dimensional data often in a discriminative manner; we concentrate mainly on the discriminative hypotheses. In this paper, our aim is to provide an accessible historical perspective on the evolution of discriminative clustering methods and notably how the nature of assumptions of the discriminative models changed over time: from decision boundaries to invariance critics. We notably highlight how mutual information has been a historical cornerstone of the progress of (deep) discriminative clustering methods. We also show some known limitations of mutual information and how discriminative clustering methods tried to circumvent those. We then discuss the challenges that discriminative clustering faces with respect to the selection of the number of clusters. Finally, we showcase these techniques using the dedicated Python package, GemClus, that we have developed for discriminative clustering.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Mixtures of dirichlet processes with applications to bayesian nonparametric problems
Charles E Antoniak. Mixtures of dirichlet processes with applications to bayesian nonparametric problems. The annals of statistics, pages 1152--1174, 1974
1974
-
[2]
Devon Hjelm, and William Buchwalter
Philip Bachman, R. Devon Hjelm, and William Buchwalter. Learning Representations by Maximizing Mutual Information Across Views . In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019
2019
-
[3]
Model- Based Gaussian and Non - Gaussian Clustering
Jeffrey D Banfield and Adrian E Raftery. Model- Based Gaussian and Non - Gaussian Clustering . Biometrics, pages 803--821, 1993. Publisher: JSTOR
1993
-
[4]
The IM Algorithm : A Variational Approach to Information Maximization
David Barber and Felix V Agakov. The IM Algorithm : A Variational Approach to Information Maximization . In NIPS , 2003
2003
-
[5]
Community Detection in Social Networks
Punam Bedi and Chhavi Sharma. Community Detection in Social Networks . WIREs Data Mining and Knowledge Discovery, 6 0 (3): 0 115--135, 2016. doi:10.1002/widm.1178
-
[6]
Mutual Information Neural Estimation
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual Information Neural Estimation . In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 531--540. PMLR, July 2018
2018
-
[7]
Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood
Christophe Biernacki, Gilles Celeux, and Gérard Govaert. Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood . IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 0 (7): 0 719--725, 2000. doi:10.1109/34.865189
-
[8]
Christopher M. Bishop. Pattern Recognition and Machine Learning ( Information Science and Statistics ) . Springer, 1 edition, 2007. ISBN 0-387-31073-8
2007
Show all 107 references
-
[9]
Leticia Bode, Alexander Hanna, Junghwan Yang, and Dhavan V. Shah. Candidate Networks , Citizen Clusters , and Political Expression : Strategic Hashtag Use in the 2010 Midterms . The ANNALS of the American Academy of Political and Social Science, 659 0 (1): 0 149--165, 2015. do...
2010 doi
-
[10]
Simultaneous model-based clustering and visualization in the fisher discriminative subspace
Charles Bouveyron and Camille Brunet. Simultaneous model-based clustering and visualization in the fisher discriminative subspace. Statistics and Computing, 22 0 (1): 0 301--324, 2012
2012
-
[11]
Model- Based Clustering and Classification for Data Science : With Applications in R
Charles Bouveyron, Gilles Celeux, T Brendan Murphy, and Adrian E Raftery. Model- Based Clustering and Classification for Data Science : With Applications in R . in Series in Statistical and Probabilistic Mathematics . Cambridge University Press, 2019
2019
-
[12]
Unsupervised Classifiers , Mutual Information and ' Phantom ' Targets
John Bridle, Anthony Heading, and David MacKay. Unsupervised Classifiers , Mutual Information and ' Phantom ' Targets . In J. Moody, S. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems , volume 4. Morgan-Kaufmann, 1992
1992
-
[13]
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised Learning of Visual Features by Contrasting Cluster Assignments . In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Informati...
2020
-
[14]
A Simple Framework for Contrastive Learning of Visual Representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations . In International Conference on Machine Learning , pages 1597--1607. PMLR, 2020
2020
-
[15]
A Review of Medical Image Data Augmentation Techniques for Deep Learning Applications
Phillip Chlap, Hang Min, Nym Vandenberg, Jason Dowling, Lois Holloway, and Annette Haworth. A Review of Medical Image Data Augmentation Techniques for Deep Learning Applications . Journal of Medical Imaging and Radiation Oncology, 65 0 (5): 0 545--563, 2021. Publisher: Wiley O...
2021
-
[16]
On Information Regularization
Adrian Corduneanu and Tommi Jaakkola. On Information Regularization . In Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence , pages 151--158, 2002
2002
-
[17]
Nearest Neighbor Matching for Deep Clustering
Zhiyuan Dang, Cheng Deng, Xu Yang, Kun Wei, and Heng Huang. Nearest Neighbor Matching for Deep Clustering . In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition , pages 13693--13702, 2021
2021
-
[18]
Davies and Donald W
David L. Davies and Donald W. Bouldin. A Cluster Separation Measure . IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1 0 (2): 0 224--227, 1979. doi:10.1109/TPAMI.1979.4766909
1979
-
[19]
Discriminative cluster analysis
Fernando De la Torre and Takeo Kanade. Discriminative cluster analysis. In Proceedings of the 23rd International Conference on Machine Learning, ICML '06, page 241–248, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595933832. doi:10.1145/1143844.1143875
2006
-
[20]
A Divisive Information Theoretic Feature Clustering Algorithm for Text Classification
Inderjit S Dhillon, Subramanyam Mallela, and Rahul Kumar. A Divisive Information Theoretic Feature Clustering Algorithm for Text Classification . JMLR, 3: 0 1265--1287, 2003. Publisher: JMLR. org
2003
-
[21]
Kernel K - Means : Spectral Clustering and Normalized Cuts
Inderjit S Dhillon, Yuqiang Guan, and Brian Kulis. Kernel K - Means : Spectral Clustering and Normalized Cuts . In Proceedings of the Tenth Acm Sigkdd International Conference on Knowledge Discovery and Data Mining , pages 551--556, 2004
2004
-
[22]
Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
Nat Dilokthanakul, Pedro AM Mediano, Marta Garnelo, Matthew CH Lee, Hugh Salimbeni, Kai Arulkumaran, and Murray Shanahan. Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders . arXiv preprint arXiv:1611.02648, 2016
2016 arXiv
-
[23]
Clustering by Maximizing Mutual Information Across Views
Kien Do, Truyen Tran, and Svetha Venkatesh. Clustering by Maximizing Mutual Information Across Views . In Proceedings of the IEEE / CVF International Conference on Computer Vision , pages 9928--9938, 2021
2021
-
[24]
Well- Separated Clusters and Optimal Fuzzy Partitions
Joseph C Dunn. Well- Separated Clusters and Optimal Fuzzy Partitions . Journal of Cybernetics, 4 0 (1): 0 95--104, 1974. Publisher: Taylor & Francis
1974
-
[25]
Using the Triangle Inequality to Accelerate K - Means
Charles Elkan. Using the Triangle Inequality to Accelerate K - Means . In Proceedings of the 20th International Conference on Machine Learning (icml-03) , pages 147--153, 2003
2003
-
[26]
Why so Many Clustering Algorithms : A Position Paper
Vladimir Estivill-Castro. Why so Many Clustering Algorithms : A Position Paper . Acm Sigkdd Explorations Newsletter, 4 0 (1): 0 65--75, 2002. Publisher: ACM New York, NY, USA
2002
-
[27]
Kernel K - Groups via Hartigan ’s Method
Guilherme França, Maria L Rizzo, and Joshua T Vogelstein. Kernel K - Groups via Hartigan ’s Method . IEEE Transactions on Pattern Analysis and Machine Intelligence, 43 0 (12): 0 4411--4425, 2020. Publisher: IEEE
2020
-
[28]
Data Augmentation for Medical Imaging : A Systematic Literature Review
Fabio Garcea, Alessio Serra, Fabrizio Lamberti, and Lia Morra. Data Augmentation for Medical Imaging : A Systematic Literature Review . Computers in Biology and Medicine, 152: 0 106391, 2023. ISSN 0010-4825. doi:doi.org/10.1016/j.compbiomed.2022.106391
2023
-
[29]
Generative Adversarial Nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets . In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Pr...
2014
-
[30]
Exploratory Latent Structure Analysis Using Both Identifiable and Unidentifiable Models
Leo A Goodman. Exploratory Latent Structure Analysis Using Both Identifiable and Unidentifiable Models . Biometrika, 61 0 (2): 0 215--231, 1974. Publisher: Oxford University Press
1974
-
[31]
A Kernel Two - Sample Test
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A Kernel Two - Sample Test . The Journal of Machine Learning Research, 13 0 (1): 0 723--773, 2012
2012
-
[32]
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition . In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770--778, 2016
2016
-
[33]
What are the True Clusters ? Pattern Recognition Letters, 64: 0 53--62, 2015
Christian Hennig. What are the True Clusters ? Pattern Recognition Letters, 64: 0 53--62, 2015. ISSN 0167-8655. doi:10.1016/j.patrec.2015.04.009
2015 doi
-
[34]
Smith, Lee Rainie, Ben Shneiderman, and Camila Espina
Itai Himelboim, Marc A. Smith, Lee Rainie, Ben Shneiderman, and Camila Espina. Classifying Twitter Topic - Networks Using Social Network Analysis . Social Media + Society, 3 0 (1): 0 2056305117691545, 2017. doi:10.1177/2056305117691545
2017 doi
-
[35]
Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio
R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning Deep Representations by Mutual Information Estimation and Maximization . In International Conference on Learning Representations , 2019
2019
-
[36]
Kernel Methods in Machine Learning
Thomas Hofmann, Bernhard Schölkopf, and Alexander J Smola. Kernel Methods in Machine Learning . The Annals of Statistics, 36 0 (3): 0 1171--1220, 2008
2008
-
[37]
Learning Discrete Representations via Information Maximizing Self - Augmented Training
Weihua Hu, Takeru Miyato, Seiya Tokui, Eiichi Matsumoto, and Masashi Sugiyama. Learning Discrete Representations via Information Maximizing Self - Augmented Training . In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learni...
2017
-
[38]
Deep Semantic Clustering by Partition Confidence Maximisation
Jiabo Huang, Shaogang Gong, and Xiatian Zhu. Deep Semantic Clustering by Partition Confidence Maximisation . In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition , pages 8849--8858, 2020
2020
-
[39]
Comparing Partitions
Lawrence Hubert and Phipps Arabie. Comparing Partitions . Journal of Classification, 2 0 (1): 0 193--218, 1985. Publisher: Springer
1985
-
[40]
Deep Clustering : On the Link Between Discriminative Models and K -means
Mohammed Jabi, Marco Pedersoli, ar Mitiche, A, and Ismail Ben Ayed. Deep Clustering : On the Link Between Discriminative Models and K -means. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1--1, 2019. doi:10.1109/TPAMI.2019.2962683
2019
-
[41]
Jain, Narasimha N
Anil K. Jain, Narasimha N. Murty, and Patrick J. Flynn. Data clustering: a review. ACM Comput. Surv., 31 0 (3): 0 264–323, September 1999. ISSN 0360-0300. doi:10.1145/331499.331504
1999
-
[42]
Categorical Reparameterization with Gumbel - Softmax
Eric Jang, Shixiang Gu, and Ben Poole. Categorical Reparameterization with Gumbel - Softmax . In International Conference on Learning Representations , 2017
2017
-
[43]
Invariant Information Clustering for Unsupervised Image Classification and Segmentation
Xu Ji, Joao F Henriques, and Andrea Vedaldi. Invariant Information Clustering for Unsupervised Image Classification and Segmentation . In Proceedings of the IEEE / CVF International Conference on Computer Vision , pages 9865--9874, 2019
2019
-
[44]
Variational Deep Embedding : An Unsupervised and Generative Approach to Clustering
Zhuxi Jiang, Yin Zheng, Huachun Tan, Bangsheng Tang, and Hanning Zhou. Variational Deep Embedding : An Unsupervised and Generative Approach to Clustering . In Proceedings of the 26th International Joint Conference on Artificial Intelligence , IJCAI '17, pages 1965--1972. AAAI ...
1965
-
[45]
Customer Segmentation Using K - Means Clustering
Tushar Kansal, Suraj Bahuguna, Vishal Singh, and Tanupriya Choudhury. Customer Segmentation Using K - Means Clustering . In 2018 International Conference on Computational Techniques , Electronics and Mechanical Systems (ctems) , pages 135--139. IEEE, 2018
2018
-
[46]
Exact Bayesian Modeling for Bivariate Poisson Data and Extensions
Dimitris Karlis and Panagiotis Tsiamyrtzis. Exact Bayesian Modeling for Bivariate Poisson Data and Extensions . Statistics and Computing, 18: 0 27--40, 2008. Publisher: Springer
2008
-
[47]
Customer Segmentation using Clustering and Data Mining Techniques
Kishana R Kashwan and CM Velu. Customer Segmentation using Clustering and Data Mining Techniques . International Journal of Computer Theory and Engineering, 5 0 (6): 0 856, 2013. Publisher: IACSIT Press
2013
-
[48]
Adam: A Method for Stochastic Optimization
Diederik P Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization . arXiv preprint arXiv:1412.6980, 2014
2014 arXiv
-
[49]
An Impossibility Theorem for Clustering
Jon Kleinberg. An Impossibility Theorem for Clustering . In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems , volume 15. MIT Press, 2003
2003
-
[50]
Probabilistic graphical models: principles and techniques
Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009
2009
-
[51]
Discriminative Clustering and Feature Selection for Brain MRI Segmentation
Youyong Kong, Yue Deng, and Qionghai Dai. Discriminative Clustering and Feature Selection for Brain MRI Segmentation . IEEE Signal Processing Letters, 22 0 (5): 0 573--577, 2015. doi:10.1109/LSP.2014.2364612
2015
-
[52]
Discriminative Clustering by Regularized Information Maximization
Andreas Krause, Pietro Perona, and Ryan Gomes. Discriminative Clustering by Regularized Information Maximization . In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems , volume 23. Curran Associates,...
2010
-
[53]
Imagenet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet Classification with Deep Convolutional Neural Networks . Advances in Neural Information Processing Systems, 25, 2012
2012
-
[54]
Yoon, Hack-Lyoung Kim, Hyung-Kwan Kim, Yong-Jin Kim, Goo-Yeong Cho, Dae-Won Sohn, Sungho Won, and Seung-Pyo Lee
Soongu Kwak, Yunhwan Lee, Taehoon Ko, Seokhun Yang, In-Chang Hwang, Jun-Bean Park, Yeonyee E. Yoon, Hack-Lyoung Kim, Hyung-Kwan Kim, Yong-Jin Kim, Goo-Yeong Cho, Dae-Won Sohn, Sungho Won, and Seung-Pyo Lee. Unsupervised Cluster Analysis of Patients With Aortic Stenosis Reveals...
2020 doi
-
[55]
Unsupervised Visual Representation Learning via Mutual Information Regularized Assignment
Dong Hoon Lee, Sungik Choi, Hyunwoo J Kim, and Sae-Young Chung. Unsupervised Visual Representation Learning via Mutual Information Regularized Assignment . In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing ...
2022
-
[56]
InfoMax - GAN : Improved Adversarial Image Generation via Information Maximization and Contrastive Learning
Kwot Sin Lee, Ngoc-Trung Tran, and Ngai-Man Cheung. InfoMax - GAN : Improved Adversarial Image Generation via Information Maximization and Contrastive Learning . In Proceedings of the IEEE / CVF Winter Conference on Applications of Computer Vision ( WACV ) , pages 3942--3952, ...
2021
-
[57]
Contrastive Clustering
Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive Clustering . In 2021 AAAI Conference on Artificial Intelligence ( AAAI ) , 2021
2021
-
[58]
Aristidis Likas, Nikos Vlassis, and Jakob J. Verbeek. The Global K - Means Clustering Algorithm . Pattern Recognition, 36 0 (2): 0 451--461, 2003. ISSN 0031-3203. doi:doi.org/10.1016/S0031-3203(02)00060-2
2003 doi
-
[59]
Self- Organization in a Perceptual Network
Ralph Linsker. Self- Organization in a Perceptual Network . Computer, 21 0 (3): 0 105--117, 1988. doi:10.1109/2.36
1988 doi
-
[60]
Least Squares Quantization in PCM
Stuart Lloyd. Least Squares Quantization in PCM . Ieee Transactions on Information Theory, 28 0 (2): 0 129--137, 1982. Publisher: IEEE
1982
-
[61]
Automated Gating of Flow Cytometry Data via Robust Model - Based Clustering
Kenneth Lo, Ryan Remy Brinkman, and Raphael Gottardo. Automated Gating of Flow Cytometry Data via Robust Model - Based Clustering . Cytometry Part a: The Journal of the International Society for Analytical Cytology, 73 0 (4): 0 321--332, 2008. Publisher: Wiley Online Library
2008
-
[62]
VAEM : a Deep Generative Model for Heterogeneous Mixed Type Data
Chao Ma, Sebastian Tschiatschek, Richard Turner, José Miguel Hernández-Lobato, and Cheng Zhang. VAEM : a Deep Generative Model for Heterogeneous Mixed Type Data . In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Proces...
2020
-
[63]
Variable Selection for Model - Based Clustering using the Integrated Complete - Data Likelihood
Matthieu Marbac and Mohammed Sedki. Variable Selection for Model - Based Clustering using the Integrated Complete - Data Likelihood . Statistics and Computing, 27 0 (4): 0 1049--1063, 2017. Publisher: Springer
2017
-
[64]
Model- Based Clustering of Gaussian Copulas for Mixed Data
Matthieu Marbac, Christophe Biernacki, and Vincent Vandewalle. Model- Based Clustering of Gaussian Copulas for Mixed Data . Communications in Statistics-Theory and Methods, 46 0 (23): 0 11635--11656, 2017. Publisher: Taylor & Francis
2017
-
[65]
Formal Limitations on the Measurement of Mutual Information
David McAllester and Karl Stratos. Formal Limitations on the Measurement of Mutual Information . In Silvia Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , volume 108 of Proceedings of M...
2020
-
[66]
The EM Algorithm and Extensions
Geoffrey J McLachlan and Thriyambakam Krishnan. The EM Algorithm and Extensions . John Wiley & Sons, 2007
2007
-
[67]
McLachlan, Richard W
Geoffrey J. McLachlan, Richard W. Bean, and David Peel. A Mixture Model - Based Approach to the Clustering of Microarray Expression Data . Bioinformatics, 18 0 (3): 0 413--422, March 2002. ISSN 1367-4803. doi:10.1093/bioinformatics/18.3.413
2002 doi
-
[68]
Daniel P. M. de Mello, Renato M. Assunção, and Fabricio Murai. Top- Down Deep Clustering with Multi - Generator GANs . In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 7770--7778, June 2022. doi:10.1609/aaai.v36i7.20745
2022 doi
-
[69]
A Survey of Clustering With Deep Learning : From the Perspective of Network Architecture
Erxue Min, Xifeng Guo, Qiang Liu, Gen Zhang, Jianjing Cui, and Jun Long. A Survey of Clustering With Deep Learning : From the Perspective of Network Architecture . IEEE Access, PP: 0 1--1, 2018. doi:10.1109/ACCESS.2018.2855437
2018
-
[70]
Discriminative Models , not Discriminative Training
Tom Minka. Discriminative Models , not Discriminative Training . Technical report, Technical Report MSR-TR-2005-144, Microsoft Research, 2005
2005
-
[71]
Virtual Adversarial Training : A Regularization Method for Supervised and Semi - Supervised Learning
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual Adversarial Training : A Regularization Method for Supervised and Semi - Supervised Learning . IEEE Transactions on Pattern Analysis and Machine Intelligence, 41 0 (8): 0 1979--1993, 2018
1979
-
[72]
On Spectral Clustering : Analysis and an Algorithm
Andrew Ng, Michael Jordan, and Yair Weiss. On Spectral Clustering : Analysis and an Algorithm . Advances in Neural Information Processing Systems, 14, 2001
2001
-
[73]
An I ntroduction to D eep C lustering
Gopi Chand Nutakki, Behnoush Abdollahi, Wenlong Sun, and Olfa Nasraoui. An I ntroduction to D eep C lustering. Clustering Methods for Big Data Analytics: Techniques, Toolboxes and Applications, pages 73--89, 2019
2019
-
[74]
Generalised Mutual Information for Discriminative Clustering
Louis Ohl, Pierre-Alexandre Mattei, Charles Bouveyron, Warith Harchaoui, Mickaël Leclercq, Arnaud Droit, and Frederic Precioso. Generalised Mutual Information for Discriminative Clustering . In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advance...
2022
-
[75]
Generalised Mutual Information : a Framework for Discriminative Clustering , 2023 a
Louis Ohl, Pierre-Alexandre Mattei, Charles Bouveyron, Warith Harchaoui, Mickaël Leclercq, Arnaud Droit, and Frédéric Precioso. Generalised Mutual Information : a Framework for Discriminative Clustering , 2023 a . \_eprint: 2309.02858
2023 arXiv
-
[76]
GemClus , November 2023 b
Louis Ohl, Pierre-Alexandre Mattei, and Frederic Precioso. GemClus , November 2023 b
2023
-
[77]
Sparse and geometry-aware generalisation of the mutual information for joint discriminative clustering and feature selection
Louis Ohl, Pierre-Alexandre Mattei, Charles Bouveyron, Mickaël Leclercq, Arnaud Droit, and Frédéric Precioso. Sparse and geometry-aware generalisation of the mutual information for joint discriminative clustering and feature selection. Statistics and Computing, 34 0 (5): 0 155...
2024 doi
-
[78]
Kernel KMeans clustering splits for end-to-end unsupervised decision trees, 2024 b
Louis Ohl, Pierre-Alexandre Mattei, Mickaël Leclercq, Arnaud Droit, and Frédéric Precioso. Kernel KMeans clustering splits for end-to-end unsupervised decision trees, 2024 b
2024
-
[79]
Improving Unsupervised Image Clustering With Robust Learning
Sungwon Park, Sungwon Han, Sundong Kim, Danu Kim, Sungkyu Park, Seunghoon Hong, and Meeyoung Cha. Improving Unsupervised Image Clustering With Robust Learning . In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition , pages 12278--12287, 2021
2021
-
[80]
Robust Mixture Modelling using the t Distribution
David Peel and Geoffrey J McLachlan. Robust Mixture Modelling using the t Distribution . Statistics and computing, 10: 0 339--348, 2000. Publisher: Springer
2000
-
[81]
Computational Optimal Transport : With Applications to Data Science
Gabriel Peyré and Marco Cuturi. Computational Optimal Transport : With Applications to Data Science . Foundations and Trends® in Machine Learning, 11 0 (5-6): 0 355--607, 2019. ISSN 1935-8237. Publisher: Now Publishers, Inc
2019
-
[82]
On Variational Bounds of Mutual Information
Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On Variational Bounds of Mutual Information . In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings o...
2019
-
[83]
Deep clustering: A comprehensive survey
Yazhou Ren, Jingyu Pu, Zhimeng Yang, Jie Xu, Guofeng Li, Xiaorong Pu, S Yu Philip, and Lifang He. Deep clustering: A comprehensive survey. IEEE Transactions on Neural Networks and Learning Systems, 2024
2024
-
[84]
Stochastic Backpropagation and Approximate Inference in Deep Generative Models
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic Backpropagation and Approximate Inference in Deep Generative Models . In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning , volume 32 of Proceedings...
2014
-
[85]
Finder, and Oren Freifeld
Meitar Ronen, Shahaf E. Finder, and Oren Freifeld. DeepDPM : Deep Clustering With an Unknown Number of Clusters . In 2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition ( CVPR ) , pages 9851--9860, 2022. doi:10.1109/CVPR52688.2022.00963
2022
-
[86]
Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis
Peter J Rousseeuw. Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis . Journal of Computational and Applied Mathematics, 20: 0 53--65, 1987. Publisher: Elsevier
1987
-
[87]
Stop Using the Elbow Criterion for K - Means and How to Choose the Number of Clusters Instead
Erich Schubert. Stop Using the Elbow Criterion for K - Means and How to Choose the Number of Clusters Instead . SIGKDD Explor. Newsl., 25 0 (1): 0 36--42, July 2023. ISSN 1931-0145. doi:10.1145/3606274.3606278. Place: New York, NY, USA Publisher: Association for Computing Machinery
2023
-
[88]
Estimating the Dimension of a Model
Gideon Schwarz. Estimating the Dimension of a Model . The Annals of Statistics, 6 0 (2): 0 461 -- 464, 1978. doi:10.1214/aos/1176344136. Publisher: Institute of Mathematical Statistics
1978
-
[89]
Normalized Cuts and Image Segmentation
Jianbo Shi and Jitendra Malik. Normalized Cuts and Image Segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 0 (8): 0 888--905, 2000. Publisher: IEEE
2000
-
[90]
Document C lustering using W ord C lusters via the I nformation B ottleneck M ethod
Noam Slonim and Naftali Tishby. Document C lustering using W ord C lusters via the I nformation B ottleneck M ethod. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 208--215, 2000
2000
-
[91]
Information- Based Clustering
Noam Slonim, Gurinder Singh Atwal, Gašper Tkačik, and William Bialek. Information- Based Clustering . Proceedings of the National Academy of Sciences, 102 0 (51): 0 18297--18302, 2005. Publisher: National Academy of Sciences
2005
-
[92]
Data C lustering and S imilarity
Julien Soler, Fabien Tenc \'e , Laurent Gaubert, and C \'e dric Buche. Data C lustering and S imilarity. In The Twenty-Sixth International FLAIRS Conference, 2013
2013
-
[93]
Unsupervised and Semi - Supervised Learning with Categorical Generative Adversarial Networks
Jost Tobias Springenberg. Unsupervised and Semi - Supervised Learning with Categorical Generative Adversarial Networks . arXiv preprint arXiv:1511.06390, 2015
2015 arXiv
-
[94]
Genesis: Cluster Analysis of Microarray Data
Alexander Sturn, John Quackenbush, and Zlatko Trajanoski. Genesis: Cluster Analysis of Microarray Data . Bioinformatics, 18 0 (1): 0 207--208, 2002. Publisher: Oxford University Press
2002
-
[95]
On information-maximization clustering: Tuning parameter selection and analytic solution
Masashi Sugiyama, Makoto Yamada, Manabu Kimura, and Hirotaka Hachiya. On information-maximization clustering: Tuning parameter selection and analytic solution. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 65--72, 2011
2011
-
[96]
PerMetrics: A Framework of Performance Metrics for Machine Learning Models
Nguyen Van Thieu. PerMetrics: A Framework of Performance Metrics for Machine Learning Models . Journal of Open Source Software, 9 0 (95): 0 6143, March 2024. doi:10.21105/joss.06143
2024 doi
-
[97]
Estimating the Number of Clusters in a Data Set via the Gap Statistic
Robert Tibshirani, Guenther Walther, and Trevor Hastie. Estimating the Number of Clusters in a Data Set via the Gap Statistic . Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63 0 (2): 0 411--423, 2001. ISSN 13697412. doi:10.1111/1467-9868.00293
2001
-
[98]
The Information Bottleneck Method
Naftali Tishby, Fernando C Pereira, and William Bialek. The Information Bottleneck Method . arXiv preprint physics/0004057, 2000
2000 arXiv
-
[99]
Rubenstein, Sylvain Gelly, and Mario Lucic
Michael Tschannen, Josip Djolonga, Paul K. Rubenstein, Sylvain Gelly, and Mario Lucic. On Mutual Information Maximization for Representation Learning . In International Conference on Learning Representations , 2020
2020
-
[100]
Representation Learning with Contrastive Predictive Coding
Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning with Contrastive Predictive Coding . arXiv e-prints, pages arXiv--1807, 2018
2018
-
[101]
Relative C lustering V alidity C riteria: A C omparative O verview
Lucas Vendramin, Ricardo JGB Campello, and Eduardo R Hruschka. Relative C lustering V alidity C riteria: A C omparative O verview. Statistical analysis and data mining: the ASA data science journal, 3 0 (4): 0 209--235, 2010. Publisher: Wiley Online Library
2010
-
[102]
Demystifying Information - Theoretic Clustering
Greg Ver Steeg, Aram Galstyan, Fei Sha, and Simon DeDeo. Demystifying Information - Theoretic Clustering . In International Conference on Machine Learning , pages 19--27. PMLR, 2014
2014
-
[103]
A Tutorial on Spectral Clustering
Ulrike von Luxburg. A Tutorial on Spectral Clustering . Statistics and Computing, 17 0 (4): 0 395--416, December 2007. ISSN 1573-1375. doi:10.1007/s11222-007-9033-z
2007 doi
-
[104]
An Overview on Deep Clustering
Xiuxi Wei, Zhihui Zhang, Huajuan Huang, and Yongquan Zhou. An Overview on Deep Clustering . Neurocomputing, 590: 0 127761, 2024. ISSN 0925-2312. doi:10.1016/j.neucom.2024.127761
2024
-
[105]
Unsupervised Deep Embedding for Clustering Analysis
Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised Deep Embedding for Clustering Analysis . In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning , volume 48 of Proceedings of Machine Learning Res...
2016
-
[106]
Discriminative k-means for clustering
Jieping Ye, Zheng Zhao, and Mingrui Wu. Discriminative k-means for clustering. Advances in neural information processing systems, 20, 2007
2007
-
[107]
A comprehensive survey on deep clustering: Taxonomy, challenges, and future directions
Sheng Zhou, Hongjia Xu, Zhuonan Zheng, Jiawei Chen, Zhao Li, Jiajun Bu, Jia Wu, Xin Wang, Wenwu Zhu, and Martin Ester. A comprehensive survey on deep clustering: Taxonomy, challenges, and future directions. ACM Computing Surveys, 2022
2022
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.