Information-Geometric Decomposition of Generalization Error in Unsupervised Learning

Gilhan Kim

arxiv: 2604.12340 · v1 · submitted 2026-04-14 · 📊 stat.ML · cond-mat.stat-mech· cs.IT· cs.LG· math.IT· math.ST· stat.TH

Information-Geometric Decomposition of Generalization Error in Unsupervised Learning

Gilhan Kim This is my paper

Pith reviewed 2026-05-10 15:10 UTC · model grok-4.3

classification 📊 stat.ML cond-mat.stat-mechcs.ITcs.LGmath.ITmath.STstat.TH

keywords generalization errorinformation geometryunsupervised learningKL divergenceepsilon-PCAmodel selectionphase diagram

0 comments

The pith

The Kullback-Leibler generalization error of unsupervised learning decomposes exactly into model error, data bias, and variance for e-flat model classes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that the expected KL divergence from the data distribution to a trained unsupervised model splits into three non-negative terms whose sum recovers the total error. The split holds exactly for any e-flat model class by combining the generalized Pythagorean theorem with a dual e-mixture variance identity from information geometry. If the decomposition is valid, analysts can separately quantify how much error comes from model restrictions, finite-sample bias, and estimation variance, which in turn supplies an analytic route to optimal model complexity.

Core claim

The Kullback-Leibler generalization error of unsupervised learning decomposes exactly into model error, data bias, and variance for any e-flat model class via the generalized Pythagorean theorem and a dual e-mixture variance identity. Applied to a technical reformulation of rank-constrained epsilon-PCA on isotropic Gaussian data, the components admit closed forms; the optimal rank is the cutoff at the noise floor epsilon, and the behavior falls into three regimes separated by the lower Marchenko-Pastur edge and an analytically computable collapse threshold epsilon_*(alpha).

What carries the argument

Exact three-component decomposition of KL generalization error (model error + data bias + variance) for e-flat models, obtained from the generalized Pythagorean theorem together with the dual e-mixture variance identity.

If this is right

For any e-flat unsupervised model the three components are non-negative and sum exactly to the observed generalization error.
In the epsilon-PCA demonstration the optimal rank equals the number of empirical eigenvalues exceeding the noise floor epsilon.
Model selection exhibits three distinct regimes (retain-all, interior optimum, collapse) whose boundaries are the Marchenko-Pastur edge and the collapse threshold epsilon_*(alpha).
The optimal cutoff balances marginal reduction in model error against the increase in data bias.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same decomposition could be used to compare competing unsupervised architectures by estimating their separate error components on held-out data.
Because the identities are information-geometric, the framework may extend to exponential-family models beyond PCA once suitable e-flat embeddings are identified.
Numerical confirmation on Gaussian data suggests that approximate versions of the decomposition might still be useful for non-Gaussian or misspecified settings.

Load-bearing premise

The model class must be e-flat, or admit an exact reformulation that preserves total generalization error, for the three-term decomposition to hold without remainder.

What would settle it

Generate samples from a known distribution, fit an explicitly e-flat model, compute the empirical total KL generalization error, and verify whether the separately calculated model-error, bias, and variance terms sum to that total within sampling error.

Figures

Figures reproduced from arXiv: 2604.12340 by Gilhan Kim.

**Figure 2.** Figure 2: Three-regime phase diagram of the optimal [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗

**Figure 3.** Figure 3: Joint numerical verification of Lemma 1, Theorem 2, and Theorem 3, on isotropic Gaussian data P = N (0, INV ) at NV = 64, D = 96 (α = 2/3), ϵ = 0.5, sample-averaged over 800 Wishart realizations. Black: empirical total generalization error of the textbook eigen-ϵ-PCA model Qm = U mDm(U m) ⊤ of (18), computed in full NV × NV matrix form (the rotation U m is used explicitly). Red, green, blue: the three comp… view at source ↗

read the original abstract

We decompose the Kullback--Leibler generalization error (GE) -- the expected KL divergence from the data distribution to the trained model -- of unsupervised learning into three non-negative components: model error, data bias, and variance. The decomposition is exact for any e-flat model class and follows from two identities of information geometry: the generalized Pythagorean theorem and a dual e-mixture variance identity. As an analytically tractable demonstration, we apply the framework to $\epsilon$-PCA, a regularized principal component analysis in which the empirical covariance is truncated at rank $N_K$ and discarded directions are pinned at a fixed noise floor $\epsilon$. Although rank-constrained $\epsilon$-PCA is not itself e-flat, it admits a technical reformulation with the same total GE on isotropic Gaussian data, under which each component of the decomposition takes closed form. The optimal rank emerges as the cutoff $\lambda_{\mathrm{cut}}^{*} = \epsilon$ -- the model retains exactly those empirical eigenvalues exceeding the noise floor -- with the cutoff reflecting a marginal-rate balance between model-error gain and data-bias cost. A boundary comparison further yields a three-regime phase diagram -- retain-all, interior, and collapse -- separated by the lower Marchenko--Pastur edge and an analytically computable collapse threshold $\epsilon_{*}(\alpha)$, where $\alpha$ is the dimension-to-sample-size ratio. All claims are verified numerically.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper gives an exact three-component decomposition of KL generalization error for unsupervised learning on e-flat models via standard info-geo identities, then applies it to a reformulated ε-PCA to get closed forms, an optimal cutoff at the noise floor, and a three-regime phase diagram.

read the letter

The paper's key result is an exact decomposition of the Kullback-Leibler generalization error in unsupervised learning into model error, data bias, and variance. This holds for any e-flat model class and rests on the generalized Pythagorean theorem along with a dual e-mixture variance identity. They show how this works by applying it to ε-PCA. Since the rank-constrained version is not e-flat, they use a technical reformulation on isotropic Gaussian data that keeps the total generalization error unchanged. Under this setup, each component gets a closed form, the optimal rank cutoff turns out to be exactly at the noise floor ε, and they map out three regimes in a phase diagram based on the Marchenko-Pastur edge and a collapse threshold. The decomposition itself is a solid application of known information geometry tools, and the closed-form analysis plus numerical verification give it concrete traction. The phase diagram is a nice addition that shows when you should retain all components, use an interior rank, or collapse. The main soft spot is whether the reformulation truly preserves the original problem's structure. The paper notes that rank-constrained ε-PCA is not e-flat but admits this adjustment with matching total error. If the reformulation shifts the effective model or the divergence measure in ways that don't align with the standard algorithm, the derived cutoff and regimes may not directly translate back. The abstract says all claims are verified numerically, but the independence of the reformulation from the target result needs close inspection in the full text. This paper is for people interested in information-geometric methods for analyzing generalization in unsupervised settings or in theoretical aspects of regularized PCA. A reader looking for geometric organizing principles for error components will get value from the framework and the example. It deserves a serious referee because the core claim is formally grounded in standard identities, the application produces new closed forms and a phase diagram, and the numerical checks are there to back it up. I would recommend sending it to peer review.

Referee Report

2 major / 2 minor

Summary. The manuscript claims an exact decomposition of the Kullback-Leibler generalization error (expected KL from data distribution to trained model) in unsupervised learning into three non-negative components—model error, data bias, and variance—for any e-flat model class, derived from the generalized Pythagorean theorem and a dual e-mixture variance identity. As demonstration, it applies the framework to rank-constrained ε-PCA (truncated covariance at rank N_K with noise floor ε) on isotropic Gaussian data via a technical reformulation that preserves total GE, yielding closed-form components, optimal cutoff λ_cut^*=ε from marginal-rate balance, and a three-regime phase diagram (retain-all, interior, collapse) separated by the Marchenko-Pastur edge and ε_*(α), all verified numerically.

Significance. If the decomposition holds and the ε-PCA reformulation is valid without altering the original problem's effective model or KL measure, the work supplies a principled information-geometric tool for dissecting generalization error beyond standard bias-variance analyses. The closed-form results and phase diagram for ε-PCA could inform regularization choices in high-dimensional unsupervised learning, with the exact non-negativity and marginal-rate interpretation offering falsifiable predictions.

major comments (2)

[Abstract and ε-PCA application section] Abstract and the ε-PCA section: the technical reformulation of rank-constrained ε-PCA (explicitly noted as not e-flat) to an e-flat class that preserves the scalar total GE is load-bearing for applying the decomposition and deriving λ_cut^*=ε plus the phase diagram. The mapping must be specified in full (including how learned parameters, effective model, and KL divergence are handled) to confirm the closed forms and conclusions apply to the original ε-PCA rather than a proxy.
[Decomposition derivation section] Decomposition derivation section: the step-by-step application of the generalized Pythagorean theorem and dual e-mixture variance identity to obtain the exact three-component split requires explicit assumptions on the model class, data distribution, and any auxiliary variables; without these, exactness and non-negativity cannot be independently verified.

minor comments (2)

[Notation throughout] Clarify notation for ε, N_K, α, and λ_cut at first use and ensure consistency across equations and text.
[Numerical verification section] Expand the numerical verification section with specific simulation parameters (e.g., values of α, ε, sample sizes) and reproducibility details such as code or raw eigenvalue data.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their careful and constructive review of our manuscript. The comments highlight important points regarding the clarity of the technical reformulation and the explicitness of the derivation assumptions. We address each major comment below and will incorporate the suggested clarifications in the revised version.

read point-by-point responses

Referee: [Abstract and ε-PCA application section] Abstract and the ε-PCA section: the technical reformulation of rank-constrained ε-PCA (explicitly noted as not e-flat) to an e-flat class that preserves the scalar total GE is load-bearing for applying the decomposition and deriving λ_cut^*=ε plus the phase diagram. The mapping must be specified in full (including how learned parameters, effective model, and KL divergence are handled) to confirm the closed forms and conclusions apply to the original ε-PCA rather than a proxy.

Authors: We agree that additional detail on the reformulation is warranted to eliminate any ambiguity. In the revised manuscript, we will expand the relevant section to provide a complete specification of the mapping: the original rank-constrained ε-PCA (with truncated covariance and noise-floor pinning) is reformulated by embedding the learned parameters into an e-flat exponential family model whose mixture coordinates are chosen so that the total KL generalization error is identically preserved for isotropic Gaussian data. We will explicitly state how the empirical eigenvalues and eigenvectors are mapped to the natural parameters of the e-flat model, define the effective model distribution, and verify that the KL divergence from the data distribution remains unchanged. This ensures the closed-form components, the optimal cutoff λ_cut^*=ε, and the three-regime phase diagram apply directly to the original ε-PCA problem rather than a distinct proxy. revision: yes
Referee: [Decomposition derivation section] Decomposition derivation section: the step-by-step application of the generalized Pythagorean theorem and dual e-mixture variance identity to obtain the exact three-component split requires explicit assumptions on the model class, data distribution, and any auxiliary variables; without these, exactness and non-negativity cannot be independently verified.

Authors: We appreciate the referee's emphasis on verifiability. The derivation assumes an e-flat model class (so that the generalized Pythagorean theorem applies in the mixture coordinates), data drawn from a distribution belonging to the same exponential family (isotropic Gaussians in the ε-PCA case), and the dual e-mixture representation for the variance identity. In the revision we will add an explicit subsection (or appendix) that enumerates these assumptions, provides the step-by-step derivation with direct citations to the information-geometric identities, and confirms the non-negativity of each component under the stated conditions. This will allow independent verification without altering the original claims. revision: yes

Circularity Check

0 steps flagged

Minor self-citation risk but core derivation independent; reformulation noted but not shown to reduce by construction

full rationale

The central decomposition is explicitly derived from two standard information-geometry identities (generalized Pythagorean theorem and dual e-mixture variance identity) that hold for any e-flat model class; these are external mathematical facts, not fitted or self-defined within the paper. The ε-PCA demonstration relies on a 'technical reformulation' that preserves total GE on isotropic Gaussians, but the provided text does not exhibit an equation-by-equation reduction showing that the component expressions or the λ_cut^*=ε result are forced by the reformulation itself rather than by the geometry. No self-citations, ansatzes smuggled via prior work, or renaming of known results appear in the abstract or description. This yields a low but non-zero score to acknowledge that the reformulation's independence from the target quantities cannot be fully verified from the given material alone.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The central claim rests on two standard information-geometry identities plus a technical reformulation of ε-PCA that preserves total GE; no new entities are postulated and free parameters are limited to the model's own hyperparameters.

free parameters (2)

epsilon
Noise floor hyperparameter that pins discarded directions in ε-PCA; the optimal cutoff is derived to equal this value.
rank N_K
Truncation rank chosen to retain empirical eigenvalues above the noise floor.

axioms (2)

standard math Generalized Pythagorean theorem of information geometry
Invoked to obtain the exact additive decomposition of KL divergence.
standard math Dual e-mixture variance identity
Invoked to isolate the variance component of the generalization error.

pith-pipeline@v0.9.0 · 5565 in / 1623 out tokens · 90652 ms · 2026-05-10T15:10:52.823200+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

22 extracted references · 22 canonical work pages · 1 internal anchor

[1]

Springer, 2016

Shun-ichi Amari.Information Geometry and Its Applications. Springer, 2016

work page 2016
[2]

American Math- ematical Society, 2000

Shun-ichi Amari and Hiroshi Nagaoka.Methods of Information Geometry. American Math- ematical Society, 2000

work page 2000
[3]

T. W. Anderson.An Introduction to Multivariate Statistical Analysis. Wiley-Interscience, Hoboken, NJ, 3rd edition, 2003

work page 2003
[4]

Silverstein.Spectral Analysis of Large Dimensional Random Matrices

Zhidong Bai and Jack W. Silverstein.Spectral Analysis of Large Dimensional Random Matrices. Springer, 2nd edition, 2010

work page 2010
[5]

Phase transition of the largest eigen- value for nonnull complex sample covariance matrices.Annals of Probability, 33(5):1643– 1697, 2005

Jinho Baik, G´ erard Ben Arous, and Sandrine P´ ech´ e. Phase transition of the largest eigen- value for nonnull complex sample covariance matrices.Annals of Probability, 33(5):1643– 1697, 2005

work page 2005
[6]

The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices.Advances in Mathematics, 227(1):494–521, 2011

Florent Benaych-Georges and Raj Rao Nadakuditi. The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices.Advances in Mathematics, 227(1):494–521, 2011

work page 2011
[7]

Cover and Joy A

Thomas M. Cover and Joy A. Thomas.Elements of Information Theory. Wiley- Interscience, 2nd edition, 2006

work page 2006
[8]

Imre Csisz´ ar.I-divergence geometry of probability distributions and minimization prob- lems.Annals of Probability, 3(1):146–158, 1975. 20

work page 1975
[9]

Eigenvalues and condition numbers of random matrices.SIAM Journal on Matrix Analysis and Applications, 9(4):543–560, 1988

Alan Edelman. Eigenvalues and condition numbers of random matrices.SIAM Journal on Matrix Analysis and Applications, 9(4):543–560, 1988

work page 1988
[10]

Matan Gavish and David L. Donoho. The optimal hard threshold for singular values is 4/ √ 3.IEEE Transactions on Information Theory, 60(8):5040–5053, 2014

work page 2014
[11]

Gr¨ unwald.The Minimum Description Length Principle

Peter D. Gr¨ unwald.The Minimum Description Length Principle. MIT Press, 2007

work page 2007
[12]

Boltzmann Sampling by Diabatic Quantum Annealing

Ju-Yeon Gyhm, Gilhan Kim, Hyukjoon Kwon, and Yongjoo Baek. Boltzmann sampling by diabatic quantum annealing. arXiv:2409.18126, 2024. Co-first authors: J.-Y. Gyhm and G. Kim

work page internal anchor Pith review Pith/arXiv arXiv 2024
[13]

Geoffrey E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771–1800, 2002

work page 2002
[14]

Johnstone

Iain M. Johnstone. On the distribution of the largest eigenvalue in principal components analysis.Annals of Statistics, 29(2):295–327, 2001

work page 2001
[15]

Tradeoff of generalization er- ror in unsupervised learning.Journal of Statistical Mechanics: Theory and Experiment, 2023(8):083401, 2023

Gilhan Kim, Hojun Lee, Junghyo Jo, and Yongjoo Baek. Tradeoff of generalization er- ror in unsupervised learning.Journal of Statistical Mechanics: Theory and Experiment, 2023(8):083401, 2023. arXiv:2303.05718; Editor’s Highlight

work page arXiv 2023
[16]

V. A. Marˇ cenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices.Mathematics of the USSR-Sbornik, 1(4):457–483, 1967

work page 1967
[17]

Asymptotics of sample eigenstructure for a large dimensional spiked co- variance model.Statistica Sinica, 17(4):1617–1642, 2007

Debashis Paul. Asymptotics of sample eigenstructure for a large dimensional spiked co- variance model.Statistica Sinica, 17(4):1617–1642, 2007

work page 2007
[18]

Information processing in dynamical systems: foundations of harmony theory

Paul Smolensky. Information processing in dynamical systems: foundations of harmony theory. In David E. Rumelhart and James L. McClelland, editors,Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1. MIT Press, 1986

work page 1986
[19]

Tipping and Christopher M

Michael E. Tipping and Christopher M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61(3):611–622, 1999

work page 1999
[20]

Novikov, Daan Christiaens, Benjamin Ades-Aron, Jan Sijbers, and Els Fieremans

Jelle Veraart, Dmitry S. Novikov, Daan Christiaens, Benjamin Ades-Aron, Jan Sijbers, and Els Fieremans. Denoising of diffusion MRI using random matrix theory.NeuroImage, 142:394–406, 2016

work page 2016
[21]

Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory.Journal of Machine Learning Research, 11:3571–3594, 2010

Sumio Watanabe. Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory.Journal of Machine Learning Research, 11:3571–3594, 2010

work page 2010
[22]

Max Welling, Michal Rosen-Zvi, and Geoffrey E. Hinton. Exponential family harmoniums with an application to information retrieval. InAdvances in Neural Information Processing Systems (NIPS), 2005. 21

work page 2005

[1] [1]

Springer, 2016

Shun-ichi Amari.Information Geometry and Its Applications. Springer, 2016

work page 2016

[2] [2]

American Math- ematical Society, 2000

Shun-ichi Amari and Hiroshi Nagaoka.Methods of Information Geometry. American Math- ematical Society, 2000

work page 2000

[3] [3]

T. W. Anderson.An Introduction to Multivariate Statistical Analysis. Wiley-Interscience, Hoboken, NJ, 3rd edition, 2003

work page 2003

[4] [4]

Silverstein.Spectral Analysis of Large Dimensional Random Matrices

Zhidong Bai and Jack W. Silverstein.Spectral Analysis of Large Dimensional Random Matrices. Springer, 2nd edition, 2010

work page 2010

[5] [5]

Phase transition of the largest eigen- value for nonnull complex sample covariance matrices.Annals of Probability, 33(5):1643– 1697, 2005

Jinho Baik, G´ erard Ben Arous, and Sandrine P´ ech´ e. Phase transition of the largest eigen- value for nonnull complex sample covariance matrices.Annals of Probability, 33(5):1643– 1697, 2005

work page 2005

[6] [6]

The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices.Advances in Mathematics, 227(1):494–521, 2011

Florent Benaych-Georges and Raj Rao Nadakuditi. The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices.Advances in Mathematics, 227(1):494–521, 2011

work page 2011

[7] [7]

Cover and Joy A

Thomas M. Cover and Joy A. Thomas.Elements of Information Theory. Wiley- Interscience, 2nd edition, 2006

work page 2006

[8] [8]

Imre Csisz´ ar.I-divergence geometry of probability distributions and minimization prob- lems.Annals of Probability, 3(1):146–158, 1975. 20

work page 1975

[9] [9]

Eigenvalues and condition numbers of random matrices.SIAM Journal on Matrix Analysis and Applications, 9(4):543–560, 1988

Alan Edelman. Eigenvalues and condition numbers of random matrices.SIAM Journal on Matrix Analysis and Applications, 9(4):543–560, 1988

work page 1988

[10] [10]

Matan Gavish and David L. Donoho. The optimal hard threshold for singular values is 4/ √ 3.IEEE Transactions on Information Theory, 60(8):5040–5053, 2014

work page 2014

[11] [11]

Gr¨ unwald.The Minimum Description Length Principle

Peter D. Gr¨ unwald.The Minimum Description Length Principle. MIT Press, 2007

work page 2007

[12] [12]

Boltzmann Sampling by Diabatic Quantum Annealing

Ju-Yeon Gyhm, Gilhan Kim, Hyukjoon Kwon, and Yongjoo Baek. Boltzmann sampling by diabatic quantum annealing. arXiv:2409.18126, 2024. Co-first authors: J.-Y. Gyhm and G. Kim

work page internal anchor Pith review Pith/arXiv arXiv 2024

[13] [13]

Geoffrey E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771–1800, 2002

work page 2002

[14] [14]

Johnstone

Iain M. Johnstone. On the distribution of the largest eigenvalue in principal components analysis.Annals of Statistics, 29(2):295–327, 2001

work page 2001

[15] [15]

Tradeoff of generalization er- ror in unsupervised learning.Journal of Statistical Mechanics: Theory and Experiment, 2023(8):083401, 2023

Gilhan Kim, Hojun Lee, Junghyo Jo, and Yongjoo Baek. Tradeoff of generalization er- ror in unsupervised learning.Journal of Statistical Mechanics: Theory and Experiment, 2023(8):083401, 2023. arXiv:2303.05718; Editor’s Highlight

work page arXiv 2023

[16] [16]

V. A. Marˇ cenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices.Mathematics of the USSR-Sbornik, 1(4):457–483, 1967

work page 1967

[17] [17]

Asymptotics of sample eigenstructure for a large dimensional spiked co- variance model.Statistica Sinica, 17(4):1617–1642, 2007

Debashis Paul. Asymptotics of sample eigenstructure for a large dimensional spiked co- variance model.Statistica Sinica, 17(4):1617–1642, 2007

work page 2007

[18] [18]

Information processing in dynamical systems: foundations of harmony theory

Paul Smolensky. Information processing in dynamical systems: foundations of harmony theory. In David E. Rumelhart and James L. McClelland, editors,Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1. MIT Press, 1986

work page 1986

[19] [19]

Tipping and Christopher M

Michael E. Tipping and Christopher M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61(3):611–622, 1999

work page 1999

[20] [20]

Novikov, Daan Christiaens, Benjamin Ades-Aron, Jan Sijbers, and Els Fieremans

Jelle Veraart, Dmitry S. Novikov, Daan Christiaens, Benjamin Ades-Aron, Jan Sijbers, and Els Fieremans. Denoising of diffusion MRI using random matrix theory.NeuroImage, 142:394–406, 2016

work page 2016

[21] [21]

Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory.Journal of Machine Learning Research, 11:3571–3594, 2010

Sumio Watanabe. Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory.Journal of Machine Learning Research, 11:3571–3594, 2010

work page 2010

[22] [22]

Max Welling, Michal Rosen-Zvi, and Geoffrey E. Hinton. Exponential family harmoniums with an application to information retrieval. InAdvances in Neural Information Processing Systems (NIPS), 2005. 21

work page 2005