Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
Reviewed by Pithpith:UTDCN77Copen to challenge →
read the original abstract
A central question of representation learning asks under which conditions it is possible to reconstruct the true latent variables of an arbitrarily complex generative process. Recent breakthrough work by Khemakhem et al. (2019) on nonlinear ICA has answered this question for a broad class of conditional generative processes. We extend this important result in a direction relevant for application to real-world data. First, we generalize the theory to the case of unknown intrinsic problem dimension and prove that in some special (but not very restrictive) cases, informative latent variables will be automatically separated from noise by an estimating model. Furthermore, the recovered informative latent variables will be in one-to-one correspondence with the true latent variables of the generating process, up to a trivial component-wise transformation. Second, we introduce a modification of the RealNVP invertible neural network architecture (Dinh et al. (2016)) which is particularly suitable for this type of problem: the General Incompressible-flow Network (GIN). Experiments on artificial data and EMNIST demonstrate that theoretical predictions are indeed verified in practice. In particular, we provide a detailed set of exactly 22 informative latent variables extracted from EMNIST.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Unsupervised Disentanglement Without Compromises : How Functional Orthogonality Enforces Identifiability
Enforcing local orthogonality on the Jacobian of the generative mapping yields identifiability for general nonlinear models when the latent domain has full combinatorial support.
-
Diverse Dictionary Learning
Diverse dictionary learning identifies intersections, complements, and dependency structures of latent variables from data X = g(Z) up to indeterminacies, and full identifiability when structural diversity is sufficient.
-
What Makes a Representation Good for Single-Cell Perturbation Prediction?
PerturbedVAE disentangles perturbation-specific signals from invariant gene expression structure to recover causal representations and improve out-of-distribution prediction in single-cell perturbation modeling.
-
How Optimality Structures Sparse Dictionaries: A Theory for Understanding SAE Representations
Derives optimality constraints for nonnegative joint dictionary learning that explain observed SAE behaviors such as feature splitting, absorption, and dense antipodal features.
-
Learning Latent Dynamical Causal Processes for Single-Cell Perturbation Prediction
Proposes CITE-VAE, a latent dynamical causal VAE with identifiability analysis for single-cell perturbation prediction, claiming better OOD generalization on CRISPR data.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.