Pith. sign in

REVIEW 4 major objections 7 minor 2 cited by

A Revisit of Total Correlation in Disentangled Variational Auto-Encoder with Partial Disentanglement

T0 review · 4 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A variational auto-encoder can enforce group-wise independence of latent factors by replacing total correlation with partial correlation, recovering entangled groups that full disentanglement misses and reducing to standard VAE at one…

desk verdict A clean PC generalization of TC-VAE with a solid IS batch estimator and honest synthetic evidence, but the user-supplied group configuration is load-bearing and its misspecification is never tested. read the letter →

arxiv 2502.02279 v1 pith:AH6OTOMR submitted 2025-02-04 cs.LG q-bio.NC

Chengrui Li , Yunmiao Wang , Yule Wang , Weihan Li , Dieter Jaeger , Anqi Wu This is my paper · ORCID
classification cs.LGq-bio.NC
keywords partialdisentanglementvariationalauto-encodertotalcorrelationgroup-wiseindependenceimportancesamplingbatchapproximationindependentcomponentanalysisdisentangledrepresentationlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper develops PDisVAE, a variational auto-encoder that replaces the total correlation (TC) penalty—which forces all K latent dimensions to be independent—with a partial correlation (PC) penalty that forces only user-specified groups of dimensions to be independent. The PC term is the KL divergence between the aggregated posterior q(z) and the product of group marginals; it vanishes exactly when groups are independent and leaves within-group dependence untouched. The authors argue that on data where factors are entangled in non-separable ways, this is the right prior, and they show that the method reduces to the standard VAE when there is one group and to a fully disentangled VAE when each group is a single dimension. They further derive an importance-sampling batch estimator for TC and PC and prove it has lower variance than the existing minibatch stratified sampling estimator.

What carries the argument

The machinery is the partial correlation (PC) term, $\mathrm{PC} = \mathrm{KL}\big(q(z) \,\|\, \prod_{g=1}^G q(z_g)\big)$, a generalization of total correlation ($\mathrm{TC} = \mathrm{KL}(q(z) \| \prod_{k=1}^K q(z_k))$). Added to the ELBO with weight $\beta$, it drives the aggregated posterior $q(z) = \frac{1}{N}\sum_n q(z|x^{(n)})$ to factorize across groups while permitting arbitrary dependence inside each group. The second piece is an importance-sampling (IS) batch estimator that replaces the minibatch weighted and stratified sampling estimators used in earlier TC-VAEs: when a latent $z$ is drawn from a specific example's posterior, the other $M-1$ batch points are reweighted to represent the rest of the dataset. The paper proves the IS estimator is unbiased with variance $(N-M)^2/(M^2(M-1))$, which is strictly smaller than the MSS estimator's variance for $M>2$.

What would settle it

Generate synthetic data with true group-wise independence $(z_1,z_3) \perp (z_2,z_4)$ and train PDisVAE with $G=2$, $H=2$ using the contiguous groups $(z_1,z_2)$ and $(z_3,z_4)$. The paper's formulation assumes groups are contiguous blocks, so it predicts the partial correlation cannot be driven to zero; observing it driven to zero would show the penalty is more flexible than its derivation, while observing it remain high would confirm the contiguous-block limitation.

Watch

Extended reading notes

Core claim

The central claim is that optimizing a VAE objective with the additional penalty $-\beta \, \mathrm{KL}(q(z) \| \prod_{g=1}^G q(z_g))$ enforces group-wise independence of the latent representation. For $G=1$ the penalty is identically zero and the objective is the standard VAE; for $G=K$ it is exactly the TC penalty, so the fully disentangled VAE appears as a special case. In synthetic experiments with three independent but internally entangled latent groups, PDisVAE attains lower partial correlation and higher latent recovery $R^2$ than standard VAE, logcosh-prior ICA, ISA-VAE, and $\beta$-TCVAE. On the partial-dsprites dataset, PDisVAE is the only method whose groups align with the true entangled location group and the independent size factor; on CelebA it expresses attributes such as background color as multi-dimensional manifolds rather than forced one-dimensional codes, and on voltage imaging data a six-group configuration separates cortical regions more clearly than full disentanglement.

Load-bearing premise

The method presupposes that the user knows the number of latent groups G and the group rank H (with K = G × H), that the true latent structure can be expressed by contiguous coordinate blocks, and that the chosen partition matches that structure; the paper states in Section 5.2 that there is no guidance for this choice and automatic group-rank reduction is left to future work.

Editorial extensions

If this is right

  • On synthetic data with three independent rank-2 groups, PDisVAE recovers the true latent structure with lower partial correlation and higher latent $R^2$ than VAE, ICA, ISA-VAE, and $\beta$-TCVAE.
  • PDisVAE automatically handles rank deficiency: if a group's true rank is below $H$, it learns dummy components, so it reduces smoothly to full disentanglement when the data are fully independent.
  • The importance-sampling batch estimator is unbiased and has lower variance than minibatch stratified sampling, which should make TC and PC penalties more stable during training.
  • On CelebA, grouping dimensions allows attributes like background color to be represented by a 2D or 3D manifold, which a fully disentangled single-component code cannot express.
  • On mouse dorsal cortex voltage imaging, a six-group configuration separates cortical regions (S1-bf, M2-m, M1) into independent groups more clearly than coarser or fully disentangled configurations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Outside the paper's setup, the same PC penalty could be attached to any VAE-style objective that uses an aggregated posterior, and the variance comparison suggests existing TC-based VAEs could improve stability by switching to the IS estimator.
  • The contiguous-coordinate-block definition of groups (Eq. 4) is a real constraint; if true latent groups interleave coordinates, a learned permutation or grouping layer would be needed, and the paper gives no guidance there.
  • The rank-deficiency experiments imply a simple automatic-group-selection heuristic: monitor within-group PCA explained variance or Gaussianity to trim $H$ during training, which the paper explicitly leaves to future work.
  • The semantic-vs-statistical discussion suggests that even when a single semantic label exists, it may live in a group of several latent dimensions; evaluating disentanglement by matching one dimension to one label could systematically misjudge group-based methods.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes PDisVAE, a variational auto-encoder objective that replaces the total correlation (TC) penalty in fully disentangled VAEs with a partial correlation (PC) penalty, KL(q(z) || ∏_g q(z_g)), defined with respect to a user-specified partition of the latent space into G groups of rank H (Eq. 6). The objective reduces to the standard VAE when G=1 and to a TC-penalized VAE when G=K. The paper also introduces an importance-sampling batch approximation for estimating PC/TC, claims it is unbiased and lower-variance than minibatch stratified sampling, and validates PDisVAE on synthetic group-wise independent data, a partially entangled dsprites variant, CelebA, and mouse voltage imaging data. The central claimed contribution is a flexible, penalty-based way to enforce group-wise independence that can also accommodate dummy latent dimensions within groups.

Significance. If the central claim holds, PDisVAE is a natural and potentially useful generalization of TC-based disentanglement, and the reduction property (G=1 and G=K) is elegant. The paper gives real evidence for the core mechanism: in synthetic experiments with known ground truth, PDisVAE achieves higher latent R2 than VAE, logcosh ICA, ISA-VAE, and beta-TCVAE, and the full-independence experiment in Appendix A.4.3 shows that PDisVAE can reduce to a fully disentangled solution when the true rank is deficient. The importance-sampling estimator, if correctly analyzed, would also be a practical contribution. However, the advertised flexibility is tested only under oracle group configurations, and the optimality claim for the estimator is not rigorously supported, so the strength of the contribution is currently below what the paper claims.

major comments (4)
  1. [Sections 4.1, 4.2, 5.2] The flexibility of PDisVAE is demonstrated only under oracle group configurations. Equation (4) fixes a partition into G contiguous blocks of equal rank H, and every controlled experiment uses the true (K,G,H): K=6, G=3, H=2 in Section 4.1; K=4, G=2, H=2 in Section 4.2; and K=6, G=3 with true underlying K=3 in Appendix A.4.3. No experiment varies G or H away from the true values, and Section 5.2 explicitly concedes that 'we may not have guidance on this information.' This matters because, with a misspecified partition, the PC penalty can be minimized by a wrong grouping, for example by splitting one true group across two supplied groups or by placing a dummy near-Gaussian dimension in an oversized group, so a low PC value alone would not certify that the recovered grouping is correct. I ask for misspecification experiments with over- and under-specified G, unequal true group ranks, and non-contiguous true groups, with latent R2 as the primary metric; these experiments are needed to support the paper's central flexibility claim.
  2. [Fig. 2(b), Table 5] The PC metric used as a headline comparison is the same objective that PDisVAE minimizes, so comparisons on PC are partly circular; beta-TCVAE minimizes TC, not PC, and standard VAE does not minimize either. The paper does provide independent evidence through latent R2 against known true latents in the synthetic experiments, and that is the strongest part of the empirical support. To avoid the appearance of circularity, the paper should present PC as a diagnostic of the objective and make latent R2 (or another equally independent metric such as MIG) the primary basis for cross-method claims, especially because the real-world conclusions in Section 4.3 are qualitative and are not supported by repeated-seed statistics.
  3. [Appendix A.3, Eqs. (17)-(19)] The optimality claim for the IS batch estimator is not established. The variance comparison computes the variances of the inverse importance weight sets IS0 and MSS0, not the variances of the estimators \hat q(z); the latter depend on the conditional densities q(z|n), which are not constant across n, and on the distribution of the sampled latent z. The paper also announces a 'theoretical proof of its optimality' in Section 3.5, but no minimization over proposal distributions appears in the appendix. Either provide a correct variance bound for the actual estimator, or revise the claim to state only that the estimator is unbiased and has lower weight variance than MSS.
  4. [Section 4.3] The real-world experiments on CelebA and mouse voltage imaging are used to support the versatility claim, but they are presented through selected qualitative reconstructions and brain maps, with no repeated seeds, no quantitative comparison across group configurations, and no uncertainty quantification. Since these data have no ground-truth latent structure, the 'valuable information' claim is difficult to verify, and the text's own Section 5.1 acknowledges that semantic-statistical correspondences cannot be rigorously certified. Please either add quantitative, replicated comparisons or explicitly mark Section 4.3 as exploratory illustration rather than validation.
minor comments (7)
  1. [Section 5.2] The word 'expecially' should be 'especially'.
  2. [Section 4.2 and Fig. 14] The name 'β-BTCV AE' in Section 4.2 is inconsistent with the 'β-TCVAE' notation used elsewhere, and Fig. 14's caption says 'fully entangled VAE' where it appears to mean 'fully disentangled VAE'.
  3. [Table 5] The 'adapted mutual information gap (MIG)' is not defined; please specify how it is computed and over how many seeds the means and standard deviations are reported.
  4. [Fig. 6] The horizontal axis labels in Fig. 6 are rendered as '100 102' and should be displayed as powers of 10.
  5. [Appendix A.1] In the related-work discussion, the sentence 'This approach has been proven to be worse than β-VAE and FactorVAE' is likely missing a 'TC' qualifier and is confusing as written.
  6. [References] Reference [9] (Bhowal et al.) lacks a year and venue, and reference [10] (Hsu et al.) would benefit from the full publication venue; also, the appendix says 'see code for details' but no code repository URL is provided.
  7. [Appendix A.3.3] The empirical evaluation of the batch estimators reports variances over 1000 repeats for a toy 10-point dataset, but the text does not specify how the 'true' TC/PC value is computed or why the empirical variance of IS is lower than MSS; a short explanation would improve reproducibility.

Circularity Check

1 steps flagged · score 4.0 of 10

The PC evaluation metric is the same functional as the PC penalty being trained, so part of the claimed superiority is built into the objective; independent latent-R2 benchmarks keep the central claim partially grounded.

  1. self definitional [Section 3.3 Eq. (6); Section 4.1 Results, Fig. 2(b); Appendix A.4.4 Table 5]
    "we develop the partially disentangled VAE (PDisVAE) that achieves the group-wise independence by an extra penalty term to the loss. Its target function L = 1/N sum_n ELBO(x(n)) - beta * KL(q(z) || prod_{g=1}^G q(z_g)) ... The PC box plot in Fig. 2(b) shows that PDisVAE achieves the lowest PC, implying that PDisVAE disentangles latent in groups the best. ... Table 5. The PC, latent R2, latent MSS, and adapted MIG evaluated for different methods on the dsprites dataset. PC down, R2 up ... PDisVAE 0.68 ... 0.54 ..."

    The PC reported in Fig. 2(b) and Table 5 is exactly the penalty term in Eq. (6) that PDisVAE is trained, with beta = 4, to minimize. Thus reporting that PDisVAE achieves the lowest PC is partly a restatement of its training objective: the success metric is the same functional as the loss term. This is not fully circular because the same experiments also evaluate latent R2 against the known true groups, an external benchmark that PDisVAE does not directly optimize, and Appendix A.4.4 reports independent metrics such as MIG. The circularity is therefore partial: the 'best group disentanglement' claim leans on a self-referential metric, while the central method still has independent synthetic validation.

full rationale

The paper's derivation of the PC objective in Eq. (6) is mathematically self-contained: replacing TC by KL(q(z) || prod_g q(z_g)) is a natural generalization, and the G=1 and G=K reductions follow by definition rather than by fitted parameters. No load-bearing self-citation chain appears: the cited prior work on TC, FactorVAE, beta-TCVAE, and ISA-VAE is external, and the paper's own previous work (Wang et al. 2024) is only an application reference. The main circularity concern is evaluative: the PC metric used to demonstrate superiority is the same quantity PDisVAE is trained to minimize, so the 'lowest PC' result is partly by construction. This would be a serious circularity if PC were the only evidence, but the synthetic experiments also report latent R2 after alignment to the true latent, which is an independent correctness check not contained in the loss; pdsprites additionally reports MSE and MIG. The acknowledged need for user-supplied (G, H) and the absence of misspecified-group experiments (Section 5.2) is a real limitation and correctness risk, but it is not circularity: the paper explicitly concedes that guidance on the group count may be unavailable. The claim that the IS estimator is 'optimal' is stronger than the variance comparison against MWS/MSS proves, but this is an overclaim rather than a circular reduction. Overall, the central derivation has independent content and is partially validated by external latent-R2 benchmarks, so the circularity score is moderate rather than high.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard VAE assumptions, a user-specified group partition, and the reliability of the minibatch estimator. The only structural assumptions beyond standard variational inference are the contiguous grouping and the hyperparameters beta, K, G, which are chosen by hand.

free parameters (3)
  • PC/TC penalty coefficient beta = 4 (ablation recommends 2 to 10)
    Weights the PC term in Eq. (6); fixed by cross-validation on synthetic experiments and transferred to real datasets.
  • Number of latent groups G = varied across {1,2,3,4,6,12} depending on experiment
    User-specified structural assumption; defines which latent dimensions are forced to be independent. No automatic selection procedure.
  • Latent dimension K and group rank H=K/G = K=6 or 4 in synthetic, K=12 in real-world experiments
    Architectural choices that, with G, determine the contiguous group assignments in Eq. (4).
assumptions (4)
  • standard math Variational inference with a Gaussian observation likelihood and the ELBO provides a tractable approximation q(z|x) and aggregated posterior q(z).
    Invoked in Eq. (1) and throughout; standard VAE machinery.
  • domain assumption The minibatch importance-sampling estimator for q(z) (Eqs. 7 and 14) is unbiased and lower-variance than MSS, so gradients of PC computed with it are reliable.
    Used to train PDisVAE; derivation in Appendix A.3 shows lower variance than the two baselines, not global optimality.
  • domain assumption Minimizing KL(q(z) || prod_g q(z_g)) drives the learned representation toward group-wise independence, and the variational family is expressive enough to realize this.
    Core mechanism of PDisVAE; depends on the density estimators and optimization actually reaching the penalty-minimizing solution.
  • ad hoc to paper True latent groups are arranged as contiguous blocks z_(g-1)H+1 ... z_gH, with user-specified G and H.
    Eq. (4) defines groups by contiguous coordinates; wrong G, H, or group ordering will enforce an incorrect independence structure. The paper acknowledges lack of guidance in Section 5.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Revisit of Total Correlation in Disentangled Variational Auto-Encoder with Partial Disentanglement." pith.science (2026). https://pith.science/paper/AH6OTOMR

@misc{pith2026250202279,
  author       = {Pith},
  title        = {Pith review of: A Revisit of Total Correlation in Disentangled Variational Auto-Encoder with Partial Disentanglement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AH6OTOMR}},
  note         = {Machine review of arXiv:2502.02279}
}
read the original abstract

A fully disentangled variational auto-encoder (VAE) aims to identify disentangled latent components from observations. However, enforcing full independence between all latent components may be too strict for certain datasets. In some cases, multiple factors may be entangled together in a non-separable manner, or a single independent semantic meaning could be represented by multiple latent components within a higher-dimensional manifold. To address such scenarios with greater flexibility, we develop the Partially Disentangled VAE (PDisVAE), which generalizes the total correlation (TC) term in fully disentangled VAEs to a partial correlation (PC) term. This framework can handle group-wise independence and can naturally reduce to either the standard VAE or the fully disentangled VAE. Validation through three synthetic experiments demonstrates the correctness and practicality of PDisVAE. When applied to real-world datasets, PDisVAE discovers valuable information that is difficult to find using fully disentangled VAEs, implying its versatility and effectiveness.

Figures

Figures reproduced from arXiv: 2502.02279 by the authors.

Figure 1
Figure 1. Visual illustrations for the desired behavior of the PDisVAE. In each case, the left plot is the estimated latent (ˆzi, zˆj ) and the right plot is the true latent (zi, zj ). become zero under any linear transformation. • Case 2: Rank-deficient. Consider that PDisVAE has identified an estimated group (ˆzi , zˆj ) in the left plot of case 2. Although they are dependent, they exhibit a clear linear relationship, which… view at source ↗
Figure 2
Figure 2. (a): The true latent z ∈ R 6 where three groups are (z1, z2) ⊥ (z3, z4) ⊥ (z5, z6), but within-groups are highly entangled (top row). Latent components in different groups are marginally independent (bottom row). (b): The PC of the estimated latent and the latent R 2 after alignment to the true latent in (a), with pair-wise t-test showing the significance level (***: p ⩽ 0.001, ****: p ⩽ 0.0001). (c): The estimated … view at source ↗
Figure 3
Figure 3. Estimated latent after aligning to the true latent (Fig.2(a)) for various methods. Left three columns: the three independent groups; right one column: a between-group component pair. VAE and ICA results are in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (11 more)
Figure 6
Figure 6. Figure 6: Three metrics w.r.t. the PC coefficient β in PDisVAE. Flexibly reduce to the fully independent case. To vali￾date that PDisVAE can flexibly get the same results as from a fully disentangled VAE when the latent is fully independent, we create a dataset that is generated…
Figure 4
Figure 4. Figure 4: (a): Latent and observation generating process. Locations (z1, z2) are entangled, and uniformly distributed in a restricted region. Color represents the location information, with the upper and lower gray triangular areas being empty. The size z3 is evenly distributed …
Figure 5
Figure 5. Figure 5: The latent plot after alignment for the group 1 (z1, z2) and group 2 (z3, z4 ≈ 0) from different methods, and their corresponding PC and latent R 2 . The color representation for location is the same as the color representation in [PITH_FULL_IMAGE:figures/full_fig_p00…
Figure 7
Figure 7. Figure 7: (a): Reconstructed images are shown by varying one of the K = 12 latent dimensions from PDisVAE applied to the CelebA dataset, with different numbers of groups G ∈ {4, 6, 12}. Each row corresponds to varying one latent component (dimension) while fixing all others to 0…
Figure 8
Figure 8. Figure 8: Brain maps  z n g 50×50 n=1 and the corresponding time series A:,g from the learned groups by different PDisVAE configurations (K, G), i.e., K components, G groups, and the group rank is H = K/G. Some groups contain dummy dimensions, so the effective group rank is low…
Figure 9
Figure 9. Figure 9: Left: Predicted mean of the latent z = (z1, z2) and its kernel density estimation. Right: 1000 repeats of batch approximations by the three methods, their empirical variance across the 1000 repeats. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Latent alignment results of different methods. Each group is aligned and matched to the true latent shown in [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: (a): The true latent z ∈ R 3 coded by RGB = z1z2z3, where three components are z1 ⊥ z2 ⊥ z3. (b): The PC of the estimated latent and the latent R 2 after alignment to the true latent in (a). The t-test between PDisVAE and others shows that PDisVAE is similar to β-TCVA…
Figure 12
Figure 12. Figure 12: Estimated and true latent distribution after alignment to the true latent shown in [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: The latent plot after alignment in latent space (z1, z3) and (z2, z3) for different methods. The color representation for location is the same as the color representation in [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: The reconstructed images by varying one of the K = 12 disentangled latent from applying PDisVAE to the CelebA dataset with the different number of groups G ∈ {1, 2, 3, 4, 6, 12}. When G = 1, PDisVAE becomes the standard VAE; when G = K = 12, PDisVAE becomes the fully …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Factorized Low-Rank RNN Framework for Uncovering Independent Neural Latent Dynamics and Connectivity

    q-bio.NC 2025-11 conditional novelty 6.0 of 10

    DisRNN, a VAE-based low-rank RNN with a group-wise independence penalty, learns disentangled latent trajectories and interpretable sub-connectivity from neural population recordings.

  2. BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics

    eess.SP 2025-05 conditional novelty 5.0 of 10

    BrainStratify's coarse-to-fine disentanglement, electrode clustering plus decoupled product quantization, modestly improves speech decoding over prior methods on sEEG and epidural ECoG datasets.

Reference graph

Works this paper leans on

15 extracted references · 9 canonical work pages · cited by 2 Pith papers

  1. [1]

    However, the choice of the non-Gaussian prior is critical and might be too rigid, hurting the flexibility of the method

    ICA (Hyv¨arinen & Oja, 2000): Traditional ICA uses a non-Gaussian prior to achieving full disentanglement since independence is non-Gaussian from the statistical perspective. However, the choice of the non-Gaussian prior is critical and might be too rigid, hurting the flexibility of the method. •

    work page 2000

  2. [4]

    It utilizes a group-wise independent prior called Lp-nested distribution to achieve the partial disentanglement

    ISA-V AE (St¨uhmer et al., 2020): ISA-V AE realized the commonly existing group-wise independence (partial disentanglement) in the real-world data. It utilizes a group-wise independent prior called Lp-nested distribution to achieve the partial disentanglement. However, they did not validate their approach on partially disentangled synthetic datasets, but ...

    work page 2020

  3. [5]

    This approach has been proven to be worse than β-V AE and FactorV AE

    β-V AE (Burgess et al., 2018): Directly penalize the KL divergence of the V AE ELBO loss, in which TC (in Eq.(??)) is implicitly penalized. This approach has been proven to be worse than β-V AE and FactorV AE. •

    work page 2018

  4. [6]

    This also gives us a hint that full disentanglement might be a strong and inappropriate assumption and could result in poor latent interpretation

    (Locatello et al., 2019): This research presented common challenges in finding disentangled latent through an unsupervised approach, implying supervision with semantic latent labels might be necessary under the assumption of full latent disentanglement. This also gives us a hint that full disentanglement might be a strong and inappropriate assumption and ...

    work page 2019

  5. [7]

    (Ahuja et al., 2022): This paper uses weak supervision from observations generated by sparse perturbations of the latent variables, which requires auxiliary information to the latent variables. •

    work page 2022

  6. [8]

    The only difference between these two papers is their implementations of minimizing TC

    [3]β-TCV AE (Chen et al., 2018): These two papers start from the statistical definition of full independence to add an extra total correlation to achieve full independence rigorously. The only difference between these two papers is their implementations of minimizing TC. •

    work page 2018

  7. [10]

    group-wise independence

    (Hsu et al., 2024): The full disentanglement is achieved by a technique called latent quantization. The approach is quantizing the latent space into discrete code vectors with a separate learnable scalar codebook per dimension. Besides, weight decay is also applied to the model regularization for better full disentanglement. 12 A Revisit of Total Correlat...

    work page 2024

  8. [13]

    (Meo et al., 2024): This paper replace the traditional TC term with a novel TC lower bound to achieve not only disentanglement but generalized observation diversity. •

    work page 2024

Show all 15 references
  1. [15]

    z1 z2 z3 p(z1, z2, z3) 0 0 1 0.25 0 1 0 0.25 1 0 0 0.25 1 1 1 0.25 A.3

    The distribution table of p(z1, z2, z3). z1 z2 z3 p(z1, z2, z3) 0 0 1 0.25 0 1 0 0.25 1 0 0 0.25 1 1 1 0.25 A.3. Batch approximation A.3.1. I MPORTANCE SAMPLING Although Eq. (7) in the main text intuitively gives the batch approximation, we still need a rigorous derivation to ...

    2018

  2. [2010]

    Disentanglement by nonlinear ica with general incompressible-flow networks (gin)

    Sorrenson, P., Rother, C., and K¨othe, U. Disentanglement by nonlinear ica with general incompressible-flow networks (gin). arXiv preprint arXiv:2001.04872,

    2001 arXiv

  3. [2013]

    Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

    arXiv

  4. [2017]

    P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A

    Burgess, C. P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A. Understanding dis- entangling in β-vae. arXiv preprint arXiv:1804.03599,

    arXiv

  5. [2018]

    Kingma, D. P. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114,

    arXiv

  6. [2022]

    A., Fischer, I., Dillon, J

    Alemi, A. A., Fischer, I., Dillon, J. V ., and Murphy, K. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410,

    arXiv

  7. [2023]

    Adversarial autoencoders

    Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., and Frey, B. Adversarial autoencoders. arXiv preprint arXiv:1511.05644,

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.