REVIEW 4 major objections 6 minor 1 cited by
Likelihood Contribution based Multi-scale Architecture for Generative Flows
T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that replacing static masking with a data-dependent split—early-gaussianizing the dimensions that contribute least to log-likelihood—improves flow-based density estimation and sample quality on standard image benchmarks.
desk verdict A cheap, sensible data-dependent split rule for multi-scale RealNVP, with consistently better bits/dim than a static mask, but the evidence is thin and the key mask-transfer assumption is untested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the per-dimension log-determinant tensor $[L_d^{(l)}]_{s\times s\times c}$, the cumulative sum of $\log|\det(\partial y_i/\partial y_{i-1}^T)|$ over flow layers $1$ through $l$, viewed as a tensor aligned with the spatial and channel dimensions. It serves as a proxy for each dimension's contribution to total log-likelihood. The method squeezes this tensor to $\frac{s}{2}\times\frac{s}{2}\times 4c$ via local max- and min-pooling, splits it into high- and low-log-det halves, factorizes the low half out early, and passes the high half into the next flow stage. The factorization is decided before the multiscale training starts and then kept fixed, so non-invertible pooling operations can be used without breaking the change-of-variables formula.
What would settle it
Run the LCMA pipeline multiple times from different random initializations of the pre-trained no-multiscale model and record which dimensions get factored at each layer. If the selected masks differ substantially between runs, or if the bits/dim advantage over the fixed-mask baseline vanishes under a different seed, the claim that data-dependent factorization is the cause of the improvement is falsified.
Extended reading notes
Core claim
On the paper's terms, the central discovery is that per-dimension log-determinant scores from a flow trained without multi-scale factorization rank dimensions by their contribution to log-likelihood, and factoring out the low-scoring dimensions early while exposing high-scoring dimensions to more flow layers improves density estimation and sample quality. Local max- and min-pooling of the log-determinant tensor preserves spatial structure when deciding the split, so the high- and low-contribution halves both retain representative pixels. In experiments the proposed LCMA architecture improves bits/dim over the fixed RealNVP baseline on all four benchmarks, with the largest gain on CelebA (3.02 to 2.71 bits/dim), and produces smoother latent-space interpolations.
Load-bearing premise
The argument assumes that the per-dimension log-determinant scores measured on a flow trained without any early factorization stay a trustworthy ranking of which dimensions deserve more flow layers once the architecture is changed to factor dimensions out early; the paper does not demonstrate that these rankings are stable across random seeds or architectures.
Editorial extensions
If this is right
- RealNVP-style models can lower bits/dim on standard image benchmarks by replacing static masking with the log-determinant heuristic, without changing coupling layers or optimizer settings.
- The heuristic is obtained as a byproduct of flow training, so the same data-dependent factorization can be tried on other multi-scale flows that currently use fixed channel splits.
- The ablation indicates ordering matters in the expected direction: gaussianizing high-log-det dimensions early is worse than gaussianizing low-log-det dimensions early, and random permutation loses spatial quality.
- Following the paper's own proposal, the factorization could eventually be learned on the fly, reducing the need for a separate pre-training pass.
Reading between the lines
- If the log-determinant rankings are stable across initializations and architectures, the heuristic becomes a general architecture-search signal for flows, not just a one-time split for RealNVP.
- The paper leaves untested whether the pre-training step can be removed or amortized; if the mask is recomputed during training, the reported gains might change.
- A natural stress test is to apply the same factorization rule within a Glow-style model with 1x1 convolutions, where the mixing of dimensions could alter which log-det scores are stable; the paper does not report that experiment.
- The CelebA result suggests redundancy matters: datasets with structured, repetitive content may benefit the most from data-dependent factorization, which is testable on other face or object datasets.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a data-dependent multi-scale architecture for normalizing flows, called LCMA (Likelihood Contribution based Multi-scale Architecture). Instead of the static dimension factorization (e.g., checkerboard or channel-wise splitting) used in RealNVP and Glow, LCMA determines which dimensions to factor out early by computing the per-dimension contribution to the total log-determinant (and hence log-likelihood) from a pretrained flow that has no multi-scale architecture. The per-dimension log-det scores are then reduced via local max/min pooling to select low-log-det dimensions for early gaussianization, and the resulting mask is fixed before the final training run. Experiments on CIFAR-10, ImageNet 32x32, ImageNet 64x64, and CelebA report bits/dim improvements over the RealNVP baseline (e.g., 3.02 to 2.71 on CelebA), qualitative sample comparisons, latent-space interpolations, and an ablation study comparing several factorization choices.
Significance. If the reported gains are robust, the paper makes a useful contribution: it introduces a simple, architecture-agnostic heuristic—per-dimension log-det—that can guide multi-scale factorization without introducing fitted constants, and the ablation indicates that gaussianizing low-log-det dimensions early is better than the reverse or random splitting. The heuristic is indeed computed from quantities that are available during flow training, which makes the idea appealing. However, the empirical support is currently thin: single runs with no error bars, no released code, and an unvalidated transfer assumption from the pretrained non-multi-scale network to the final LCMA architecture. The central claim of the paper is therefore plausible but not yet convincingly established.
major comments (4)
- [Section 3 (LCMA construction; Figure 2)] The central assumption of the method is that per-dimension log-det scores computed from a pretrained flow with no multi-scale architecture remain valid for deciding the factorization in a different architecture that performs early gaussianization. The paper does not test this transfer: after dimensions are factored out, the tensors seen by later layers are different from those seen by the pretrained network, and the paper provides no evidence that the log-det ranking is stable across pretraining seeds, random initializations, or architectural changes. Since Table 1 reports single bits/dim values with no error bars, the reported gains could be an artifact of the particular pretrained mask. I ask the authors to report (i) the overlap of the selected dimensions across multiple pretraining runs, (ii) mean and standard deviation of bits/dim over several final training runs, and (iii) at least one result where the mask is derived from an independent pretrained model.
- [Section 3 (factorization operation; Figure 1)] The description of the pooling procedure is dimensionally inconsistent: the text says that local max-pooling and min-pooling convert the s×s×c tensor [L(l)d] into an s/2×s/2×4c tensor, but each of max-pooling and min-pooling on s×s×c yields s/2×s/2×c, so concatenating the two results gives 2c channels, not 4c. The mechanism that produces 4c channels, and the subsequent split into equal halves of high- and low-log-det dimensions, is therefore unclear. Since the factorization mask is defined by this operation, the authors should specify the exact tensor operations and shapes (including how the 4c channels are partitioned into the two halves) so that the method is reproducible.
- [Abstract] The abstract on the first page states 'We present such implementations for several state-of-the-art flow models', but the full text presents only an implementation for the RealNVP flow (Section 5), and the paper's own abstract in the body says 'We present such an implementation for the original flow introduced in Dinh et al. (2016)'. The claim of multiple implementations is not supported by the experiments. Please align the abstract with the actual scope of the paper.
- [Section 5.3 (Table 2)] The ablation study does not state which dataset it uses; the numbers (3.02 for the baseline and 2.71 for LCMA) match the CelebA row of Table 1, but the table should label the dataset explicitly. In addition, the three non-LCMA variants are each evaluated with a single bits/dim number (3.05, 3.10, 3.02), so the reader cannot tell whether the ordering is meaningful. Please report multiple runs with standard deviations (or at least seed counts) for all entries in the table.
minor comments (6)
- [Section 3, Eq. (9)] In Eq. (9), the quantity [L(l)d]_{s×s×c} is written as a sum of log|det(∂y_i/∂y_{i-1}^T)|, which is a scalar log-determinant, but the text uses it as a per-dimension tensor. Please define the per-dimension log-absolute-derivative (e.g., for the triangular Jacobians of affine coupling layers) explicitly to make the heuristic precise.
- [Section 4] Section 4 states 'we essentially have a heuristic for free', but the proposed pipeline requires a full pretraining run of a flow without multi-scaling for each dataset (as described in Section 5). This computational cost should be acknowledged when comparing with static factorization methods.
- [Section 5.2] The qualitative comparison in Section 5.2 and Figure 3 is based solely on visual inspection; given the small quantitative gains, an independent quantitative assessment of sample quality (e.g., FID or at least multiple fixed-seed samples) would strengthen the claim.
- [Section 3] The term 'data-dependent factorization' is used to describe a mask that is computed once from a pretrained model and then frozen; since the mask does not adapt during training, a more precise descriptor would be 'precomputed, data-dependent'.
- [Table 2] The column header 'RealNVP(Dinh et al., 2016)' refers to the conventional multi-scale architecture; the table would be clearer if it explicitly named the factorization type (checkerboard and channel-wise) rather than only the model name.
- [Appendix A] Appendix A says that more architecture details will be given in a source code release, but no code is provided; please include code or detailed hyperparameters to ensure reproducibility.
Circularity Check
No circularity: the log-det heuristic is a data-dependent architectural prior, and the reported bits/dim gains come from fresh training runs against an external RealNVP baseline.
full rationale
Walked the derivation chain of Section 3. Equation (4) decomposes log-likelihood into a latent-density term plus the sum of per-layer log-det terms, and Equation (9) defines the per-dimension cumulative log-det tensor used as the factorization heuristic. This is not a fitted parameter later relabeled as a prediction: the mask is computed once from a pre-trained no-multi-scale model and frozen before the final LCMA training run, so the bits/dim values in Table 1 are measured on a model trained with that fixed architectural choice, not read off from the heuristic itself. The comparisons are made against Dinh et al.'s RealNVP baseline and against ablation variants, which are external or contrasting configurations. The proposition that high-log-det dimensions should receive more flow layers is a heuristic assumption, not a consequence of the likelihood identity; hence the reported gains are not forced by construction. The main weakness, that transfer of the log-det ranking from a no-multi-scale model to the LCMA architecture is not validated for stability across pretraining runs or random seeds, is a robustness and reproducibility concern rather than a circularity. There are no load-bearing self-citations: the cited prior work is used as baseline or background material.
Assumptions & free parameters
free parameters (2)
- Factor split ratio at each scaling step =
1:1 (half of dimensions gaussianized early, half passed on)
- Number of scaling stages per dataset =
CIFAR-10: 1; ImageNet 32: 3; ImageNet 64 and CelebA: 4
assumptions (4)
- standard math Change-of-variables formula for invertible flows (Eq. 2 and Eq. 4)
- domain assumption Per-dimension log-det score is a valid proxy for per-dimension importance
- domain assumption Log-det rankings transfer from a pre-trained no-multiscale model to the final multiscale architecture
- domain assumption Local max/min pooling of the log-det tensor preserves enough spatial structure for good samples
Cite this review
Pith. "Pith review of Likelihood Contribution based Multi-scale Architecture for Generative Flows." pith.science (2026). https://pith.science/paper/JO2FEZP3
@misc{pith2026190801686,
author = {Pith},
title = {Pith review of: Likelihood Contribution based Multi-scale Architecture for Generative Flows},
year = {2026},
howpublished = {\url{https://pith.science/paper/JO2FEZP3}},
note = {Machine review of arXiv:1908.01686}
}
read the original abstract
Deep generative modeling using flows has gained popularity owing to the tractable exact log-likelihood estimation with efficient training and synthesis process. However, flow models suffer from the challenge of having high dimensional latent space, the same in dimension as the input space. An effective solution to the above challenge as proposed by Dinh et al. (2016) is a multi-scale architecture, which is based on iterative early factorization of a part of the total dimensions at regular intervals. Prior works on generative flow models involving a multi-scale architecture perform the dimension factorization based on static masking. We propose a novel multi-scale architecture that performs data-dependent factorization to decide which dimensions should pass through more flow layers. To facilitate the same, we introduce a heuristic based on the contribution of each dimension to the total log-likelihood which encodes the importance of the dimensions. Our proposed heuristic is readily obtained as part of the flow training process, enabling the versatile implementation of our likelihood contribution based multi-scale architecture for generic flow models. We present such implementations for several state-of-the-art flow models and demonstrate improvements in log-likelihood score and sampling quality on standard image benchmarks. We also conduct ablation studies to compare the proposed method with other options for dimension factorization.
Figures
Figures from the paper (6 more)
Forward citations
Cited by 1 Pith paper
-
Normalizing Flows: An Introduction and Review of Current Methods
A survey that organizes normalizing flow methods into a taxonomy and reviews their mathematical foundations, reported performance, and open problems.
Reference graph
Works this paper leans on
-
[1]
Semi- conditional normalizing flows for semi-supervised learning
Andrei Atanov, Alexandra V olokhova, Arsenii Ashukha, Ivan Sosnovik, and Dmitry Vetrov. Semi- conditional normalizing flows for semi-supervised learning. arXiv preprint arXiv:1905.00505,
arXiv 1905
-
[3]
In this section, for the ease of access, we summarize the experimental settings. Datasets: We perform experiments on four benchmarked image datasets: CIFAR-10 (Krizhevsky, 2009), Imagenet (Russakovsky et al.,
work page 2009
-
[7]
Generating sequences with recurrent neural networks
Alex Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850,
-
[9]
Emerging Convolutions for Generative Normalizing Flows
Emiel Hoogeboom, Rianne van den Berg, and Max Welling. Emerging convolutions for generative normalizing flows. arXiv preprint arXiv:1901.11137,
work page Pith review arXiv 1901
-
[10]
A style-based generator architecture for generative adversarial networks
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. arXiv preprint arXiv:1812.04948,
-
[12]
Auto-encoding variational bayes
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114,
-
[14]
Semi-supervised learning with generative adversarial networks
Augustus Odena. Semi-supervised learning with generative adversarial networks. arXiv preprint arXiv:1606.01583,
-
[15]
Conditional image generation with pixelcnn decoders
Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelcnn decoders. In Advances in neural information processing systems, pp. 4790–4798, 2016a. Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. arXiv preprint arXiv:1601.06759, 2016b. Scott...
Show all 20 references
-
[17]
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P
URL http://arxiv.org/abs/1409.0575. Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P. Kingma. Pixelcnn++: Improving the pixel- cnn with discretized logistic mixture likelihood and other modifications. CoRR, abs/1701.05517,
-
[18]
Benigno Uria, Iain Murray, and Hugo Larochelle
URL http://arxiv.org/abs/1701.05517. Benigno Uria, Iain Murray, and Hugo Larochelle. Rnade: The real-valued neural autoregressive density-estimator. In Advances in Neural Information Processing Systems, pp. 2175–2183,
-
[19]
10 A E XPERIMENTAL SETTINGS For direct comparison with Dinh et al. (2016), data pre-processing, optimizer parameters as well as flow architectural details (coupling layers, residual blocks) are kept the same, except that the factorization of dimensions at each flow layer is perf...
2016
-
[21]
Pre-processing: For CelebA, we take a central crop of 148× 148 then resize it to 64×
(downsampled to 32× 32 and 64× 64), and CelebFaces Attributes (CelebA) (Liu et al., 2015). Pre-processing: For CelebA, we take a central crop of 148× 148 then resize it to 64×
2015
-
[64]
The sample allocation for training and validation were done as per the official allocation for the datasets
For dequantization of images (whose values lies in [0, 256]D), the data is transformed to logit(α + (1− α)⊙ x 256 ), whereα = 0.05. The sample allocation for training and validation were done as per the official allocation for the datasets. Flow model architecture: We use affine...
2016
-
[1995]
doi: 10.1162/neco.1995.7.6.1129
ISSN 0899-7667. doi: 10.1162/neco.1995.7.6.1129. Xi Chen, Nikhil Mishra, Mostafa Rohaninejad, and Pieter Abbeel. Pixelsnail: An improved autore- gressive generative model. arXiv preprint arXiv:1712.09763,
1995 arXiv
-
[2009]
Videoflow: A flow-based generative model for video
Manoj Kumar, Mohammad Babaeizadeh, Dumitru Erhan, Chelsea Finn, Sergey Levine, Laurent Dinh, and Durk Kingma. Videoflow: A flow-based generative model for video. arXiv preprint arXiv:1903.01434,
1903 arXiv
-
[2013]
Flow++: Improving flow- based generative models with variational dequantization and architecture design
Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, and Pieter Abbeel. Flow++: Improving flow- based generative models with variational dequantization and architecture design. arXiv preprint arXiv:1902.00275,
1902 arXiv
-
[2015]
Nice: Non-linear independent components estimation
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516,
-
[2016]
Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios
URL http://arxiv.org/abs/1605.08803. Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. Cubic-spline flows. arXiv preprint arXiv:1906.02145,
1906 arXiv
-
[2018]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,
-
[2019]
Gansynth: Adversarial neural audio synthesis
Jesse Engel, Kumar Krishna Agrawal, Shuo Chen, Ishaan Gulrajani, Chris Donahue, and Adam Roberts. Gansynth: Adversarial neural audio synthesis. arXiv preprint arXiv:1902.08710,
1902 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.