Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization

David I. Inouye; Jim Lim; Ziyu Gong

REVIEW 3 major objections 7 minor 59 references

Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Distribution matching only needs the prior's score function, not its density—and a CLIP-space Gromov-Wasserstein regularizer preserves geometry.

desk verdict A useful, sensible idea for non-adversarial distribution matching with score-based priors, but the unbiasedness claim is stronger than the implemented estimator actually supports. read the letter →

arxiv 2506.14607 v1 pith:KI7Y2PZT submitted 2025-06-17 cs.LG cs.CY

Ziyu Gong , Jim Lim , David I. Inouye This is my paper · ORCID

classification cs.LGcs.CY

keywords score-basedpriorsdistributionmatchingdenoisingscorefunctionsubstitutionvariationalalignmentupperboundGromov-Wassersteinregularizationsemanticpreservationdomainadaptation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that likelihood-based distribution matching can be trained using only the score function of the prior—the gradient of its log-density—instead of the prior density itself. It introduces the Score Function Substitution (SFS) trick, which rewrites the cross-entropy term in the variational alignment upper bound so that encoder and decoder gradients are computed from a detached score evaluation and remain unbiased. The prior is then updated by denoising score matching in alternation, which tightens the bound without requiring sampling, normalization, or a tractable density. The authors argue this both avoids the bias of fixed priors like Gaussians in VAEs and the instability of adversarial or diffusion-based prior training like LSGM, while enabling geometry-preserving Gromov-Wasserstein regularization in a CLIP semantic space. If correct, expressive priors become practical for fair classification, domain adaptation, and domain translation without adversarial training.

What carries the argument

The machinery is the Score Function Substitution (SFS) trick: the identity in Proposition 3.1 that replaces the log-prior evaluation in the cross-entropy term with a dot product against the prior's score, with the score's input detached from the encoder's computational graph. This single substitution is load-bearing because it keeps the encoder/decoder gradient exact while eliminating density evaluation, normalization, prior sampling, and the Jacobian of the diffusion network; it is also what licenses the alternating optimization, since the prior's own parameters are updated separately by denoising score matching (Eq. 9). Around this core sit the VAUB objective being optimized, the score network parameterizing the prior, and the Gromov-Wasserstein loss (GW-EP with Euclidean distances, GW-SP with CLIP semantic distances) that injects geometry preservation.

What would settle it

Pick a tractable prior and posterior, such as Gaussian mixtures, train a score model to convergence at several noise levels, and compare the exact cross-entropy gradient $\nabla_\theta \mathbb{E}[-\log Q(z)]$ against the SFS gradient computed with the same samples; if the relative error does not shrink toward zero as $\sigma_{\min}$ decreases, or if SAUB's objective drifts from VAUB's as $\sigma_{\min} \to 0$, the unbiasedness claim collapses.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that gradient-based distribution matching through the variational alignment upper bound (VAUB) needs only the prior's score, not its density. Its Proposition 3.1 gives the SFS identity: $\nabla_\theta \mathbb{E}_{z_\theta \sim q_\theta(z|x)}[-\log Q_\psi(z_\theta)] = \nabla_\theta \mathbb{E}[- (\nabla_{\bar z} \log Q_\psi(\bar z)|_{\bar z=z_\theta})^\top z_\theta]$, where the score evaluation is treated as constant with respect to $\theta$. This yields the SAUB objective, whose encoder and decoder gradients coincide with VAUB's but which never calls the density or backpropagates through a diffusion network. Because the substitution does not preserve prior-parameter gradients, the paper alternates: it updates the encoder and decoder with SAUB, then updates the score-based prior with denoising score matching on the encoder's marginal posterior, making the bound tight when the lower-level problem is solved. It further claims that adding a Gromov-Wasserstein structural-preservation loss with CLIP embeddings as the latent metric—GW-SP—improves semantic preservation and downstream performance, reporting accuracy gains on MNIST-to-USPS adaptation, better accuracy at equal demographic parity on Adult, and better image retrieval on CelebA translation.

Load-bearing premise

The encoder and decoder gradients are unbiased only if the denoising score model at the smallest chosen noise level accurately approximates the clean score of the prior; the paper's stability experiment shows both SAUB and LSGM degrade at $\sigma_{\min}=0.001$, so this approximation quality is the load-bearing premise.

Editorial extensions

If this is right

Expressive, non-normalized priors become usable in VAE-style distribution matching, so the latent prior can be as complex as the encoder's marginal posterior without flow constraints or adversarial training.
Small-noise score models—which give better clean-score approximations—no longer destabilize training the way diffusion-Jacobian backpropagation does in LSGM, so practitioners can safely operate near $\sigma_{\min} \approx 0.01$.
Geometry-preserving regularization moves from pixel and Euclidean spaces to semantic embedding spaces, which should transfer to any task where a pretrained embedding defines meaningful distances.
Alternating denoising score matching makes the variational upper bound tight when the score model converges, giving likelihood-based distribution matching a principled non-adversarial way to track distribution shift.
Memory and time savings from omitting the diffusion Jacobian grow with latent dimension, making score-based priors more favorable for high-dimensional representations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The SFS identity is essentially a pathwise gradient estimator for a log-prior cost, so it should port to any latent-variable objective with a cross-entropy-style term—hierarchical VAEs, regularized autoencoders, or causal representation learning—whenever only the score is available.
Because the paper updates the prior only through denoising score matching, it never optimizes the prior's likelihood directly; a testable extension would be a stochastic estimator for the prior parameters that closes the remaining gap between SAUB and the original VAUB.
Replacing CLIP with another pretrained distance, such as a self-supervised image feature or a medical embedding, is a direct recipe for semantic preservation in domains where CLIP is weak.
If the clean-score approximation at the minimum noise level is the bottleneck, an annealed noise schedule that starts large and decreases $\sigma_{\min}$ during training should further tighten the bound while avoiding the $\sigma_{\min}=0.001$ instability the paper documents.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You'll want to know two things about this one. First, the core idea is real: train a score-based prior with denoising score matching, then use only the score (not the density) to compute encoder gradients inside the VAUB objective. That is a natural and practical extension of LSGM to non-adversarial distribution matching, and the detached-score trick does avoid the Jacobian instability that plagues LSGM at low noise levels. The CLIP-based Gromov-Wasserstein regularizer is also a pragmatic way to inject semantic structure, and the paper shows it helps in several tasks. Second, the paper's central claim of "exact" or "unbiased" gradients is over the top. The implemented estimator evaluates the score at z + σmin ε and detaches the score network; it is an unbiased gradient estimator only if the score network equals the clean score at that point. In practice, DSM trains toward the score of the noisy marginal, so there is an O(σmin²) bias plus score approximation error, and the paper gives no bound or correction. Its own stability experiment (Fig. 1) shows both SAUB and LSGM degrade at σmin = 0.001, and the main experiments do not report which σmin was used. That is a genuine soft spot, but not a fatal one: for small enough σmin the bias is likely negligible, and the method clearly stabilizes training compared to LSGM. The other soft spots are smaller. Table 1 has no error bars and the VAUB baseline (40.7%) looks far below what one would expect, making the gains look larger than they are. The GW-SP results, especially on MNIST-USPS, rely on powerful CLIP features, so comparisons with non-CLIP baselines are not apples-to-apples; that should be stated more honestly. Appendix A's proof also has sign inconsistencies and mixes exact equality with the detached estimator; a careful revision would clean that up. The self-referential prior-posterior loop is standard alternating optimization, not circular reasoning, so the circularity burden is low. This is a paper for people working on domain adaptation, fairness, and score-based latent models. I would send it to peer review — it clearly deserves a serious referee — but I would ask for the empirical gaps to be filled: error bars, the σmin setting for every result, a bias analysis or at least a sensitivity sweep, and a discussion of what CLIP is contributing. With those, the conditional acceptance becomes a straightforward accept.

Referee Report

3 major / 7 minor

Summary. This paper proposes Score Function Substitution (SFS) for likelihood-based distribution matching in the VAUB framework. Instead of modeling the domain-invariant prior density explicitly, the prior is represented by its score function, trained by denoising score matching. The authors claim (Prop. 3.1) that the encoder-gradient of the cross-entropy term can be expressed using a detached score evaluation, so encoder/decoder updates need not backpropagate through the diffusion score network. A bilevel alternating algorithm (SAUB) alternates VAUB-style encoder/decoder updates with DSM prior updates. A Gromov-Wasserstein regularizer, computed in Euclidean or CLIP semantic space, is added to preserve geometry. Experiments cover synthetic D-shape separation, Adult fairness, MNIST-USPS domain adaptation, and CelebA domain translation, comparing with Gaussian/MoG/VampPrior/LSGM priors and VAUB/adversarial baselines.

Significance. If the core identity and empirical results hold, SFS is a useful contribution: it sidesteps density normalization and diffusion Jacobians, making expressive learned priors practical for distribution matching, and the CLIP-space GW regularizer gives a sensible way to inject semantics. The paper ships code, includes a stability comparison against LSGM, reports computational efficiency, and uses separate CLIP models for training and evaluation in the retrieval metric. However, the exactness/unbiasedness claim is stronger than what is demonstrated: with a finite sigma_min and approximate DSM scores, the SAUB gradient is biased relative to the VAUB objective, and the paper's own stability experiment shows sensitivity at small sigma_min. The missing error bars and anomalous VAUB baseline in Table 1 also weaken the empirical conclusions as printed.

major comments (3)

[Sections 3.1-3.2, Eqs. (6)-(9), Alg. 1] The SFS identity in Proposition 3.1 is exact only for the clean score, i.e., S_psi(z) = grad_z log Q_psi(z); the implemented method evaluates S_psi at z* = z + sigma_min * epsilon on detached samples (Alg. 1, line 7) and trains S_psi with the DSM objective (Eq. 9) on the noisy marginal posterior. For any sigma_min > 0 and any imperfect DSM fit, the SAUB gradient is a biased estimate of the VAUB cross-entropy gradient, and no bias bound or bias correction is provided. The paper's own Fig. 1 shows that both SAUB and LSGM degrade at sigma_min = 0.001, so the choice of sigma_min is empirically consequential, yet Tables 1 and 2 do not report the sigma_min used. Moreover, the stability study in Section 3.3 freezes the prior and trains only the encoder/decoder, so it does not test the full alternating SAUB loop. Please add a quantitative bias analysis (at minimum an O(sigma_min^2) statement with explicit smoothness assumptions), report sigma_min for every experiment, and revise the claims of exactness/unbiasedness to approximate guarantees.
[Appendix A, Eqs. (19)-(20)] As printed, the proof of Proposition 3.1 has a sign inconsistency. Eq. (19) states E_epsilon grad_theta [ (grad_zbar log Q_psi(zbar)|_{zbar=g_theta(epsilon)})^T g_theta(epsilon) ], while Eq. (20) states grad_theta E_epsilon [ -(grad_zbar log Q_psi(zbar)|_{zbar=g_theta(epsilon)})^T g_theta(epsilon) ]. If the score is treated as constant with respect to theta, the derivative of -a^T g_theta is -J_g^T a, so the two displayed expressions are negatives of each other unless the score is zero. The intended identity is correct by a standard pathwise-derivative argument, but the proof as written must be corrected (either the missing minus sign in Eq. (19) or an explicit statement that the score is frozen before differentiation). Since Proposition 3.1 is the central theoretical claim, this is load-bearing despite being local.
[Table 1] Table 1 reports no error bars or number of seeds, and the VAUB baseline entries (40.7% for MNIST to USPS, 45.3% for USPS to MNIST) are far below what one would expect from a VAE-based distribution-matching method on this task, including the cited VAUB work. If these numbers are accurate, the architecture and hyperparameter choices that produce them should be described; if they are not, the comparison against the proposed method is not meaningful. Please add multi-seed standard deviations for all methods and reconcile the VAUB numbers with the prior literature or explain the discrepancy (e.g., different encoder, latent dimension, or training budget).

minor comments (7)

[Eq. (5)] Equation (5) contains LaTeX artifacts such as '/bracehtipupleft/bracehtipdownright' that should be cleaned before publication.
[Section 3.2] In Section 3.2, 'it's density' should be 'its density'.
[Appendix F, Eq. (25)] The notation z* is used inconsistently: earlier z* = z + sigma_min * epsilon, but Eq. (25) writes z* = z. Please align the definitions.
[Table 2] Table 2 reports standard deviations for retrieval metrics but not for SSIM or LPIPS; please report repeated-run variability for all metrics or state explicitly why it is omitted.
[Fig. 4] Figure 4 shows a fairness-accuracy trade-off without error bars or the number of runs; please add them or state that the curves are single runs.
[Section 5.2] The sentence 'our method not only retains the advantages of the SAUB method' is confusing because SAUB is introduced in this paper; this should read 'VAUB' or be rephrased.
[References] References Song et al. 2021a and 2021b are duplicate entries for the same paper; consolidate them.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SFS is a chain-rule identity, the DSM/SAUB alternation is an explicit approximation, and the VAUB self-citation is to published prior work.

full rationale

No circular step found. Proposition 3.1 (Eq. 6) is a reparameterization/chain-rule identity: both sides equal E[(∂zθ/∂θ)ᵀ(−∇_z log Qψ(zθ))], so the claim that encoder gradients can be computed with the score evaluated as a detached constant is a mathematical equivalence, not an input defined in terms of the output. The implemented upper-level objective (Eq. 8) replaces the exact score by Sψ(z*, σmin) with z* = z + σmin ε, and Eq. 9 trains Sψ by denoising score matching; this is an explicit approximation with acknowledged finite-σmin error, and the paper's Section 3.3 stability study measures the behavior of that approximation rather than silently assuming it away. The lower-level/upper-level alternation is a standard EM-style fixed-point scheme: the prior is learned to match the encoder's marginal posterior, and the encoder is then trained against that prior; this is an optimization loop, not a derivation that assumes its conclusion. The only self-citation is the VAUB objective of Gong et al. (2024), a published, externally checkable bound; the present paper's claims concern the score-based training algorithm and its experimental evaluation, not the validity of VAUB itself. Finite-σmin score error is a correctness and robustness concern, not a circularity, and no fitted constant is presented as a prediction.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The method rests on standard score matching and reparameterization machinery plus the self-authored VAUB bound. The main hand-chosen quantities are β, λGW, noise levels, and the inner-loop length L, none of which are fully reported. The CLIP metric assumption is a strong external input, and the normalizability of the implicit score-based prior is left unverified.

free parameters (5)

β = 0.1 reported in stability experiment; not reported across all tasks
VAUB regularization coefficient in Eq. 5 that controls mutual information between latent and data. It is chosen by hand and directly shapes the optimized objective.
λGW = not reported
Weight on the Gromov-Wasserstein loss in Eq. 10. No values are given for any experiment, so the strength of the structural preservation term is unstated.
σmin and σmax = σmax = 1; σmin values 0.001, 0.01, 0.1, 0.2 in stability study; not reported elsewhere
Noise levels for the denoising score matching objective in Eq. 9. The clean-score approximation used by SFS depends directly on σmin.
L = not reported
Number of score model update loops per encoder step in Algorithm 1. It controls the alternating optimization balance and is not specified.
Weight initialization scale = not reported
Appendix C states that grid search was used to find an optimal weight scale, meaning the results depend on manually tuned initialization.

assumptions (6)

domain assumption The VAUB objective in Eq. 1 is a valid upper bound on a Jensen-Shannon divergence up to a constant.
The whole method optimizes this bound, and the paper inherits it from the self-authored Gong et al. 2024 work without re-deriving it.
standard math Denoising score matching at sufficiently small noise recovers the clean score of the data distribution.
Used in Eq. 9 and in the claim that σ0 ≈ 0 gives the clean score function of the marginal posterior.
domain assumption The learned score network corresponds to a valid normalized density Qψ whose score is Sψ.
The SFS gradient substitutes ∇ log Q with Sψ, which is meaningful only if Qψ is a normalizable density. The paper does not ensure or verify this.
standard math The reparameterization trick applies to the encoder distributions used in all experiments.
The proof of Proposition 3.1 explicitly relies on zθ = gθ(ε), which requires reparameterizable posteriors.
domain assumption Simple alternating optimization converges to a good solution of the bi-level problem in Eqs. 8 and 9.
The paper selects alternating updates for simplicity and cites empirical success, but provides no convergence guarantee.
domain assumption CLIP embeddings define a valid metric for the Gromov-Wasserstein loss in Eq. 12.
GW-SP uses distances in a pretrained CLIP space and assumes those distances align with task-relevant semantic structure.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization." pith.science (2026). https://pith.science/paper/KI7Y2PZT

@misc{pith2026250614607,
  author       = {Pith},
  title        = {Pith review of: Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KI7Y2PZT}},
  note         = {Machine review of arXiv:2506.14607}
}

read the original abstract

Distribution matching (DM) is a versatile domain-invariant representation learning technique that has been applied to tasks such as fair classification, domain adaptation, and domain translation. Non-parametric DM methods struggle with scalability and adversarial DM approaches suffer from instability and mode collapse. While likelihood-based methods are a promising alternative, they often impose unnecessary biases through fixed priors or require explicit density models (e.g., flows) that can be challenging to train. We address this limitation by introducing a novel approach to training likelihood-based DM using expressive score-based prior distributions. Our key insight is that gradient-based DM training only requires the prior's score function -- not its density -- allowing us to train the prior via denoising score matching. This approach eliminates biases from fixed priors (e.g., in VAEs), enabling more effective use of geometry-preserving regularization, while avoiding the challenge of learning an explicit prior density model (e.g., a flow-based prior). Our method also demonstrates better stability and computational efficiency compared to other diffusion-based priors (e.g., LSGM). Furthermore, experiments demonstrate superior performance across multiple tasks, establishing our score-based method as a stable and effective approach to distribution matching. Source code available at https://github.com/inouye-lab/SAUB.

Figures

Figures reproduced from arXiv: 2506.14607 by the authors.

**Figure 1.** The reconstruction loss and negative log-likelihood are presented on a logarithmic scale for improved visualization. The experiment uses consistent hyperparameters (β = 0.1), an identical VAE architecture, and the same pretrained score model. of the data distribution, which has been empirically shown to be unstable at low noise levels (Poole et al., 2022). Conversely, our Score Function Substitution (SFS) trick eli… view at source ↗

**Figure 2.** The dataset consists of two domains: Domain 1 (left nested ’D-shaped’) and Domain 2 (right flipped ’D-shaped’). In each domain, the outer ’D’ corresponds to Label 1, and the inner ’D’ to Label 2. The shared latent spaces are visualized for models trained with varying data sizes (n = 20, 100, 500 samples) using Gaussian(Kingma et al., 2019), Mixture of Gaussians(Gong et al., 2024), Vampprior(Tomczak & Welling, 2018),… view at source ↗

**Figure 3.** [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (17 more)

**Figure 5.** Figure 5: All models use the same architecture. Refer to Appendix I for details on the neural network and CLIP model. Applying GW loss in the CLIP semantic space shows superior semantic preservation in both (a) and (b). The samples are selectively chosen to represent diverse v…

**Figure 6.** Figure 6: Target/Prior Distribution for the stability analysis in subsection 3.3. The prior distribution is the target distribution projected onto the Z-space. E.1. Results As shown in Figure [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

**Figure 6.** Figure 6: (a) MNIST to USPS [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: (a) MNIST to USPS (LSDA). (b) USPS to MNIST (LSDA). (c) UMAP with SP. (d) UMAP without SP. All labeled data is randomly selected from the source dataset and tested on the target dataset, with results averaged over 10 trials. Both (a) and (b) demonstrate that with SP lo…

**Figure 8.** Figure 8: Histogram of the pairwise distance between data samples within a class and between different classes for three datasets: MNIST, USPS, and CelebA. The amount of separation of two histogram is computed by using the AUROC score which being measured by a binary classifier …

**Figure 9.** Figure 9: Multi-domain adaptation: MNIST images rotated at various angles. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

**Figure 10.** Figure 10: These figures show the translated dataset, reconstructed dataset, as well as the latent space under sample size 20. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]

**Figure 11.** Figure 11: This figures show the translated dataset, reconstructed dataset, as well as the latent space under sample size 50. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]

**Figure 12.** Figure 12: This figures show the translated dataset, reconstructed dataset, as well as the latent space under sample size 100. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_12.png]

**Figure 13.** Figure 13: This figures show the translated dataset, reconstructed dataset, as well as the latent space under sample size 200. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_13.png]

**Figure 14.** Figure 14: This figures show the translated dataset, reconstructed dataset, as well as the latent space under sample size 500. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_14.png]

**Figure 15.** Figure 15: MNIST to USPS translated image trained with SP. L. FairFace Image Translation This experimental setting is conducted in a fully unsupervised manner without SP loss. We compare our proposed score-based prior (SAUB) with a multi-Gaussian-based learning prior (VAUB) to e…

**Figure 16.** Figure 16: In this experiment, both models are trained in an unsupervised manner (i.e., SAUB is trained without GW-SP loss). SAUB clearly exhibits superior semantic preservation in both (a) and (b), particularly with respect to features such as skin color, race, and age. Notably…

**Figure 17.** Figure 17: Random samples from the FairFace experiment using our method. Top three rows translate from male to female and the bottom three rows translate from female to male. First row is original, second is reconstructed, and third is translated. 28 [PITH_FULL_IMAGE:figures/fu…

**Figure 18.** Figure 18: Random Samples from Black to Blonde Hair Female 29 [PITH_FULL_IMAGE:figures/full_fig_p029_18.png]

**Figure 19.** Figure 19: Random Samples from Blonde to Black Hair Female 31 [PITH_FULL_IMAGE:figures/full_fig_p031_19.png]

**Figure 20.** Figure 20: Fairness-accuracy trade-off comparison across different methods with and without Gromov-Wasserstein regularization. The plot shows accuracy (x-axis) versus demographic parity gap ∆DP (y-axis) for VAUB, LSGM, and SAUB methods. Lower ∆DP values indicate better fairness …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 30 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A

Burgess, C. P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A. Understanding disentangling in -vae, 2018. URL https://arxiv.org/abs/1804.03599

arXiv 2018
[3]

A., Nekrashevich, M., Mokrov, P., Burnaev, E., and Korotin, A

Carrasco, X. A., Nekrashevich, M., Mokrov, P., Burnaev, E., and Korotin, A. Uncovering challenges of solving the continuous gromov-wasserstein problem, 2024. URL https://arxiv.org/abs/2303.05978

arXiv 2024
[4]

Learning flat latent manifolds with vaes

Chen, N., Klushyn, A., Ferroni, F., Bayer, J., and Van Der Smagt, P. Learning flat latent manifolds with vaes. arXiv preprint arXiv:2002.04881, 2020

arXiv 2002
[5]

Chen, R. T. Q., Li, X., Grosse, R., and Duvenaud, D. Isolating sources of disentanglement in variational autoencoders, 2019. URL https://arxiv.org/abs/1802.04942

arXiv 2019
[6]

Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems

Chen, T., Sun, Y., and Yin, W. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. Advances in Neural Information Processing Systems, 34: 0 25294--25307, 2021

2021
[7]

Cho, W., Gong, Z., and Inouye, D. I. Cooperative distribution alignment via jsd upper bound. In Neural Information Processing Systems (NeurIPS), dec 2022 a

work page 2022
[8]

Cho, W., Gong, Z., and Inouye, D. I. Cooperative Distribution Alignment via JSD Upper Bound , 2022 b . URL https://arxiv.org/abs/2207.02286

work page Pith review arXiv 2022

Show all 59 references

[9]

Data determines distributional robustness in contrastive language image pre-training (clip), 2022

Fang, A., Ilharco, G., Wortsman, M., Wan, Y., Shankar, V., Dave, A., and Schmidt, L. Data determines distributional robustness in contrastive language image pre-training (clip), 2022. URL https://arxiv.org/abs/2205.01397

2022 arXiv
[10]

and Ozdaglar, A

Farnia, F. and Ozdaglar, A. Do GAN s always have N ash equilibria? In III, H. D. and Singh, A. (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp.\ 3029--3039. PMLR, 13--18 Jul 2020. URL http...

2020
[11]

Interpolating between optimal transport and mmd using sinkhorn divergences

Feydy, J., S \'e journ \'e , T., Vialard, F.-X., Amari, S.-i., Trouv \'e , A., and Peyr \'e , G. Interpolating between optimal transport and mmd using sinkhorn divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.\ 2681--2690. PMLR, 2019

2019
[12]

Domain-adversarial training of neural networks

Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., and Lempitsky, V. Domain-adversarial training of neural networks. The journal of machine learning research, 17 0 (1): 0 2096--2030, 2016

2016
[13]

Gong, Z., Usman, B., Zhao, H., and Inouye, D. I. Towards practical non-adversarial distribution matching. In International Conference on Artificial Intelligence and Statistics (AISTATS), May 2024

2024
[14]

Generative adversarial nets

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. Advances in neural information processing systems, 27, 2014

2014
[15]

Isometric autoencoders

Gropp, A., Atzmon, M., and Lipman, Y. Isometric autoencoders. arXiv preprint arXiv:2006.09289, 2020

2006 arXiv
[16]

M., Dilkina, B., and Ver Steeg, G

Gupta, U., Ferber, A. M., Dilkina, B., and Ver Steeg, G. Controllable guarantees for fair outcomes via contrastive information estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.\ 7610--7619, 2021

2021
[17]

Isometric representation learning for disentangled latent space of diffusion models

Hahm, J., Lee, J., Kim, S., and Lee, J. Isometric representation learning for disentangled latent space of diffusion models. arXiv preprint arXiv:2407.11451, 2024

2024 arXiv
[18]

FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods , 2023

Han, X., Chi, J., Chen, Y., Wang, Q., Zhao, H., Zou, N., and Hu, X. FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods , 2023

2023
[19]

Hoffman, M. D. and Johnson, M. J. Elbo surgery: yet another way to carve up the variational evidence lower bound. In Workshop in Advances in Approximate Bayesian Inference, NIPS, volume 1, 2016

2016
[20]

When is unsupervised disentanglement possible? Advances in Neural Information Processing Systems, 34: 0 5150--5161, 2021

Horan, D., Richardson, E., and Weiss, Y. When is unsupervised disentanglement possible? Advances in Neural Information Processing Systems, 34: 0 5150--5161, 2021

2021
[21]

J., Duvenaud, D., Wiltschko, A

Johnson, M. J., Duvenaud, D., Wiltschko, A. B., Datta, S. R., and Adams, R. P. Composing graphical models with neural networks for structured representations and fast inference, 2017

2017
[22]

B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models, 2020. URL https://arxiv.org/abs/2001.08361

2020 arXiv
[23]

and Mnih, A

Kim, H. and Mnih, A. Disentangling by factorising. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.\ 2649--2658. PMLR, 10--15 Jul 2018. URL https://proceedings.mlr....

2018
[24]

P., Welling, M., et al

Kingma, D. P., Welling, M., et al. An introduction to variational autoencoders. Foundations and Trends in Machine Learning , 12 0 (4): 0 307--392, 2019

2019
[25]

The GAN landscape: Losses, architectures, regularization, and normalization, 2019

Kurach, K., Lucic, M., Zhai, X., Michalski, M., and Gelly, S. The GAN landscape: Losses, architectures, regularization, and normalization, 2019. URL https://openreview.net/forum?id=rkGG6s0qKQ

2019
[26]

Lee, Y., Yoon, S., Son, M., and Park, F. C. Regularized autoencoders for isometric representation learning. In International Conference on Learning Representations, 2022

2022
[27]

The variational fair autoencoder

Louizos, C., Swersky, K., Li, Y., Welling, M., and Zemel, R. The variational fair autoencoder. arXiv preprint arXiv:1511.00830, 2015

2015 arXiv
[28]

Are gans created equal? a large-scale study

Lucic, M., Kurach, K., Michalski, M., Gelly, S., and Bousquet, O. Are gans created equal? a large-scale study. In Advances in neural information processing systems, pp.\ 700--709, 2018

2018
[29]

Learning adversarially fair and transferable representations

Madras, D., Creager, E., Pitassi, T., and Zemel, R. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pp.\ 3384--3393. PMLR, 2018

2018
[30]

Adversarial autoencoders, 2016

Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., and Frey, B. Adversarial autoencoders, 2016

2016
[31]

dsprites: Disentanglement testing sprites dataset

Matthey, L., Higgins, I., Hassabis, D., and Lerchner, A. dsprites: Disentanglement testing sprites dataset. https://github.com/deepmind/dsprites-dataset/, 2017

2017
[32]

Invariant representations without adversarial training

Moyer, D., Gao, S., Brekelmans, R., Galstyan, A., and Ver Steeg, G. Invariant representations without adversarial training. Advances in Neural Information Processing Systems, 31, 2018

2018
[33]

Domain generalization via invariant feature representation

Muandet, K., Balduzzi, D., and Schölkopf, B. Domain generalization via invariant feature representation. In Dasgupta, S. and McAllester, D. (eds.), Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pp.\...

2013
[34]

Gromov-wasserstein autoencoders, 2023

Nakagawa, N., Togo, R., Ogawa, T., and Haseyama, M. Gromov-wasserstein autoencoders, 2023. URL https://arxiv.org/abs/2209.07007

2023 arXiv
[35]

and Patel, A

Nie, W. and Patel, A. B. Towards a better understanding and regularization of gan training dynamics. In Uncertainty in Artificial Intelligence, pp.\ 281--291. PMLR, 2020

2020
[36]

T., Rezende, D

Papamakarios, G., Nalisnick, E. T., Rezende, D. J., Mohamed, S., and Lakshminarayanan, B. Normalizing flows for probabilistic modeling and inference. J. Mach. Learn. Res., 22 0 (57): 0 1--64, 2021

2021
[37]

T., and Mildenhall, B

Poole, B., Jain, A., Barron, J. T., and Mildenhall, B. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022

2022 arXiv
[38]

W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision, 2021. URL https://arxiv.org/abs/2103.00020

2021 arXiv
[39]

Denoising diffusion implicit models, 2022

Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models, 2022. URL https://arxiv.org/abs/2010.02502

2022 arXiv
[40]

and Ermon, S

Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019

2019
[41]

Maximum likelihood training of score-based diffusion models

Song, Y., Durkan, C., Murray, I., and Ermon, S. Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems, 34: 0 1415--1428, 2021 a

2021
[42]

Maximum likelihood training of score-based diffusion models, 2021 b

Song, Y., Durkan, C., Murray, I., and Ermon, S. Maximum likelihood training of score-based diffusion models, 2021 b . URL https://arxiv.org/abs/2101.09258

2021 arXiv
[43]

P., Kumar, A., Ermon, S., and Poole, B

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021 c . URL https://openreview.net/forum?id=PxTIG12RRHS

2021
[44]

and Zhang, K

Spirtes, P. and Zhang, K. Causal discovery and inference: concepts and recent methodological advances. In Applied informatics, volume 3, pp.\ 1--28. Springer, 2016

2016
[45]

and Welling, M

Tomczak, J. and Welling, M. Vae with a vampprior. In International conference on artificial intelligence and statistics, pp.\ 1214--1223. PMLR, 2018

2018
[46]

Truong, T.-D., Chappa, N. V. S. R., Nguyen, X. B., Le, N., Dowling, A., and Luu, K. Otadapt: Optimal transport-based approach for unsupervised domain adaptation, 2022. URL https://arxiv.org/abs/2205.10738

2022 arXiv
[47]

Disentangled representation learning with the gromov-monge gap, 2024

Uscidda, T., Eyring, L., Roth, K., Theis, F., Akata, Z., and Cuturi, M. Disentangled representation learning with the gromov-monge gap, 2024. URL https://arxiv.org/abs/2407.07829

2024 arXiv
[48]

Score-based generative modeling in latent space

Vahdat, A., Kreis, K., and Kautz, J. Score-based generative modeling in latent space. Advances in neural information processing systems, 34: 0 11287--11302, 2021

2021
[49]

A connection between score matching and denoising autoencoders

Vincent, P. A connection between score matching and denoising autoencoders. Neural Computation, 23 0 (7): 0 1661--1674, 2011. doi:10.1162/NECO_a_00142

2011 doi
[50]

G., Xing, E., and Hu, Z

Wu, Y., Zhou, P., Wilson, A. G., Xing, E., and Hu, Z. Improving gan training with probability ratio clipping and sample reweighting. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp...

2020
[51]

An alternating optimization method for bilevel problems under the polyak- ojasiewicz condition

Xiao, Q., Lu, S., and Chen, T. An alternating optimization method for bilevel problems under the polyak- ojasiewicz condition. Advances in Neural Information Processing Systems, 36: 0 63847--63873, 2023

2023
[52]

Do vision-language pretrained models learn composable primitive concepts?, 2023

Yun, T., Bhalla, U., Pavlick, E., and Sun, C. Do vision-language pretrained models learn composable primitive concepts?, 2023. URL https://arxiv.org/abs/2203.17271

2023 arXiv
[53]

Central moment discrepancy (cmd) for domain-invariant representation learning

Zellinger, W., Grubinger, T., Lughofer, E., Natschl \"a ger, T., and Saminger-Platz, S. Central moment discrepancy (cmd) for domain-invariant representation learning. In International Conference on Learning Representations, 2017

2017
[54]

Learning fair representations

Zemel, R., Wu, Y., Swersky, K., Pitassi, T., and Dwork, C. Learning fair representations. In Dasgupta, S. and McAllester, D. (eds.), Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pp.\ 325--333, Atla...

2013
[55]

M., Costeira, J

Zhao, H., Zhang, S., Wu, G., Moura, J. M., Costeira, J. P., and Gordon, G. J. Adversarial multiple source domain adaptation. Advances in neural information processing systems, 31, 2018

2018
[56]

Zhao, H., Coston, A., Adel, T., and Gordon, G. J. Conditional learning of fair representations. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Hkekl0NFPr

2020
[57]

@esa (Ref

\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...
[58]

\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...
[59]

The 3rd Workshop On Tractable Probabilistic Modeling at ICML 2019, https://sites.google.com/view/icmltpm2019/home

@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

2020 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A

Burgess, C. P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A. Understanding disentangling in -vae, 2018. URL https://arxiv.org/abs/1804.03599

arXiv 2018

[3] [3]

A., Nekrashevich, M., Mokrov, P., Burnaev, E., and Korotin, A

Carrasco, X. A., Nekrashevich, M., Mokrov, P., Burnaev, E., and Korotin, A. Uncovering challenges of solving the continuous gromov-wasserstein problem, 2024. URL https://arxiv.org/abs/2303.05978

arXiv 2024

[4] [4]

Learning flat latent manifolds with vaes

Chen, N., Klushyn, A., Ferroni, F., Bayer, J., and Van Der Smagt, P. Learning flat latent manifolds with vaes. arXiv preprint arXiv:2002.04881, 2020

arXiv 2002

[5] [5]

Chen, R. T. Q., Li, X., Grosse, R., and Duvenaud, D. Isolating sources of disentanglement in variational autoencoders, 2019. URL https://arxiv.org/abs/1802.04942

arXiv 2019

[6] [6]

Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems

Chen, T., Sun, Y., and Yin, W. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. Advances in Neural Information Processing Systems, 34: 0 25294--25307, 2021

2021

[7] [7]

Cho, W., Gong, Z., and Inouye, D. I. Cooperative distribution alignment via jsd upper bound. In Neural Information Processing Systems (NeurIPS), dec 2022 a

work page 2022

[8] [8]

Cho, W., Gong, Z., and Inouye, D. I. Cooperative Distribution Alignment via JSD Upper Bound , 2022 b . URL https://arxiv.org/abs/2207.02286

work page Pith review arXiv 2022