FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
read the original abstract
A promising class of generative models maps points from a simple distribution to a complex distribution through an invertible neural network. Likelihood-based training of these models requires restricting their architectures to allow cheap computation of Jacobian determinants. Alternatively, the Jacobian trace can be used if the transformation is specified by an ordinary differential equation. In this paper, we use Hutchinson's trace estimator to give a scalable unbiased estimate of the log-density. The result is a continuous-time invertible generative model with unbiased density estimation and one-pass sampling, while allowing unrestricted neural network architectures. We demonstrate our approach on high-dimensional density estimation, image generation, and variational inference, achieving the state-of-the-art among exact likelihood methods with efficient sampling.
This paper has not been read by Pith yet.
Forward citations
Cited by 22 Pith papers
-
Denoising Diffusion Implicit Models
DDIMs construct non-Markovian diffusion processes that share DDPM training objectives but allow much faster reverse sampling, demonstrated empirically at 10-50x wall-clock speedup.
-
Kurtosis-Guided Denoising Score Matching for Tabular Anomaly Detection
K-DSM uses per-feature kurtosis to set noise scales in DSM, enabling effective single-scale anomaly detection on tabular benchmarks in both semi-supervised and unsupervised settings.
-
Generative Modeling with Orbit-Space Particle Flow Matching
OGPP is a particle flow-matching method using orbit-space canonicalization and geometric paths that achieves lower error and fewer steps than prior approaches on 3D benchmarks.
-
Differentiable free energy surface: a variational approach to directly observing rare events using generative deep-learning models
VaFES constructs a latent space from reversible collective variables and variationally optimizes a tractable-density generative model to produce a continuous free energy surface from which rare events are directly sampled.
-
Training-Free Refinement of Flow Matching with Divergence-based Sampling
Flow Divergence Sampler refines flow matching by computing velocity field divergence to correct ambiguous intermediate states during inference, improving fidelity in text-to-image and inverse problem tasks.
-
Constructive conditional normalizing flows
Explicit constructions approximate diffeomorphisms and pushforward measures via continuity equation flows with perceptron velocity fields of piecewise constant weights, using polar-like decompositions and probabilisti...
-
Application of deep neural networks for computing the renormalization group flow of the two-dimensional phi^4 field theory
RGFlow uses flow-based neural networks to learn bijective real-space RG transformations for the 2D phi^4 theory, identifying a Wilson-Fisher-like critical point and estimating the correlation length exponent.
-
pop-cosmos: Star formation over 12 Gyr from generative modelling of a deep infrared-selected galaxy catalogue
A score-based diffusion generative model on deep infrared galaxy photometry yields a star formation rate density peaking at z=1.3 and shows distinct non-parametric star formation histories plus AGN activity peaking du...
-
Progressive Distillation for Fast Sampling of Diffusion Models
Progressive distillation halves sampling steps repeatedly in diffusion models, reaching 4 steps with FID 3.0 on CIFAR-10 from 8192-step samplers.
-
Conservative Flows: A New Paradigm of Generative Models
Conservative flows generate by running probability-preserving stochastic dynamics initialized at data points rather than noise, using corrected Langevin or predictor-corrector mechanisms on top of any pretrained flow ...
-
Quantum Dynamics via Score Matching on Bohmian Trajectories
Neural networks learn the score of the probability density on Bohmian trajectories to recover exact Schrödinger dynamics via self-consistent minimization for nodeless wave functions, demonstrated on double-well splitt...
-
Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing
RC-GRPO-Editing constrains GRPO exploration to editing regions via localized noise and attention rewards, improving instruction adherence and non-target preservation in flow-based image editing.
-
MIOFlow 2.0: A unified framework for inferring cellular stochastic dynamics from single cell and spatial transcriptomics data
MIOFlow 2.0 learns stochastic cellular trajectories from transcriptomics data via neural SDEs, unbalanced optimal transport for growth, and a joint latent space unifying gene expression with spatial features.
-
Saving Foundation Flow-Matching Priors for Inverse Problems
FMPlug adapts foundation flow-matching models into practical priors for inverse problems by combining instance-guided warm-start with sharp Gaussianity regularization, showing superior results on image restoration and...
-
A Differentiable Programming System to Bridge Machine Learning and Scientific Computing
Zygote is a differentiable programming system in Julia that supports gradients for nearly all language constructs while generating high-performance code without user refactoring.
-
From Snapshots to Trajectories: Learning Single-Cell Gene Expression Dynamics via Conditional Flow Matching
scFM learns bidirectional velocity fields from entropically regularized OT couplings between snapshots, with added alignment and regularization to reduce drift in long-horizon predictions of single-cell trajectories.
-
Divergence-Suppressing Couplings for Rectified Flow
Divergence-suppressing couplings attenuate the divergent part of the velocity field when generating training couplings for Rectified Flow, yielding straighter paths and better generation quality at no extra inference cost.
-
Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies
Unsupervised behavioral mode discovery combined with mutual information rewards enables RL fine-tuning of multimodal generative policies that achieves higher success rates without losing action diversity.
-
The Score-Difference Flow for Implicit Generative Modeling
Score-difference flow reduces KL divergence between distributions and is formally equivalent to denoising diffusion models and a hidden subproblem in optimal GAN training under stated conditions.
-
A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
Diffusion, score-based, and flow matching models are unified as instances of learning time-dependent vector fields inducing marginal distributions governed by continuity and Fokker-Planck equations.
-
Flemme: A Flexible and Modular Learning Platform for Medical Images
Flemme is a modular platform separating encoders (conv/transformer/SSM) from encoder-decoder architectures for medical images, with a hierarchical pyramid loss yielding reported average gains of 5.6% Dice and 5.57% PSNR.
-
Flow Matching Guide and Code
Flow Matching is a generative modeling framework with mathematical foundations, design choices, extensions, and open-source PyTorch code for applications like image and text generation.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.