REVIEW 8 cited by
Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In deep latent Gaussian models, the latent variable is generated by a time-inhomogeneous Markov chain, where at each time step we pass the current state through a parametric nonlinear map, such as a feedforward neural net, and add a small independent Gaussian perturbation. This work considers the diffusion limit of such models, where the number of layers tends to infinity, while the step size and the noise variance tend to zero. The limiting latent object is an It\^o diffusion process that solves a stochastic differential equation (SDE) whose drift and diffusion coefficient are implemented by neural nets. We develop a variational inference framework for these \textit{neural SDEs} via stochastic automatic differentiation in Wiener space, where the variational approximations to the posterior are obtained by Girsanov (mean-shift) transformation of the standard Wiener process and the computation of gradients is based on the theory of stochastic flows. This permits the use of black-box SDE solvers and automatic differentiation for end-to-end inference. Experimental results with synthetic data are provided.
Forward citations
Cited by 8 Pith papers
-
Pathwise Learning of Stochastic Dynamical Systems with Partial Observations
A pathwise Zakai-equation control formulation is used to train conditional neural SDEs that amortize nonlinear filtering of partially observed stochastic dynamics.
-
Data-to-Energy Stochastic Dynamics
A new data-to-energy iterative proportional fitting algorithm trains Schrödinger bridges when endpoint distributions are known only through unnormalised densities.
-
Efficient Training of Neural SDEs Using Stochastic Optimal Control
A hybrid variational inference method for neural SDEs that initializes the control function with the closed-form optimal linear control, then learns a neural residual.
-
Fourier Neural Operators for Non-Markovian Processes:Approximation Theorems and Experiments
A mirror-padded Fourier neural operator can approximate, with proven error bounds, solution maps of path-dependent SDEs and Lipschitz transformations of fractional Brownian motion.
-
Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis
The authors derive information-theoretic generalization bounds for VAEs and diffusion models that expose a trade-off in the diffusion time T, and propose using the computable bound to select T and regularize training.
-
Numerical PDE solvers outperform neural PDE solvers
DeepFDM, a differentiable finite-difference solver that learns PDE coefficients, achieves 10 to 40 times lower error than FNO, U-Net and ResNet on five scalar time-dependent PDEs in one to three dimensions.
-
Quantum-Inspired Differentiable Integral Neural Networks (QIDINNs): A Feynman-Based Architecture for Continuous Learning Over Streaming Data
QIDINNs define parameter updates as kernel-weighted integrals of past gradients, essentially continuous-time momentum, and claim superior streaming learning without providing the backpropagation cost they claim to avoid.
-
Deep Neural Networks Inspired by Differential Equations
A review of differential-equation-inspired neural networks that compiles known results into a taxonomy, with no new experiments or theory.
Discussion (0). Continue with ORCID to comment.