GAIA introduces a geometry-adaptive integral autoencoder that unifies forward, boundary-value, and inverse PDE operator learning on arbitrary domains via geometry tokens and cross-attention.
mega hub Mixed citations
Adam: A Method for Stochastic Optimization
Mixed citation behavior. Most common role is method (50%).
2073
Pith papers citing it
Method
50% of classified citations
abstract
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.
hub tools
citation-role summary
method 117
background 97
other 9
baseline 8
dataset 2
citation-polarity summary
claims ledger
- abstract We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little
authors
mega hub controls
export citing contexts JSON
export graph JSON
export full bundle JSON
open full Pith review
annotated reader queued
Recognition alignment
counterfactual ablation
co-cited works
representative citing papers
ShardNet enforces non-convex polyhedral safety constraints in neural controllers by construction via a differentiable projection layer, achieving 100% verified safety and over 3x larger safe sets than prior methods on double integrator benchmarks.
The paper establishes the first finite-time convergence rate of 1/T^{2/13} for classical Adam (with bias correction, no extra steps) in nonsmooth nonconvex optimization under heavy-tailed noise with β1=β2.
Machine learning discovers a tube-seeding strategy for IBP reduction of Feynman integrals that scales linearly with numerator power, demonstrated on rank-20 2-loop 5-point integrals.
TAKO demonstrates real-time adversarial takeover of robotic diffusion policies via reusable universal patches on visual inputs, achieving 100% success in steering attacker-chosen trajectories across multiple tasks, encoders, and diffusion methods.
Forward gradient framework for PQCs unifies SPSA and parameter-shift as limits, introduces QUIVER adaptive optimizer with closed-form measurement allocation, and demonstrates efficient training of 60-qubit circuits on ECG5000 and MNIST.
Introduces REST-ASMR multimodal dataset of PPG, stimuli, and continuous annotations for ASMR research, validated with 97% responder rate, significant agreement, PPG deceleration, and BiLSTM achieving 75.51% frame-level accuracy under strict subject-video independent 4-fold CV.
PINNs are used to non-parametrically infer the neutron star EOS from NICER and pulsar data, producing M_max = 2.06 M_sun, R_1.4 = 12.85 km, and a reproducible speed-of-sound softening at 2-4 rho_0 consistent with quark-hadron crossover.
OpenVMR uses normalizing flow to detect out-of-distribution queries and performs moment retrieval only on in-distribution queries.
Derives geodesic ridge regularization and Riemannian Gibbs Process prior for feature-learning wide neural networks, generalizing kernel-regime results via function-space axiomatization.
Ensembits is the first tokenizer of protein conformational ensembles that outperforms static tokenizers on RMSF prediction and matches them on function and mutation tasks while using less pretraining data.
In the high-dimensional limit the spherical Boltzmann machine admits exact equations for training dynamics, Bayesian evidence, and cascades of phase transitions tied to mode alignment with data, which connect to generative phenomena including double descent and out-of-equilibrium biases.
Attention and LoRA regression losses induce Poincaré inequalities under mild regularization, so SGD-mimicking SDEs converge to minimizers with no assumptions on data or model size.
SLayerGen generates crystals invariant to any space or layer group via autoregressive lattice and Wyckoff sampling plus equivariant diffusion, achieving gains over bulk models on diperiodic materials after correcting a prior loss inconsistency for hexagonal groups.
3DSS is the first differentiable surface splatting renderer that recovers shape, spatially-varying BRDF materials, and HDR illumination from multi-view images via a coverage-based compositing model derived from reconstruction kernels.
PF-AGD is the first parameter-free deterministic accelerated first-order method with Õ(ε^{-5/3} log(1/ε)) complexity for smooth non-convex optimization.
STARE uses step-wise RL to attack multimodal models, achieving 68% higher attack success rate while revealing that adversarial optimization concentrates conceptual toxicity early and detail toxicity late in the generation trajectory.
Qvine uses vine copula-inspired quantum circuit structures to achieve linear or quadratic depth scaling for loading high-dimensional distributions with high approximation quality.
Neural networks optimized solely on crossing symmetry reconstruct CFT correlators from minimal input data to few-percent accuracy across generalized free fields, minimal models, Ising, N=4 SYM, and AdS diagrams.
MMGait provides a new multi-sensor gait dataset and OmniGait baseline to support single-modal, cross-modal, and unified multi-modal person identification from walking patterns.
Neural simulation-based inference on unbinned top-quark pair data at 13 TeV yields improved gluon PDF precision over traditional binned analyses while incorporating experimental and theoretical uncertainties.
Adam-HNAG is a splitting-based reformulation of Adam that yields the first convergence proof for Adam-type methods, including accelerated rates, in convex smooth optimization.
The paper introduces the CMCC-ReID task, constructs the SYSU-CMCC benchmark dataset, and proposes the PIA network with disentangling and prototype modules that outperforms prior methods on combined modality and clothing variations.
Quantitative Bayesian inference using a deep-learning emulator detects 0.018-0.020 M_sun of helium in the Type Ic supernova 2014L.
citing papers explorer
-
Canonical Regularisation of Wide Feature-Learning Neural Networks
Derives geodesic ridge regularization and Riemannian Gibbs Process prior for feature-learning wide neural networks, generalizing kernel-regime results via function-space axiomatization.
-
Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
A solvable hierarchical model with power-law feature strengths yields explicit power-law scaling of prediction error through sequential recovery of latent directions by a layer-wise spectral algorithm.
-
On Hallucinations in Inverse Problems: Fundamental Limits and Provable Assessment Methods
Hallucinations in inverse problem reconstructions are fundamental to ill-posedness, with necessary and sufficient conditions plus computable bounds depending only on the forward model.
-
Online Learning-to-Defer with Varying Experts
Presents first online L2D algorithm for multiclass classification with bandit feedback and varying experts, achieving O((n+n_e)T^{2/3}) regret generally and O((n+n_e)√T) under low noise.
-
Optimality of Sub-network Laplace Approximations: New Results and Methods
Sub-network Laplace approximations always underestimate full-model predictive variance, and two new gradient-based and greedy selection rules provide theoretically grounded improvements.
-
The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Models
Higher-variance classes are learned first in diffusion models; strong class imbalance reverses the order and imposes distinct delayed learning times on minority classes.
-
Tuning Derivatives for Causal Fairness in Machine Learning
New framework formalizes causal fairness for continuous protected attributes via path-specific derivatives and introduces a tuning algorithm for fair predictors.
-
Diffusion and Flow-based Copulas: Forgetting and Remembering Dependencies
Diffusion and flow processes forget dependencies to define valid copulas then learn to remember them for density estimation and sampling, outperforming prior copula methods on complex datasets.
-
Preconditioned Regularized Wasserstein Proximal Sampling
A preconditioned regularized Wasserstein proximal sampling algorithm is introduced for particle-based approximation of Gibbs distributions, featuring a PDE-derived kernel formulation and non-asymptotic convergence analysis for quadratic potentials.
-
Graph neural networks for residential location choice: connection to classical logit models
GNN-DCMs apply graph neural networks to discrete choice modeling, recovering nested logit and spatially correlated logit via message passing on utilities and demonstrating better predictive performance for residential location choices in Chicago.
-
Sequential Off-Policy Learning with Logarithmic Smoothing
Introduces logarithmic smoothing algorithms for sequential off-policy learning that match batch SOTA and outperform prior methods under sequential updates.
-
Mini-batch Estimation for Deep Cox Models: Statistical Foundations and Practical Guidance
Mini-batch SGD optimizes a different objective than full partial likelihood in Cox models, but the resulting mb-MPLE is still consistent with optimal rates for neural nets and asymptotic normality for linear models.
-
Variational Sequential Optimal Experimental Design using Reinforcement Learning
vsOED uses a variational one-point reward and RL policy optimization to provide a lower bound on expected information gain for sequential experimental design, supporting nuisance parameters, implicit likelihoods, and multiple design goals.
-
Importance Sparsification for Sinkhorn Algorithm
Spar-Sink applies importance sampling based on upper bounds of transport plans to create a sparse approximation of the Sinkhorn kernel, yielding consistent estimators for regularized OT and UOT with near-linear iteration cost.
-
Graph Attention Networks
Graph Attention Networks compute learnable attention coefficients over node neighborhoods to produce weighted feature aggregations, achieving state-of-the-art results on citation networks and inductive protein-protein interaction graphs.
-
Dynamic Prediction of Alternating Recurrent Events via Neural Network
Presents a neural network model with IPW pseudo-observations for dynamic prediction of alternating recurrent events, with simulation and medical resident mood application results.
-
Data Evolution by Wittgenstein's Rule Following
WRF generates new datasets by extrapolating structural descriptor trajectories from prior datasets and averaging them for family resemblance, then scoring mixture-based candidates.
-
Parameter-Free and Group Conditional Online Conformal Prediction
A parameter-free algorithm for group-conditional online conformal prediction that achieves optimal coverage guarantees without learning-rate tuning.
-
Density-Ratio Losses for Post-Hoc Learning to Defer
Post-hoc learning to defer is cast as density-ratio learning between model and expert ideal distributions, producing DR CPE losses that recover Chow's rule for KL-based ideals and support adjustable deferral via thresholding.
-
Generalized Functional ANOVA in Closed-Form: A Unified View of Additive Explanations
Introduces a Riesz basis for explicit generalized functional ANOVA decomposition under input dependence and an associated data-driven estimation procedure.
-
Keeping Score: Efficiency Improvements in Neural Likelihood Surrogate Training via Score-Augmented Loss Functions
Score-augmented loss functions with adaptive weighting improve neural likelihood surrogate quality in simulation-based inference at lower simulation cost for structured stochastic process models.
-
Spatial Adapter: Structured Spatial Decomposition and Closed-Form Covariance for Frozen Predictors
The Spatial Adapter equips frozen predictors with a spatially regularized orthonormal basis for residuals and derives a closed-form low-rank-plus-noise covariance for spatial prediction and kriging.
-
Amortized Variational Inference for Joint Posterior and Predictive Distributions in Bayesian Uncertainty Quantification
An amortized variational framework jointly targets the posterior and posterior-predictive distributions via a KL upper bound and moment regularization, yielding more accurate predictions at lower online cost than two-stage variational inference.
-
Laplace Approximation for Bayesian Tensor Network Kernel Machines
LA-TNKM applies a linearized Laplace approximation to tensor network kernel machines for Bayesian inference, matching or exceeding Gaussian processes and Bayesian neural networks on UCI regression tasks.
-
FedSPDnet: Geometry-Aware Federated Deep Learning with SPDnet
FedSPDnet uses manifold projections and retractions to average Stiefel-constrained parameters in federated SPDnet, outperforming standard federated EEGnet on EEG motor imagery benchmarks in F1 score and robustness.
-
Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification
Ensemble-based method of moments on softmax outputs produces stable Dirichlet predictive distributions that improve uncertainty-guided tasks like selective classification over evidential deep learning.
-
Generalizing Score-based generative models for Heavy-tailed Distributions
Early stopping plus normalizing flow initialization extends diffusion models to any target distribution with proven KL convergence, and a hybrid flow-SGM pipeline captures heavy tails without tail-specific assumptions.
-
Efficient Inference for Coupled Hidden Markov Models in Continuous Time and Discrete Space
Proposes Latent Interacting Particle Systems with an efficient parameterization of twist potentials to enable approximate posterior inference for coupled continuous-time hidden Markov models via twisted sequential Monte Carlo, demonstrated on a latent SIRS graph model and real wildfire data.
-
Geometric Autoencoder Priors for Bayesian Inversion: Learn First Observe Later
GABI learns geometry-conditioned latent priors from multi-geometry physical response datasets for use in Bayesian inversion, yielding geometry-adapted posteriors via ABC sampling.
-
Energy-Weighted Flow Matching: Unlocking Continuous Normalizing Flows for Efficient and Scalable Boltzmann Sampling
Energy-Weighted Flow Matching reformulates conditional flow matching with importance sampling to enable continuous normalizing flows to model Boltzmann distributions from energy evaluations alone, with iterative and annealed variants showing competitive performance on benchmarks.
-
Overfitting has a limitation: a model-independent generalization gap bound based on R\'enyi entropy
A model-independent upper bound on generalization gap is established that depends solely on the Rényi entropy of the data-generating distribution for histogram-determined algorithms such as ERM.
-
Operator Learning for Schr\"{o}dinger Equation: Unitarity, Error Bounds, and Time Generalization
A linear estimator for the Schrödinger evolution operator is introduced that enforces weak unitarity, supplies uniform prediction error bounds and time-extrapolation bounds, and reports up to 100x lower relative error than FNO and DeepONet on hydrogen, ion-trap, and optical-lattice Hamiltonians.
-
MissNODAG: Differentiable Cyclic Causal Graph Learning from Incomplete Data
MissNODAG is a differentiable cyclic causal graph learner that jointly recovers graph structure and missingness mechanism from incomplete data including MNAR via additive noise model and EM.
-
Recurrent Adversarial Service Times
RNN for arrivals paired with recurrent GAN for service times to model queuing dynamics without assuming specific inter-event distributions.
-
Deep Mixture Point Processes: Spatio-temporal Event Prediction with Rich Contextual Information
DMPP models spatio-temporal event intensity as a deep NN-weighted mixture of kernels to incorporate high-dimensional context while keeping likelihood integration tractable.
-
Demystifying MMD GANs
MMD GANs have unbiased critic gradients but biased generator gradients from sample-based learning, and the Kernel Inception Distance provides a practical new measure for GAN convergence and dynamic learning rate adaptation.
-
Automated Residual Plot Assessment With the R Package autovi and the Shiny Application autovi.web
An R package uses computer vision to predict visual signal strength for automating residual plot assessment in linear model diagnostics.
-
Renewable Lasso without Batch-Number Constraints: A Gradient-Enhanced Approach
Introduces gradient-enhanced renewable Lasso for high-dimensional GLMs that eliminates batch-number constraints, derives non-asymptotic bounds, and extends to distributed streaming with only gradient exchanges.
-
Discrete Causal Representations from Heterogeneous Domains: A Bayesian Approach with Social Survey Applications
A Bayesian approach with SMC inference learns discrete causal representations from heterogeneous domains, demonstrated on social survey data.
-
A data-driven Fourier-mixture neural-network method for density estimation
A Fourier-trained neural network using positive Gaussian-Laplace mixtures estimates fixed-horizon densities from empirical characteristic functions, with derived L2 error bounds and competitive performance on benchmarks and equity returns.
-
Consistency Regularised Gradient Flows for Inverse Problems
A consistency-regularized Euclidean-Wasserstein-2 gradient flow performs joint posterior sampling and prompt optimization in latent space for efficient low-NFE inverse problem solving with diffusion models.
-
Missingness-aware Data Imputation via AI-powered Bayesian Generative Modeling
MissBGM jointly models data generation and missingness in a Bayesian neural generative framework to produce consistent imputations with principled posterior uncertainty.
-
Probabilistic Graphical Model using Graph Neural Networks for Bayesian Inversion of Discrete Structural Component States
A probabilistic graphical model framework with graph neural network inference computes Bayesian posteriors for discrete structural states, claimed to match traditional Bayesian results while scaling to high-dimensional problems via topology-informed learning and scale-adaptive training.
-
EmDT: Embedding Diffusion Transformer for Tabular Data Generation in Fraud Detection
EmDT combines UMAP clustering with a Transformer-based diffusion process to create synthetic fraud samples that improve XGBoost classification on credit card fraud data while preserving correlations and privacy.
-
Efficient Deconvolution in Populational Inverse Problems
A methodology for populational inverse problems that simultaneously deconvolves unknown observational noise and recovers parameter distributions via structured gradient descent and adaptive empirical measure-based active learning for surrogates.
-
'In-Between' Uncertainty in Bayesian Neural Networks
MFVI in BNNs underestimates uncertainty between data regions, leading to overconfident OOD predictions, while linearised Laplace approximation performs better.
-
eXact-Prior Variational Autoencoder (X-VAE): Learning Data-Adaptive Gaussian Mixture Priors for Latent Distributions
X-VAE uses empirical statistics from a pretrained autoencoder to set a data-adaptive Gaussian prior and introduces a latent scaling factor for controllable generation.
-
A Survey on Data-Dependent Worst-Case Generalization Bounds
The survey unifies extensions of PAC-Bayesian theory to data-dependent sets, geometric and topological complexity measures of optimization trajectories, and stability replacements for information terms into one template inequality with comparative evaluation.
-
A numerical study into neural network surrogate model performance for uncertainty propagation
Numerical study comparing feedforward NN and DeepONet with data-driven and physics-informed losses on stochastic heat equation, highlighting larger errors at distribution tails due to extrapolation.
-
Neural Networks as Linear Regression: An Introduction for Statisticians
The paper introduces neural networks by framing them as approximations to linear regression with common customizations for statisticians.