REVIEW 6 cited by
On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Inducing point Gaussian process approximations are often considered a gold standard in uncertainty estimation since they retain many of the properties of the exact GP and scale to large datasets. A major drawback is that they have difficulty scaling to high dimensional inputs. Deep Kernel Learning (DKL) promises a solution: a deep feature extractor transforms the inputs over which an inducing point Gaussian process is defined. However, DKL has been shown to provide unreliable uncertainty estimates in practice. We study why, and show that with no constraints, the DKL objective pushes "far-away" data points to be mapped to the same features as those of training-set points. With this insight we propose to constrain DKL's feature extractor to approximately preserve distances through a bi-Lipschitz constraint, resulting in a feature space favorable to DKL. We obtain a model, DUE, which demonstrates uncertainty quality outperforming previous DKL and other single forward pass uncertainty methods, while maintaining the speed and accuracy of standard neural networks.
Forward citations
Cited by 6 Pith papers
-
Metric Non-Collapse in Learned World Models for Control: Approximation Theory, Finite-Sample Geometric Guarantees, and Deterministic Planning Transfer
A mathematically justified local-global metric regularizer converts approximately optimized empirical world models into provably non-collapsed encoders with controlled planning transfer for deterministic nonlinear control.
-
Direct Bethe Free Energy Minimization for Bayesian Neural Networks
Training a Gaussian last layer by direct minimization of a shared-cavity predictive log-loss yields single-pass NLL/calibration parity-or-better with tuned and ensembled baselines on 7/8 UCI and 4/5 large/deep benchmarks.
-
ConceptVAE: Self-Supervised Fine-Grained Concept Disentanglement from 2D Echocardiographies
ConceptVAE learns to discretize echocardiograms into fine-grained anatomical concepts and per-concept styles without labels, and reports gains over a VICReg baseline on retrieval, segmentation, and OOD detection.
-
Distance-informed Neural Processes
A neural process with a bi-Lipschitz-regularized local encoder achieves better uncertainty calibration and OOD detection than existing NP variants.
-
On Efficient Bayesian Exploration in Model-Based Reinforcement Learning
Information-gain exploration bonuses are claimed to converge and to make model-based exploration sample-efficient, but the central mathematical proof does not hold together.
-
A Simple and Effective Method for Uncertainty Quantification and OOD Detection
A single-model OOD detection method that measures feature-space density with a Gaussian kernel (IPF) reports AUROC 93.18 on CIFAR-10 vs SVHN, a marginal gain over DDU's 92.90, with the kernel width selected to maximize AUROC.
Discussion (0). Continue with ORCID to comment.