Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Amortized In-Context Bayesian Posterior Estimation

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that amortized Bayesian posterior estimators trained with a reverse-KL objective consistently beat forward-KL (neural posterior estimation) estimators on predictive metrics, with the largest gains under model…

desk verdict The paper's headline — reverse-KL beats forward-KL for predictive tasks — is largely an artifact of scoring per-sample modes rather than the posterior predictive; the benchmark work is extensive but the central claim needs re-evaluation. read the letter →

arxiv 2502.06601 v1 pith:GX2EQCVK submitted 2025-02-10 cs.LG cs.AIstat.ML

Sarthak Mittal , Niels Leif Bracher , Guillaume Lajoie , Priyank Jaini , Marcus Brubaker This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords amortizedBayesianinferencein-contextlearningreverseKLdivergenceneuralposteriorestimationpermutation-invariantarchitecturesnormalizingflowsmodelmisspecificationzero-shottransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper makes the case for a specific design choice in amortized Bayesian inference: when a neural network is trained to output a posterior over model parameters directly from a dataset given in context, the training objective should be reverse KL, not the forward-KL objective borrowed from neural posterior estimation. Across regression, classification, and unsupervised benchmark tasks, the reverse-KL estimator produces better predictive L2 loss and accuracy than its forward-KL counterpart, and the gap widens under model misspecification and zero-shot transfer to real-world tabular datasets. The reverse-KL objective is the enabling mechanism, because its gradient does not require paired (parameter, dataset) simulations from the assumed model, so the estimator can be trained on any data distribution, including real data with no parameter labels. A reader should care because the work converts a library of competing heuristics into a testable rule: for fast, forward-looking posterior estimation, minimize reverse KL over the parameters of a transformer-conditioned flow.

What carries the argument

The load-bearing object is the reverse-KL training objective for a dataset-conditioned approximating family $q_\varphi(\theta \mid D)$: $$ \varphi^* = \arg\min_\varphi \mathbb{E}_{D \sim \chi}\, \mathbb{E}_{\$\theta$ \sim q_\varphi(\cdot \mid D)} \left[ \log q_\varphi(\$\theta$ \mid D) - \log p(D, \$\theta$) \right] . $$ Unlike the forward-KL (neural posterior estimation) objective, which requires paired $(\theta, D)$ draws from the assumed generative model, this objective needs only samples from the current approximate posterior and evaluations of the joint density $p(D, \theta)$, so the dataset distribution $\chi$ can be any source, including unlabeled real data. The paper combines it with permutation-invariant encoders (a transformer without positional encodings using a [CLS] token, or a DeepSets mean-pooling network) and diagonal-Gaussian or normalizing-flow density families, plus a masking scheme that embeds low-dimensional problems into a fixed 100-dimensional space so one model serves all feature dimensionalities.

What would settle it

Two concrete checks would settle it: on a heteroscedastic or heavy-tailed dataset where the assumed Gaussian likelihood is plainly wrong, see whether the reverse-KL predictive gain over forward-KL persists or flips; and measure the posterior mass a variable-dimension estimator places on the zero-padded coordinates of a low-dimensional task, since any mass away from zero invalidates the masking assumption on which its zero-shot generalization rests.

Watch

Extended reading notes

Core claim

The paper's central empirical claim is that on predictive metrics, namely expected L2 loss and classification accuracy of the posterior predictive, the reverse-KL amortized in-context estimator outperforms the forward-KL (NPE-style) estimator across every task family tested: Gaussian mean estimation, Gaussian mixtures, linear and nonlinear regression, and linear and nonlinear classification. The advantage is largest exactly where the assumptions are strained: under model misspecification, where the data-generating process differs from the assumed likelihood, and in zero-shot transfer from simulated to real tabular benchmark datasets. The authors argue the mechanism is the flexibility of the training distribution $\chi$: forward KL is tied to paired $(\theta, D)$ simulations from the assumed model, while reverse KL can be trained on any data stream, so it adapts when the real world does not match the model. Within the design space, transformer conditioning with normalizing flows gives the best predictive results for reverse KL, while forward KL retains a narrow edge at capturing low-dimensional multimodality in the Gaussian-mixture task.

Load-bearing premise

The load-bearing premise is that the synthetic likelihood and prior families used at training time, for instance standard-normal parameter priors with a known noise variance, sit close enough to the real processes behind the public tabular benchmark tasks that the posterior predictive produced by the learned estimator is a meaningful quantity rather than purely an inductive-bias artifact.

Editorial extensions

If this is right

  • A practitioner building an amortized posterior estimator for prediction should default to the reverse-KL objective: in the paper's tables it is never worse than forward KL on predictive metrics and is far better on nonlinear regression and classification.
  • Bayesian inference becomes trainable without paired (parameter, dataset) simulations: reverse-KL estimators can be trained on raw observation data, and the paper's 'switched data' experiments show that training on the real target distribution improves out-of-distribution performance further.
  • Transformer conditioning without positional encodings, combined with normalizing flows, is the strongest tested configuration for the reverse-KL objective, while added flow capacity helps forward KL only marginally.
  • A single variable-dimension estimator trained purely on simulated data transfers zero-shot to real tabular regression and classification benchmarks, and its posterior provides a far better starting point for MAP fine-tuning than prior initialization.
  • In low-dimensional multimodal problems such as Gaussian mixtures, forward KL keeps an advantage at capturing multiple modes, so reverse KL's win is specific to high-dimensional, prediction-oriented settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The mode-seeking property of reverse KL likely drives its predictive edge: prediction rewards any single parameter region that explains the data, so a posterior that concentrates on one good mode wins on L2 loss and accuracy even though its uncertainty is understated, a trade-off the paper's Gaussian-mixture results expose without resolving.
  • A testable extension is to carry the estimator into the application domains the introduction names, namely poll aggregation or compartment-style epidemiological models, and check whether the reverse-KL advantage survives richer hierarchical priors and time dependence; the paper only studies fixed likelihood families.
  • The masking-to-100-dimensional trick suggests a scaling path the paper leaves implicit: one network trained with reverse KL could serve as a general-purpose Bayesian engine across heterogeneous model classes, since the estimator already spans many dimensionalities and, with reverse KL, any data distribution.
  • Because the reported metrics score point predictions rather than uncertainty, a natural follow-up test is calibration, whether predictive intervals drawn from the reverse-KL posterior cover real held-out outcomes at their nominal rates, since mode-seeking objectives may trade coverage for accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a unified benchmark and training framework for amortized in-context Bayesian posterior estimation, comparing forward-KL (neural posterior estimation) and reverse-KL (ELBO-style) objectives across three architectures (GRU, DeepSets, Transformer) and two density parameterizations (diagonal Gaussian, normalizing flow). The core empirical claim is that reverse-KL-trained estimators achieve better predictive performance (L2 loss, accuracy) than forward-KL estimators, especially with Transformers and normalizing flows, and that they generalize zero-shot to real-world tabular data and to misspecified models. The paper also introduces a zero-padding technique to handle variable feature dimensionality.

Significance. If the central claim holds, the paper provides a practically useful recommendation: when building amortized Bayesian posterior estimators for predictive tasks, use the reverse-KL objective. The benchmark is extensive, with many combinations of models, objectives, architectures, and metrics, and the objectives are derived from well-known principles (NPE and amortized VI). The paper also makes a useful conceptual contribution by noting that reverse-KL allows training on datasets without paired parameter labels, which matters for misspecified or real data. However, the strength of the central claim depends critically on how 'predictive performance' is measured, and the paper's own appendix contains a qualification that contradicts the strong architectural claims in the main text.

major comments (3)
  1. [Section 4, Eq. (15); Appendix D, Eqs. (48)-(49)] The predictive metric used throughout the paper is not the posterior predictive defined in Eq. (8). Equation (15) evaluates E_{theta~q_phi}[METRIC(y_hat, y*)] with y_hat = Mode[p(y*|x*, theta)], and Appendix D's Eqs. (48)-(49) compute, e.g., E_theta[(y_i - Mode[p(y_i|x_i, theta)])^2]. For squared loss, this equals (y* - E_theta[f_theta(x*)])^2 + Var_theta(f_theta(x*)), so any approximate posterior with excess variance is penalized. Forward-KL (NPE) is known to produce overdispersed, mode-covering approximations, while reverse-KL is mode-seeking and underdispersed; the reported L2/accuracy gains of reverse-KL may therefore be an artifact of scoring per-sample modes rather than averaging them into a true posterior predictive. Since the abstract's headline is 'superiority of the reverse KL estimator for predictive problems,' the authors must either (i) re-evaluate with the actual posterior predictive, e.g., Monte Carlo E_{theta~q}[p(y*|x*, theta)] and a corresponding point prediction, or (ii) clearly reframe the claim as 'expected loss of per-sample modal predictions' and justify why that is the relevant decision-theoretic quantity. The appendix tables' labels 'according to the posterior predictive' are inaccurate under the current formulas.
  2. [Section 5 (Architectural Choices) vs. Appendix H.4] The main text and abstract claim that Transformer architectures outperform DeepSets and GRUs (e.g., 'especially when combined with the transformer architecture and normalizing flows' in the abstract, and 'Transformers outperform both DeepSets and GRUs' in Section 5). However, Appendix H.4 explicitly states: 'A definitive conclusion cannot be drawn regarding the superiority of one backbone over the other, i.e. between DeepSets or Transformer. However, amortization models with DeepSets as the backbone tend towards better generalization regarding OoD datasets.' These statements are in direct tension. The authors should either weaken the main-text and abstract claims to match the appendix's more cautious conclusion, or provide additional evidence that establishes Transformer superiority on the tabular/OoD benchmarks where the appendix says no definitive conclusion is possible.
  3. [Section 4.2 (Generalizing to Variable Feature Dimensions)] The zero-padding/masking procedure is underspecified in a way that affects the validity of the variable-dimension experiments. The text says low-dimensional problems are embedded into 100 dimensions 'with the extra features and parameters set to 0.' But if the prior over all 100 parameters is N(0,I), then for zero-padded features the likelihood does not depend on the inactive parameters, so the exact posterior over those parameters is the prior N(0,1), not a point mass at zero. A network trained to output zero (or a degenerate distribution) on those dimensions would not be recovering the Bayesian posterior. The paper should clarify exactly what prior/likelihood is used for the masked dimensions during training, and provide evidence that the marginal posterior over the active parameters is preserved (e.g., compare active-dimension posteriors against a fixed-dimensional model or against MCMC). Without this, the variable-dimension generalization results and the zero-shot tabular experiments built on them are not fully supported.
minor comments (5)
  1. [References and Appendix A.3] The reference to 'V on Oswald' is a typo; it should be 'von Oswald'.
  2. [Appendix F.1 and F.2] The warmup iteration counts are misprinted: '2, 5000' should be '25,000', and '12, 5000' should be '12,500' or '12,5000' is likely '12,500'.
  3. [Appendix H.4] There is a missing space in 'withXAVIER -INIT initialization'; also 'XAVIER' should be 'Xavier' for consistency.
  4. [Throughout] The paper uses 'TANH' in some places and 'TanH'/'tanh' in others; standardize the notation.
  5. [Section 4, Eq. (15)] The notation in Eq. (15) uses 'METRIC(y_hat, y*)' but the definition of y_hat is not repeated; consider making the metric definition self-contained, especially since the appendix's formulas are the primary reference.

Circularity Check

1 steps flagged · score 6.0 of 10

Predictive superiority claim is constructed by a per-sample-mode metric that penalizes dispersion by definition.

  1. self definitional [Section 4, Metrics, Eq. (15); Appendix D, Eqs. (D.48)-(D.49)]
    "For the former, we consider L2 loss and accuracy as applicable, in the following manner E(x∗,y∗),D∼χEθ∼qφ(·|D)METRIC (ˆy, y∗) (15) where ˆy is the mode of the distribution p(·|x∗, θ) ... (N−)LRL2 = ED∼χEθ∼qφ(·|D)[ Σ_i (y_i − Mode [p(y_i|x_i, θ)])^2 ] (48)"

    The metric in Eq. (15) is not the posterior predictive p(y*|D) defined in Eq. (8), which is E_{θ|D}[p(y*|θ)]. For squared loss, E_{θ∼q}[(y* − f_θ(x))^2] = (y* − E_q[f_θ(x)])^2 + Var_q(f_θ(x)). Thus the evaluation penalizes any approximate posterior with excess dispersion. Forward-KL is the mode-covering objective (Eq. 7/12) and is known to over-disperse; reverse-KL is the mode-seeking objective (Eq. 5/14) and is underdispersed. The claimed 'superiority of the reverse KL estimator for predictive problems' is therefore generated by the metric definition itself, and the Appendix captions even rename this per-sample-mode average as 'expected L2 loss according to the posterior predictive'.

full rationale

Most of the derivation chain is self-contained. Equations (11)-(14) derive the forward- and reverse-KL amortized objectives from first principles, the baselines (HMC, Langevin, optimization, random) are external references, and the sample-based metrics (2-Wasserstein and symmetric KL in Tables 5-6) provide genuinely independent evaluation. The reverse-KL advantage under misspecification is a structural consequence of Eq. (14) allowing arbitrary χ, not a fitted result. The only load-bearing circularity is the predictive metric: Eq. (15) evaluates per-sample modes of p(y|x*,θ) averaged over θ∼q_φ, which by the bias-variance decomposition penalizes dispersion; since reverse-KL is by definition mode-seeking, its advertised predictive superiority is effectively baked into the evaluation. The self-citation to Kobyzev et al. (co-authored by Brubaker) is only a normalizing-flow review and is not load-bearing. The objectives and posterior-quality claims retain independent content, but the central predictive claim reduces by construction to the chosen metric, so the score is 6.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical or model entities; 'in-context posterior estimator' is a name for a method architecture, not a postulated entity. The free parameters are training and architectural hyperparameters, all chosen by hand rather than fitted to the evaluation target. The axioms are standard Bayesian modeling assumptions plus the untested zero-padding assumption from Section 4.2.

free parameters (6)
  • KL warmup schedule = linear 0 to 1 over 5,000 to 125,000 iterations, task-dependent
    The beta-VAE style linear KL scaling is a hand-chosen training hyperparameter that affects convergence and final posterior quality; the paper does not tune it per task.
  • Number of test datasets and posterior samples = 100 test datasets, 25 posterior samples per dataset
    Evaluation protocol fixed by hand; the reported means and standard errors depend on these counts.
  • Transformer architecture = 4 layers, 256 dimensions, 4 heads, 1024 feed-forward
    Chosen to match DeepSets parameter count; affects capacity and results.
  • DeepSets architecture = 4 plus 4 layers, 627 hidden units
    Chosen to match Transformer parameter count.
  • Normalizing flow architecture = 6 coupling blocks, 128 hidden units
    FrEIA AllInOneBlock; capacity choice affects flexibility of q_phi.
  • GRU architecture = 4 layers, 256 hidden units
    Chosen for parameter parity with the other backbones.
assumptions (6)
  • standard math Observations are iid, so the posterior p(theta|D) is invariant to permutations of D.
    The permutation-invariant architectures (DeepSets, Transformer without positional encoding) are justified by this; Section B.
  • domain assumption The assumed likelihood and prior (e.g., N(0,I) prior, known variance sigma^2=0.25) define the target posterior.
    All amortized estimators approximate the posterior of the assumed model, not the true generating process; Section C.
  • standard math For forward KL, training data must be sampled from the assumed model's prior predictive; for reverse KL, arbitrary chi is allowed.
    Equations 10-12 versus Equation 14; this is the mathematical basis of the misspecification advantage.
  • ad hoc to paper Zero-padding or masking low-dimensional problems into 100D preserves the posterior over the relevant parameters.
    Section 4.2 assumes embedding extra features and parameters as zero leaves the posterior of the modeled parameters unchanged; not proved.
  • domain assumption MCMC (Langevin, HMC) samples provide a reliable reference posterior for W2 and symmetric KL metrics.
    The paper itself calls this a 'crude proxy' in Section 4.5.
  • domain assumption Real OpenML datasets are reasonable testbeds for models trained on synthetic N(0,I) inputs.
    Section 4.4 and Appendix G assume the assumed model class is meaningful for these tasks after normalization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Amortized In-Context Bayesian Posterior Estimation." pith.science (2026). https://pith.science/paper/GX2EQCVK

@misc{pith2026250206601,
  author       = {Pith},
  title        = {Pith review of: Amortized In-Context Bayesian Posterior Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GX2EQCVK}},
  note         = {Machine review of arXiv:2502.06601}
}
read the original abstract

Bayesian inference provides a natural way of incorporating prior beliefs and assigning a probability measure to the space of hypotheses. Current solutions rely on iterative routines like Markov Chain Monte Carlo (MCMC) sampling and Variational Inference (VI), which need to be re-run whenever new observations are available. Amortization, through conditional estimation, is a viable strategy to alleviate such difficulties and has been the guiding principle behind simulation-based inference, neural processes and in-context methods using pre-trained models. In this work, we conduct a thorough comparative analysis of amortized in-context Bayesian posterior estimation methods from the lens of different optimization objectives and architectural choices. Such methods train an amortized estimator to perform posterior parameter inference by conditioning on a set of data examples passed as context to a sequence model such as a transformer. In contrast to language models, we leverage permutation invariant architectures as the true posterior is invariant to the ordering of context examples. Our empirical study includes generalization to out-of-distribution tasks, cases where the assumed underlying model is misspecified, and transfer from simulated to real problems. Subsequently, it highlights the superiority of the reverse KL estimator for predictive problems, especially when combined with the transformer architecture and normalizing flows.

Figures

Figures reproduced from arXiv: 2502.06601 by the authors.

Figure 1
Figure 1. Amortized Bayesian Posterior Estimation: Illustration of predictions from the reverse KL in-context estimator. Model predictions, true predictions and sample points are shown in red, black and blue respectively. Additionally for classification, we label sample points with their ground-truth class, and draw the decision boundary according to the model. where D is a notion of divergence between two distributions. Once… view at source ↗
Figure 2
Figure 2. Tabular Experiments: Initializing parameters from the proposed amortized model leads to good zero-shot performance and often optimal initialization across (non-)linear regression and classification tasks. χreal (→) qφ Data Linear MLP GP Nonlinear Nonlinear χsim (→) Model NLR LR NLR Fwd-KL Gaussian 15.454±0.246 2.216±0.097 14.733±0.513 Rev-KL 0.382±0.003 1.892±0.113 0.155±0.006 + switched data 0.367±0.006 1.226±0.001… view at source ↗
Figure 3
Figure 3. Left: Estimation of the means of a GMM, where red and green samples denote the first and second mean vectors. Unlike in reverse KL, the cluster labels switch in forward KL, highlighting its ability to capture underlying multi-modality. Right: Kernel density estimation of the true posterior, overlaid with estimates from forward and reverse KL systems, for different probabilistic models. W2 2 (↓) Objective qφ Model Ga… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Trends of Performance over different Dimensions in Variable Dimensionality Setup: We see that our proposed reverse KL methodology outperforms the forward KL one. 0 20 40 60 80 100 Dimensionality 20 0 20 40 60 80 100 120 140 Loss Linear Regression 0 20 40 60 80 100 Dime…
Figure 5
Figure 5. Figure 5: Trends of Performance over different Dimensions in Variable Dimensionality Setup: We see that transformer models generalize better to different dimensional inputs than DeepSets. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_5.png]
Figure 6
Figure 6. Figure 6: Trends of Performance over different Dimensions in Variable Dimensionality Setup: We see that normalizing flows leads to similar performances than Gaussian based variational approximation. as well as different number of clusters and classes, respectively, for the GMM a…
Figure 7
Figure 7. Figure 7: Tabular Experiments | Linear Regression with Diagonal Gaussian: For every regression dataset from the OpenML platform considered, we initialize the parameters of a linear regression-based probabilistic model with the amortized inference models which were trained with a…
Figure 8
Figure 8. Figure 8: Tabular Experiments | Linear Regression with Normalizing Flow: For every regression dataset from the OpenML platform considered, we initialize the parameters of a linear regression-based probabilistic model with the amortized inference models which were trained with a …
Figure 9
Figure 9. Figure 9: Tabular Experiments | Nonlinear Regression with Diagonal Gaussian: For every regression dataset from the OpenML platform considered, we initialize the parameters of a nonlinear regression-based probabilistic model with the amortized inference models which were trained …
Figure 10
Figure 10. Figure 10: Tabular Experiments | Nonlinear Regression with Normalizing Flow: For every regression dataset from the OpenML platform considered, we initialize the parameters of a nonlinear regression-based probabilistic model with the amortized inference models which were trained …
Figure 11
Figure 11. Figure 11: Tabular Experiments | Linear Classification with Diagonal Gaussian: For every classification dataset from the OpenML platform considered, we initialize the parameters of a linear classification-based probabilistic model with the amortized inference models which were t…
Figure 12
Figure 12. Figure 12: Tabular Experiments | Linear Classification with Normalizing Flow: For every classification dataset from the OpenML platform considered, we initialize the parameters of a linear classification-based probabilistic model with the amortized inference models which were tr…
Figure 13
Figure 13. Figure 13: Tabular Experiments | Nonlinear Classification with Diagonal Gaussian: For every classification dataset from the OpenML platform considered, we initialize the parameters of a nonlinear classification-based probabilistic model with the amortized inference models which …
Figure 14
Figure 14. Figure 14: Tabular Experiments | Nonlinear Classification with Normalizing Flow: For every classification dataset from the OpenML platform considered, we initialize the parameters of a linear classification-based probabilistic model with the amortized inference models which were…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Universal priors: solving empirical Bayes via Bayesian inference and pretraining

    stat.ML 2026-02 conditional novelty 8.0 of 10

    A simple random prior-on-prior lets pretrained transformers achieve near-optimal empirical Bayes regret uniformly over all test priors, and length generalization matches α-posterior inference.

  2. Reinforced sequential Monte Carlo for amortised sampling

    cs.LG 2025-10 conditional novelty 6.0 of 10

    A method that trains neural samplers using SMC-collected off-policy samples and an importance-weighted replay buffer improves mode coverage on multi-modal targets.

  3. The Milky Way - Large Magellanic Cloud Interaction with Simulation Based Inference

    astro-ph.GA 2025-10 conditional novelty 5.0 of 10

    Simulation-based inference on outer-halo star velocities gives a Milky Way reflex speed of 26.4 km/s and an LMC enclosed mass of 9.2×10^10 solar masses within 50 kpc.

Reference graph

Works this paper leans on

71 extracted references · 34 canonical work pages · cited by 3 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    FrEIA : Framework for easily invertible architectures, 2018

    Ardizzone, L., Bungert, T., Draxler, F., K \"o the, U., Kruse, J., Schmier, R., and Sorrenson, P. FrEIA : Framework for easily invertible architectures, 2018. URL https://github.com/vislearn/FrEIA

    work page 2018

  3. [3]

    Framework for Easily Invertible Architectures (FrEIA) , 2018-2022

    Ardizzone, L., Bungert, T., Draxler, F., Köthe, U., Kruse, J., Schmier, R., and Sorrenson, P. Framework for Easily Invertible Architectures (FrEIA) , 2018-2022. URL https://github.com/vislearn/FrEIA

    work page 2018

  4. [4]

    A unified perspective on natural gradient variational inference with gaussian mixture models

    Arenz, O., Dahlinger, P., Ye, Z., Volpp, M., and Neumann, G. A unified perspective on natural gradient variational inference with gaussian mixture models. arXiv preprint arXiv:2209.11533, 2022

    arXiv 2022

  5. [5]

    J., Kourentzes, N., and Panagiotelis, A

    Athanasopoulos, G., Hyndman, R. J., Kourentzes, N., and Panagiotelis, A. Forecast reconciliation: A review. International Journal of Forecasting, 2023. ISSN 0169-2070. doi:https://doi.org/10.1016/j.ijforecast.2023.10.010. URL https://www.sciencedirect.com/science/article/pii/S0169207023001097

    doi:10.1016/j.ijforecast.2023.10.010 2023

  6. [6]

    Flow network based generative models for non-iterative diverse candidate generation

    Bengio, E., Jain, M., Korablyov, M., Precup, D., and Bengio, Y. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34: 0 27381--27394, 2021

    2021

  7. [7]

    P., Jankowiak, M., Obermeyer, F., Pradhan, N., Karaletsos, T., Singh, R., Szerlip, P., Horsfall, P., and Goodman, N

    Bingham, E., Chen, J. P., Jankowiak, M., Obermeyer, F., Pradhan, N., Karaletsos, T., Singh, R., Szerlip, P., Horsfall, P., and Goodman, N. D. Pyro: Deep universal probabilistic programming. The Journal of Machine Learning Research, 20 0 (1): 0 973--978, 2019

    work page 2019

  8. [8]

    G., van Rijn, J

    Bischl, B., Casalicchio, G., Feurer, M., Hutter, F., Lang, M., Mantovani, R. G., van Rijn, J. N., and Vanschoren, J. Openml benchmarking suites. arXiv:1708.03731v2 [stat.ML], 2019

    arXiv 2019

Show all 71 references
  1. [9]

    Bishop, C. M. and Nasrabadi, N. M. Pattern recognition and machine learning, volume 4. Springer, 2006

    2006

  2. [10]

    Amortized inference for gaussian process hyperparameters of structured kernels

    Bitzer, M., Meister, M., and Zimmer, C. Amortized inference for gaussian process hyperparameters of structured kernels. arXiv preprint arXiv:2306.09819, 2023

    2023 arXiv

  3. [11]

    M., Kucukelbir, A., and McAuliffe, J

    Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. Variational inference: A review for statisticians. Journal of the American statistical Association, 112 0 (518): 0 859--877, 2017

    2017

  4. [12]

    Markov chain monte carlo method and its application

    Brooks, S. Markov chain monte carlo method and its application. Journal of the royal statistical society: series D (the Statistician), 47 0 (1): 0 69--100, 1998

    1998

  5. [13]

    D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M

    Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M. A., Guo, J., Li, P., and Riddell, A. Stan: A probabilistic programming language. Journal of statistical software, 76, 2017

    2017

  6. [14]

    K., Zhou, J., Lu, P., Molaei, S., and Clifton, D

    Chauhan, V. K., Zhou, J., Lu, P., Molaei, S., and Clifton, D. A. A brief review of hypernetworks in deep learning. arXiv preprint arxiv:2306.06955, 2023

    2023 arXiv

  7. [15]

    Chen, Y., Garnett, R., and Montgomery, J. M. Polls, context, and time: A dynamic hierarchical bayesian forecasting model for us senate elections. Political Analysis, 31 0 (1): 0 113–133, 2023. doi:10.1017/pan.2021.42

    2023 doi

  8. [16]

    Cooper, I., Mondal, A., and Antonopoulos, C. G. A sir model assumption for the spread of covid-19 in different communities. Chaos, Solitons & Fractals, 139: 0 110057, 2020. ISSN 0960-0779. doi:https://doi.org/10.1016/j.chaos.2020.110057. URL https://www.sciencedirect.com/scien...

    2020

  9. [17]

    The frontier of simulation-based inference

    Cranmer, K., Brehmer, J., and Louppe, G. The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117 0 (48): 0 30055–30062, May 2020. ISSN 1091-6490. doi:10.1073/pnas.1912789117. URL http://dx.doi.org/10.1073/pnas.1912789117

    2020 doi

  10. [18]

    Density estimation using real NVP

    Dinh, L., Sohl-Dickstein, J., and Bengio, S. Density estimation using real NVP . 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings, 2017. URL http://arxiv.org/abs/1605.08803

    2017 arXiv

  11. [19]

    Model-agnostic meta-learning for fast adaptation of deep networks

    Finn, C., Abbeel, P., and Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp.\ 1126--1135. PMLR, 2017

    2017

  12. [20]

    F., Feurer, M., and Bischl, B

    Fischer, S. F., Feurer, M., and Bischl, B. Open ML - CTR 23 a curated tabular regression benchmarking suite. In AutoML Conference 2023 (Workshop), 2023. URL https://openreview.net/forum?id=HebAOoMm94

    2023

  13. [21]

    Q., Bindel, D., and Wilson, A

    Gardner, J., Pleiss, G., Weinberger, K. Q., Bindel, D., and Wilson, A. G. Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. Advances in neural information processing systems, 31, 2018

    2018

  14. [22]

    S., and Valiant, G

    Garg, S., Tsipras, D., Liang, P. S., and Valiant, G. What can transformers learn in-context? a case study of simple function classes. Advances in Neural Information Processing Systems, 35: 0 30583--30598, 2022

    2022

  15. [23]

    W., Rezende, D., and Eslami, S

    Garnelo, M., Rosenbaum, D., Maddison, C., Ramalho, T., Saxton, D., Shanahan, M., Teh, Y. W., Rezende, D., and Eslami, S. A. Conditional neural processes. In International conference on machine learning, pp.\ 1704--1713. PMLR, 2018 a

    2018

  16. [24]

    J., Eslami, S., and Teh, Y

    Garnelo, M., Schwarz, J., Rosenbaum, D., Viola, F., Rezende, D. J., Eslami, S., and Teh, Y. W. Neural processes. arXiv preprint arXiv:1807.01622, 2018 b

    2018 arXiv

  17. [25]

    Compositional score modeling for simulation-based inference

    Geffner, T., Papamakarios, G., and Mnih, A. Compositional score modeling for simulation-based inference. 2023

    2023

  18. [26]

    Gelfand, A. E. Gibbs sampling. Journal of the American statistical Association, 95 0 (452): 0 1300--1304, 2000

    2000

  19. [27]

    B., Stern, H

    Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. Bayesian Data Analysis, Third Edition . CRC Press, November 2013. ISBN 9781439840955. URL https://play.google.com/store/books/details?id=ZXL6AQAAQBAJ

    2013

  20. [28]

    M., Nogales, E., and Chiu, W

    Glaeser, R. M., Nogales, E., and Chiu, W. Single-particle Cryo-EM of Biological Macromolecules. 2053-2563. IOP Publishing, 2021. ISBN 978-0-7503-3039-8. doi:10.1088/978-0-7503-3039-8. URL https://dx.doi.org/10.1088/978-0-7503-3039-8

    2021 doi

  21. [29]

    P., Foong, A

    Gordon, J., Bruinsma, W. P., Foong, A. Y., Requeima, J., Dubois, Y., and Turner, R. E. Convolutional conditional neural processes. arXiv preprint arXiv:1910.13556, 2019

    1910 arXiv

  22. [30]

    Recasting gradient-based meta-learning as hierarchical bayes

    Grant, E., Finn, C., Levine, S., Darrell, T., and Griffiths, T. Recasting gradient-based meta-learning as hierarchical bayes. arXiv preprint arXiv:1801.08930, 2018

    2018 arXiv

  23. [31]

    beta- VAE : Learning basic visual concepts with a constrained variational framework

    Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. beta- VAE : Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, 2017. URL https://openreview.n...

    2017

  24. [32]

    D., Blei, D

    Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J. Stochastic variational inference. Journal of Machine Learning Research, 2013

    2013

  25. [33]

    D., Gelman, A., et al

    Hoffman, M. D., Gelman, A., et al. The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. J. Mach. Learn. Res., 15 0 (1): 0 1593--1623, 2014

    2014

  26. [34]

    Tabpfn: A transformer that solves small tabular classification problems in a second

    Hollmann, N., M \"u ller, S., Eggensperger, K., and Hutter, F. Tabpfn: A transformer that solves small tabular classification problems in a second. arXiv preprint arXiv:2207.01848, 2022

    2022 arXiv

  27. [35]

    Meta-learning in neural networks: A survey

    Hospedales, T., Antoniou, A., Micaelli, P., and Storkey, A. Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis & Machine Intelligence, 44 0 (09): 0 5149--5169, sep 2022. ISSN 1939-3539. doi:10.1109/TPAMI.2021.3079209

    2022

  28. [36]

    Kim, H., Mnih, A., Schwarz, J., Garnelo, M., Eslami, A., Rosenbaum, D., Vinyals, O., and Teh, Y. W. Attentive neural processes. arXiv preprint arXiv:1901.05761, 2019

    1901 arXiv

  29. [37]

    Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  30. [38]

    Kingma, D. P. and Dhariwal, P. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018

    2018

  31. [39]

    Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  32. [40]

    P., Welling, M., et al

    Kingma, D. P., Welling, M., et al. An introduction to variational autoencoders. Foundations and Trends in Machine Learning , 12 0 (4): 0 307--392, 2019

    2019

  33. [41]

    J., and Brubaker, M

    Kobyzev, I., Prince, S. J., and Brubaker, M. A. Normalizing flows: An introduction and review of current methods. IEEE transactions on pattern analysis and machine intelligence, 43 0 (11): 0 3964--3979, 2020

    2020

  34. [42]

    Siamese neural networks for one-shot image recognition

    Koch, G., Zemel, R., Salakhutdinov, R., et al. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2. Lille, 2015

    2015

  35. [43]

    Bayesian hypernetworks

    Krueger, D., Huang, C.-W., Islam, R., Turner, R., Lacoste, A., and Courville, A. Bayesian hypernetworks. arXiv preprint arxiv:1710.04759, 2017

    2017 arXiv

  36. [44]

    Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., and Teh, Y. W. Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning, pp.\ 3744--3753. PMLR, 2019

    2019

  37. [45]

    Lin, W., Schmidt, M., and Khan, M. E. Handling the positive-definite constraint in the bayesian learning rule. In International conference on machine learning, pp.\ 6116--6126. PMLR, 2020

    2020

  38. [46]

    J., and Adams, R

    Liu, S., Sun, X., Ramadge, P. J., and Adams, R. P. Task-agnostic amortized inference of gaussian process hyperparameters. Advances in Neural Information Processing Systems, 33: 0 21440--21452, 2020

    2020

  39. [47]

    Amortized inference for causal structure learning

    Lorch, L., Sussex, S., Rothfuss, J., Krause, A., and Sch \"o lkopf, B. Amortized inference for causal structure learning. Advances in Neural Information Processing Systems, 35: 0 13104--13118, 2022

    2022

  40. [48]

    Minka, T. P. Expectation propagation for approximate bayesian inference. arXiv preprint arXiv:1301.2294, 2013

    2013 arXiv

  41. [49]

    Recognition networks for approximate inference in bn20 networks

    Morris, Q. Recognition networks for approximate inference in bn20 networks. arXiv preprint arXiv:1301.2295, 2013

    2013 arXiv

  42. [50]

    P., Grabocka, J., and Hutter, F

    M \"u ller, S., Hollmann, N., Arango, S. P., Grabocka, J., and Hutter, F. Transformers can do bayesian inference. arXiv preprint arXiv:2112.10510, 2021

    2021 arXiv

  43. [51]

    and Wood, F

    Paige, B. and Wood, F. Inference networks for sequential monte carlo in graphical models. In International Conference on Machine Learning, pp.\ 3040--3049. PMLR, 2016

    2016

  44. [52]

    Neural clustering processes

    Pakman, A., Wang, Y., Mitelut, C., Lee, J., and Paninski, L. Neural clustering processes. In International Conference on Machine Learning, pp.\ 7455--7465. PMLR, 2020

    2020

  45. [53]

    J., Mohamed, S., and Lakshminarayanan, B

    Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., and Lakshminarayanan, B. Normalizing flows for probabilistic modeling and inference. The Journal of Machine Learning Research, 22 0 (1): 0 2617--2680, 2021

    2021

  46. [54]

    T., Mertens, U

    Radev, S. T., Mertens, U. K., Voss, A., Ardizzone, L., and K \"o the, U. Bayesflow: Learning complex stochastic models with invertible neural networks. IEEE transactions on neural networks and learning systems, 33 0 (4): 0 1452--1466, 2020

    2020

  47. [55]

    and Mohamed, S

    Rezende, D. and Mohamed, S. Variational inference with normalizing flows. In International conference on machine learning, pp.\ 1530--1538. PMLR, 2015

    2015

  48. [56]

    J., Mohamed, S., and Wierstra, D

    Rezende, D. J., Mohamed, S., and Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning, pp.\ 1278--1286. PMLR, 2014

    2014

  49. [57]

    Simpson, F., Davies, I., Lalchand, V., Vullo, A., Durrande, N., and Rasmussen, C. E. Kernel identification through transformers. Advances in Neural Information Processing Systems, 34: 0 10483--10495, 2021

    2021

  50. [58]

    P., Kumar, A., Ermon, S., and Poole, B

    Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  51. [59]

    Learning stochastic inverses

    Stuhlm \"u ller, A., Taylor, J., and Goodman, N. Learning stochastic inverses. Advances in neural information processing systems, 26, 2013

    2013

  52. [60]

    Hypernetworks with statistical filtering for defending adversarial examples

    Sun, Z., Ozay, M., and Okatani, T. Hypernetworks with statistical filtering for defending adversarial examples. arXiv preprint arxiv:1711.01791, 2017

    2017 arXiv

  53. [61]

    H., and Hospedales, T

    Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H., and Hospedales, T. M. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 1199--1208, 2018

    2018

  54. [62]

    Tang, L., Zhou, Y., Wang, L., Purkayastha, S., Zhang, L., He, J., Wang, F., and Song, P. X.-K. A review of multi-compartment infectious disease models. International Statistical Review, 88 0 (2): 0 462--513, 2020. doi:https://doi.org/10.1111/insr.12402. URL https://onlinelibra...

    2020 doi

  55. [63]

    Denoising diffusion samplers

    Vargas, F., Grathwohl, W., and Doucet, A. Denoising diffusion samplers. arXiv preprint arXiv:2302.13834, 2023

    2023 arXiv

  56. [64]

    N., Kaiser, ., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  57. [65]

    Matching networks for one shot learning

    Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016

    2016

  58. [66]

    Transformers learn in-context by gradient descent

    Von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., and Vladymyrov, M. Transformers learn in-context by gradient descent. In International Conference on Machine Learning, pp.\ 35151--35174. PMLR, 2023

    2023

  59. [67]

    Uncovering mesa-optimization algorithms in transformers

    von Oswald, J., Niklasson, E., Schlegel, M., Kobayashi, S., Zucchet, N., Scherrer, N., Miller, N., Sandler, M., Vladymyrov, M., Pascanu, R., et al. Uncovering mesa-optimization algorithms in transformers. arXiv preprint arXiv:2309.05858, 2023

    2023 arXiv

  60. [68]

    and Teh, Y

    Welling, M. and Teh, Y. W. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11), pp.\ 681--688, 2011

    2011

  61. [69]

    Zaheer, M., Kottur, S., Ravanbhakhsh, S., P \'o czos, B., Salakhutdinov, R., and Smola, A. J. Deep sets. In Advances in Neural Information Processing Systems, volume 2017-December, 2017

    2017

  62. [70]

    and Chen, Y

    Zhang, Q. and Chen, Y. Path integral sampler: a stochastic control approach for sampling. arXiv preprint arXiv:2111.15141, 2021

    2021 arXiv

  63. [71]

    Past, present, and future of software for bayesian inference, 2023

    Štrumbelj, E., Bouchard-Côté, A., Corander, J., Gelman, A., Rue, H., Murray, L., Pesonen, H., Plummer, M., and Vehtari, A. Past, present, and future of software for bayesian inference, 2023. URL http://hdl.handle.net/10754/694575

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.