Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

This paper shows ICON, a transformer trained on condition–solution pairs, implicitly computes the mean of a Bayesian posterior predictive distribution, and that a generative extension can sample the full distribution to quantify uncertainty

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 05:29 UTC pith:O7HAV35L

load-bearing objection A clean Bayesian reinterpretation of ICON with a generative extension; the theory is correct, the empirical support is thin, but the paper deserves refereeing. the 3 major comments →

arxiv 2509.05186 v2 pith:O7HAV35L submitted 2025-09-05 stat.ML cs.LGcs.NAmath.NA

Probabilistic operator learning: generative modeling and uncertainty quantification for foundation models of differential equations

Benjamin J. Zhang , Siting Liu , Stanley J. Osher , Markos A. Katsoulakis This is my paper
classification stat.ML cs.LGcs.NAmath.NA MSC 60H2562F1568T07
keywords In-context operator learningProbabilistic operator learningRandom differential equationsBayesian inferencePosterior predictive distributionGenerative modelingUncertainty quantificationTransformer foundation models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that in-context operator networks silently perform Bayesian inference: ICON's point prediction is not just a learned map, but the conditional expectation of the solution given the prompt and the example pairs. The authors formalize the data-generating process through random differential equations, so that each ICON prompt is generated by a hidden operator and independent conditions. The key theorem identifies the least-squares training objective's minimizer with the posterior predictive mean, meaning ICON bypasses explicit posterior computation entirely. Building on this, the paper constructs generative ICON (GenICON), which samples from the posterior predictive distribution and thereby yields uncertainty bands around predictions. If correct, this recasts a broad class of foundation-model operator learners as amortized likelihood-free Bayesian inference, and makes uncertainty quantification a byproduct of the same training pipeline.

Core claim

On its own terms, the paper establishes that ICON approximates the mean of the posterior predictive distribution over solution operators, conditioned on the provided context. Conditions, parameters, and quantities of interest are modeled as Banach- or Hilbert-space-valued random variables, and the ICON joint distribution factorizes as a prior over hidden operators times independent likelihoods. Theorem 3.1 shows that the unique L2 minimizer of the ICON training objective is exactly the conditional expectation of the new solution given the new condition and the demo pairs; since that same conditional distribution coincides with the Bayesian posterior predictive, ICON performs implicit Bayesia

What carries the argument

The load-bearing object is the ICON conditional distribution Q_{z^J | y^J, {(y^j,z^j)}}, obtained by marginalizing the hidden operator parameters out of a random differential equation, together with the Hilbert-space projection theorem that equates least-squares regression with conditional expectation. The generative extension relies on a pushforward map G(·, y^J, context) such that G(·, y^J, context)♯Pη equals that conditional distribution; existence follows from a kernel-representation lemma for Polish spaces, and training uses an (f,Γ)-divergence GAN objective.

Load-bearing premise

The key assumption is that the finite transformer trained on the empirical loss actually reaches the global least-squares minimizer T* in Theorem 3.1; without approximation or optimization error bounds, the Bayesian interpretation applies exactly only if training has converged.

What would settle it

Train ICON on a random ODE whose true posterior predictive is analytically computable, such as a linear ODE with Gaussian coefficients and Gaussian observation noise, and compare ICON's output to the exact conditional mean on held-out prompts. If the trained model systematically deviates while the training loss is small, the equivalence fails. Separately, average many GenICON samples on the same prompt and compare the average to ICON's output; a significant gap would contradict Theorem 4.2 in practice.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • ICON's predictions are meaningful for both forward and inverse problems, even when the posterior over parameters is degenerate or non-identifiable, because the conditional mean is well-defined.
  • Averaging many GenICON samples reproduces the ICON point prediction, so the generative model extends, rather than replaces, existing ICON checkpoints.
  • The Bayesian view gives guidance on context size J: for identifiable problems with exact likelihoods, even a single demo can suffice, which the paper demonstrates numerically.
  • Adding zero-mean observation noise to the quantity of interest leaves the conditional mean unchanged, explaining observed denoising behavior with partial, noisy observations.
  • Sampling from GenICON provides empirical variances and credible bands for PDE solutions, delivering uncertainty quantification without redesigning the solver.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same random-differential-equation reading could be exported to other multi-operator foundation models, turning their deterministic predictions into posterior means and proposing generative counterparts; the paper only gestures at this.
  • GenICON's sample variance gives a testable calibration target: train on a family with a known posterior and check that the empirical variance matches the true posterior variance, not just the observation-noise level.
  • Since Theorem 4.1 is agnostic to the generator architecture, GenICON could be combined with diffusion or normalizing-flow generators on function spaces rather than GANs alone.
  • A more ambitious extension is using GenICON as a likelihood-free posterior sampler for model parameters: conditioning on quantities of interest and sampling latent codes could estimate posterior distributions on coefficients, an inverse-problem use case the paper raises but does not implement.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper develops a probabilistic operator-learning framework based on random differential equations, shows that the least-squares objective used to train in-context operator networks is the L2 projection onto the conditional expectation of the posterior predictive distribution, and introduces GenICON, a conditional generative model to sample from that distribution. The main theoretical results (Theorems 3.1, 4.1, 4.2) establish the conditional-expectation interpretation, the existence of a generative map, and the equivalence of its mean to the deterministic ICON. Numerical experiments on 1D ODE/PDE problems illustrate robustness to non-identifiability and noise and demonstrate that GenICON's empirical variance tracks the injected noise level.

Significance. The probabilistic interpretation is a useful conceptual contribution: it connects ICON to Bayesian inference and likelihood-free inference in function spaces, and it motivates a generative extension with potential UQ benefits. The theorems are correctly stated and their proofs are standard. The existence theorem for infinite-dimensional conditional generative models is simple but worth stating. However, the practical UQ claims are not supported by the current experimental evidence, and the central identification relies on an unquantified approximation.

major comments (3)
  1. [Section 3.1–3.2, Theorem 3.1 and Eq. (16)] The paper claims ICON computes/approximates the posterior predictive mean. Theorem 3.1 characterizes the population minimizer T* over all Σ-measurable functions, but the trained model T_θ solves the finite-sample, parametric least-squares problem (16). No approximation or generalization error bound is given, and no validation criterion checks whether T_θ is close to T*. The same gap affects GenICON: Theorem 4.1 guarantees an exact generative map, while (31) trains a parametric GAN on finite data. Section 6 acknowledges 'approximation and finite-sample error,' but the abstract and Section 3.3 state the identification without this caveat. The claims should be reformulated as approximate/implicit, or supported by consistency/generalization analysis.
  2. [Section 5.3, Table 2 and Section 5.4] The quantitative evaluation of GenICON's UQ reduces to comparing the average empirical variance of generated samples to the prescribed observation-noise variance. This is a single-moment, self-consistency check; it does not demonstrate that the conditional distribution is calibrated. No coverage of credible intervals, proper scoring rules, or distributional distance to the true posterior predictive is reported. Section 5.4 reports boundary-variance behavior, but again no calibration metric. Given that the paper's central practical contribution is 'principled uncertainty quantification,' this evidence is insufficient. Either add proper UQ metrics or substantially temper the claims.
  3. [Section 5 and Appendix A] All numerical demonstrations are on 1D ODE/PDE problems, and no code or data are released. The claims about foundation-model capabilities and general applicability would be better supported by a benchmark on a higher-dimensional problem and by releasing code/data for reproducibility. The absence of baselines (e.g., ensemble ICON, dropout UQ, Gaussian processes) makes it hard to judge the benefit of GenICON over simpler alternatives.
minor comments (4)
  1. [Theorem 4.1] Notation mismatch: the theorem states a reference random variable η∼P_η on H, but the pushforward is written as P_x. Use P_η consistently throughout the statement and proof.
  2. [Section 5.2] The text refers to '1000 source terms k(x,ω)', but in the random BVP of Section 2.3 k is a coefficient/permeability field and c is the source term. Please clarify the terminology.
  3. [Figure 7 caption] The boundary values are given with excessive precision (u_0 = -0.38746364, u_1 = -0.14212306). Round to a reasonable number of digits.
  4. [Appendix A] The target Lipschitz constant L appearing in Eq. (32) is never specified; only the penalty weight λ=0.1 is listed in Table 4. State the value of L (presumably 1) and the precise form of the noise input dimension.

Circularity Check

0 steps flagged

No significant circularity: the main claims follow from standard L2 projection theory and explicit definitions, while the acknowledged approximation gaps are correctness risks rather than circular reasoning.

full rationale

The paper's central derivation, that ICON approximates the posterior predictive mean, rests on Theorem 3.1, which is the standard Hilbert-space characterization of conditional expectation as the minimizer of mean-squared error. This is a mathematical identity, not an assumption equivalent to the conclusion. The training objective (16) is an empirical Monte Carlo version of that L2 minimization; the paper does not claim that finite transformers provably attain the true minimizer, and indeed Section 6 explicitly states: 'Our method assumes the GenICON model exactly characterizes the posterior predictive, but practical implementations incur approximation and finite-sample error.' That is a transparent limitation, not a disguised circular premise. The identification of the ICON conditional measure (19) with the posterior predictive distribution (25) follows directly from the definitions of the generative model and Bayes' rule; it is an interpretive equivalence, not an input to the training argument. Theorem 4.1 is a standard measurable-map existence result for non-atomic probability measures, and Theorem 4.2 is explicitly proved 'immediately by the definition of a pushforward measure' — a consistency check, not a predictive claim. The cited prior work by the authors, including the original ICON papers and the (f,Γ)-divergence, supplies architectures and divergence tools; it is not the load-bearing justification for the Bayesian inference claim. No fitted parameter is renamed as a prediction, and the GenICON uncertainty estimates are empirically compared against known noise levels. Therefore, no circular step meeting the stated evidentiary standard is present.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central mathematical claims rely on standard measure-theoretic results and on the modeling assumption that the training data arises from random differential equations with independent parameters and conditions. The only free parameters are implementation hyperparameters for the GAN training; they do not enter the main theorems.

free parameters (3)
  • gradient penalty coefficient lambda = 0.1
    Chosen by hand for GAN training in Section 4.2 and Appendix A, Table 4. It is not fitted to data but is an ad hoc hyperparameter that affects the training objective.
  • Lipschitz constant L = not specified
    The target Lipschitz constant for the discriminator in Eq. (32) is never assigned a concrete value; the paper only states L > 0. This is a free hyperparameter that is left undefined.
  • training ratio (discriminator:generator) = 5:1
    Chosen by hand in Appendix A, Table 4. It is an implementation choice that can materially affect GAN convergence and the resulting generative distribution.
axioms (5)
  • standard math Separability of the Banach and Hilbert spaces (A, Y, Z, H) is required for regular conditional probabilities and Bochner integration.
    Invoked in Section 3.1, where the paper states separability allows for disintegration of measures. This is standard for probability on Banach spaces.
  • standard math Hilbert space projection theorem characterizes conditional expectation as an L2 projection.
    Used in the proof of Theorem 3.1, which is essential to show that ICON's least-squares objective yields the conditional expectation.
  • standard math Borel isomorphism theorem for non-atomic probability measures on Polish spaces.
    Used in the proof of Theorem 4.1 to map the reference measure to uniform on [0,1]. This is a classical result in probability theory.
  • standard math Kernel representation lemma for regular conditional probability kernels.
    Used in the proof of Theorem 4.1 to construct a measurable map f that pushes forward uniform measure to the posterior predictive. This is standard.
  • domain assumption The data generating process for ICON follows the random differential equation formalism with independent parameters and conditions.
    Assumed in Section 2.1 and formalized in Eq. (12) and (17). If real-world training data is not generated by such independent draws from an RDE, the Bayesian interpretation does not apply.

pith-pipeline@v1.4.0-alltime-deepseek-medium · 24360 in / 7231 out tokens · 71256 ms · 2026-08-05T05:29:26.849439+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Probabilistic operator learning: generative modeling and uncertainty quantification for foundation models of differential equations." pith.science (2026). https://pith.science/paper/O7HAV35L

@misc{pith2026250905186,
  author       = {Pith},
  title        = {Pith review of: Probabilistic operator learning: generative modeling and uncertainty quantification for foundation models of differential equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O7HAV35L}},
  note         = {Machine review of arXiv:2509.05186}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In-context operator networks (ICON) are a class of operator learning methods based on the novel architectures of foundation models. Trained on a diverse set of datasets of initial and boundary conditions paired with corresponding solutions to ordinary and partial differential equations (ODEs and PDEs), ICON learns to map example condition-solution pairs of a given differential equation to an approximation of its solution operator. Here, we present a probabilistic framework that reveals ICON as implicitly performing Bayesian inference, where it computes the mean of the posterior predictive distribution over solution operators conditioned on the provided context, i.e., example condition-solution pairs. The formalism of random differential equations provides the probabilistic framework for describing the tasks ICON accomplishes while also providing a basis for understanding other multi-operator learning methods. This probabilistic perspective provides a basis for extending ICON to \emph{generative} settings, where one can sample from the posterior predictive distribution of solution operators. The generative formulation of ICON (GenICON) captures the underlying uncertainty in the solution operator, which enables principled uncertainty quantification in the solution predictions in operator learning.

Figures

Figures reproduced from arXiv: 2509.05186 by Benjamin J. Zhang, Markos A. Katsoulakis, Siting Liu, Stanley J. Osher.

Figure 1
Figure 1. Figure 1: The ICON graphical model, corresponding to the probability measure described in ( [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: ICON is a model that performs Bayesian inference. Given a collection of example [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the capabilities of ICON vs generative ICON. Standard ICON returns a [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: ICON is able to recover the correct ODE behavior using only a single example condition [PITH_FULL_IMAGE:figures/full_fig_p020_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: shows a representative prediction: although ICON is never exposed to clean solution data during training, its prediction (red +) aligns closely with the true noise-free solution (black +) [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Posterior predictive samples generated by GenICON trained on [PITH_FULL_IMAGE:figures/full_fig_p022_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: GenICON predictions on elliptic PDEs with a random right boundary. (a) Predicted QoIs [PITH_FULL_IMAGE:figures/full_fig_p024_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Harness In-Context Operator Learning with Chain of Operators

    cs.LG 2026-06 unverdicted novelty 6.0

    CHOP reduces relative inference error on OOD operator tasks for scalar conservation laws and mean-field control by composing frozen ICON with explicit closed-form elementary operators that remain interpretable.

  2. Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning

    cs.LG 2026-05 unverdicted novelty 5.0

    Multiple Neural Operators achieve near-optimal approximation and generalization rates for multi-task operator learning, matching single-task scaling laws and performing similarly to a multi-task DeepONet extension.

Reference graph

Works this paper leans on

44 extracted references · 26 canonical work pages · cited by 2 Pith papers · 2 internal anchors

  1. [1]

    Fourier Neural Operator for parametric partial differential equations

    Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Kaushik Bhattacharya, An- drew Stuart, Anima Anandkumar, et al. Fourier Neural Operator for parametric partial differential equations. InInternational Conference on Learning Representations, 2021

    work page 2021

  2. [2]

    DeepONet: Learning nonlinear oper- ators for identifying differential equations based on the universal approximation theorem of operators.arXiv preprint arXiv:1910.03193, 2019

    Lu Lu, Pengzhan Jin, and George Em Karniadakis. DeepONet: Learning nonlinear oper- ators for identifying differential equations based on the universal approximation theorem of operators.arXiv preprint arXiv:1910.03193, 2019. 25

    Pith/arXiv arXiv 1910

  3. [3]

    In-context operator learning with data prompts for differential equation problems.Proceedings of the National Academy of Sciences, 120(39):e2310142120, 2023

    Liu Yang, Siting Liu, Tingwei Meng, and Stanley J Osher. In-context operator learning with data prompts for differential equation problems.Proceedings of the National Academy of Sciences, 120(39):e2310142120, 2023

    2023

  4. [4]

    Fine-tune language models as multi-modal differ- ential equation solvers.arXiv preprint arXiv:2308.05061, 2023

    Liu Yang, Siting Liu, and Stanley J Osher. Fine-tune language models as multi-modal differ- ential equation solvers.arXiv preprint arXiv:2308.05061, 2023

    Pith/arXiv arXiv 2023

  5. [5]

    PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Laws

    Liu Yang and Stanley J Osher. PDE generalization of in-context operator networks: A study on 1D scalar nonlinear conservation laws.arXiv preprint arXiv:2401.07364, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    Poseidon: Efficient foundation models for PDEs

    Maximilian Herde, Bogdan Raoni´ c, Tobias Rohner, Roger K¨ appeli, Roberto Molinaro, Em- manuel de B´ ezenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for PDEs. arXiv preprint arXiv:2405.19101, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Towards a foundation model for partial differential equation: Multi-operator learning and extrapolation.arXiv preprint arXiv:2404.12355, 2024

    Jingmin Sun, Yuxuan Liu, Zecheng Zhang, and Hayden Schaeffer. Towards a foundation model for partial differential equation: Multi-operator learning and extrapolation.arXiv preprint arXiv:2404.12355, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior.Advances in Neural Information Pro- cessing Systems, 36, 2024

    Shashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji, Dmitriy Morozov, Michael W Mahoney, and Amir Gholami. Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior.Advances in Neural Information Pro- cessing Systems, 36, 2024

    work page 2024

  9. [9]

    Prose: Predicting multiple operators and symbolic expressions using multimodal transformers.Neural Networks, 180:106707, 2024

    Yuxuan Liu, Zecheng Zhang, and Hayden Schaeffer. Prose: Predicting multiple operators and symbolic expressions using multimodal transformers.Neural Networks, 180:106707, 2024

    2024

  10. [10]

    BCAT: A block causal transformer for PDE foundation models for fluid dynamics.arXiv preprint arXiv:2501.18972, 2025

    Yuxuan Liu, Jingmin Sun, and Hayden Schaeffer. BCAT: A block causal transformer for PDE foundation models for fluid dynamics.arXiv preprint arXiv:2501.18972, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    A foundation model for the Earth system.Nature, pages 1–8, 2025

    Cristian Bodnar, Wessel P Bruinsma, Ana Lucic, Megan Stanley, Anna Allen, Johannes Brand- stetter, Patrick Garvan, Maik Riechert, Jonathan A Weyn, Haiyu Dong, et al. A foundation model for the Earth system.Nature, pages 1–8, 2025

    work page 2025

  12. [12]

    Dpot: Auto-regressive denoising operator transformer for large-scale PDE pre-training.arXiv preprint arXiv:2403.03542, 2024

    Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, and Jun Zhu. Dpot: Auto-regressive denoising operator transformer for large-scale PDE pre-training.arXiv preprint arXiv:2403.03542, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Multiple physics pretraining for physical surrogate models.arXiv preprint arXiv:2310.02994, 2023

    Michael McCabe, Bruno R´ egaldo-Saint Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, et al. Multiple physics pretraining for physical surrogate models.arXiv preprint arXiv:2310.02994, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Operator learning: Algorithms and analysis.arXiv preprint arXiv:2402.15715, 2024

    Nikola B Kovachki, Samuel Lanthaler, and Andrew M Stuart. Operator learning: Algorithms and analysis.arXiv preprint arXiv:2402.15715, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    Language models are unsupervised multitask learners.OpenAI Blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI Blog, 1(8):9, 2019

    work page 2019

  16. [16]

    Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877– 1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877– 1901, 2020. 26

    work page 1901

  17. [17]

    Springer, 1995

    Ludwig Arnold.Random Dynamical Systems. Springer, 1995

    work page 1995

  18. [18]

    An explanation of in-context learning as implicit Bayesian inference

    Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit Bayesian inference. InInternational Conference on Learning Representations, 2022

    work page 2022

  19. [19]

    The frontier of simulation-based inference

    Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117(48):30055–30062, 2020

    work page 2020

  20. [20]

    Katsoulakis, and Luc Rey-Bellet

    Panagiota Birmpa, Jinchao Feng, Markos A. Katsoulakis, and Luc Rey-Bellet. Model uncer- tainty and correctability for directed graphical models.SIAM/ASA Journal on Uncertainty Quantification, 10(4):1461–1512, 2022

    work page 2022

  21. [21]

    An explanation of in-context learning as implicit Bayesian inference.arXiv preprint arXiv:2111.02080, 2021

    Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit Bayesian inference.arXiv preprint arXiv:2111.02080, 2021

    Pith/arXiv arXiv 2021

  22. [22]

    Transformers can do Bayesian inference

    Samuel M¨ uller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do Bayesian inference. InInternational Conference on Learning Represen- tations, 2022

    work page 2022

  23. [23]

    Neural operator: Learning maps between function spaces with applications to PDEs.Journal of Machine Learning Research, 24(89):1–97, 2023

    Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to PDEs.Journal of Machine Learning Research, 24(89):1–97, 2023

    2023

  24. [24]

    Springer

    Vladimir I Bogachev.Measure Theory. Springer

    work page

  25. [25]

    Springer Science & Business Media, 2012

    Nicholas Vakhania, Vazha Tarieladze, and Sergei Chobanyan.Probability Distributions on Banach Spaces, volume 14. Springer Science & Business Media, 2012

    work page 2012

  26. [26]

    Springer Cham, 3 edition, 2021

    Olav Kallenberg.Foundations of Modern Probability, volume 99 ofProbability Theory and Stochastic Modelling. Springer Cham, 3 edition, 2021

    work page 2021

  27. [27]

    Springer Science & Business Media, 2013

    Michel Ledoux and Michel Talagrand.Probability in Banach Spaces: Isoperimetry and Pro- cesses. Springer Science & Business Media, 2013

    2013

  28. [28]

    Exchangeable se- quence models can naturally quantify uncertainty over latent concepts.arXiv preprint arXiv:2408.03307, 2024

    Naimeng Ye, Hanming Yang, Andrew Siah, and Hongseok Namkoong. Exchangeable se- quence models can naturally quantify uncertainty over latent concepts.arXiv preprint arXiv:2408.03307, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Is in-context learning in large language models Bayesian? a martingale perspective.arXiv preprint arXiv:2406.00793, 2024

    Fabian Falck, Ziyu Wang, and Chris Holmes. Is in-context learning in large language models Bayesian? a martingale perspective.arXiv preprint arXiv:2406.00793, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Amortized variational inference: A systematic review.Journal of Artificial Intelligence Research, 78:167–215, 2023

    Ankush Ganguly, Sanjana Jain, and Ukrit Watchareeruetai. Amortized variational inference: A systematic review.Journal of Artificial Intelligence Research, 78:167–215, 2023

    work page 2023

  31. [31]

    Condi- tional sampling with monotone GANs: From generative models to likelihood-free inference

    Ricardo Baptista, Bamdad Hosseini, Nikola B Kovachki, and Youssef M Marzouk. Condi- tional sampling with monotone GANs: From generative models to likelihood-free inference. SIAM/ASA Journal on Uncertainty Quantification, 12(3):868–900, 2024

    work page 2024

  32. [32]

    Conditional optimal trans- port on function spaces.SIAM/ASA Journal on Uncertainty Quantification, 13(1):304–338, 2025

    Bamdad Hosseini, Alexander W Hsu, and Amirhossein Taghvaei. Conditional optimal trans- port on function spaces.SIAM/ASA Journal on Uncertainty Quantification, 13(1):304–338, 2025. 27

    work page 2025

  33. [33]

    Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784, 2014

    Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784, 2014

    Pith/arXiv arXiv 2014

  34. [34]

    Cocogen: Physically consistent and conditioned score-based generative models for forward and inverse problems.SIAM Journal on Scientific Computing, 47(2):C399–C425, 2025

    Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy. Cocogen: Physically consistent and conditioned score-based generative models for forward and inverse problems.SIAM Journal on Scientific Computing, 47(2):C399–C425, 2025

    work page 2025

  35. [35]

    Generative adversarial neural operators.Transactions on Machine Learning Research, 2022

    Md Ashiqur Rahman, Manuel A Florez, Anima Anandkumar, Zachary E Ross, and Kamyar Azizzadenesheli. Generative adversarial neural operators.Transactions on Machine Learning Research, 2022

    work page 2022

  36. [36]

    Infinite-dimensional diffusion models.Journal of Machine Learning Research, 25(414):1–52, 2024

    Jakiw Pidstrigach, Youssef Marzouk, Sebastian Reich, and Sven Wang. Infinite-dimensional diffusion models.Journal of Machine Learning Research, 25(414):1–52, 2024

    2024

  37. [37]

    Score-based diffusion models in function space.arXiv preprint arXiv:2302.07400, 2023

    Jae Hyun Lim, Nikola B Kovachki, Ricardo Baptista, Christopher Beckham, Kamyar Aziz- zadenesheli, Jean Kossaifi, Vikram Voleti, Jiaming Song, Karsten Kreis, Jan Kautz, et al. Score-based diffusion models in function space.arXiv preprint arXiv:2302.07400, 2023

    Pith/arXiv arXiv 2023

  38. [38]

    Multilevel diffusion: Infinite dimensional score-based diffusion models for image gener- ation.SIAM Journal on Mathematics of Data Science, 7(3):1337–1366, 2025

    Paul Hagemann, Sophie Mildenberger, Lars Ruthotto, Gabriele Steidl, and Nicole Tianjiao Yang. Multilevel diffusion: Infinite dimensional score-based diffusion models for image gener- ation.SIAM Journal on Mathematics of Data Science, 7(3):1337–1366, 2025

    work page 2025

  39. [39]

    MIT Press Cambridge, MA, 2006

    Christopher KI Williams and Carl Edward Rasmussen.Gaussian Processes for Machine Learn- ing, volume 2. MIT Press Cambridge, MA, 2006

    work page 2006

  40. [40]

    (f,Γ)-divergences: Interpolating betweenf-divergences and integral probability metrics.Jour- nal of Machine Learning Research, 23(39):1–70, 2022

    Jeremiah Birrell, Paul Dupuis, Markos A Katsoulakis, Yannis Pantazis, and Luc Rey-Bellet. (f,Γ)-divergences: Interpolating betweenf-divergences and integral probability metrics.Jour- nal of Machine Learning Research, 23(39):1–70, 2022

    work page 2022

  41. [41]

    Wasserstein generative adversarial networks

    Martin Arjovsky, Soumith Chintala, and L´ eon Bottou. Wasserstein generative adversarial networks. InInternational Conference on Machine Learning, pages 214–223. PMLR, 2017

    work page 2017

  42. [42]

    The Wiener–Askey polynomial chaos for stochastic differential equations.SIAM Journal on Scientific Computing, 24(2):619–644, 2002

    Dongbin Xiu and George Em Karniadakis. The Wiener–Askey polynomial chaos for stochastic differential equations.SIAM Journal on Scientific Computing, 24(2):619–644, 2002

    work page 2002

  43. [43]

    Polynomial Chaos Expansion for Operator Learning

    Himanshu Sharma, Luk´ aˇ s Nov´ ak, and Michael D Shields. Polynomial chaos expansion for operator learning.arXiv preprint arXiv:2508.20886, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  44. [44]

    A generative modeling/Physics-Informed Neural Network approach to random differential equations.arXiv preprint arXiv:2507.01687, 2025

    Georgios Arampatzis, Stylianos Katsarakis, and Charalambos Makridakis. A generative modeling/Physics-Informed Neural Network approach to random differential equations.arXiv preprint arXiv:2507.01687, 2025. 28 A Generative ICON architecture and training We adopt a GAN framework where both the generator and discriminator are decoder-only Trans- formers with...

    work page arXiv 2025