REVIEW 2 major objections 4 minor 33 references
NestyNet. II. Coherent Function-Space Posteriors from Scientific Neural Surrogates (or How to Avoid Expensive MCMC)
T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper shows that the uncertainty of a fitted neural surrogate lives in a low-dimensional space of coherent function deformations, with four modes carrying 99% of the force posterior variance in its 800-parameter flagship fit.
desk verdict A well-built first-order function-space posterior for neural surrogates, with honest validation, but derivative deliverable widths are understated by ~36% on a warm branch and the flagship never checks that ratio. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the measurement-to-function transport $T_X := J_{f,X} H_{\mathrm{GN}}^+ J_m^{\top}$ (Eq. 5), the linearized derivative of the whole fitting map: it tells how a small perturbation of whitened measurements moves the converged fitted function. Its weighted singular value decomposition yields orthonormal deformation modes $\psi_k$ and singular values $s_k$; the generative sister model $F_\zeta(x)=\hat{f}(x)+\sum_{k=1}^{K} s_k \zeta_k \psi_k(x)$ (Eq. 15) turns those into smooth function draws with independent standard-normal coefficients. The same machinery gives scalar intervals at one curvature solve, e.g. $\sigma_M^2 = m^{\top} H^{-1} m$, and propagates correlated deliverables jointly, using the Jacobians and Gauss-Newton curvature the NestyNet second-order optimizer already computes at convergence plus the evidence-prior curvature that makes the 800-parameter posterior proper.
What would settle it
Perform a warm perturbed-data refit ensemble at reduced noise ($\gamma=1/8$) on the rotation-curve benchmark and check whether the ratio of refit to transport function-band width approaches 1; the paper's own data show a persistent residual-to-signal RMS of 0.30-0.38 even at small noise, so a clean falsifier is whether the residual vanishes or is absorbed by branch scatter as $\gamma$ is further reduced.
Extended reading notes
Core claim
On its own terms, the central claim is that the generative sister model $F_\zeta(x)=\hat{f}(x)+\sum_{k=1}^{K} s_k \zeta_k \psi_k(x)$ with $\zeta\sim\mathcal{N}(0,I_K)$ is a valid low-dimensional posterior over the fitted function, so that any derived quantity, including derivative- or integral-dependent ones, inherits the posterior coherently. The $\psi_k$ are the leading singular functions of the measurement-to-function transport $T_X = J_{f,X} H^+ J_m^{\top}$, the derivative of the data-to-fit stationary branch, and $s_k$ their singular values; truncation after $K$ minimizes expected function-space error by the Eckart-Young property. The paper distinguishes the repeated-experiment covariance $T_{\mathrm{rep}}$ from the local Gauss-Newton/Laplace posterior $T_{\mathrm{post}}$ and shows empirically in the flagship example that they differ, with the posterior band wider by the prior term. It validates the tangent approximation against an independent spline-MCMC posterior on a mock rotation curve and against warm and cold refits of perturbed data; fixed-branch function widths agree at the 10-15% level, while cold starts reveal extra branch scatter.
Load-bearing premise
The construction rests on the assumption that if the measurements were wiggled by the size of their quoted noise, the refitted curve would move proportionally, with no significant curvature and no jump to a different solution of the fit; where that fails, the reported bands understate the refit scatter.
Editorial extensions
If this is right
- For any scientific deliverable computed from a converged surrogate fit, the posterior of the deliverable can be obtained from coherent function draws, including deliverables that require derivatives or integrals, at the cost of a curvature solve plus a few Jacobian-vector products.
- The effective dimension of the posterior is set by the data, prior, model, and chosen deliverable, not by the number of network parameters: an $P=800$ fit collapses to $K_{1\%}=4$ for the vertical-force posterior, and the count is stable when the force basis is refined from 8 to 32 splines.
- The construction separates repeated-experiment estimator scatter from the Bayesian posterior; on the flagship fit the refit ensemble matches the repeated-experiment transport within sampling scatter while the posterior sister band is wider by the prior term.
- Convergence of every fit is certifiable by a stationarity check ($\eta_N, \eta_f \leq 10^{-3}$), and tangent-inadequate directions are flagged in advance by the $Q$ diagnostic, so the method carries its own validity conditions.
- Because draws are functions, downstream nonlinear compressions such as $M(r_0)=r_0 v^2/G$, epicyclic frequency, and the collisionless-Boltzmann tower can be pushed exactly through each draw, preserving correlations among deliverables.
Reading between the lines
- A reader might infer that the same construction transfers to other differentiable surrogate classes beyond the segmented NestyNet model, as long as the optimizer exposes converged Jacobians and curvature and the fit is deterministic; the transport identity itself is generic response analysis.
- The four-mode compression suggests that for derivative-dependent science, the relevant posterior lives near a low-dimensional manifold of function space, so a natural test is whether non-Gaussian coefficient laws over those same $K$ coordinates (which the paper says would leave the basis unchanged) resolve the persistent midplane-density derivative-gap systematic.
- The cold-start excess (kappa-width ratio 2.274 versus 1.356 warm) implies that for problems with multiple basins, the local linearized posterior is only part of the uncertainty budget; a complete scientific error report would need to combine the sister with an explicit basin-averaging or model-averaging step.
- Extending the method to time-dependent or field-valued surrogates would give each draw a coherent spatiotemporal field, allowing uncertainty-aware simulations whose initial conditions and forcing inherit calibrated correlated perturbations.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper constructs a low-dimensional, function-space posterior for a fitted neural surrogate trained with second-order optimization. The central object is the measurement-to-function transport of Eq. (5), T_X = J_f H^+ J_m^T, obtained by linearizing the converged stationary condition (Eq. (3)) about the fit. Its SVD gives deformation modes, and the truncated "generative sister" of Eq. (15) generates coherent function draws with independent standard-normal coefficients. Draws are pushed through nonlinear downstream quantities (mass, epicyclic frequency, density, force), so interval estimates of derived quantities inherit the function-space posterior. Validation on a mock rotation curve compares the sister to an independent spline+MCMC posterior and to 48 warm and 12 cold perturbed-data refits; a flagship P=800 phase-space distribution fit builds a vertical-force and density tower, with four modes reported to capture 99% of the force posterior variance.
Significance. If the construction were fully calibrated, it would be a useful contribution: it gives a practical route to coherent derivative- and integral-aware uncertainty propagation for high-dimensional neural surrogates without MCMC, it separates repeated-experiment from posterior covariance, and it packages the uncertainty as a low-rank generative model. The paper's controlled validation is a genuine strength: the rotation-curve mass posterior agrees with an independent spline+MCMC reference to a few percent, the comparison to full nonlinear refits is an unusual and welcome calibration check, and the four-mode stability under force-basis refinement is a real empirical finding. The public code and explicit enumeration of conditioning choices further support reproducibility. The main weakness is that the derivative-dependent part of the claim, which is central to the abstract and to the flagship deliverables, is under-calibrated in the controlled test and not quantitatively checked in the flagship.
major comments (2)
- [§5.6] The controlled calibration shows the tangent transport under-estimates the derivative deliverable width: for κ, the transport width is 0.0107, the warm-refit width is 0.0145 (ratio 1.356, 95% interval [1.021,1.665]), and the cold-refit width is 0.0243 (ratio 2.274, interval [1.036,3.010]). Both 95% intervals exclude unity. Since the central claim—abstract and §1—is that "any derived quantity, including those requiring derivatives or integrals... inherits the posterior," and since §5.3 explicitly differentiates every draw for κ, A, and B, this is a quantitative failure of the advertised property for derivative deliverables. The manuscript rules out rank truncation, metric reweighting, and replacing H_GN by H_stat, and then leaves "cutoff response, tangent nonlinearity, and optimization-branch response" as candidates; no correction or calibration factor is supplied. The claim should be weakened to a first-order tangent posterior with a known derivative under-coverage, or the interval construction should be corrected and recalibrated.
- [§6.4] The sentence "the exact-ρ(0) distributions from the three constructions coincide" is not a quantitative calibration. The headline deliverables ρ(0) and ν(0) require the deepest derivative rung of the fitted surface (§6.1–6.2), exactly the rung where §5.6 found the transport under-calibrated, yet no width ratio, coverage interval, or bootstrap error is reported. The four-mode 99% variance of Fig. 3e is an internal property of the empirical re-solved force draw ensemble, not a comparison to refits. Without an analogue of the §5.6 ratio for ρ(0) and ν(0), the flagship does not demonstrate derivative inheritance at the scale advertised in the abstract. Please report refit-to-transport width ratios for these deliverables, ideally with bootstrap intervals, and state explicitly whether the κ under-coverage persists.
minor comments (4)
- [Abstract/§6.3/§8] The "four uncertainty coordinates" of the flagship are empirical re-solved force modes whose coefficient law is not the independent-Gaussian law of Eq. (15); make this distinction at the first occurrence so the abstract is not read as saying Eq. (15) generates the four-mode force posterior.
- [Figure 2c] The caption labels the panel "(M, κ) joint cloud" but the panel title printed in the figure is "(c) (M, ) joint cloud"; the missing "κ" should be restored.
- [References] References to R. A. Ibata et al. 2026a and 2026b are to unpublished/in-prep papers; since §1 says the optimizer and SPLA machinery come from these papers, please confirm availability or summarize the needed properties in an appendix.
- [§6.5] The empirical-Bayes grid uses the same data to select α and to produce the flagship figures; the paper states this conditioning, but the abstract's unqualified "posterior" should say more visibly that the reported object is conditional on the evidence-selected prior strength.
Circularity Check
The core transport construction is independently benchmarked; only the disclosed empirical-Bayes selection of the prior strength on the same data prevents a fully clean bill.
-
other
[§6.1 and §6.5 (evidence-selected α used for the flagship posterior)]
"The prior uses one seed-anchored Gaussian block per segment (relative scale 0.25, floor 10−3), and its strength is set by the data through the Laplace evidence (§6.5), which prefers α≃0.125 independent of sample size. Figures 3 and 4 and the four-mode result below all use that evidence-selected value."
The prior strength α is the hyperparameter that makes the flagship curvature H=J_m^T J_m+P full rank and therefore gives the advertised posterior its existence. §6.1 states that α is 'set by the data through the Laplace evidence,' and §6.5 computes that evidence on the same mock catalogue, assigning weight 0.895 to α=0.125 and using that value for the headline figures. The reported flagship intervals are therefore p(M|D,S,α̂) with α̂ fitted to D, not p(M|D) with prior strength marginalized.
full rationale
The paper's core derivation is self-contained and independently checked. The transport T_X = J_f,X H^+ J_m^T (Eq. 5) is assembled from converged Jacobians and Gauss–Newton curvature; the generative sister of Eq. (15) is a Karhunen–Loève expansion of that transport with independent standard-normal coefficients, and Eq. (14) is the exact Eckart–Young truncation error. The central claims are validated against an external cubic-spline plus MCMC posterior in §5.2 and against 48 warm and 12 cold nonlinear refits in §5.6, with finite-ensemble error intervals. The flagship four-mode result is an empirical SVD summary of the re-solved force draws, explicitly not the linearized transport modes, and it is tested for stability under force-basis refinement in §6.3. I find no load-bearing uniqueness theorem, no hidden ansatz smuggled in by citation, and no fitted parameter renamed as a prediction in the main construction. The only genuine circularity is the disclosed empirical-Bayes selection of the regularization strength α: the same data set the prior strength and then condition the reported posterior. The paper conditions on this choice rather than marginalizing over it, and it is a mild caveat that does not collapse the central derivation. The self-citations to Paper I and the SPLA machinery provide computational scaffolding, but the transport and its calibration are re-derived and externally benchmarked in this paper, so the citation is not the load-bearing argument. Overall circularity is low: score 2.
Assumptions & free parameters
free parameters (6)
- Eigenvalue truncation threshold tau_eig =
1e-10 (rotation), 1e-12 (flagship)
- Convergence certificate thresholds eta_N, eta_f =
1e-3
- Evidence-selected prior strength alpha =
approximately 0.125
- Segment prior relative scale and floor =
0.25, 1e-3
- Tikhonov ridge for CBE solve =
1e-8 times mean normal-matrix diagonal
- Differentiated-block weight w1 =
1
assumptions (5)
- domain assumption The data-to-fitted-function map is locally linear at the scale of the measurement noise.
- domain assumption The Gauss-Newton curvature H_GN equals the stationary-point Hessian for posterior purposes.
- domain assumption The persistent regularization rows act as a proper Gaussian prior for the posterior flavor.
- domain assumption The posterior over functions is Gaussian with a low-dimensional dominant subspace.
- domain assumption The anchor measure and output metric W_mu are declared and fixed.
Cite this review
Pith. "Pith review of NestyNet. II. Coherent Function-Space Posteriors from Scientific Neural Surrogates (or How to Avoid Expensive MCMC)." pith.science (2026). https://pith.science/paper/SST5KU3A
@misc{pith2026260806274,
author = {Pith},
title = {Pith review of: NestyNet. II. Coherent Function-Space Posteriors from Scientific Neural Surrogates (or How to Avoid Expensive MCMC)},
year = {2026},
howpublished = {\url{https://pith.science/paper/SST5KU3A}},
note = {Machine review of arXiv:2608.06274}
}
read the original abstract
Scientific analyses increasingly use flexible neural networks, but their thousands of correlated parameters make it challenging to interpret the associated uncertainties. Here we develop a low-dimensional posterior for the fitted function itself, for scientific neural surrogates trained with second-order optimization. Linearizing the fitting procedure with respect to the randomized residual rows gives a measurement-to-function transport, the linear map, assembled from the converged Jacobians and Gauss--Newton curvature, that carries measurement perturbations into the function perturbations that refitting would produce. Its leading singular functions define coherent deformation modes. Independent Gaussian coefficients then generate smooth function draws, so that any derived quantity, including those requiring derivatives or integrals of the draw, inherits the posterior. The construction distinguishes repeated-experiment covariance from the local Gauss--Newton/Laplace posterior and propagates both to correlated quantities of scientific interest. The result is conditional on the fit's declared choices (architecture, hyperparameters, active set, and optimization branch), and every fit is certified as converged by checking that a further optimization step would change the fitted predictions by less than a chosen small fraction of the measurement errors. Our primary example is an 800-parameter phase-space distribution function fit for a mock stellar disk. Four uncertainty coordinates, two orders of magnitude fewer than the fitted parameters and stable under refinement of the force basis, capture 99\% of the vertical-force posterior variance, and 4000 coherent draws propagate through the force, total-density, surface-density, and frequency calculations in 0.8s. The method provides a highly efficient route to uncertainty propagation for derivative-dependent scientific inference.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
and Tenachi, Wassim and Diakogiannis, Foivos and Ibata, Neil and Shankar, Anirudh , title =
Ibata, Rodrigo A. and Tenachi, Wassim and Diakogiannis, Foivos and Ibata, Neil and Shankar, Anirudh , title =. arXiv e-prints , year =
-
[2]
and Tenachi, Wassim and Diakogiannis, Foivos and Ibata, Neil and Shankar, Anirudh , title =
Ibata, Rodrigo A. and Tenachi, Wassim and Diakogiannis, Foivos and Ibata, Neil and Shankar, Anirudh , title =. in preparation , year =
-
[3]
and Tenachi, Wassim and Diakogiannis, Foivos and Ibata, Neil and Shankar, Anirudh , title =
Ibata, Rodrigo A. and Tenachi, Wassim and Diakogiannis, Foivos and Ibata, Neil and Shankar, Anirudh , title =. Physical Review E , year =
- [4]
-
[5]
Annals of Statistics , volume =
Efron, Bradley , title =. Annals of Statistics , volume =. 1979 , doi =
work page 1979
-
[6]
MacKay, David J. C. , title =. Neural Computation , volume =. 1992 , doi =
work page 1992
-
[7]
Communications in Applied Mathematics and Computational Science , volume =
Goodman, Jonathan and Weare, Jonathan , title =. Communications in Applied Mathematics and Computational Science , volume =. 2010 , doi =
work page 2010
-
[8]
and Lang, Dustin and Goodman, Jonathan , title =
Foreman-Mackey, Daniel and Hogg, David W. and Lang, Dustin and Goodman, Jonathan , title =. Publications of the Astronomical Society of the Pacific , volume =. 2013 , doi =
work page 2013
Show all 33 references
-
[9]
and Solonen, Antti and Spantini, Alessio , title =
Cui, Tiangang and Martin, James and Marzouk, Youssef M. and Solonen, Antti and Spantini, Alessio , title =. Inverse Problems , volume =. 2014 , doi =
2014
-
[10]
SIAM Journal on Scientific Computing , volume =
Spantini, Alessio and Solonen, Antti and Cui, Tiangang and Martin, James and Tenorio, Luis and Marzouk, Youssef , title =. SIAM Journal on Scientific Computing , volume =. 2015 , doi =
2015
-
[11]
and Gelman, Andrew , title =
Hoffman, Matthew D. and Gelman, Andrew , title =. Journal of Machine Learning Research , volume =
-
[12]
Bayesian Analysis , volume =
Vehtari, Aki and Gelman, Andrew and Simpson, Daniel and Carpenter, Bob and B\"urkner, Paul-Christian , title =. Bayesian Analysis , volume =. 2021 , doi =
2021
-
[13]
, title =
Freeman, Kenneth C. , title =. Astrophysical Journal , volume =. 1970 , doi =
1970
-
[14]
2008 , isbn =
Binney, James and Tremaine, Scott , title =. 2008 , isbn =
2008
-
[15]
2020 , doi =
Astronomy & Astrophysics , volume =. 2020 , doi =
2020
-
[16]
Psychometrika , volume =
Eckart, Carl and Young, Gale , title =. Psychometrika , volume =. 1936 , doi =
1936
-
[17]
Annales Academi
Karhunen, Kari , title =. Annales Academi
-
[18]
Probability Theory
Lo. Probability Theory
-
[19]
Immer, Alexander and Korzepa, Maciej and Bauer, Matthias , title =. Proc. 24th Int. Conf. on Artificial Intelligence and Statistics (AISTATS), PMLR 130 , pages =. 2021 , archivePrefix =
2021
-
[20]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
Daxberger, Erik and Kristiadi, Agustinus and Immer, Alexander and Eschenhagen, Runa and Bauer, Matthias and Hennig, Philipp , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =. 2021 , archivePrefix =
2021
-
[21]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
Khan, Mohammad Emtiyaz and Immer, Alexander and Abedi, Ehsan and Korzepa, Maciej , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =. 2019 , archivePrefix =
2019
-
[22]
SIAM Journal on Scientific Computing , volume =
Bui-Thanh, Tan and Ghattas, Omar and Martin, James and Stadler, Georg , title =. SIAM Journal on Scientific Computing , volume =. 2013 , doi =
2013
-
[23]
, title =
Hampel, Frank R. , title =. Journal of the American Statistical Association , volume =. 1974 , doi =
1974
-
[24]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
Blondel, Mathieu and Berthet, Quentin and Cuturi, Marco and Frostig, Roy and Hoyer, Stephan and Llinares-L\'opez, Felipe and Pedregosa, Fabian and Vert, Jean-Philippe , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =. 2022 , archivePrefix =
2022
-
[25]
Quarterly of Applied Mathematics , volume =
Sirovich, Lawrence , title =. Quarterly of Applied Mathematics , volume =. 1987 , doi =
1987
-
[26]
and Dow, Eric and Wang, Qiqi , title =
Constantine, Paul G. and Dow, Eric and Wang, Qiqi , title =. SIAM Journal on Scientific Computing , volume =. 2014 , doi =
2014
-
[27]
and Tenorio, Luis and Marzouk, Youssef M
Spantini, Alessio and Cui, Tiangang and Willcox, Karen E. and Tenorio, Luis and Marzouk, Youssef M. , title =. SIAM Journal on Scientific Computing , volume =. 2017 , doi =
2017
-
[28]
SIAM Journal on Scientific Computing , volume =
Lieberman, Chad and Willcox, Karen and Ghattas, Omar , title =. SIAM Journal on Scientific Computing , volume =. 2010 , doi =
2010
-
[29]
Advances in Neural Information Processing Systems 32 , year =
Paszke, Adam and Gross, Sam and Massa, Francisco and others , title =. Advances in Neural Information Processing Systems 32 , year =
-
[30]
and Millman, K
Harris, Charles R. and Millman, K. Jarrod and van der Walt, St. Array programming with. Nature , year =
-
[31]
and others , title =
Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E. and others , title =. Nature Methods , year =
-
[32]
, title =
Hunter, John D. , title =. Computing in Science & Engineering , year =
-
[33]
and Jankowiak, Martin and others , title =
Bingham, Eli and Chen, Jonathan P. and Jankowiak, Martin and others , title =. Journal of Machine Learning Research , year =
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.