{"id":"881fc515-0334-49af-acee-ff9aeaba55ef","arxiv_id":"2501.00952","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Partially Bayesian neural networks with only the first hidden and output layers probabilistic match fully Bayesian networks on active learning benchmarks while cutting compute by about four times.","lead":"This paper tests partially Bayesian neural networks, where only a few layers track uncertainty, for active learning on molecular and materials datasets. It finds that making the first hidden layer and the output layer probabilistic gives accuracy and uncertainty close to a fully Bayesian network at roughly a quarter of the compute.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The PBNN-vs-Full-BNN comparison is not yet supported: freezing deterministic layers at SWA values is validated only by aggregate R-hat and visual overlap, not by a quantitative check of the resulting predictive distributions.","rationale":"The reader and I both locate the load-bearing risk in the conditional-posterior approximation: PBNN conditions on point estimates of the deterministic layers rather than marginalizing over them. The empirical results are plausible and the code and data are released, so I would not reject the paper. However, the central claim that PBNNs achieve uncertainty estimates comparable to Full BNNs rests on this approximation, and the current evidence is indirect: aggregated R-hat statistics only assess convergence of the sampled probabilistic weights, not the validity of freezing the others, while visual overlap and aggregate NLPD/coverage curves do not establish that the predictive distributions are close in the high-uncertainty regions that active learning selects. The proposed three-way ablation directly tests the approximation by unfreezing the deterministic layers, and it is a concrete, feasible check using the authors' own scripts. Since the reader already issued a CONDITIONAL verdict and this concern does not by itself invalidate the results, no change in verdict is needed; the condition should be that the authors either supply this quantitative comparison or soften the 'comparable to Full BNN' claim.","tokens_in":15941,"tokens_out":6855,"duration_ms":67502,"concrete_test":"Use the provided FreeSolv or ESOL scripts to run three models with identical data-dependent priors: (i) Full BNN with all layers sampled, (ii) PBNN(0,4) exactly as in Algorithm 1, and (iii) Full BNN warm-started from the same θ_det so that the 'frozen' layers are also sampled. Compare per-point predictive variances U(x) and the rank correlation of acquisition scores argmax U over the unlabeled pool at AL steps 1, 20, and 100; also report the Wasserstein distance between the predictive distributions on a fixed test set. If the rank correlation between (i) and (ii) is below 0.9, or if (iii) yields materially different U from (ii), the conditional-posterior approximation is not faithful and the comparison to Full BNN is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest link is the unexamined point-mass approximation in Algorithm 1 and Section 2.1.1: after deterministic SWA training, the non-probabilistic layers are frozen at θ_det and HMC/NUTS is run only over the selected layers. The resulting predictive distribution (Eq. 6) is a conditional posterior p(y|x, D, θ_det), not an approximation to the full posterior predictive p(y|x,D) = ∫ p(y|x,D,θ_D) p(θ_D|D) dθ_D unless the deterministic-layer posterior is tightly concentrated. The paper's only evidence for this concentration is the aggregate R-hat histograms in Appendix 1, which are computed on sampled probabilistic weights and cannot detect bias in frozen deterministic weights, and visual overlap of error bands in Figures 3–5. NLPD and coverage are aggregate calibration scores; they can match Full BNN even if the PBNN predictive distribution is narrower or differently shaped in the high-uncertainty regions that the acquisition function targets. If freezing removes a substantial part of the weight uncertainty, the headline 'comparable uncertainty estimates' fails.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes partially Bayesian neural networks (PBNNs), in which a deterministic network is first trained with stochastic weight averaging and then HMC/NUTS sampling is applied only to selected layers, with the remaining weights frozen at their SWA values. It compares PBNN configurations against a fully Bayesian network on active learning for a nonstationary toy problem, two molecular datasets (ESOL, FreeSolv), and two materials datasets (Steel fatigue/NIMS, Conductivity/HTEM), using RMSE, NLPD, and 95% coverage. It further tests transfer learning by initializing the prior means with weights from a deterministic model pretrained on computational data, sweeping the prior width tau, on a noisy version of FreeSolv and a Materials Project/experimental bandgap dataset. The main claims are that PBNN(0,4) matches Full BNN accuracy and uncertainty at roughly one quarter of the computational cost, and that theory-pretrained priors improve early active-learning performance and calibration.","tokens_in":16094,"tokens_out":6264,"duration_ms":57156,"significance":"If the claims hold, the paper provides a practical and timely result: it offers a concrete rule of thumb for MLP architectures in active learning (make the first hidden layer and the output layer probabilistic), demonstrates that Bayesian inference over a subset of layers can provide useful uncertainty quantification, and shows a principled way to inject computational priors into active-learning workflows. The study is reproducible in principle: code and data are linked, five random seeds are used, R-hat diagnostics are reported, and the number of probabilistic-layer configurations is varied systematically. The consistency of the layer-placement trend across four datasets and a toy problem is a genuine strength. However, the key comparison to Full BNN and the practical cost claim require additional quantitative validation before the headline conclusions can be accepted.","major_comments":[{"comment":"The central parity claim is supported only by a conditional predictive distribution. After freezing the deterministic layers at theta_det, Eq. (6) computes p(y|x*, D, theta_det), not the full posterior predictive p(y|x*, D) = integral p(y|x*, theta, sigma) p(theta, sigma|D) dtheta dsigma. The paper does not quantify how much weight uncertainty is removed by this point-mass approximation. The R-hat histograms in Appendix 1 are computed on the sampled probabilistic weights only, so they cannot detect bias or variance loss in the frozen layers, and the NLPD and coverage metrics in Figures 4 and 5 are aggregate calibration scores that can coincide for distributions with different shapes in the high-uncertainty regions selected by the acquisition function. I request a quantitative check: for example, report the distribution of predictive variances (epistemic and aleatoric components) for PBNN versus Full BNN over the acquisition-selected inputs, coverage conditional on uncertainty bins, or the sensitivity of PBNN predictions to different deterministic SWA checkpoints. Without such a check, the headline claim that PBNN uncertainty estimates are comparable to those of fully Bayesian networks is not established.","section":"Section 2.1.1, Algorithm 1, Eq. (6)"},{"comment":"The prior for probabilistic layers is centered at theta_det obtained by training on the same dataset D used for the subsequent HMC/NUTS inference. This is a data-dependent (empirical-Bayes) prior, and the manuscript does not discuss it as such. If the Full BNN baseline uses standard zero-centered priors, the PBNN-versus-Full-BNN comparison conflates layer partialization with prior information. Please either use a prior that does not depend on the AL training targets, or compare PBNN against a Full BNN with an equally data-informed prior, and discuss the effect of double use of D.","section":"Algorithm 1, lines 10 and 14"},{"comment":"The statement that active learning with PBNN achieves accuracy comparable to (ESOL) or better than (FreeSolv) that of standard deterministic ML models cites external MoleculeNet numbers [77]. Those numbers were obtained with different splits, features, preprocessing, and training protocols, and they are not directly comparable to the active-learning curves shown here. Please add a deterministic ML model trained and evaluated under the identical protocol, including the same random 5% initial subsets and the same RMSE computation, so that the accuracy claim is testable.","section":"Section 3.2, Figures 4(a) and 4(b)"},{"comment":"The statement 'it decreased the overall computational time by nearly a factor of four' is a core practical claim but is not supported by any measurement in the paper. Please report wall-clock times (or relative training and sampling times) per active-learning iteration for PBNN(0,4), other PBNN configurations, and Full BNN, together with hardware and sampler settings, and specify whether the factor refers to total end-to-end time or MCMC time only.","section":"Section 3.2"},{"comment":"The transfer-learning experiments make the last two hidden layers and the output layer probabilistic, rather than the PBNN(0,4) configuration that Sections 3.1 to 3.3 identify as the best. The text gives no rationale for this change. Because the transfer-learning comparisons use a different probabilistic-layer configuration, it is unclear whether the observed benefits of theory-pretrained priors generalize to the recommended architecture or are specific to the last-layer setup. Please test transfer learning with PBNN(0,4) as well, or justify the alternative configuration.","section":"Section 3.5, Figures 7 and 8"}],"minor_comments":[{"comment":"The word 'inverval' should be 'interval'.","section":"Section 2.2.1"},{"comment":"As written, 'Sample weights theta_l ~ p(theta_l)' reads as drawing from the prior rather than from the posterior; please clarify that NUTS generates posterior samples from the specified prior and likelihood.","section":"Algorithm 1, line 15"},{"comment":"Reference [54] should cite Hoffman and Gelman, not Homan and Gelman; reference [61] has an author-name formatting error ('Alp Kucukelbir David M. Blei and Jon D. McAuliffe').","section":"References"},{"comment":"There are typos in 'the the range' and 'traditonally'.","section":"Appendix 1"},{"comment":"The Noisy-FreeSolv dataset is described as experimental after synthetic noise is added; please clarify explicitly that the 'experimental' component is simulated so that readers do not mistake it for a real experimental dataset.","section":"Section 3.5"},{"comment":"The notation PBNN(0,4) appears in the code snippet before it is defined in Section 3.1; please define the notation earlier or add a pointer.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope and the empirical trends are interesting, but the main comparison to Full BNN and the computational-cost claim need stronger quantitative support. The transfer-learning section also needs to be reconciled with the architectural recommendation from the earlier sections. I recommend major revision rather than rejection because the missing evidence can plausibly be supplied within the paper's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid empirical benchmark paper. The novelty isn't PBNNs themselves—Sharma et al. and Harrison et al. already ran partially stochastic layers—but the systematic layer-placement comparison and the theory-pretrained prior recipe. On the evidence here, early-layer PBNNs track Full BNNs on four datasets and a toy problem at roughly a quarter of the compute. I buy it.\n\nWhat it does well: same architecture and training parameters across all datasets, five seeds, R-hat diagnostics, code and data in the repo, and the Full BNN comparison is same-protocol, not lifted from external papers. The writing is clear and the limitations are mostly acknowledged.\n\nThe soft spots are proportional. The stress-test note is correct: freezing deterministic layers at SWA values is a point-mass approximation, and the validation is indirect—aggregate R-hat on sampled weights plus visual overlap of error bands. NLPD and coverage can look fine even if the predictive distribution is mis-shaped precisely in the high-uncertainty regions that the acquisition function targets. I'd want a direct check of the predictive distribution, e.g., comparing PBNN and Full BNN posterior samples on held-out points away from the training distribution. That would tighten the main claim. Also, the confidence interval construction is never specified (normal sigma from Eq. 6 is likely, but it should be stated), only one small MLP is tested, and the deterministic baseline comparison relies on fixed MoleculeNet numbers instead of a same-protocol run. Transfer learning has a confound: Noisy-FreeSolv reuses the same molecules with added noise, and the bandgap task has theory and experiment in near-identical feature space, so part of the early gain could be memorization rather than a robust prior effect.\n\nWho it's for: practitioners in materials and chemical informatics who need UQ-guided AL under compute constraints. It's not a theory paper; it's an empirical recipe. I'd send it to a serious referee. The central claim needs a qualification or a direct predictive-distribution check, and the CI construction should be spelled out, but the recipe is plausible, reproducible, and useful.","headline":"A solid, reproducible empirical recipe for PBNN layer placement and theory-pretrained priors in active learning, with a real gap in validating the frozen-layer approximation before claiming full-BNN parity.","tokens_in":16674,"tokens_out":3228,"would_cite":true,"duration_ms":29283,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Only the first hidden and output layers need to be Bayesian to match full Bayesian active-learning accuracy, and theory-pretrained priors make early exploration faster.","keywords":["active learning","Bayesian neural networks","uncertainty quantification","partially Bayesian neural networks","transfer learning","materials informatics","cheminformatics","Hamiltonian Monte Carlo"],"falsifier":"A layer-ablation calculation on the four benchmark datasets would settle it: estimate the predictive variance contributed by each layer's weights by sampling only that layer at a time, and compare the sum against a full-network sampling. If the frozen deeper layers contribute a large share of the full Bayesian network's total predictive variance, the PBNN(0,4) uncertainty estimate is not faithful, and its comparable coverage would be a coincidence rather than a consequence of the method.","tokens_in":15672,"feed_emoji":"🧪","tokens_out":14404,"duration_ms":106773,"temperature":0.7,"pith_summary":"The paper asks whether a neural network must be fully Bayesian to guide active learning in materials and chemistry, and answers no. It shows that treating only the first hidden layer and the output layer as probabilistic, with all other weights frozen after stochastic-weight-averaged training, reproduces the accuracy, negative log predictive density, and 95% coverage of a fully Bayesian network across four molecular and materials benchmark datasets, at nearly a quarter of the computational cost. It further shows that initializing the Bayesian priors with weights pre-trained on theoretical calculations such as molecular dynamics or density functional theory makes active learning of experimental data more efficient, with the largest gains in the first tens of exploration steps. The practical upshot is that uncertainty-driven experimental exploration can be made affordable in the small-data, noisy-data regime typical of physical science.","feed_headline":"Two Bayesian layers match full networks at a quarter of the cost","feed_subtitle":"The first hidden layer and the output layer carry enough uncertainty for active learning, at a fraction of the cost.","key_machinery":"The load-bearing object is the partially Bayesian neural network (PBNN), defined by a two-stage training procedure in Algorithm 1: first a deterministic MLP is trained with stochastic weight averaging, then Hamiltonian Monte Carlo with the No-U-Turn Sampler is run only over a chosen subset of layers -- here the first hidden layer and the output layer, or one of the later hidden layers plus the output layer -- with priors centered on the pre-trained deterministic weights, while all other weights stay frozen. The quantity doing the work is the posterior predictive uncertainty $U^{post} = \\frac{1}{N}\\sum_{i=1}^N (y^*_i - \\mu^{post})^2$, which combines weight-sampling variability with the observation-noise samples $\\sigma_i$; the paper's claim is that this quantity computed over the small probabilistic subset faithfully approximates the full posterior predictive distribution of Eq. (4). The transfer-learning variant uses the same machinery, with the prior means set to the weights of a deterministic network pre-trained on simulated data and the prior width $\\tau$ controlling how strongly the theory constrains the posterior.","core_discovery":"The central discovery is architectural: where stochasticity is placed in a Bayesian neural network matters more than how much of the network is stochastic. With a five-layer MLP, the configuration PBNN(0,4) -- probabilistic first hidden layer plus probabilistic output layer -- tracks a fully Bayesian network's RMSE and uncertainty quality on ESOL, FreeSolv, steel fatigue, and conductivity datasets, while configurations with probabilistic later hidden layers (PBNN(1,4), PBNN(2,4), PBNN(3,4)) show unstable, oscillatory uncertainty calibration. The same favorable configuration also cuts overall computation by nearly a factor of four. The transfer-learning result is that priors centered on weights pre-trained on simulation data are a form of domain knowledge injection: they produce better-calibrated uncertainties and faster early progress than zero-centered priors, with the prior width $\\tau$ acting as a tunable degree of trust in the theory.","pith_inferences":["The apparent layer-position effect suggests a testable hypothesis the authors only gesture at: the first hidden layer's stochasticity is what lets the model adapt its input representation under sparse data, while later-layer weights are tightly constrained by the data and add little to predictive uncertainty; if true, the same rule should transfer to convolutional or graph networks, with the analo","One way to make the prior-width trade-off fully adaptive would be a scheduled 'cooling' policy that starts with a tight theory-informed prior and widens $\\tau$ as experimental data accumulate; the paper's two datasets hint at such a schedule but do not implement it.","Because the authors used one MLP architecture and one set of hyperparameters across all four datasets, PBNN(0,4) could plausibly serve as a default configuration for similar descriptor-based active-learning tasks, but this generalization is an extrapolation beyond the tested benchmark scope."],"forward_implications":["Active learning with uncertainty-driven acquisition becomes practical for small, noisy materials and chemistry datasets: PBNN(0,4) delivers full-Bayesian-quality coverage at roughly a quarter of the compute.","Autonomous experimental workflows can run end-to-end without human oversight of the deterministic stage, since the MAP-prior penalty added in Algorithm 1 is designed to prevent overfitting before the sampling stage begins.","Theory-to-experiment transfer shortens the early exploration phase: pre-trained priors give lower RMSE and better calibrated coverage in the first tens of active learning steps than uninformative priors.","The prior width $\\tau$ provides a single dial for trust in computational models: narrow priors hold the model close to theory, wider priors let experimental data take over, and the paper notes the possibility of relaxing $\\tau$ dynamically as data accumulate.","The layer-position rule -- stochastic first hidden layer plus stochastic output layer -- gives a concrete default architecture for future PBNN applications to tabular scientific data."],"supporting_citations":[{"why":"The earlier demonstration that selectively stochastic BNNs can match fully stochastic performance; this is the premise the paper extends to active learning.","marker":"[63]"},{"why":"Supplies the variational Bayesian last-layers treatment that motivates making only the output layer probabilistic.","marker":"[64]"},{"why":"Provides stochastic weight averaging, the procedure used for the deterministic training stage in Algorithm 1.","marker":"[65]"},{"why":"The No-U-Turn Sampler is the MCMC algorithm used to sample the posterior over the chosen probabilistic layers.","marker":"[54]"},{"why":"Defines the non-stationary toy benchmark used to compare PBNN configurations against the full Bayesian network baseline.","marker":"[76]"},{"why":"Provides the standard deterministic baselines and benchmark context for the molecular datasets that the active-learning results are compared against.","marker":"[77]"}],"fun_headline_variants":["Placement of Bayesian layers cuts cost 4x without losing quality","Stochastic first and output layers match full Bayesian accuracy","Partial Bayesian nets: quarter cost, same active learning results","Simulation-pretrained priors speed up experimental active learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison to a fully Bayesian network assumes that freezing all unsampled layers at their stochastic-weight-averaged values does not discard a substantial share of the total weight uncertainty, so that the posterior predictive uncertainty computed over the two probabilistic layers faithfully approximates the full posterior predictive distribution.","fun_headline_variants_meta":{"raw":{"variants":["Placement of Bayesian layers cuts cost 4x without losing quality","Stochastic first and output layers match full Bayesian accuracy","Partial Bayesian nets: quarter cost, same active learning results","Simulation-pretrained priors speed up experimental active learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000345,"raw_usage":{"total_tokens":1869,"prompt_tokens":896,"completion_tokens":973,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":904}},"tokens_in":512,"tokens_out":973,"duration_ms":8775,"temperature":1.0,"reasoning_tokens":904,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:38:33.071098+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A layer-ablation calculation on the four benchmark datasets would settle it: estimate the predictive variance contributed by each layer's weights by sampling only that layer at a time, and compare the sum against a full-network sampling. If the frozen deeper layers contribute a large share of the full Bayesian network's total predictive variance, the PBNN(0,4) uncertainty estimate is not faithful, and its comparable coverage would be a coincidence rather than a consequence of the method.","supporting_citations":[{"cited_title":"The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo","cited_arxiv_id":null,"evidence_quote":"The No-U-Turn Sampler is the MCMC algorithm used to sample the posterior over the chosen probabilistic layers."}],"review_version":1}