{"id":"766fe925-1863-453d-b573-a126d212741c","arxiv_id":"2506.14828","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A prior-guided deep Gaussian process with auxiliary tasks achieved the best predictive accuracy for most experimental mechanical properties in an 8-element high-entropy alloy dataset.","lead":"This paper benchmarks four families of surrogate models on a new high-entropy alloy dataset and reports that a deep Gaussian process seeded with an encoder-decoder prior predicts several mechanical properties more accurately than standard alternatives. The result is a candidate recipe for material scientists who need uncertainty-aware predictions from sparse, multi-source data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported error bars omit the uncertainty of the deterministic encoder-decoder prior; without a calibration check, the 'uncertainty-aware' half of the central claim is unsupported.","rationale":"The reader's weakest-assumption analysis and my stress-test point to the same structural issue: HDGP P-All's reported predictive distribution is built from a deterministic prior plus a residual DGP, and the uncertainty of the prior is never propagated or validated. This is the most load-bearing concern because the paper's title, abstract, and discussion explicitly advertise uncertainty-awareness, not just point-prediction accuracy. If a calibration test showed coverage near nominal, the concern would be resolved; if it shows systematic under-coverage, the central claim's uncertainty component fails even though the accuracy tables may stand. I do not see an internal inconsistency in the accuracy comparison itself: the tables support the claim that HDGP P-All is best for most experimental properties among the models tested, and the 5-split averaging is a reasonable practice. I also note the paper's lack of an MTGP baseline and the absence of a commit hash, but these weaken scope rather than threaten the argument's correctness. The proposed test is direct, feasible with the existing code, and would settle whether the uncertainty claim survives. The verdict remains CONDITIONAL, hence no adjustment to the reader's recommendation is needed.","tokens_in":14935,"tokens_out":3883,"duration_ms":54520,"concrete_test":"On the same five held-out test splits used for Tables 2–3, compute empirical interval coverage for HDGP P-All, HDGP P-Main, HDGP NP-All, and cGP: for every test target, record whether the true value falls within the model's reported 1-sigma and 2-sigma predictive intervals; then pool across tasks and also report per-task coverage. If HDGP P-All's 1-sigma coverage is substantially below 68% (for example, below 50%) while cGP's coverage is close to nominal, the omission of prior uncertainty is empirically confirmed. As a secondary diagnostic, deliberately shift the encoder-decoder prior on held-out inputs by one training-set residual standard deviation and check whether the reported interval still contains the true value; this isolates the prior's contribution to the under-coverage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim has two parts: HDGP P-All is both the most accurate surrogate and genuinely uncertainty-aware. The accuracy part is reasonably supported by Tables 2–3, but the uncertainty part is not. In Section 2.3 the prior is injected by 'subtracting predicted prior values from the training data outputs,' and the final prediction is obtained by re-adding the encoder-decoder output. The DGP then models only the residual. The error bars shown in Figure 5 are explicitly described as 'the standard deviation as predicted by the model,' which for this construction is the predictive variance of the residual DGP alone. No variance or model-form uncertainty from the encoder-decoder prior is propagated. The encoder-decoder is itself a parametric model trained on finite, noisy data, so treating its output as a known mean function understates total predictive uncertainty whenever the prior is miscalibrated. This is not a philosophical quibble: the paper's own discussion says that in out-of-distribution regions the model 'converges to the prior,' so the final prediction is dominated by the prior exactly where the prior's uncertainty matters most, yet the reported interval does not include that uncertainty. The paper provides no calibration analysis, coverage test, or comparison of predictive intervals against empirical residuals. Since 'uncertainty-aware' appears in the title and abstract, and since overconfident error bars can mislead Bayesian optimization/acquisition in materials design, this omission is load-bearing for the central claim. A deterministic prior could in principle be treated as fixed, but then the uncertainty would be conditional on that prior being exactly correct; the paper does not state or test that assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper benchmarks four surrogate model families—conventional Gaussian processes (cGP), two-layer variational deep Gaussian processes (DGP), an encoder-decoder neural network, and XGBoost—on the BIRDSHOT high-entropy alloy dataset, which contains multiple experimental and computational property tasks. It introduces 'HDGP' configurations in which a DGP is trained on residuals after subtracting the predictions of a pre-trained encoder-decoder 'prior,' with variants using either all tasks or only main tasks. The central claim is that the prior-guided DGP using all tasks (HDGP P-All) gives the best predictive accuracy for most experimental properties while also providing predictive uncertainty estimates. Experiments on five 80/20 random splits report R², RMSE, MAE, SMAPE, and Spearman correlation; parity plots with model-predicted standard deviations are shown for one split. The paper concludes that HDGP P-All is the optimal modeling choice for correlated HEA properties.","tokens_in":15106,"tokens_out":7274,"duration_ms":82025,"significance":"If fully supported, the work would provide a useful benchmark and a practical recipe for combining deterministic neural priors with deep GPs in sparse multi-output materials data. A clear strength is the public release of code and data, as well as the use of multiple random splits with reported standard deviations. However, the 'uncertainty-aware' half of the claim is not established: the reported intervals do not include the prior's uncertainty, and no calibration or coverage analysis is provided. The accuracy claim is also overstated relative to the tables, which show that HDGP P-All is not the best model for several auxiliary tasks. The core idea is defensible, but the manuscript needs revision to align its claims with the evidence.","major_comments":[{"comment":"The predictive uncertainty shown in Figure 5 is described as 'the standard deviation as predicted by the model.' Because the encoder-decoder prior is re-added deterministically at test time and is not a probabilistic model, the reported variance is only the residual DGP's predictive variance; the prior's parameter and model-form uncertainty is never propagated. The paper itself states that in out-of-distribution regions the model 'converges to the prior' (Section 2.3), meaning the final point prediction is dominated by the prior exactly where the prior's uncertainty matters most, yet the reported interval excludes that uncertainty. The paper provides no calibration analysis, coverage test, or comparison of predictive intervals against empirical residuals. Since 'uncertainty-aware' appears in the title and abstract, this is a load-bearing gap. Please either propagate the prior's uncertainty (e.g., treating the prior as a GP or using an ensemble/Bayesian neural network) or provide calibration evidence showing that the residual-DGP variance alone is adequate.","section":"Section 2.3, Figure 5"},{"comment":"The statement 'HDGP P-All emerges as the optimal modeling choice' (Section 2.3) is not supported by the tables for all tasks. For VarvYS, HDGP P-All has RMSE 133.2±25.1, which is worse than XGB (63.1±43.6) and cGP (85.3±60.2); for VEC, HDGP P-All (R²=0.676) is far worse than cGP (R²=1.000); for SFE, HDGP P-All (R²=0.19) is not better than HDGP NP-All (0.29). Thus the optimal-model claim should be restricted to the main experimental mechanical properties (yield strength, UTS, elongation, hardness, modulus, dynamic hardness) where the improvement is more consistent. Moreover, with only five random splits, many differences are within one standard deviation (e.g., UTS/YS R² is 0.803±0.131 for HDGP P-All vs 0.785±0.028 for cGP), and no significance tests are provided, so 'consistently superior' is not statistically established.","section":"Tables 2 and 3"},{"comment":"The selection of the DGP reduction parameter is described only as choosing the value that gave the highest Spearman coefficient and lowest RMSE in 'cross-validation accuracy metrics,' without stating whether this cross-validation was performed on the training folds only, whether the same 80/20 splits were reused, or what the chosen reduction parameter values were. If the reduction parameter was selected using the same test splits that are later averaged, the reported metrics are optimistically biased. Please specify the inner validation scheme and report the selected reduction parameter for each DGP configuration.","section":"Section 4.3"},{"comment":"There is an inconsistency about which tasks are included in the prior: Table 1 says HDGP P-All uses 'priors derived from an encoder-decoder model trained on main tasks,' while the text says HDGP P-All 'had encoder-decoder priors for all tasks except Varveillien yield strength, SFE and VEC.' Clarify the encoder-decoder's output space and how the auxiliary computationally predicted properties (VarvYS, SFE, VEC, depth of penetration) enter the DGP model and the prior injection.","section":"Table 1 and Section 2.3"}],"minor_comments":[{"comment":"Please provide the encoder-decoder architecture and training hyperparameters (number of hidden layers and units, activation, regularization strength, optimizer, epochs, learning rate); the code link is useful but the manuscript should be self-contained.","section":"Section 4.4"},{"comment":"The dataset description would benefit from the exact number of compositions, the number of samples per task, and the missingness pattern across tasks; these are needed to assess the claimed heterotopic data challenge.","section":"Section 2.1"},{"comment":"The '-' entries for HDGP NP-Main and HDGP P-Main on the first three tasks are not explained; state that these models were only trained on main tasks and therefore do not predict auxiliary tasks.","section":"Tables 2 and 3"},{"comment":"The ELBO expression is incomplete: the variables Q, p(Y,H,F) are undefined and the equation as written is not a usable description; either define it properly or omit it.","section":"Section 4.2, Eq. (9)"},{"comment":"There are typographical errors such as 'quantificdaation' and 'ans' in Section 1; please proofread the manuscript.","section":"Introduction"},{"comment":"The caption lists panels (a)-(k) but the text does not describe which train-test split is shown; state whether this is a representative split and note that the error bars are the predictive standard deviations (see also the major comment about uncertainty).","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central accuracy claim for the main mechanical properties (yield strength, UTS, elongation, hardness, modulus, dynamic hardness) is reasonably supported by Tables 2 and 3, and the hybrid prior idea is interesting. The main risk is that the 'uncertainty-aware' promise is not backed by calibration evidence; if the authors can add a calibration/coverage analysis and refine the accuracy claim to the tasks where it actually holds, this could become acceptable. The paper fits the journal's scope; no ethical concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a legitimate empirical comparison, and the central accuracy claim—HDGP P-All is the best surrogate for most principal properties in the BIRDSHOT HEA dataset—is reasonably supported by Tables 2 and 3. What is not supported is the 'uncertainty-aware' part of the title. The error bars in Figure 5 are the predictive variance of the residual DGP only; the encoder-decoder prior is treated as a fixed, known function, and its uncertainty is never propagated. Since the model is designed to converge to the prior in out-of-distribution regions, that is exactly where the reported intervals are most likely to be too narrow. No calibration check, coverage test, or comparison against empirical residuals appears. This is a real gap, load-bearing for one of the two headline claims, but not fatal to the paper as a whole.\n\nThe paper earns credit. The combination of residual learning around an encoder-decoder prior inside a multi-task DGP is a new configuration, and the systematic comparison across seven setups on the BIRDSHOT data is new. The authors are transparent about fixing the train-test split between prior and DGP to prevent leakage, and the 5-split averaging with standard deviations is reasonable practice. The accuracy gains on yield strength, UTS, elongation, hardness, and modulus are visible and consistent across tasks.\n\nSoft spots, in proportion: the uncertainty gap is the main one. The VEC auxiliary task is also an odd case—cGP essentially nails it (R2 ~1.0) while HDGP P-All gets ~0.68, and the summary does not acknowledge this. No MTGP baseline is included despite multi-task GP being a stated motivation. Code and data links are given but without a commit hash, so reproducibility is promised rather than pinned down. These are all addressable in revision.\n\nWho this is for: materials informatics practitioners who want a concrete model comparison on a real, sparse, multi-property dataset. Not a methodological breakthrough, but a useful data point for model selection in Bayesian optimization loops.\n\nRecommendation: send it to peer review. A serious referee can push for calibration analysis and the missing baseline. The accuracy claim is worth publishing; the uncertainty claim needs to be substantiated or tempered.","headline":"A solid empirical benchmark with a real gap: the accuracy claim for HDGP P-All mostly holds up, but the 'uncertainty-aware' half of the title is not demonstrated.","tokens_in":15817,"tokens_out":2308,"would_cite":true,"duration_ms":128390,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that a two-layer multi-task deep Gaussian process trained on residuals around an encoder-decoder prior, with auxiliary computational tasks, predicts experimental high-entropy alloy properties most accurately while also…","keywords":["deep Gaussian processes","multi-task learning","high-entropy alloys","uncertainty quantification","surrogate modeling","heterotopic data","materials informatics","encoder-decoder prior"],"falsifier":"Take a held-out set of compositions far from the training region where the encoder-decoder prior is known to be inaccurate, compute HDGP-P-All's predictive intervals, and check how often the true values fall within two standard deviations; if coverage is far below the nominal 95% and uncertainties do not grow outside the training domain, the claim that the model is uncertainty-aware in the stated sense fails.","tokens_in":1657,"feed_emoji":"⚙️","tokens_out":4546,"duration_ms":119807,"temperature":0.7,"pith_summary":"The paper asks which surrogate model best predicts several correlated mechanical properties of high-entropy alloys from composition alone, when data are sparse, noisy, and incomplete. It compares conventional Gaussian processes (cGP), deep Gaussian processes (DGP), an encoder-decoder neural network, and XGBoost on a curated dataset of more than 100 Al-Co-Cr-Cu-Fe-Mn-Ni-V alloys that mixes experimental measurements with computational descriptors. Its central claim is that a two-layer variational multi-task DGP which models residuals around an encoder-decoder prior, and which also trains on auxiliary computational tasks, is the overall best: it gives the lowest errors and highest rank correlations for most experimental properties while producing uncertainty estimates. If this is right, materials scientists gain a practical recipe for uncertainty-aware surrogate modeling in alloy design, where experiments are costly and the composition space is vast.","feed_headline":"Prior-guided deep GP beats six rivals on alloy properties","feed_subtitle":"A residual deep GP with a neural prior predicts alloy strength and hardness best, with error bars.","key_machinery":"The load-bearing mechanism is a two-layer variational multi-task deep Gaussian process with residual prior injection. Each layer is a multi-task GP with 10 latent functions, and a reduction parameter controls the dimensionality of the latent space between layers; the prior is a regularized dense encoder-decoder network trained on the same tasks. The prior's predictions are subtracted from the training outputs so that the DGP models only residuals, then added back to form final predictions and variances. This multi-task structure lets the model share information across correlated properties, handle heterotopic (partially missing) data by evaluating the likelihood only on observed outputs, and avoid defaulting to a constant mean in regions far from training data.","core_discovery":"The paper's central discovery is that injecting a deterministic neural-network prior into a deep Gaussian process changes the quality of multi-task property prediction. Instead of learning raw outputs, the DGP learns residuals: the encoder-decoder's predictions are subtracted from the training outputs, the DGP is fit to those residuals, and the prior is added back at prediction time. In the best configuration, called HDGP-P-All, the DGP is trained on all tasks, including the six measured experimental properties plus auxiliary computed descriptors such as valence electron concentration, stacking fault energy, model-based yield strength, and simulated depth of penetration, and it outperforms the other six configurations on most experimental tasks, with substantial RMSE reductions relative to conventional GP and XGBoost on yield strength, ultimate tensile strength, hardness, modulus, and elongation. The paper states directly that HDGP-P-All emerges as the optimal modeling choice for predicting correlated material properties within HEA datasets, and it reports model-predicted standard deviations as error bars.","pith_inferences":["The same residual-prior recipe likely transfers to other small-data multi-task materials datasets, such as steels or oxides, whenever a cheap deterministic surrogate can be trained first; the expected gain should grow as the prior improves.","A direct testable extension would be to propagate uncertainty from the encoder-decoder prior into the DGP (for example by treating the prior as a random function or ensembling priors); if predictive interval coverage improves, the paper's uncertainty estimates are incomplete as currently reported.","For out-of-distribution compositions, bias in the encoder-decoder prior may dominate the prediction error, so acquisition functions in Bayesian optimization that rely on the reported variances could over-exploit if that prior is inaccurate.","The dataset's planned release of uncertainty estimates for experimental replicates and computed descriptors would allow direct calibration of the model-predicted standard deviations, something the present study does not perform."],"forward_implications":["For sparse multi-task materials datasets, a prior-guided deep Gaussian process is a better default than conventional single-task Gaussian processes and XGBoost when both accuracy and uncertainty are needed.","Adding auxiliary computational descriptors as extra tasks improves prediction of the main experimental properties, so experimental and simulation data should be modeled jointly rather than separately.","The hybrid structure, deterministic neural prior plus probabilistic GP residual, offers a way to obtain error bars from deep models without full Bayesian neural networks.","Because the DGP handles heterotopic and incomplete data natively, no imputation is required before modeling, which simplifies practical workflows.","The best configuration's predictive error bars make it suitable for active learning and Bayesian optimization loops aimed at discovering new alloy compositions."],"supporting_citations":[{"why":"Defines deep Gaussian processes as hierarchical compositions of GPs, the model class under test.","marker":"[9]"},{"why":"Supplies the encoder-decoder tabular regression model that is both a benchmark and the source of the prior.","marker":"[11]"},{"why":"Provides the doubly stochastic variational inference procedure used to train the deep GP.","marker":"[22]"},{"why":"Introduces the prior-injection-by-residual modeling idea that the paper adopts.","marker":"[26]"},{"why":"Formulates multi-task Gaussian process prediction for correlated outputs.","marker":"[8]"},{"why":"Gives the co-regionalized multi-output kernel framework that supports heterotopic data handling.","marker":"[21]"},{"why":"Supplies the first experimental campaign's alloys and measurements that form the dataset.","marker":"[23]"},{"why":"Provides the Curtin-Varvenne solid solution strengthening model used to compute predicted yield strength.","marker":"[20]"},{"why":"Provides the machine learning model for stacking fault energy, an auxiliary computational task.","marker":"[19]"}],"fun_headline_variants":["Neural prior boosts deep GP for alloy properties","Residual deep GP with prior wins on HEA tasks","Uncertainty-aware deep GP bests alloy surrogates","Prior-guided deep GP cuts errors in alloys","Deep GP with neural prior tops multi-task HEA models"],"cache_read_input_tokens":17792,"weakest_assumption_plain":"The weakest load-bearing premise is that the DGP's predictive variance on the residuals is the full uncertainty of the final prediction: the encoder-decoder prior is treated as a deterministic function whose own error is never propagated, so if the prior is wrong far from the training data, the reported error bars will be too small.","fun_headline_variants_meta":{"raw":{"variants":["Neural prior boosts deep GP for alloy properties","Residual deep GP with prior wins on HEA tasks","Uncertainty-aware deep GP bests alloy surrogates","Prior-guided deep GP cuts errors in alloys","Deep GP with neural prior tops multi-task HEA models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":1210,"prompt_tokens":932,"completion_tokens":278,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":201}},"tokens_in":548,"tokens_out":278,"duration_ms":4041,"temperature":1.0,"reasoning_tokens":201,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:59:29.605785+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of compositions far from the training region where the encoder-decoder prior is known to be inaccurate, compute HDGP-P-All's predictive intervals, and check how often the true values fall within two standard deviations; if coverage is far below the nominal 95% and uncertainties do not grow outside the training domain, the claim that the model is uncertainty-aware in the stated sense fails.","supporting_citations":[{"cited_title":"Damianou, N","cited_arxiv_id":null,"evidence_quote":"Defines deep Gaussian processes as hierarchical compositions of GPs, the model class under test."},{"cited_title":"Attari, R","cited_arxiv_id":null,"evidence_quote":"Supplies the encoder-decoder tabular regression model that is both a benchmark and the source of the prior."},{"cited_title":"Salimbeni, M","cited_arxiv_id":null,"evidence_quote":"Provides the doubly stochastic variational inference procedure used to train the deep GP."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the prior-injection-by-residual modeling idea that the paper adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Formulates multi-task Gaussian process prediction for correlated outputs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the co-regionalized multi-output kernel framework that supports heterotopic data handling."},{"cited_title":"Hastings, M","cited_arxiv_id":null,"evidence_quote":"Supplies the first experimental campaign's alloys and measurements that form the dataset."},{"cited_title":"Varvenne, A","cited_arxiv_id":null,"evidence_quote":"Provides the Curtin-Varvenne solid solution strengthening model used to compute predicted yield strength."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the machine learning model for stacking fault energy, an auxiliary computational task."}],"review_version":1}