{"id":"86e83819-e034-4331-add1-405be2647d68","arxiv_id":"2501.01148","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ATAIS is an adaptive importance sampler that alternates between sampling nonlinear-model parameters and analytically updating the noise covariance matrix, then reweights old samples to approximate the joint posterior over both.","lead":"This paper proposes two Monte Carlo sampling schemes, ATAIS and ILIS, for jointly estimating the parameters of a nonlinear multioutput model and the covariance matrix of its noise, including uncertainty intervals. The main method alternates importance sampling on the model parameters with a fast covariance estimate, then reuses the samples for a Bayesian analysis of the covariance.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The second-stage reweighting in Section 5 may not be a valid importance approximation of the joint posterior: the θ-proposals are tuned to Σ_ML and are not shown to dominate p(θ|Y,Σ_j) for all Σ_j in the Wishart prior support.","rationale":"The reader's weakest assumption is the convergence of the alternating fixed-point iteration in Section 3. That is a real concern, but it is not the most load-bearing for the paper's headline contribution. The central novelty is the 'complete posterior' approximation over {θ,Σ} without additional model evaluations, and that claim rests on the statistical validity of the second-stage importance reweighting in Section 5. Even if the alternating optimization converged perfectly to the global pair (θMAP,ΣML), the recycled θ-samples are drawn from proposals adapted to p(θ|Y,ΣML), and the proposal support may not cover the conditional posteriors p(θ|Y,Σ_j) for matrices Σ_j in the Wishart prior/proposal support. The paper's own Section 4.4 acknowledges that 'the standard IS denominator provides instability and high variance,' yet the complete-posterior weights in Eq. (48) use exactly that denominator, and no effective-sample-size or variance diagnostic is provided for the joint weights. The empirical Bayes choice of the prior (Section 5.4) further means the 'posterior' intervals are not calibrated under a fixed prior, so the uncertainty analysis over Σ is not substantiated. These issues directly affect the central claim of robust Bayesian inference for covariance matrices and are independent of the fixed-point convergence question. The paper is well-written and the experiments are suggestive, but the missing support for the complete-posterior approximation is a conditionally addressable gap rather than a proven flaw, so CONDITIONAL remains the appropriate verdict.","tokens_in":32069,"tokens_out":7779,"duration_ms":85053,"concrete_test":"Use a linear-Gaussian model y_r = Hθ + v_r, v_r ∼ N(0,Σ), with K=2 and θ∈R^2, where the exact posterior p(θ,Σ|Y) is analytically available via conjugate priors. Run complete ATAIS with T=50, N=100, ν=5, J=1000, computing the joint weights in Eq. (48). Record the effective sample size ESS = (Σβ)^2 / Σβ^2 for each of 100 simulated datasets, and compare the 95% credible interval for Σ with the exact posterior interval. If ESS drops below about 1% of NTJ in any run, or if the empirical coverage of the credible interval differs from 95% by more than the Monte Carlo error, the recycled-sample approximation does not support the claimed complete posterior. Additionally, evaluate the tail ratio sup_θ p(θ|Y,Σ_j)/q(θ|µ_t,Λ_t) on a fine grid for the drawn Σ_j; if this ratio is unbounded, the importance weights lack finite variance.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that complete ATAIS approximates the joint posterior p(θ,Σ|Y) by recycling θ-samples from proposals q(θ|µ_t,Λ_t) that were adapted to the conditional posterior at Σ_ML. The weights in Eq. (48) use the individual proposal as denominator; this is valid only if q dominates p(θ|Y,Σ_j) for every Σ_j in the support of the Wishart prior/proposal qΣ (Section 5.4). That condition is neither proved nor checked. For Σ_j with substantially larger scale or different orientation than Σ_ML, the conditional posterior is broader or shifted, and the Gaussian proposal can have lighter tails, producing importance weights with very large or infinite variance. The paper itself warns in Section 4.4 that the standard IS denominator yields instability and high variance and recommends a mixture denominator, but the second stage still uses the standard denominator. No ESS or variance diagnostic is reported for the joint weights β or the marginal Σ weights λ_j, and the empirical Bayes prior (Section 5.4) is itself data-dependent, so the reported credible intervals are not validated as posterior intervals. Consequently, even if the alternating fixed-point iteration in Section 3 converged perfectly, the claimed approximation of the complete posterior of {θ,Σ} could be degenerate, invalidating the uncertainty analysis over Σ.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript introduces adaptive target adaptive importance sampling (ATAIS) for Bayesian inversion problems with an unknown noise covariance. The first stage alternates an importance-sampling approximation of the conditional posterior p(θ|Y,Σ_ML) with a closed-form maximum-likelihood update of Σ given the current particle maximum; after T iterations it outputs weighted θ particles and estimates (θ_MAP,Σ_ML). The second stage draws covariance matrices from a Wishart prior/proposal whose scale is set to Σ_ML/ν, reweights the existing θ particles with joint importance weights, and thereby claims an approximation of the complete posterior p(θ,Σ|Y) with no additional model evaluations. An alternative scheme, ILIS, is described. Numerical experiments cover localization, a multi-output model, mini-batches, Student-t noise, a biology ODE system, graph topology estimation, and comparisons with MCMC benchmarks.","tokens_in":32453,"tokens_out":5304,"duration_ms":52199,"significance":"The recycling idea is attractive: if the second-stage weights were valid, complete Bayesian inference over θ and Σ would be obtained at negligible extra cost relative to the first stage. The numerical experiments show low MAE in several settings and a favorable comparison against MCMC benchmarks with comparable model evaluations. The paper also demonstrates flexibility with non-Gaussian noise and mini-batches. However, the central claim requires stronger theoretical and numerical support than currently given; in particular, the convergence of the alternating optimization and the validity of the recycled importance weights are not established. With those gaps addressed, the method would be a useful contribution.","major_comments":[{"comment":"Convergence of the alternating optimization is load-bearing and is not demonstrated. The paragraph 'On the convergence of the alternating optimization' asserts that the error can be bounded by increasing N and cites Akyildiz et al. [38] for an O(N^{-1/(2(M+1))}) rate, but [38] analyzes a different sampler and the argument does not show contraction, uniqueness, or global convergence of the fixed-point map (18)-(19). Because each iteration uses the particle maximum θ_max^(t) rather than the exact θ_MAP, the sequence could in principle cycle or converge to a local fixed point; if it does, the final Σ_ML^(T) and every downstream reweighted posterior and credible interval are affected. Please provide a convergence theorem under explicit assumptions, or at minimum a systematic empirical study of restart dependence and path dependence of Σ_ML^(T) with convergence diagnostics, and state the conditions under which the fixed point is global.","section":"Section 3, Eqs. (18)-(19) and Table 1"},{"comment":"The recycled joint weights are not shown to be valid importance weights. The denominator q(θ|μ_t,Λ_t) was adapted to the conditional posterior at Σ_ML, while the numerator targets p(θ|Y,Σ^(j)) for Wishart draws Σ^(j) with potentially larger scale or different orientation. The required domination condition q ≫ p(θ|Y,Σ^(j)) g_θ for all Σ^(j) in the support of q_Σ is neither proved nor checked; if it fails, the weights β have infinite or very large variance. Note that Section 4.4 itself warns that the standard IS denominator produces instability and high variance and recommends a mixture denominator, yet the second stage still uses the standard denominator. Please provide a boundedness/domination argument and report ESS or variance diagnostics for the joint weights β and the marginal weights λ_j.","section":"Section 5.2, Eqs. (48)-(49)"},{"comment":"The prior and proposal for Σ are set to Φ=Σ_ML^(T)/ν, where Σ_ML^(T) is estimated from the same dataset used in the first stage and in the subsequent reweighting. The paper acknowledges this as empirical Bayes, but the reported 95% intervals in Eqs. (67), (70), and (73) are then not calibrated Bayesian posterior intervals; they are conditional on a data-dependent prior location. The claim in Sections 5.2 and 8 that complete ATAIS provides an approximation of the complete posterior should be qualified, and the intervals should be validated by a repeated-sampling coverage study or by a two-stage analysis that accounts for the estimation of Φ.","section":"Section 5.4, Eq. (59) and Section 5"},{"comment":"The graph topology experiment omits essential data-generation details, so the reported 69.7% recovery rate is not interpretable. The text does not specify R, the number of observation vectors, the time points τ, the true covariance Σ_true, or the noise level used to generate Y from Eq. (74); it also does not justify the ad hoc threshold 0.3 applied to P=Σ^{-1} or report sensitivity to that threshold. Without these details, and without a baseline method, the experiment cannot support the claim that ATAIS estimates graph topology well.","section":"Section 7.6"}],"minor_comments":[{"comment":"Typographical errors such as 'di fferent', 'Vwhicular Technology' in reference [10], 'Statistical Sicence' in [47], and 'Statistic Computing' in [59] should be corrected.","section":"Throughout"},{"comment":"Notation should be made more consistent: 'unit matrix' in Eq. (21) should be 'identity matrix'; the normalized weights are written as wbar in Eq. (26) and wbar^(n)_t in Table 8; and the denominator in Eq. (28) should be defined precisely when the sample is indexed by t.","section":"Eqs. (21), (26), (28), Table 8"},{"comment":"In the multi-output model, the row with N=100 and T=50 reports a θtrue MAE of 0.009 while neighboring rows report values around 0.001; please check whether this is a typo.","section":"Table 5"},{"comment":"The marginal likelihood estimator divides by JNT, the product of the number of matrices and all particles; clarify whether N and T include samples discarded by the criterion in Section 4.3, since that criterion suggests saving only a subset of particles.","section":"Section 5.2, Eq. (51)"}],"recommendation":"major_revision","confidential_remarks":"I would not reject the manuscript on the basis of the current experiments, but the central claim in the abstract and Section 8, namely that complete ATAIS provides an approximation of the complete posterior, is stronger than what is proved. The two load-bearing issues, alternating convergence and importance-weight domination, need to be resolved before publication. The paper would also benefit from a reproducibility statement or release of code and data for the numerical experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the first stage of this paper is genuinely useful - alternating IS over theta with closed-form ML updates of Sigma, in a gradient-free setting, for nonlinear multioutput models - and the sample-recycling idea in the second stage is practically nice. But the headline claim, that complete ATAIS approximates the joint posterior p(theta,Sigma|Y), is not backed by the analysis as written, and the paper gives no diagnostics to show the second-stage weights are well behaved.\n\nWhat is actually new: the extension from the authors' earlier scalar-noise ATAIS to a full unknown covariance matrix is non-trivial and useful, and the two-block structure with recycling (no additional evaluations of the expensive model f) is a real practical virtue. The experimental coverage is decent: sensor localization, a multi-output nonlinear model, an ODE biology system, and graph topology estimation with total dimension 59. The MAE tables mostly show expected monotone improvement with N and T, and the method is extended to Student-t noise and mini-batches. The citation pattern is fine; the work is properly anchored to the authors' prior ATAIS papers. Credit is also due for flagging the empirical-Bayes nature of Eq. (59) and for admitting that the convergence argument is heuristic.\n\nSoft spots, in order of size:\n\n1. The joint reweighting in Section 5 is not valid importance sampling as stated. The theta-samples come from proposals tuned to the conditional posterior at Sigma_ML. For a Wishart draw Sigma_j with larger scale or different orientation, nothing guarantees that q(theta|mu_t,Lambda_t) dominates p(theta|Y,Sigma_j), so the weights in Eq. (48) can have large or infinite variance. The paper itself warns in Section 4.4 that the plain IS denominator is unstable and recommends the mixture denominator of Eq. (28) - then uses the plain denominator in Eq. (48). No ESS or weight-variance diagnostics are reported for the joint weights beta or the marginal Sigma weights lambda_j. In the reported credible-interval experiments nu=100, so Wishart draws concentrate near Sigma_ML and the problem may be mild there. But as a generic method, the central claim is not supported. This is addressable: report ESS for the joint weights, use a mixture or defensive denominator, or restrict the claim to settings where the Sigma-proposal is concentrated near the first-stage estimate.\n\n2. Convergence of the alternating fixed-point iteration is asserted, with a rate borrowed from a different sampler; there is no contraction argument. The convergence plots for the test problems look fine, so I read this as a theory gap, not a sign of failure.\n\n3. The graph topology experiment omits key generation details - R, the noise level, the actual Sigma used - and is not reproducible as written.\n\n4. The benchmarks are weak (naive MH variants and a 'helped' MH that uses ATAIS's own Sigma_ML). The no-gradient constraint partially justifies this, but the comparison table should not be read as strong evidence against modern adaptive IS or SMC samplers.\n\nThis paper is for practitioners who need a gradient-free sampler for joint parameter/covariance inference in costly nonlinear models. It deserves a serious referee. I would send it out, and require the second-stage diagnostics (or a softened claim) before acceptance.","headline":"Useful first-stage sampler and a nice recycling trick, but the claimed joint posterior over theta and Sigma is not validated as written - worth refereeing, with diagnostics or a softened claim required.","tokens_in":32891,"tokens_out":6621,"would_cite":true,"duration_ms":60866,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","62H12","65C05"],"pacs":[],"model":"deepseek-v4-flash","headline":"An adaptive importance sampler, ATAIS, jointly infers a nonlinear model's parameters and its noise covariance, then recycles the same particles to approximate the full posterior without extra model evaluations.","keywords":["Bayesian inversion","adaptive importance sampling","covariance matrix","uncertainty analysis","multioutput signals","tempering","sequence of posteriors"],"falsifier":"Run complete ATAIS on a two-mode likelihood whose modes have different residual covariance structures and start the algorithm from several initial matrices; if the sequence of Sigma_t cycles between two matrices or the final weighted posterior concentrates on the mode selected by the initial Sigma regardless of the data, the fixed-point assumption fails. A cleaner check is computing the marginal-likelihood estimate (Eq. (51)) from two independent restarts: if the estimates differ by more than their Monte Carlo error, the recycled cloud is not a faithful representation of the full posterior.","tokens_in":31878,"feed_emoji":"📊","tokens_out":8127,"duration_ms":71927,"temperature":0.7,"pith_summary":"This paper tries to make joint Bayesian inference over two coupled unknowns tractable: the parameters theta of a nonlinear multi-output model and the covariance matrix Sigma of the observation noise. Its proposal, complete ATAIS, splits the joint space into two blocks, runs adaptive importance sampling over theta while updating Sigma with an analytic maximum-likelihood formula, and then reuses the same theta particles to build an approximation of the full posterior over {theta, Sigma} without any additional evaluation of the expensive model. If it works, the payoff is credible intervals on covariance components, model selection via marginal likelihood, and uncertainty analysis in problems where direct sampling on the joint space is impractical. The paper also introduces a simpler competitor, ILIS, and reports numerical evidence that ATAIS outperforms it and standard MCMC benchmarks on several examples.","feed_headline":"One sampler gets parameters and noise covariance together","feed_subtitle":"Alternating particle updates with analytic covariance steps yields full posterior and credible intervals at no extra model cost.","key_machinery":"The load-bearing object is the alternating fixed-point pair formed by Eq. (18) and Eq. (19): at iteration t, theta_MAP is estimated from a cloud of importance samples targeting the conditional posterior with current covariance Sigma_ML^(t-1), and then Sigma_ML^(t) is set to the empirical outer-product average of the residuals at that $\\theta$ estimate. ATAIS also adapts the proposal mean to the current MAP estimate and uses a cyclic scale parameter delta_t for exploration; a final weight correction reweights all particles to the last conditional posterior. The recycling step in the second stage multiplies each particle's conditional weight rho_t^(n)(Sigma) by the prior/proposal ratio gamma_j of a drawn covariance matrix, which is what allows the full posterior and marginals to be assembled from the same $\\theta$ samples.","core_discovery":"The central claim is that the alternating scheme converges to the pair (theta_MAP, Sigma_ML) and that the resulting weighted sample cloud represents the conditional posterior p($\\theta$ | Y, Sigma_ML) after a final reweighting; drawing Wishart matrices centered at Sigma_ML and reweighting the same cloud yields particle approximations of p($\\theta$, Sigma | Y), p($\\theta$ | Y), p(Sigma | Y), and the marginal likelihood. The second stage is essentially free because the residuals e_t,r^(n) = y_r - f_r(theta_t^(n)) are saved during the first stage. Across the paper's numerical experiments, this complete procedure provides low mean absolute error in estimating $\\theta$ and Sigma, including a 59-dimensional graph-topology problem, and outperforms the benchmark algorithms tested.","pith_inferences":["Editorial inference: the recycling step implicitly conditions on the theta-cloud and reweights over Sigma only, so the variance of the resulting posterior estimators should be smaller than sampling the joint space directly, especially when the theta cloud already sits near the mode.","Editorial inference: for large K, the empirical covariance update (Eq. (19)) may be unstable; plugging a shrinkage estimator into the same alternating loop would be a natural, untested variant.","Editorial inference: the Wishart prior location is set to the data-derived Sigma_ML, so for small sample sizes R the reported credible intervals may understate the true uncertainty; a hierarchical prior over the reference matrix would test this.","Editorial inference: running the complete procedure from multiple random starts and comparing the marginal-likelihood estimate (Eq. (51)) would make a practical convergence diagnostic for the alternating fixed-point step."],"forward_implications":["Complete ATAIS delivers an approximation of p(theta, Sigma | Y) at the same model-evaluation cost as sampling theta alone, because the covariance stage reuses saved residuals.","Credible intervals for individual covariance entries, such as the 95% intervals reported for the sensor-localization and biological-system examples, become computable by resampling Wishart draws with weights lambda_j.","The method extends to Student-t and other elliptical noises through a fixed-point covariance estimator, and to large datasets via mini-batches, with experiments showing small errors in both settings.","In the 59-dimensional graph-topology example the precision matrix recovered by thresholding the estimated Sigma matches the ground truth adjacency in 69.7% of runs."],"supporting_citations":[{"why":"Introduces the earlier adaptive-target importance sampling scheme that complete ATAIS extends to joint inference over theta and Sigma.","marker":"[15]"},{"why":"Companion optimization-based Bayesian inversion scheme that motivates the alternating MAP/ML update structure.","marker":"[16]"},{"why":"Provides the recycling idea of using different numbers of samples per block, which justifies reweighting NT theta-samples against J covariance samples.","marker":"[7]"},{"why":"Supplies the multiple importance sampling weighting used when the proposal denominator is a mixture, reducing estimator variance.","marker":"[47]"},{"why":"Establishes the fixed-point estimator for scale matrices of elliptical distributions used for the Student-t noise extension.","marker":"[43]"},{"why":"Cites a well-conditioned high-dimensional covariance estimator as the safeguard when K is large and the empirical Sigma_ML is unstable.","marker":"[32]"},{"why":"Supports the empirical-Bayes choice of prior densities from the data, used to center the Wishart prior at Sigma_ML.","marker":"[27]"}],"fun_headline_variants":["Sampler locks parameters and noise covariance in one pass","Adaptive sampler yields full Bayesian covariance posterior","One sampler, two blocks: parameters and noise covariance","Recycling samples gives covariance posterior for free","AIS scheme computes parameter and covariance posteriors together"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole scheme assumes that the back-and-forth updates between the best parameter point and the best noise covariance actually settle on the single globally best pair, instead of cycling or locking onto a less good local answer; the paper only argues that using more particles shrinks the error, without proving this convergence.","fun_headline_variants_meta":{"raw":{"variants":["Sampler locks parameters and noise covariance in one pass","Adaptive sampler yields full Bayesian covariance posterior","One sampler, two blocks: parameters and noise covariance","Recycling samples gives covariance posterior for free","AIS scheme computes parameter and covariance posteriors together"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000548,"raw_usage":{"total_tokens":2608,"prompt_tokens":923,"completion_tokens":1685,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":1613}},"tokens_in":539,"tokens_out":1685,"duration_ms":10916,"temperature":1.0,"reasoning_tokens":1613,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:34:05.440993+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run complete ATAIS on a two-mode likelihood whose modes have different residual covariance structures and start the algorithm from several initial matrices; if the sequence of Sigma_t cycles between two matrices or the final weighted posterior concentrates on the mode selected by the initial Sigma regardless of the data, the fixed-point assumption fails. A cleaner check is computing the marginal-likelihood estimate (Eq. (51)) from two independent restarts: if the estimates differ by more than their Monte Carlo error, the recycled cloud is not a faithful representation of the full posterior.","supporting_citations":[{"cited_title":"Martino, F","cited_arxiv_id":null,"evidence_quote":"Introduces the earlier adaptive-target importance sampling scheme that complete ATAIS extends to joint inference over theta and Sigma."},{"cited_title":"Lopez-Santiago, L","cited_arxiv_id":null,"evidence_quote":"Companion optimization-based Bayesian inversion scheme that motivates the alternating MAP/ML update structure."},{"cited_title":"Martino, V","cited_arxiv_id":null,"evidence_quote":"Provides the recycling idea of using different numbers of samples per block, which justifies reweighting NT theta-samples against J covariance samples."},{"cited_title":"Elvira, L","cited_arxiv_id":null,"evidence_quote":"Supplies the multiple importance sampling weighting used when the proposal denominator is a mixture, reducing estimator variance."},{"cited_title":"Bilodeau and D","cited_arxiv_id":null,"evidence_quote":"Establishes the fixed-point estimator for scale matrices of elliptical distributions used for the Student-t noise extension."},{"cited_title":"Ledoit and M","cited_arxiv_id":null,"evidence_quote":"Cites a well-conditioned high-dimensional covariance estimator as the safeguard when K is large and the empirical Sigma_ML is unstable."},{"cited_title":"Llorente, L","cited_arxiv_id":null,"evidence_quote":"Supports the empirical-Bayes choice of prior densities from the data, used to center the Wishart prior at Sigma_ML."}],"review_version":1}