Recognition: unknown
Online Bayesian Calibration under Gradual and Abrupt System Changes
Pith reviewed 2026-05-08 12:15 UTC · model grok-4.3
The pith
BRPC separates parameter estimation from bias correction to enable online Bayesian calibration under gradual drifts and abrupt shifts.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
BRPC extends projected calibration to the online setting by separating a discrepancy-free particle update for calibration parameters from a conditional Gaussian process update for discrepancy, preserving identifiability while enabling bias-aware adaptation under gradual system evolution, and integrates restart mechanisms to detect and handle abrupt regime shifts, with established theoretical guarantees for tracking performance and false-alarm behavior.
What carries the argument
The separation of a discrepancy-free particle update for calibration parameters from a conditional Gaussian process update for discrepancy, together with restart mechanisms that detect regime shifts.
Load-bearing premise
The separation of discrepancy-free parameter updates from discrepancy modeling continues to prevent confounding and maintain identifiability when data arrive sequentially from a nonstationary source.
What would settle it
Run the method on synthetic data with known drifting calibration parameters plus a fixed model bias; if the recovered parameter trajectories deviate persistently from the true drifts while the discrepancy term absorbs the parameter error, the separation has failed to preserve identifiability.
Figures
read the original abstract
Bayesian model calibration is central to digital twins and computer experiments, as it aligns model outputs with field observations by estimating calibration parameters and correcting systematic model bias. Classical Bayesian calibration introduces latent parameters and a discrepancy function to model bias, but suffers from parameter--discrepancy confounding and is typically formulated as an offline procedure under a stationary data-generating assumption. These limitations are restrictive in modern digital twin applications, where systems evolve over time and may exhibit gradual drift and abrupt regime shifts. While data assimilation methods enable sequential updates, they generally do not explicitly model systematic bias and are less effective under abrupt changes. We propose Bayesian Recursive Projected Calibration (BRPC), an online Bayesian calibration framework for streaming data under simulator mismatch and nonstationarity. BRPC extends projected calibration to the online setting by separating a discrepancy-free particle update for calibration parameters from a conditional Gaussian process update for discrepancy, preserving identifiability while enabling bias-aware adaptation under gradual system evolution. To handle abrupt changes, BRPC is integrated with restart mechanisms that detect regime shifts and reset the calibration process. We establish theoretical guarantees for both components, including tracking performance under gradual evolution and false-alarm and detection behavior for restart mechanisms. Empirical studies on synthetic and plant-simulation benchmarks show that BRPC improves calibration accuracy under gradual changes, while restart-augmented BRPC further improves robustness and predictive performance under abrupt regime shifts compared to sliding-window Bayesian calibration and data assimilation baselines.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes Bayesian Recursive Projected Calibration (BRPC) as an online framework for Bayesian model calibration under simulator mismatch and nonstationarity. It extends projected calibration by separating a discrepancy-free particle-filter update on calibration parameters from a conditional Gaussian-process update on the discrepancy function, adds restart mechanisms to detect and adapt to abrupt regime shifts, claims theoretical tracking and detection guarantees, and reports empirical gains over sliding-window Bayesian calibration and data-assimilation baselines on synthetic and plant-simulation benchmarks.
Significance. If the separation and restart mechanisms are shown to preserve identifiability and deliver the claimed tracking performance, the work would provide a principled online alternative to classical offline Bayesian calibration for digital-twin applications that must handle gradual drift and abrupt changes. The explicit combination of particle filtering with conditional GPs and change-point detection is a concrete technical contribution that could be adopted in streaming calibration pipelines.
major comments (2)
- [method description (post-abstract)] The central separation of a discrepancy-free particle update for calibration parameters from a conditional GP update for discrepancy (described in the method section following the abstract) is asserted to preserve identifiability, yet no explicit bound is given on the coupling error induced by finite-particle approximation or tracking lag when the underlying parameters drift continuously. This is load-bearing for the claim of unbiased online adaptation under gradual nonstationarity.
- [theory section] Theoretical guarantees for tracking performance under gradual evolution and for false-alarm/detection behavior of the restart mechanism are stated in the abstract and presumably derived in the theory section, but the manuscript provides no proof sketch or key lemma showing that the conditional GP update remains unbiased once the particle estimates are only approximate.
minor comments (2)
- [introduction] The abstract and introduction would benefit from a short paragraph contrasting BRPC with existing online calibration methods that already combine filtering and discrepancy modeling, to clarify the precise novelty.
- [empirical studies] Notation for the particle weights and the conditional GP kernel should be introduced once and used consistently; several symbols appear without prior definition in the empirical section.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed comments on our manuscript. The points raised highlight important aspects of the theoretical analysis that we agree merit further elaboration. We address each major comment below and commit to specific revisions that strengthen the presentation without altering the core contributions.
read point-by-point responses
-
Referee: [method description (post-abstract)] The central separation of a discrepancy-free particle update for calibration parameters from a conditional GP update for discrepancy (described in the method section following the abstract) is asserted to preserve identifiability, yet no explicit bound is given on the coupling error induced by finite-particle approximation or tracking lag when the underlying parameters drift continuously. This is load-bearing for the claim of unbiased online adaptation under gradual nonstationarity.
Authors: We agree that the manuscript would benefit from an explicit error bound on the finite-particle approximation under continuous parameter drift. The separation is constructed so that the particle update operates on residuals orthogonal to the discrepancy space, inheriting identifiability from the offline projected calibration framework. However, the current text invokes standard particle-filter convergence without deriving a drift-specific coupling bound. In the revision we will add a proposition (placed after the method description) that bounds the total-variation distance between the approximate and ideal parameter posterior by a term that decays with particle count and grows linearly with the supremum drift rate, under standard Lipschitz and boundedness assumptions on the likelihood. revision: yes
-
Referee: [theory section] Theoretical guarantees for tracking performance under gradual evolution and for false-alarm/detection behavior of the restart mechanism are stated in the abstract and presumably derived in the theory section, but the manuscript provides no proof sketch or key lemma showing that the conditional GP update remains unbiased once the particle estimates are only approximate.
Authors: The theory section establishes tracking and detection guarantees under the idealized assumption of exact parameter knowledge; we acknowledge that the propagation of particle-filter approximation error into the conditional GP update is not accompanied by a proof sketch or key lemma. In the revised manuscript we will insert a short lemma immediately preceding the main tracking theorem. The lemma shows that the conditional GP posterior mean remains unbiased up to an additive term controlled by the particle approximation error, because the GP update is linear in the residuals and the separation ensures that the expected residual given the approximate parameters equals the true discrepancy plus a vanishing bias term. revision: yes
Circularity Check
No significant circularity; derivation extends prior framework with independent theoretical guarantees and empirical validation
full rationale
The paper defines BRPC by separating a discrepancy-free particle update for calibration parameters from a conditional Gaussian process update for discrepancy, extending projected calibration to the online nonstationary setting while claiming identifiability preservation and providing tracking guarantees plus restart mechanisms. No equations or derivations in the provided abstract reduce any prediction or result to fitted inputs by construction, nor do they rely on self-citations as the sole unverified load-bearing premise. The central claims are supported by stated theoretical results on gradual evolution and abrupt-shift detection, plus comparisons to sliding-window and data-assimilation baselines on synthetic and plant-simulation benchmarks, keeping the chain self-contained.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Projected calibration separation of parameters and discrepancy preserves identifiability in the online streaming setting
- domain assumption Regime shifts can be reliably detected by a restart mechanism without excessive false alarms
Reference graph
Works this paper leans on
-
[1]
Bayesian Online Changepoint Detection
Ryan Prescott Adams and David JC MacKay. Bayesian online changepoint detection.arXiv preprint arXiv:0710.3742, 2007
work page Pith review arXiv 2007
-
[2]
Restarted Bayesian online change-point detector achieves optimal detection delay
Réda Alami, Odalric Maillard, and Raphael Féraud. Restarted Bayesian online change-point detector achieves optimal detection delay. InInternational Conference on Machine Learning, pages 211–221. PMLR, 2020
2020
-
[3]
Online continual learning with maximal interfered retrieval
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. Advances in Neural Information Processing Systems, 32, 2019
2019
-
[4]
Mirror descent and nonlinear projected subgradient methods for convex optimization.Operations Research Letters, 31(3):167–175, 2003
Amir Beck and Marc Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization.Operations Research Letters, 31(3):167–175, 2003
2003
-
[5]
Streaming variational Bayes.Advances in Neural Information Processing Systems, 26, 2013
Tamara Broderick, Nicholas Boyd, Andre Wibisono, Ashia C Wilson, and Michael I Jordan. Streaming variational Bayes.Advances in Neural Information Processing Systems, 26, 2013
2013
-
[6]
Dark experience for general continual learning: a strong, simple baseline.Advances in Neural Information Processing Systems, 33:15920–15930, 2020
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline.Advances in Neural Information Processing Systems, 33:15920–15930, 2020
2020
-
[7]
A continual learning survey: Defying forgetting in classification tasks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7): 3366–3385, 2021
Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7): 3366–3385, 2021
2021
-
[8]
Composite objective mirror descent
John C Duchi, Shai Shalev-Shwartz, Yoram Singer, and Ambuj Tewari. Composite objective mirror descent. InColt, volume 10, pages 14–26, 2010
2010
-
[9]
The ensemble Kalman filter: Theoretical formulation and practical implementa- tion.Ocean Dynamics, 53(4):343–367, 2003
Geir Evensen. The ensemble Kalman filter: Theoretical formulation and practical implementa- tion.Ocean Dynamics, 53(4):343–367, 2003
2003
-
[10]
The ensemble Kalman filter for combined state and parameter estimation.IEEE Control Systems Magazine, 29(3):83–104, 2009
Geir Evensen. The ensemble Kalman filter for combined state and parameter estimation.IEEE Control Systems Magazine, 29(3):83–104, 2009
2009
-
[11]
On-line inference for multiple changepoint problems.Journal of the Royal Statistical Society Series B: Statistical Methodology, 69(4):589–605, 2007
Paul Fearnhead and Zhen Liu. On-line inference for multiple changepoint problems.Journal of the Royal Statistical Society Series B: Statistical Methodology, 69(4):589–605, 2007
2007
-
[12]
A survey on concept drift adaptation.ACM Computing Surveys (CSUR), 46(4):1–37, 2014
João Gama, Indr˙e Žliobait ˙e, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation.ACM Computing Surveys (CSUR), 46(4):1–37, 2014
2014
-
[13]
Scaled Gaussian stochastic process for computer model calibration and prediction.SIAM/ASA Journal on Uncertainty Quantification, 6(4):1555–1583, 2018
Mengyang Gu and Long Wang. Scaled Gaussian stochastic process for computer model calibration and prediction.SIAM/ASA Journal on Uncertainty Quantification, 6(4):1555–1583, 2018
2018
-
[14]
Dynamical models and tracking regret in online convex pro- gramming
Eric Hall and Rebecca Willett. Dynamical models and tracking regret in online convex pro- gramming. InInternational Conference on Machine Learning, pages 579–587. PMLR, 2013
2013
-
[15]
Online convex optimization in dynamic environments
Eric C Hall and Rebecca M Willett. Online convex optimization in dynamic environments. IEEE Journal of Selected Topics in Signal Processing, 9(4):647–662, 2015
2015
-
[16]
Bayesian calibration of computer models.Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(3):425–464, 2001
Marc C Kennedy and Anthony O’Hagan. Bayesian calibration of computer models.Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(3):425–464, 2001. 10
2001
-
[17]
Overcoming catastrophic forgetting in neural networks.Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks.Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017
2017
-
[18]
Shunji Kotsuki, Koji Terasaki, Hisashi Yashiro, Hirofumi Tomita, Masaki Satoh, and Takemasa Miyoshi. Online model parameter estimation with ensemble data assimilation in the real global atmosphere: A case with the nonhydrostatic icosahedral atmospheric model (NICAM) and the global satellite mapping of precipitation data.Journal of Geophysical Research: At...
2018
-
[19]
Particle filters for high-dimensional geoscience applications: A review.Quarterly Journal of the Royal Meteorological Society, 145(723):2335–2365, 2019
Van Leeuwen, Peter Jan, Hans R Künsch, Lars Nerger, Roland Potthast, and Sebastian Reich. Particle filters for high-dimensional geoscience applications: A review.Quarterly Journal of the Royal Meteorological Society, 145(723):2335–2365, 2019
2019
-
[20]
Gradient episodic memory for continual learning
David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in Neural Information Processing Systems, 30, 2017
2017
-
[21]
Learning under concept drift: A review.IEEE Transactions on Knowledge and Data engineering, 31(12): 2346–2363, 2018
Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Joao Gama, and Guangquan Zhang. Learning under concept drift: A review.IEEE Transactions on Knowledge and Data engineering, 31(12): 2346–2363, 2018
2018
-
[22]
Stochastic model predictive control: An overview and perspectives for future research.IEEE Control Systems Magazine, 36(6):30–44, 2016
Ali Mesbah. Stochastic model predictive control: An overview and perspectives for future research.IEEE Control Systems Magazine, 36(6):30–44, 2016
2016
-
[23]
Variational continual learning.arXiv preprint arXiv:1710.10628, 2017
Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. Variational continual learning.arXiv preprint arXiv:1710.10628, 2017
-
[24]
Continuous inspection schemes.Biometrika, 41(1/2):100–115, 1954
Ewan S Page. Continuous inspection schemes.Biometrika, 41(1/2):100–115, 1954
1954
-
[25]
Experi- ence replay for continual learning.Advances in Neural Information Processing Systems, 32, 2019
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experi- ence replay for continual learning.Advances in Neural Information Processing Systems, 32, 2019
2019
-
[26]
Yvonne M Ruckstuhl and Tijana Janji´c. Parameter and state estimation with ensemble Kalman filter based algorithms for convective-scale applications.Quarterly Journal of the Royal Meteorological Society, 144(712):826–841, 2018
2018
-
[27]
Gaussian process change point models
Yunus Saatçi, Ryan D Turner, and Carl E Rasmussen. Gaussian process change point models. InInternational Conference on Machine Learning, pages 927–934, 2010
2010
-
[28]
Progress & compress: A scalable framework for continual learning
Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for continual learning. InInternational Conference on Machine Learning, pages 4528–4537. PMLR, 2018
2018
-
[29]
Online learning and online convex optimization.Foundations and Trends® in Machine Learning, 4(2):107–194, 2011
Shai Shalev-Shwartz. Online learning and online convex optimization.Foundations and Trends® in Machine Learning, 4(2):107–194, 2011
2011
-
[30]
Rui Tuo and C. F. Jeff Wu. Efficient calibration for imperfect computer models.The Annals of Statistics, 43(6):2331–2352, 2015
2015
-
[31]
Continuous calibration of a digital twin: Comparison of particle filter and Bayesian calibration approaches.Data-Centric Engineering, 2:e15, 2021
Rebecca Ward, Ruchi Choudhary, Alastair Gregory, Melanie Jans-Singh, and Mark Girolami. Continuous calibration of a digital twin: Comparison of particle filter and Bayesian calibration approaches.Data-Centric Engineering, 2:e15, 2021
2021
-
[32]
Characteriz- ing concept drift.Data Mining and Knowledge Discovery, 30(4):964–994, 2016
Geoffrey I Webb, Roy Hyde, Hong Cao, Hai Long Nguyen, and Francois Petitjean. Characteriz- ing concept drift.Data Mining and Knowledge Discovery, 30(4):964–994, 2016
2016
-
[33]
MIT Press Cambridge, MA, 2006
Christopher KI Williams and Carl Edward Rasmussen.Gaussian processes for machine learning, volume 2. MIT Press Cambridge, MA, 2006
2006
-
[34]
Bayesian projected calibration of computer models.Journal of the American Statistical Association, 116(536):1965–1982, 2021
Fangzheng Xie and Yanxun Xu. Bayesian projected calibration of computer models.Journal of the American Statistical Association, 116(536):1965–1982, 2021. 11
1965
-
[35]
Window-limited CUSUM for sequential change detection.IEEE Transactions on Information Theory, 69(9):5990–6005, 2023
Liyan Xie, George V Moustakides, and Yao Xie. Window-limited CUSUM for sequential change detection.IEEE Transactions on Information Theory, 69(9):5990–6005, 2023
2023
-
[36]
Yang Xu and Chiwoo Park. Deep jump Gaussian processes for surrogate modeling of high- dimensional piecewise continuous functions.arXiv preprint arXiv:2510.21974, 2025
-
[37]
Continual learning through synaptic intelligence
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. InInternational Conference on Machine Learning, pages 3987–3995. PMLR, 2017
2017
-
[38]
pre” for pre-update predictive quantities, superscript “sh
Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. InProceedings of the 20th International Conference on Machine Learning (ICML-03), pages 928–936, 2003. 12 Appendix Appendix organization.Appendix A gives the algorithmic details, including restart initialization and the BOCPD-style and wCUSUM-style restart procedure...
2003
-
[39]
In the sinusoidal configu- ration, θb = 11.5 + 8.5 sin 2πb 40
Drifting mode:the latent arrival-rate parameter varies smoothly. In the sinusoidal configu- ration, θb = 11.5 + 8.5 sin 2πb 40
-
[40]
Sudden-change mode:the latent arrival-rate parameter is piecewise constant with abrupt regime switches
-
[41]
Mixed mode:the latent arrival-rate parameter combines continuous variation with abrupt structural changes. The goal is to infer the latent arrival-rate parameter online from the revenue stream while maintaining synchronization between the physical system and the digital twin. This benchmark is intended to test whether the qualitative conclusions from the ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.