{"id":"9d349f28-c78b-4863-8a50-7b4f26867535","arxiv_id":"2504.17966","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A conformal prediction trigger switches robot trajectory prediction from a fast neural net to an uncertainty-aware Gaussian process Port-Hamiltonian model when test dynamics go out of distribution.","lead":"This paper proposes a plug-and-play framework that uses conformal prediction to spot when a trained neural predictor fails on out-of-distribution data, then switches to a physics-consistent Gaussian process Port-Hamiltonian model. A generalist might care because it offers a way to keep robot trajectory predictions reliable when the dynamics of obstacles change.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The conformal trigger as written requires ground-truth future states Z(t+1) to compute the nonconformity score, so the switch cannot precede the predictions it is meant to safeguard; the paper does not demonstrate a causally valid online mode.","rationale":"The reader's weakest assumption is exactly the point I would stress. The strongest claim is not the GP-dPHS model alone, which is grounded in prior work, but the plug-and-play switching mechanism that makes the framework broadly applicable; if the trigger cannot run before the target prediction, the central value proposition is reduced to an offline retraining scheme. I agree with CONDITIONAL rather than REJECT because the concern is fixable: the authors could either clearly state and evaluate a lag-1 causal trigger, or re-scope the contribution to offline detection from observed history, and correct the conformal-confidence sentence to a false-positive rate statement. The experimental section is a single trajectory with no error bars, and the promised code link is a placeholder; these strengthen the case for conditional acceptance with reproducibility requirements, but they are not the most load-bearing issue. The paper does not contain an explicit limitation section, and Assumption 2 (observability) is the only stated caveat; neither changes the analysis. If the causal test fails, I would move toward REJECT or UNVERDICTED for the online claim, but the offline version may remain conditionally acceptable.","tokens_in":9192,"tokens_out":9237,"duration_ms":96830,"concrete_test":"Run the provided code and dataset in a strictly causal mode: at each time T, compute the score using only observations up to T, e.g., R(T)=||s(T)-f(X(T-1))||, decide whether to switch, then predict s(T+1) through s(T+w) with the chosen branch; compare the resulting MSE against always-nominal and always-GP-dPHS baselines over the same horizon. Also record, on held-out ID data, the empirical frequency of R>C; if it exceeds δ, the stated OOD-probability guarantee is false. This single evaluation would settle whether the trigger is available at switching time and whether the conformal calibration is used correctly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central 'plug-and-play' mechanism hinges on the detection rule in Eq. (1) and Sec. II-A: R(t+1)=||Z(t+1)-f(X(t))||. At the moment a prediction of Z(t+1) is needed, Z(t+1) is not yet available, so R(t+1) cannot be computed before the nominal predictor f issues its output. The description in Sec. III-B.1, where R(i) is 'derived from the loss between the prediction P and real observations' at a 'future point i', and Algorithm 1, which computes the score from f(D1) 'based on the observations', both confirm that the trigger fires only after the prediction can be compared with the observed future state. The claimed ability to switch before a bad prediction is therefore not supported as stated; the method is at best a one-step-lag detector or an offline detector that first observes a history of OOD frames and only then activates the GP-dPHS branch. The experiment, which trains on the first 80% of the OOD trajectory and tests on the remaining 20%, is compatible with the offline reading but does not test the online plug-and-play claim. A second, compounding overclaim is the probabilistic interpretation: Eq. (2) gives P(R≤C)≥1−δ under exchangeability of calibration and test scores, which bounds false positives under the ID distribution; it does not imply P(OOD | R>C)≥1−δ, so the sentence 'we can determine it as OOD with probability (1−δ)' reverses the implication. Both issues sit at the triggering premise, so without a fix or a clear re-scoping to offline use, the headline contribution is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PnP-PIML, a two-branch prediction framework for robotic trajectory forecasting. A nominal data-driven predictor is used by default; a conformal-prediction routine computes the predictor's error on recent observations, and when that error exceeds a calibrated threshold R(t+1)>C, the framework switches to a Gaussian-process distributed Port-Hamiltonian system (GP-dPHS) branch. The GP-dPHS learns an unknown Hamiltonian from limited outlier-distribution observations and predicts with physical consistency and Bayesian uncertainty. The method is evaluated on a real oscillating-spring dataset, comparing against a retrained LSTM and a vanilla GP. The central claims are that the framework detects outlier dynamics and produces reliable physics-informed predictions in out-of-distribution scenarios, and that it can be plugged into any existing data-driven predictor.","tokens_in":9538,"tokens_out":6771,"duration_ms":68831,"significance":"If the switching mechanism works as advertised, PnP-PIML would be practically useful: it offers a lightweight way to keep a fast nominal predictor while resorting to a more expensive, physically structured model only when needed. The experimental demonstration is on real data, the code and dataset link is provided, and the GP-dPHS branch is grounded in prior work by the same group. I found no circularity in the evaluation: the GP-dPHS is trained on the first 80% of the OOD frames and evaluated on the held-out 20%, with the damping coefficient fitted on the training portion. The main novelty is the conformal-prediction trigger, and that is also where the paper's central claims need the most careful scrutiny. There is no new formal proof, and the results section would be stronger with repeated trials or error bars, but the core idea is defensible if the trigger is re-scoped to a causal or offline setting.","major_comments":[{"comment":"The nonconformity score R(t+1) = ||Z(t+1) - f(X(t))|| requires the ground-truth future state Z(t+1) at the moment the trigger is evaluated. At the time the nominal predictor issues its forecast, Z(t+1) is not yet available, so the switch cannot precede the bad prediction it is intended to catch. Algorithm 1 also computes Rnew from f(D1) before waking the dPHS branch, and the experiment in Section IV-D trains on the first 80% of the 234 OOD frames and tests on the last 20%, which is an offline reading. The introduction's statement that the framework can switch to the physics-consistent model when unreliable predictions are detected is therefore not supported as stated. Please either re-scope the claim to offline/one-step-lag detection, or provide a causally valid trigger (for example, a window of past errors) and demonstrate it in an online experiment.","section":"Section II-A, Eq. (1) and Algorithm 1"},{"comment":"The sentence 'as long as the new test score R(t+1)... is greater than the constant C, we can determine it as OOD with probability (1-δ)' reverses the logical direction of conformal prediction. Equation (2) bounds the probability that an exchangeable ID test score is at most C, i.e., P(R≤C) ≥ 1-δ, which controls the false-positive rate under D0. It does not imply P(OOD | R>C) ≥ 1-δ; that posterior probability depends on the prior probability of OOD and on the alternative distribution, neither of which is specified. The same reversal appears in Section III-B.2 ('we detect the OOD case under (1-δ) confidence'). Please replace this language with a statement about false-alarm control under D0, or add the assumptions that would justify a posterior claim.","section":"Section II-A, Eq. (2) and following paragraph"},{"comment":"The title and abstract promise uncertainty-quantified predictions, but the uncertainty is not propagated to the final trajectory. The procedure samples a deterministic Hamiltonian from the GP distribution and propagates that sample through the numerical solver, and the experimental section reports only MSE curves in Figure 6, with no prediction intervals or coverage evaluation for the GP-dPHS output. To support the 'reliable predictions' claim, the paper should either propagate the GP posterior covariance through the dPHS to the predicted trajectory, or generate an ensemble of trajectories and evaluate their empirical coverage. As written, the UQ stops at the Hamiltonian and does not reach the user-facing prediction.","section":"Section III-C.1 and Section IV-D"}],"minor_comments":[{"comment":"The first line uses 'non-conformality score'; the standard term is 'nonconformity score'.","section":"Algorithm 1"},{"comment":"The paragraph beginning 'We denote the temporal and spatial state variables (p, q) as x' is confusing because x is used both as the state variable and as the position variable in the definitions p = ∂x/∂t and q = ∂x/∂z. Please rename the position variable (for example, s) to make the definition of p and q unambiguous.","section":"Section III-B.1"},{"comment":"The sentence 'with the the spatial and temporal variables' repeats the article 'the'; this should be corrected.","section":"Section IV-B"},{"comment":"The quantile level (1+1/K)(1-δ) can exceed 1 for small calibration sets (roughly K < 1/δ - 1), so the intended finite-sample construction is not stated. Please use the standard CEIL((1-δ)(K+1))/K quantile level or explicitly require K sufficiently large.","section":"Equation (2)"},{"comment":"The phrase 'which is outbound of the CP (white)' is unclear; the intended meaning appears to be 'outside the CP region shown in white'.","section":"Figure 1 caption"}],"recommendation":"major_revision","confidential_remarks":"The manuscript leans heavily on the authors' own previous GP-dPHS work, so the incremental contribution is the conformal-prediction trigger and the plug-and-play framing. The major issues identified above are fixable: the trigger needs to be re-scoped to a causally valid setting or demonstrated online, the probability statement must be corrected to a false-alarm interpretation, and the UQ claim needs to be tied to the predicted trajectory. With those changes, the paper could be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe short version: the paper has a sensible hybrid idea — use a cheap nominal predictor and reserve a physics-informed GP-dPHS model for out-of-distribution cases — but the conformal trigger that makes it 'plug-and-play' does not work the way the paper claims. The score R(t+1) is defined as the error between the prediction and the ground-truth future Z(t+1), so the switch can only fire after the prediction is already made and the true future is observed. That is a one-step-lag or offline detector, not a mechanism for avoiding bad predictions. The paper also overstates the conformal guarantee: Eq. (2) bounds the probability that an in-distribution score exceeds C, but the text reverses this to claim a score above C identifies OOD with probability 1−δ.\n\nWhat is genuinely new is the integration of the two pieces. Neither the conformal trigger nor the GP-dPHS is new on its own — the latter is the authors’ prior work — but the switching architecture is a legitimate extension, and the motivation is clear: spend computation only when the cheap predictor fails. The paper is not circular; the fallback is trained on held-out frames. The experimental setup uses a real oscillating spring, which is a plus, and the physics-informed model beats retrained DNN and vanilla GP baselines. But it is a single scenario with no error bars, and the detection step is not causally validated.\n\nThe bigger issue is the framing. If the method were described as an offline switch — e.g., you have a batch of OOD frames, detect them, then train the physics model — the conformal setup would be more defensible, though the probability interpretation would still need correction. As written, the headline 'switch before bad predictions' is unsupported.\n\nI would send this to a serious reviewer for a robotics or L4DC-style venue, but only with the expectation of major revision. The authors need to either make the score causal (using only past observations) or explicitly narrow the claim to offline detection. The physics-informed model is worth engaging with; the detection guarantee is not. A conditional accept with insistence on fixing the trigger description would be appropriate.","headline":"The physics-informed fallback is real, but the conformal trigger cannot switch before the prediction it is meant to safeguard as written; the paper needs re-scoping or a causal score.","tokens_in":10070,"tokens_out":2610,"would_cite":false,"duration_ms":27386,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a conformal-prediction trigger can catch out-of-distribution dynamics and switch to a Gaussian-process Port-Hamiltonian model, producing physically consistent forecasts with uncertainty even when the original…","keywords":["conformal prediction","out-of-distribution detection","port-Hamiltonian systems","Gaussian processes","physics-informed learning","trajectory prediction","uncertainty quantification","PDE systems"],"falsifier":"Run the framework in an online loop where the true next state arrives after the predictor must be committed, such as a one-step-delayed observation. Record how many unsafe forecasts the nominal model produces before the conformal score first exceeds the threshold; if that count is positive, the protective-switch claim fails. Alternatively, feed the calibration set a sequence whose scores are non-exchangeable, such as a linear drift, and measure how often the fixed threshold $C$ misclassifies in-distribution data as out-of-distribution; if the error rate exceeds the advertised $\\delta$, the stated detection guarantee is false.","tokens_in":8975,"feed_emoji":"🧩","tokens_out":6787,"duration_ms":59997,"temperature":0.7,"pith_summary":"This paper tries to establish that a plug-and-play layer can rescue a pre-trained data-driven predictor when the dynamics it was trained on shift. The layer uses conformal prediction to flag when the predictor's error on new observations exceeds a calibrated threshold, and then hands the forecasting task to a Gaussian-process distributed Port-Hamiltonian system that learns the system's energy (Hamiltonian) from the limited new data. If the framework works as claimed, a robot facing a previously unlearned obstacle, such as a swinging cable instead of a rigid stick, could switch from a fast but unreliable neural forecast to a physics-consistent model with confidence intervals, without retraining the original network. The paper demonstrates the idea on an oscillating spring whose transversal motion is predicted from real video data.","feed_headline":"Conformal alarm hands failing predictors to a physics model","feed_subtitle":"Detects when data-driven forecasts go stale, then learns the system's energy with quantified uncertainty.","key_machinery":"The load-bearing mechanism is the pair consisting of the conformal trigger and the GP-dPHS branch. The conformal trigger is the nonconformity score $R(t+1)$ computed as the prediction error of the nominal model; its calibration quantile $C$ is set from exchangeable in-distribution scores, so exceeding $C$ is taken to flag outlier dynamics. The physics branch is the Gaussian-process distributed Port-Hamiltonian system (GP-dPHS): the unknown Hamiltonian functional $H$ of a PDE in Port-Hamiltonian form $\\partial x/\\partial t=(J-R)\\delta_x H+G_d u$ is endowed with a GP prior whose kernel is the squared exponential twisted by the differential operator $(\\hat J_\\Theta-\\hat R_\\Theta)$, giving physically structured uncertainty. A sampled Hamiltonian is then fed to a numerical solver to generate predictions.","core_discovery":"The central claim is that out-of-distribution dynamics can be detected in a deployment-agnostic way, and that a physics-informed learner using only the scarce out-of-distribution observations can out-predict retrained data-driven baselines on the new regime. Concretely, the framework computes a nonconformity score $R(t+1)=\\|Z(t+1)-f(X(t))\\|$ for the nominal predictor $f$, compares it with the $(1-\\delta)$-quantile $C$ of calibration scores, and declares the test point out-of-distribution when $R(t+1)>C$. On detection, it encodes the new observations into a distributed Port-Hamiltonian system whose Hamiltonian functional is modeled as a Gaussian process, samples a deterministic Hamiltonian from that GP, and propagates it through a PDE solver to produce the forecast. The physics prior imposes energy-conservation structure, so trajectories stay physically consistent, while the GP's Bayesian nature supplies uncertainty quantification on the prediction.","pith_inferences":["Because the nonconformity score requires the ground-truth future state $Z(t+1)$ to be evaluated, the framework as stated is reactive rather than predictive: the switch fires after an erroneous forecast is already observable, so seamless plug-and-play for online safety-critical use would need an online substitute score, such as the residual of a filter or a learned error predictor.","The energy representation learned by the GP-dPHS branch is tied to the object's physical structure, so a natural extension is to reuse or transfer the learned Hamiltonian across similar deformable objects or across different initial conditions, which would further reduce the data needed at switch time.","The conformal threshold $C$ is calibrated under exchangeability; under strong distribution shift the coverage guarantee degrades. A testable refinement is to wrap the trigger with adaptive conformal schemes that re-calibrate as shift is detected, rather than treating $C$ as fixed."],"forward_implications":["Any pre-trained predictor, such as an LSTM or a neural network, can be wrapped with this layer without altering its training, because the switch is purely a runtime decision.","In out-of-distribution regimes with scarce data, the physics-informed branch needs only a few observations to learn an energy representation, unlike retraining a deep network on the new regime.","The predictions from the physics branch come with uncertainty intervals, which can be used for risk-aware planning or collision avoidance.","The framework applies to any system that can be recast in distributed Port-Hamiltonian form, including wave-like or elastic PDE systems, so the same module generalizes across obstacle types."],"supporting_citations":[{"why":"Supplies the conformal prediction framework and the quantile-based coverage guarantee used to set the threshold C.","marker":"Angelopoulos and Bates, 2021"},{"why":"Cited for conformal prediction under distribution shifts, motivating the use of CP for out-of-distribution detection at runtime.","marker":"Zhao et al., 2024"},{"why":"Defines the distributed Port-Hamiltonian system formulation that the physics branch encodes.","marker":"Macchelli et al., 2004"},{"why":"Introduces Gaussian process Port-Hamiltonian systems, the Bayesian learning-with-physics-prior foundation extended here to the distributed (PDE) case.","marker":"Beckers et al., 2022"},{"why":"Provides the GP-dPHS model for PDE systems and the oscillating-string dPHS representation used in the experiments.","marker":"Tan et al., 2024"},{"why":"Supports modeling the Hamiltonian as a Gaussian process via invariance to linear transformations.","marker":"Jidling et al., 2017"}],"fun_headline_variants":["Conformal alarm hands failing predictors to physics","Physics rescue for out-of-distribution predictions","Uncertainty-triggered switch to physics-aware model","Detect stale dynamics, then learn with GP-physics","Port-Hamiltonian fallback with conformal safety net"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole framework leans on being able to compute the prediction error $R(t+1)=\\|Z(t+1)-f(X(t))\\|$ at the moment a decision is needed, which requires the true future state to be already known; if that ground truth is not available online, the switch cannot actually prevent bad predictions, and the claim that the method is plug-and-play in real time is unsupported.","fun_headline_variants_meta":{"raw":{"variants":["Conformal alarm hands failing predictors to physics","Physics rescue for out-of-distribution predictions","Uncertainty-triggered switch to physics-aware model","Detect stale dynamics, then learn with GP-physics","Port-Hamiltonian fallback with conformal safety net"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00039,"raw_usage":{"total_tokens":2031,"prompt_tokens":901,"completion_tokens":1130,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":1054}},"tokens_in":517,"tokens_out":1130,"duration_ms":11466,"temperature":1.0,"reasoning_tokens":1054,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:28:06.835009+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the framework in an online loop where the true next state arrives after the predictor must be committed, such as a one-step-delayed observation. Record how many unsafe forecasts the nominal model produces before the conformal score first exceeds the threshold; if that count is positive, the protective-switch claim fails. Alternatively, feed the calibration set a sequence whose scores are non-exchangeable, such as a linear drift, and measure how often the fixed threshold $C$ misclassifies in-distribution data as out-of-distribution; if the error rate exceeds the advertised $\\delta$, the stated detection guarantee is false.","supporting_citations":[],"review_version":1}