{"id":"5138386f-baa9-415a-86f8-a1f9c79e0471","arxiv_id":"2505.09119","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An adapted belief-space iLQR that penalizes parameter uncertainty outperforms regression and filtering baselines for simultaneous system identification and control in partially observable domains.","lead":"The paper recasts model identification adaptive control as a planning problem over beliefs about both system state and unknown parameters, solved with an iterative linear-quadratic controller. It shows this approach identifies parameters faster than regression or filtering baselines in cart-pole and aircraft simulations, including when parameters change suddenly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II contradicts the identification claim: in the Aircraft Partial Obs MIAC condition, BiLQR's final parameter-covariance trace (22.609±0.334) is significantly worse than MPC+EKF's (20.741±0.146), yet the text claims comparable or better identification.","rationale":"The stress-test should target the central empirical claim directly. The paper's own Table II provides a falsifying instance: in the partially observable aircraft MIAC task, the trace of the final parameter covariance is significantly larger (worse) for BiLQR than for MPC+EKF. Since the paper's abstract and Section V claim that the method yields faster and more accurate parameter identification and outperforms the baselines, this row is load-bearing. The reported numbers are not borderline: the difference is roughly five standard errors. The text's assertion of comparable performance in 'both problem domains with both full and partial observability' is therefore inaccurate. This is stronger than the reader's concern about the ML-observation EKF approximation, which only questions generalization to harder systems; here the stated domain itself contradicts the claim. The reader's CONDITIONAL verdict remains appropriate, but the condition should explicitly require correcting or explaining this discrepancy. A paired re-analysis of the 150 seeds is the minimal check.","tokens_in":9282,"tokens_out":16433,"duration_ms":168171,"concrete_test":"Using the released repository (github.com/sisl/MIAC_BiLQR/), recompute the Aircraft Partial Obs rows of Table II from the 150 per-seed trials: for each seed record tr(Σθθ,τ) under BiLQR and MPC+EKF, and compute the paired mean difference with a 95% confidence interval. If the interval for (BiLQR − MPC+EKF) excludes 0 with the reported sign, the identification claim must be weakened for this condition; if the table contains a typo, correct it and re-evaluate the narrative.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table II contradicts the paper's claim of more accurate parameter identification. In the Aircraft Partial Obs MIAC condition, the final parameter-covariance trace is 22.609±0.334 for BiLQR versus 20.741±0.146 for MPC+EKF. Lower is better, so MPC+EKF reduces residual parameter uncertainty more, by 1.868 with a standard error of about 0.365 (z≈5.1). The corresponding log-likelihoods are −7.919±0.900 (BiLQR) and −8.035±6.288 (MPC+EKF), which are statistically indistinguishable. Yet Section IV.B.2 states that in both domains and both observability settings BiLQR performed comparably to MPC+EKF, and the conclusion claims significant outperformance. This is not a generalization concern; the paper's own table refutes the 'more accurate parameter identification' subclaim for one of the four tested configurations. Any conditional acceptance must require either correcting this row or rewriting the claim.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formulates informative input design and model identification adaptive control (MIAC) as ρ-POMDPs in which the system parameters are hidden state variables, and it solves the resulting belief-space planning problem with an adapted belief-space iterative LQR (BiLQR) planner. The authors present experiments on cart-pole and steady aircraft flight domains under full and partial observability, comparing BiLQR against regression, EKF, and MPC baselines. They report that BiLQR reduces the trace of the final parameter covariance and improves log-likelihood and control reward, and they also demonstrate adaptation to sudden parameter changes with a nonzero parameter process noise. The paper closes with a conclusion that BiLQR 'significantly outperforms' the baselines, particularly under partial observability, while acknowledging the local optimality and scalability limitations of the approach.","tokens_in":9530,"tokens_out":5715,"duration_ms":54634,"significance":"If the claims hold, this is a useful conceptual contribution: casting MIAC as a ρ-POMDP over the joint state-parameter space provides a principled way to balance control and information gathering, and adapting BiLQR to this problem is a pragmatic algorithmic step. On the positive side, the paper ships a reproducibility link, uses standard POMDPs.jl infrastructure, and its informative-input-design experiments show large and clearly significant improvements over the baselines (e.g., Table I, Aircraft Full Obs: trace 7.994±0.470 vs 32.329±0.148 for Random+EKF). However, the MIAC experiments are less supportive. The primary evaluation metric is the same as the planning objective, and several key differences are not statistically significant; one table row directly contradicts the stated claims. The overall significance therefore depends on whether the claims are revised to match the evidence and whether additional statistical analysis is provided.","major_comments":[{"comment":"In the Aircraft Partial Obs MIAC condition, BiLQR's final parameter-covariance trace is 22.609±0.334, which is worse (higher) than MPC+EKF's 20.741±0.146; the difference is roughly 5 standard errors. The text in Section IV.B.2 states that BiLQR 'performed comparably to MPC with EKF updates' in both domains and both observability settings, and the conclusion claims significant outperformance. This is a direct contradiction for one of the four reported configurations. The corresponding log-likelihoods (−7.919±0.900 vs −8.035±6.288) and rewards (78.429±12.471 vs 58.033±19.242) are not statistically distinguishable, so no other metric in that row rescues the claim. The authors must either correct the table entry if it is erroneous, or revise the text and conclusion to acknowledge that BiLQR does not consistently outperform MPC+EKF on parameter identification.","section":"Table II, Section IV.B.2"},{"comment":"The paper does not report statistical significance tests or confidence intervals for the differences between methods. This is not merely a presentation issue because several headline comparisons appear non-significant. For example, in Aircraft Full Obs MIAC, the reward difference between BiLQR (77.953±12.905) and MPC+EKF (56.040±18.239) has an approximate standard error of 22.3, so the difference is less than one standard error. None of the MIAC log-likelihood differences are significant either (e.g., Aircraft Full Obs: 4.201 with standard error about 4.5). Without formal tests, the statement that BiLQR 'significantly outperforms' baselines (Section V) is unsupported. I request hypothesis tests or confidence intervals for the pairwise comparisons, or appropriately qualified language.","section":"Tables I and II, Section IV.B"},{"comment":"The primary evaluation metric, the trace of the final parameter covariance, is exactly the quantity that the planner's objective penalizes in Eqs. (14) and (15), while the baselines do not optimize that quantity. Consequently, part of the reported advantage on this metric is by construction. The independent log-likelihood and reward metrics are meant to address this, but in the MIAC experiments the log-likelihood differences are not statistically significant (as detailed above), and the reward differences are also not significant. The paper should either add an evaluation metric that is not directly optimized by the planner, or restrict the claims of superiority to the metrics that actually show significant independent support.","section":"Eqs. (14)-(15), Table II"},{"comment":"The belief forward pass propagates the covariance with an extended Kalman filter under a maximum-likelihood observation assumption (Eq. (12) and the paragraph after Eq. (13)). This approximation can produce overconfident covariance predictions when the dynamics are strongly nonlinear or the belief becomes multi-modal. Since the experiments are limited to two relatively smooth domains, the current empirical evidence does not assess this risk. The authors should at least discuss this limitation explicitly and, ideally, compare the planned covariance reduction with the realized reduction from the filtering updates to show whether the planner's uncertainty predictions are reliable. This is a generalization concern rather than an internal inconsistency, but it is relevant to the claimed practical applicability.","section":"Section III.C, Eqs. (12)-(13)"}],"minor_comments":[{"comment":"The text 'nonlinear, stochastic, and unde ractuated' appears to contain a typo; it should read 'underactuated'.","section":"Section I"},{"comment":"The empty entries for Regression under 'Partial Obs' should be explained in the caption or text; presumably regression requires full-state measurements and cannot be applied directly to the partially observable setting.","section":"Table I"},{"comment":"The reference list ordering is inconsistent: [11] appears after [6] and before [7], breaking the numerical order. Please renumber the references sequentially.","section":"References"},{"comment":"Figure 3 shows a single trajectory without error bars or multiple runs; the caption already indicates it is one example simulation, but the text should clarify that this is illustrative and not a statistically aggregated result.","section":"Figure 3"},{"comment":"The sign convention in Eq. (5) defines a reward R(b,a) that is negative of the cost in Eq. (4), but the text describes maximizing reward while minimizing the objective. It would be clearer to state explicitly that the reward function is the negative stagewise cost.","section":"Section III.B, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The central formulation is reasonable, but the paper's claims currently overreach the evidence, especially the Table II Aircraft Partial Obs row. I would ask the authors to verify that row against their code and raw results; if the number is correct, the claims must be revised. The lack of significance testing is also a barrier to acceptance at a serious journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful new thing here is the reformulation: treating unknown system parameters as hidden state and solving the joint state-parameter belief-space planning problem with an adapted BiLQR. That framing does not appear in the cited BiLQR or rho-POMDP work, and the paper does a clean job of writing down the belief dynamics, the EKF covariance recursion, and the penalty-based reward. The experiments on two domains, with full and partial observability, are more than a toy demo, and the authors ship code and experimental details. The informative-input-design results (Table I) are clearly in their favor, and the log-likelihood and reward metrics are not just the planning objective, so those provide some independent support.\n\nThe soft spots are real. First, the paper's central MIAC claim overreaches. In Table II, Aircraft Partial Obs, BiLQR's final parameter-covariance trace is 22.609 ± 0.334, while MPC+EKF gets 20.741 ± 0.146. Lower is better, and the difference is about 5 standard errors. That directly contradicts the text's statement that BiLQR performed comparably to MPC+EKF in both domains and both observability settings, and it weakens the abstract's claim of outperforming baselines. This is not a minor generalization concern; it is one of the four tested configurations, and the identification subclaim fails there. A conditional acceptance would need this fixed, either by correcting the row or softening the claim. Second, the main metric (trace of parameter covariance) is also the planning objective, so part of the advantage is by construction. The log-likelihood and reward metrics help, but the paper reports no statistical significance tests, relying on standard-error overlap, which is not enough when errors are small and differences are one-sided. Third, the EKF maximum-likelihood-observation approximation in Eq. (12) and after Eq. (13) means planned covariance reduction may be overconfident in strongly nonlinear or multimodal settings; the paper only tests smooth, near-linear domains. That is a limitation worth stating, but not a fatal flaw.\n\nWho should read this: people working on adaptive control, belief-space planning, and active parameter estimation. The paper is a solid engineering contribution that deserves serious refereeing, but not in its current form. The authors need to address the Table II contradiction and add some form of uncertainty quantification over the comparisons.\n\nRecommendation: send to peer review, but insist on a major revision addressing the contradictory table row and the overclaim before acceptance.","headline":"A solidly-engineered incremental advance (MIAC as rho-POMDP with BiLQR) whose own Table II contradicts the headline identification claim in the aircraft partial-observability case.","tokens_in":10044,"tokens_out":904,"would_cite":true,"duration_ms":11105,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper turns model identification and control into one belief-space planning problem, with unknown parameters as hidden state, and reports better identification and control than regression, EKF, and MPC baselines in cart-pole and…","keywords":["model identification adaptive control","rho-POMDP","belief-space planning","belief-space iLQR","informative input design","partial observability","cart-pole balancing","aircraft system identification"],"falsifier":"Run the same planner on a system with strongly nonlinear dynamics and a genuinely multi-modal parameter posterior, such as a cart-pole with friction or backlash where the pole mass has two plausible values; if the trace of the planned parameter covariance from Eq. (12) is systematically below the actual squared parameter estimation error over the same horizon, the central claim that planned uncertainty reduction matches realized identification fails for that regime.","tokens_in":9097,"feed_emoji":"🤖","tokens_out":5676,"duration_ms":55513,"temperature":0.7,"pith_summary":"Model identification adaptive control (MIAC) is usually split into separate estimation and control loops, which can waste information. This paper tries to end that split: it formulates MIAC as a rho-POMDP in which the unknown system parameters are hidden state variables, so the same planning objective that drives the system toward its goal also rewards shrinking parameter uncertainty. The resulting planner, an adapted belief-space iterative LQR, plans over Gaussian beliefs and executes control actions that double as informative experiments. On cart-pole and steady aircraft flight tasks, in both fully and partially observable versions and under sudden parameter jumps, the paper reports lower final parameter covariance, higher parameter log likelihood, and better control rewards than regression, EKF, and MPC baselines. If this holds, a single planner can replace the usual estimate-then-control pipeline in continuous partially observable systems.","feed_headline":"Planner learns a system's parameters as it controls","feed_subtitle":"Treating unknown parameters as hidden state beats regression, EKF, and MPC on cart-pole and aircraft tasks.","key_machinery":"The central machinery is the adapted belief-space iterative LQR (BiLQR) over a Gaussian joint belief with mean and covariance over the state and parameters. The forward pass propagates the mean using the deterministic joint dynamics and the covariance using an extended Kalman filter that assumes maximum-likelihood observations; the reward is an LQR-style stage cost plus a terminal quadratic penalty on the parameter covariance; and the backward pass linearizes the belief dynamics to compute locally optimal feedback gains. This mechanism converts the partially observed identification-and-control problem into a deterministic trajectory optimization over belief coordinates, so the same control signal serves both to balance and to probe the unknown parameters.","core_discovery":"The paper's central claim is that model identification adaptive control can be solved as a single belief-space optimization. System parameters theta are appended to the hidden state, the joint belief at time t is a Gaussian over the state and parameters, and the stage reward combines the usual state and action control cost with a penalty on the parameter covariance. Solving this rho-POMDP with a maximum-likelihood-assumption belief-space iLQR yields a policy that both stabilizes the plant and actively excites informative trajectories. In experiments the planner identifies the cart-pole pole mass and the aircraft state-space matrix columns more quickly and more accurately than regression with MPC, EKF with MPC, or random probing with EKF, and it recovers after an instantaneous mass change mid-run.","pith_inferences":["A testable extension left implicit in the paper is to replace the terminal covariance penalty with a mutual-information reward, which would change how the planner explores when the posterior is multi-modal.","The same Gaussian-belief machinery should transfer to sigma-point or particle covariance propagation, letting the planner apply to stronger nonlinearities than the cart-pole and linear aircraft examples; this is an extension, not a claim in the paper.","Because the parameter covariance is planned online, the method naturally provides a stopping rule for active identification: keep probing while the parameter covariance is above a threshold, then switch to pure control, which the paper does not explicitly implement."],"forward_implications":["A single objective can replace the estimate-then-control pipeline: by appending parameters to the state and penalizing their covariance, the same control signal both completes the task and probes the plant.","Partial observability degrades performance but does not break it: in experiments the planner still identifies parameters more accurately than the baselines when only noisy positions and angles are observed.","Abrupt parameter changes are trackable: with a nonzero process noise on the parameters, the planner re-identifies the new mass after a sudden jump.","Because the solution is locally optimal around a nominal belief trajectory, performance guarantees are local; strongly multi-modal or highly nonlinear regimes would need a global planner."],"supporting_citations":[{"why":"Introduces rho-POMDPs with belief-dependent rewards, the mathematical formulation the paper builds on for penalizing parameter covariance.","marker":"[10]"},{"why":"Supplies the belief-space iLQR algorithm with maximum-likelihood observations that the paper adapts to plan over state-and-parameter beliefs.","marker":"[11]"},{"why":"Defines informative input design for dynamic mode decomposition, the identification objective and the aircraft-domain setup the paper compares against.","marker":"[2]"},{"why":"Defines model identification adaptive control, the control paradigm the paper reformulates as a rho-POMDP and evaluates against.","marker":"[6]"},{"why":"Provides a sampling-based Bayesian reinforcement learning baseline for simultaneous active parameter estimation and control, a direct alternative the paper situates itself against.","marker":"[9]"},{"why":"Demonstrates belief-dependent rewards for active information gathering in path planning, supporting the use of such rewards in the MIAC objective.","marker":"[18]"},{"why":"Supplies the linear regression estimator used as an identification baseline in the experimental comparisons.","marker":"[12]"}],"fun_headline_variants":["Uncertain parameters as hidden state: one planner controls and identifies","Belief-space planning learns plant parameters on the fly","Control and identify in one belief-space optimization","One planner: stabilizes and identifies unknown parameters","Active learning for control: belief-space iLQR identifies as it stabilizes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"If the dynamics are strongly nonlinear or the belief becomes genuinely multi-modal, the extended Kalman filter with maximum-likelihood observations used in the forward pass can be overconfident about how much parameter uncertainty the planned actions will remove.","fun_headline_variants_meta":{"raw":{"variants":["Uncertain parameters as hidden state: one planner controls and identifies","Belief-space planning learns plant parameters on the fly","Control and identify in one belief-space optimization","One planner: stabilizes and identifies unknown parameters","Active learning for control: belief-space iLQR identifies as it stabilizes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00097,"raw_usage":{"total_tokens":4061,"prompt_tokens":816,"completion_tokens":3245,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":3165}},"tokens_in":432,"tokens_out":3245,"duration_ms":20798,"temperature":1.0,"reasoning_tokens":3165,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:38:33.088673+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same planner on a system with strongly nonlinear dynamics and a genuinely multi-modal parameter posterior, such as a cart-pole with friction or backlash where the pole mass has two plausible values; if the trace of the planned parameter covariance from Eq. (12) is systematically below the actual squared parameter estimation error over the same horizon, the central claim that planned uncertainty reduction matches realized identification fails for that regime.","supporting_citations":[{"cited_title":"A POM DP extension with belief-dependent rewards,","cited_arxiv_id":null,"evidence_quote":"Introduces rho-POMDPs with belief-dependent rewards, the mathematical formulation the paper builds on for penalizing parameter covariance."},{"cited_title":"Be- lief space planning assuming maximum likelihood observati ons,","cited_arxiv_id":null,"evidence_quote":"Supplies the belief-space iLQR algorithm with maximum-likelihood observations that the paper adapts to plan over state-and-parameter beliefs."},{"cited_title":"Informative inp ut design for dynamic mode decomposition,","cited_arxiv_id":null,"evidence_quote":"Defines informative input design for dynamic mode decomposition, the identification objective and the aircraft-domain setup the paper compares against."},{"cited_title":"Model identification ada p- tive control: implementation case studies for a high manoeu vrability aircraft,","cited_arxiv_id":null,"evidence_quote":"Defines model identification adaptive control, the control paradigm the paper reformulates as a rho-POMDP and evaluates against."},{"cited_title":"Si- multaneous active parameter estimation and control using s ampling- based Bayesian reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides a sampling-based Bayesian reinforcement learning baseline for simultaneous active parameter estimation and control, a direct alternative the paper situates itself against."},{"cited_title":"Sequential bayesian optimization for adaptive informative path planning with m ultimodal sensing,","cited_arxiv_id":null,"evidence_quote":"Demonstrates belief-dependent rewards for active information gathering in path planning, supporting the use of such rewards in the MIAC objective."},{"cited_title":"Linear regression applied t o system identification for adaptive control systems,","cited_arxiv_id":null,"evidence_quote":"Supplies the linear regression estimator used as an identification baseline in the experimental comparisons."}],"review_version":1}