{"id":"c0e6a447-e7bb-43b4-a6aa-e31c0a7b6c0b","arxiv_id":"2504.13413","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Predictive Imitation Learning trains a policy by minimizing a surrogate loss built from learned multi-step predictors and a dynamics consistency term, and the paper reports lower trajectory error than behavior cloning and rollout methods.","lead":"Scientists propose a new imitation learning algorithm, Predictive Imitation Learning, that uses learned multi-step predictors instead of rolling out trajectories one step at a time. It claims to reduce compounding errors and resist measurement noise better than standard behavior cloning in control benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 analyzes an oracle version with exact fixed predictors Gτ=(A+BK*)^τ, while the implemented PIL estimates Gτ from the same noisy data, so the central guarantee does not cover the proposed algorithm.","rationale":"The reader's weakest_assumption identifies exactly the gap I find most load-bearing: the theory requires Gτ to be the fixed, exact transition (A+BK*)^τ, while the algorithm estimates Gτ from the same noisy data. I checked whether there is an even more basic flaw in the denominator lower bound in Lemma 4/Eq. (30); there is not, because the τ=1 term includes G0=I, so the lower bound by Σ y_t y_t^T is valid. The theorem's other defects (the equality rather than inequality in Theorem 1, the incomplete Lemma 7, the undefined 'Sasha' citation, and the self-reference [20]) are real but secondary; they do not change the verdict. Because the reader already assigned CONDITIONAL on the same basis, my concern does not move the verdict. The concrete synthetic test above would settle whether the oracle-predictor assumption is actually needed or whether the estimated predictors still satisfy the bound in practice.","tokens_in":883,"tokens_out":1122,"duration_ms":83445,"concrete_test":"On the LTI setup of §V-A, generate a long training set with known A, B, K*, Σξ>0. Compute the estimated Gτ by least squares (Appendix E, Eq. 59) and bK by the PIL objective; for T = 10^3, 10^4, 10^5, compare ∥bK−K*∥ with the same computation using the oracle G*τ=(A+BK*)^τ. If the two curves differ by more than the Theorem 1 error bound, or if the estimated-G error does not approach the κ2||Σξ|| floor, the theorem does not cover the implemented estimator. Also perform the analytic check: derive the limit of the least-squares Gτ as G*τ Σ_x (Σ_x+Σξ)^{-1}, substitute into Lemma 3/Eq. (14), and exhibit the extra bias term.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim (Theorem 1) is proven only for a regime that the implemented algorithm never occupies. Lemma 3's closed form for bK (Eq. 14) and the subsequent concentration proof treat Gτ as the exact, fixed expert closed-loop transition G*τ=(A+BK*)^τ, with G0=I. In the actual method, the predictors are estimated from the same noisy measurements used to fit K (Appendix E, Eq. 59: min_Gτ Σ_t ||y_{t+τ}-Gτ y_t||²; in the general algorithm they are trained jointly with πθ in Eq. 8). With state noise ξ, the least-squares estimate is not consistent for G*τ: the estimated Gτ converges to G*τ Σ_x (Σ_x+Σξ)^{-1}, an attenuated operator, while Lemma 3's derivation uses Gτ y_t = G*τ−1 x_t to eliminate the state and leaves only η_t−K*Gτ−1 ξ_t as the noise. If the estimated Gτ is used instead, Eq. (14) acquires an additional bias term proportional to (G*τ − estimated Gτ)(Σ_t y_t y_t^T) that does not vanish with T under fixed Σξ>0 and is not included in the claimed κ2||Σξ|| floor. Thus the theorem does not establish the stated guarantee for the implemented PIL procedure; it is a guarantee about a different, oracle-predictor algorithm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Predictive Imitation Learning (PIL), a model-based imitation learning framework in which a policy and a set of multi-step state predictors are trained jointly by minimizing a surrogate objective that combines state prediction error, control prediction error, and a dynamics-consistency penalty. The authors argue that this formulation is computationally cheaper than rollout-based trajectory matching and more robust to measurement noise and distribution shift than behavior cloning. The manuscript reports experiments on linear systems, an inverted pendulum, and MuJoCo continuous-control tasks, where PIL outperforms BC and a rollout-based baseline. For LTI systems with linear expert policies, the authors provide finite-sample guarantees: Theorem 1 gives a high-probability bound on the learned feedback gain error, and Theorem 2 attempts to show that PIL has a smaller expected error than BC when state noise is not too large.","tokens_in":16666,"tokens_out":7501,"duration_ms":72380,"significance":"If the theoretical guarantees were established for the implemented algorithm, this would be a useful contribution to offline model-based imitation learning: the surrogate objective is plausible, the approach is computationally simpler than trajectory rollouts, and the empirical results are broadly positive. The manuscript also ships a public implementation, which aids reproducibility, and the experiments cover both linear and high-dimensional nonlinear tasks. However, the theoretical core as written only analyzes an oracle version of the method, and the comparison theorem is not a comparison of actual estimation errors; these gaps currently prevent the paper from supporting its central claims as strongly as stated.","major_comments":[{"comment":"Theorem 1 is proven for an oracle version of the algorithm, not for the implemented PIL. In Lemma 3, the gradient derivation replaces Gτ with the exact expert closed-loop transition by using Gτ = (A+BK*)Gτ−1 to eliminate Gτ (Eq. (16)). The implemented method, however, estimates Gτ from the same noisy data, either separately via least squares (Appendix E, Eq. (59)) or jointly with the policy (Eq. (8)). With state noise, the least-squares estimate is not consistent for G*τ; it is attenuated, Ĝτ ≈ G*τ Σx(Σx+Σξ)^{-1}. Consequently, Eq. (14) and the concentration bound in Lemma 4 do not apply to the estimated predictors used in the experiments. The theorem therefore provides a guarantee for a different, oracle-predictor algorithm. The authors should either analyze the estimated-predictor case, including a bound on the bias (G*τ − Ĝτ) in the closed form, or explicitly state and clearly label the oracle assumption in the theorem and its proof.","section":"Section IV, Appendix A, Lemma 3"},{"comment":"The statement of Theorem 1 omits a data-coverage condition that the proof requires. Theorem 1 says 'Under Assumption 1, for any T ≥ O(n log(1/δ))', but Lemma 4 additionally requires T ≳ (ψξ/ϕξ²) n log(...) (Eq. (18)) so that the empirical covariance lower bound (Eq. (30), via Lemma 5) holds. This condition is not implied by Assumption 1 when the state-noise covariance Σξ is large relative to the minimum state covariance ϕx. Without it, the claimed O(1/√T) rate plus the κ2||Σξ|| floor does not follow. Please either add the missing condition to the theorem statement or prove the bound under Assumption 1 alone.","section":"Section IV, Theorem 1 and Appendix B, Lemma 4"},{"comment":"Theorem 2 does not establish the claimed advantage of PIL over BC. The theorem compares expected norms of quantities ωPIL and ωBC without defining them, and the appendix proof only compares constants in upper bounds (Eqs. (49)–(50)) under the matrix condition (51): ∥B⊤Q(I−A)∥√trΣξ ≤ ∥B⊤QB∥√trΣη. That condition is not equivalent to the text's tr(Σξ) ≤ C tr(Ση), since C would need to depend on the system matrices in a specific way. Moreover, a comparison of upper-bound slopes does not imply the actual estimation errors satisfy E∥bKPIL − K*∥ ≤ E∥bKBC − K*∥. The claimed superiority over BC is therefore not substantiated by the theorem.","section":"Section IV, Theorem 2 and Appendix C"},{"comment":"The sample-complexity proof relies on 'Proposition 8.2 of Sasha', 'Proposition 3.1 of Sasha', and 'Equation (46) of Sasha' without any bibliographic identification. These results are load-bearing for the matrix concentration steps used in Eqs. (24), (26), (27), and (39), so the proof cannot be checked as written. Please provide full statements and proofs of the cited propositions or a precise, complete reference.","section":"Appendix B"}],"minor_comments":[{"comment":"The theorem and Lemma 4 state an equality, ||bKPIL − K*|| = κ1/√(T−H+1) + κ2||Σξ||, but the proof establishes an upper bound; the statements should use '≤'.","section":"Section IV, Theorem 1 and Lemma 4"},{"comment":"The statement of Lemma 7 is garbled: it reads '∥bK−K*∥ ≤ α+β∥∥' with a missing argument, and the proof is left incomplete with 'such that'. Please complete the lemma statement and proof.","section":"Appendix C, Lemma 7"},{"comment":"In the linear-system experiment, the predictors are first estimated by least squares and then fixed, which is not the joint training described in Section III-D and Eq. (8); the exact protocol should be clarified, as the theoretical section does not cover either variant.","section":"Appendix E, Section V.A"},{"comment":"Reference [20] is titled the same as this paper and has a placeholder arXiv number 'arXiv:XXXXX'; if it is intended to be this manuscript or a companion paper, please cite it properly.","section":"References"},{"comment":"There is a typo in the proof sketch of Theorem 1: 'its’ norm' should be 'its norm'.","section":"Appendix B, Proof Sketch"}],"recommendation":"major_revision","confidential_remarks":"The central gap is that Theorem 1 analyzes an oracle version of the algorithm while the experiments use estimated predictors; this needs to be either closed with a bias analysis or explicitly scoped. The missing 'Sasha' references and the incomplete Lemma 7 also need attention. I recommend major revision rather than rejection because the empirical contribution is meaningful and the oracle-theoretic gap may be fixable, but the current theoretical statements overclaim what is proven."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a sensible model-based imitation learning idea with a respectable empirical section, but the theory as written does not cover the actual algorithm and several appendices are incomplete. I would not desk-reject it, but it needs major revision.\n\nThe genuinely new thing is the PIL objective: instead of H-step rollout matching, they minimize a surrogate loss over parameterized multi-step predictors, with a consistency term that ties the predictors to the known dynamics. That is a legitimate trick and it seems to help. Experiments on linear systems, a nonlinear expert, an inverted pendulum, and MuJoCo tasks show lower maximum trajectory discrepancy than BC and a rollout-based baseline, and they ship code. The known-dynamics assumption is stated honestly, though it should be more prominent.\n\nThe soft spots are in the theory. Theorem 1 in the main text is only a sketch; the appendix proof rests on Lemma 3, whose closed form assumes the predictors G_tau are exactly the true closed-loop transitions (A+BK*)^tau. The implemented method estimates G_tau by least squares from the same noisy data (Appendix E), so the guarantee is for an oracle version. Under state noise the least-squares estimator is biased, meaning the claimed kappa2||Sigma_xi|| floor does not cover the actual estimate. Theorem 2 compares only upper-bound slopes, and its proof in Appendix C is literally unfinished: Lemma 7 trails off mid-sentence and the statement has a blank. There is also an undefined source \"Sasha\" and a self-citation to an unpublished placeholder (ref [20]). These are fixable, but the current claims overstate what is proven.\n\nEmpirically the comparisons are narrower than the abstract implies: only BC and one rollout-based baseline, no DAgger or GAIL, and the linear-system example is a 2-state plant. Still, the method looks plausible and the code is available for verification.\n\nWho this is for: people working on model-based IL or MPC-style surrogate losses who want a cheaper alternative to rollout matching. It deserves a serious referee, but only with major revision: rewrite the theory to analyze the estimated-predictor procedure, complete the appendices, and add a limitations paragraph. I would bring it to a reading group to discuss the gap between the oracle analysis and the practical estimator, but I would not cite the theoretical claims as they stand.","headline":"A promising model-based IL method with a solid empirical story, but the theoretical guarantees are for an oracle version and the appendix proofs are unfinished.","tokens_in":17214,"tokens_out":3341,"would_cite":true,"duration_ms":31566,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By training a policy together with learned multi-step predictors and a dynamics-consistency term, imitation learning can track an expert over long horizons with less compounding error and noise sensitivity than behavior cloning.","keywords":["imitation learning","multi-step prediction","model predictive control","behavior cloning","distribution shift","measurement noise","linear time-invariant systems","continuous control"],"falsifier":"Take the linear time-invariant setup of Theorem 1, estimate the multi-step predictors by least squares on the same noisy rollouts used for $K$, freeze them, and measure $\\|\\hat{K} - K^{\\star}\\|$ as $T$ grows. If the error does not decay like $1/\\sqrt{T-H+1}$ or is dominated by a term proportional to the predictor-estimation bias, the bound does not extend to the implemented procedure. A cleaner alternative: intentionally bias the predictors, for example by truncating the estimation horizon, and check whether the observed error matches the predictor-bias term.","tokens_in":16173,"feed_emoji":"🤖","tokens_out":9820,"duration_ms":80470,"temperature":0.7,"pith_summary":"Imitation learning from demonstrations suffers because errors made at one step feed into the next, so behavior cloning often drifts off the expert trajectory, especially when sensor noise corrupts the data. This paper proposes Predictive Imitation Learning (PIL), which trains the policy together with a set of multi-step predictors that forecast several future states at once, while a consistency term keeps those forecasts compatible with the known dynamics model. The paper's claim is that minimizing this surrogate loss reduces the maximum trajectory deviation from the expert, is more robust to measurement noise than behavior cloning, and avoids the computational cost of unrolling full rollouts. If the claim is right, model-based imitation learning becomes a cheaper and more reliable alternative for control tasks with noisy sensors and long horizons.","feed_headline":"Multi-step prediction beats behavior cloning in imitation learning","feed_subtitle":"PIL learns the policy and future-state predictors together, shrinking expert-tracking error as data grows.","key_machinery":"The load-bearing mechanism is the multi-step predictor family $G_{\\tau,\\vartheta}$. Each predictor maps the encoded current observation $z_t$ to a state prediction $\\tau$ steps ahead, $x_{t+\\tau|t}=G_{\\tau,\\vartheta}(z_t)$, so the optimizer never has to unroll the dynamics step by step. A consistency residual $w_{t+\\tau|t}=x_{t+\\tau|t}-f(x_{t+\\tau-1|t},u_{t+\\tau-1|t})$ penalizes disagreement between the predicted trajectory and the known dynamics, weighted by $P$, which grounds the learned predictors in the true system. The policy $\\pi_\\theta$ maps predicted states to controls, and all parameters are trained jointly against the surrogate loss of Eq. (8), which combines state errors, input errors, and the consistency penalty. In the LTI analysis the predictors are fixed at their ground-truth values $G^{\\star}_\\tau=(A+BK^{\\star})^\\tau$; the closed-form gain then follows from a least-squares normal equation.","core_discovery":"The central claim is that imitation learning should be framed as a multi-step prediction problem: instead of matching expert actions one state at a time, learn a policy and a family of $\\tau$-step state predictors so that the predicted closed-loop trajectory tracks the expert's observed trajectory over a horizon $H$, with a penalty for inconsistency with the known transition function. In the linear time-invariant case with a linear expert policy, the paper derives a closed-form expression for the estimated feedback gain and proves a finite-sample bound: with high probability, $\\|\\hat{K}_{\\mathrm{PIL}} - K^{\\star}\\| \\leq \\kappa_1/\\sqrt{T-H+1} + \\kappa_2\\|\\Sigma_\\xi\\|$, so the policy error decays as the number of demonstrations grows and grows linearly with the state-measurement noise covariance. For the one-step case $H=1$, it further shows that PIL's expected distance to the expert controller is no larger than behavior cloning's when state noise is small relative to input noise. Numerical comparisons on linear systems, an inverted pendulum, and MuJoCo tasks support the claim that PIL outperforms both behavior cloning and rollout-based imitation.","pith_inferences":["A direct testable extension would split the data: estimate $G_\\tau$ on one clean set, freeze it, then fit $K$; if the bound in Theorem 1 holds only in this regime, the joint-training gap is the real limiting factor.","The encoder in PIL acts as a learned denoiser, so an editor-level guess is that the same framework extends to partially observed nonlinear systems where the encoder learns a belief state rather than a filtered observation.","Because the consistency term is a soft constraint, replacing the known model $f$ with a learned dynamics model would turn PIL into a fully model-based imitation learning method that learns both dynamics and policy from demonstrations, a direction the paper lists for future work.","The $H=1$ comparison suggests that state-transition information is most valuable when action observations are noisier than state observations; an empirical sweep varying $\\mathrm{tr}(\\Sigma_\\xi)/\\mathrm{tr}(\\Sigma_\\eta)$ could test this boundary directly."],"forward_implications":["Guaranteed policy error shrinks as $O(1/\\sqrt{T-H+1})$ in the number of demonstrations, so collecting more expert trajectories directly improves long-horizon tracking.","The bound's linear dependence on $\\|\\Sigma_\\xi\\|$ means sensor quality, not just expert behavior, is a first-order determinant of imitation quality; noise-robust imitation learning should include state denoising.","Since PIL does not require differentiating through the dynamics, it stays competitive even when dynamics gradients are unavailable, which the pendulum experiments support.","In the fixed-predictor linear case the method reduces to a closed-form least-squares problem, making it substantially cheaper than rollout-based trajectory optimization.","Horizon $H$ is a tunable accuracy-computation tradeoff: longer horizons help up to a point, and the experiments show increasing prediction order reduces maximum trajectory discrepancy."],"supporting_citations":[{"why":"defines the compounding-error and distribution-shift problem that PIL is designed to solve.","marker":"[7]"},{"why":"introduces behavior cloning, the baseline that PIL is compared against and shown to beat.","marker":"[8]"},{"why":"supplies the model-based trajectory-matching approach that PIL extends with multi-step predictors.","marker":"[17]"},{"why":"is the model-based imitation-driving method whose rollout cost PIL's surrogate loss avoids.","marker":"[18]"},{"why":"provides the receding-horizon model-predictive-control template for the finite-horizon surrogate objective.","marker":"[19]"}],"fun_headline_variants":["Imitation learning reimagined as multi-step prediction","Multi-step prediction outperforms behavior cloning","Predictive imitation: robust to noise and distribution shift","Imitation via predicted trajectories: better than BC","Model-based imitation with multi-step state prediction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The theoretical guarantees treat the multi-step predictors as the exact expert closed-loop transitions that remain fixed while the policy is fitted, whereas the implemented method learns them from the same noisy data and trains them jointly with the policy.","fun_headline_variants_meta":{"raw":{"variants":["Imitation learning reimagined as multi-step prediction","Multi-step prediction outperforms behavior cloning","Predictive imitation: robust to noise and distribution shift","Imitation via predicted trajectories: better than BC","Model-based imitation with multi-step state prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000218,"raw_usage":{"total_tokens":1406,"prompt_tokens":880,"completion_tokens":526,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":456}},"tokens_in":496,"tokens_out":526,"duration_ms":4479,"temperature":1.0,"reasoning_tokens":456,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:09:17.352531+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the linear time-invariant setup of Theorem 1, estimate the multi-step predictors by least squares on the same noisy rollouts used for $K$, freeze them, and measure $\\|\\hat{K} - K^{\\star}\\|$ as $T$ grows. If the error does not decay like $1/\\sqrt{T-H+1}$ or is dominated by a term proportional to the predictor-estimation bias, the bound does not extend to the implemented procedure. A cleaner alternative: intentionally bias the predictors, for example by truncating the estimation horizon, and check whether the observed error matches the predictor-bias term.","supporting_citations":[{"cited_title":"Alvinn: An autonomous land vehicle in a neural network,","cited_arxiv_id":null,"evidence_quote":"introduces behavior cloning, the baseline that PIL is compared against and shown to beat."},{"cited_title":"Model-based imitation learning by probabilistic trajectory matching,","cited_arxiv_id":null,"evidence_quote":"supplies the model-based trajectory-matching approach that PIL extends with multi-step predictors."},{"cited_title":"Model-based imitation learning for urban driving,","cited_arxiv_id":null,"evidence_quote":"is the model-based imitation-driving method whose rollout cost PIL's surrogate loss avoids."},{"cited_title":"Model predictive control,","cited_arxiv_id":null,"evidence_quote":"provides the receding-horizon model-predictive-control template for the finite-horizon surrogate objective."}],"review_version":1}