{"id":"8eb7cb4f-028f-4b3f-835d-50bfd19a42d9","arxiv_id":"2506.15700","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"CAC is a reinforcement learning algorithm that uses a learned control contraction metric to shape the reward, claiming contraction-certified, optimal path tracking with unknown dynamics.","lead":"CAC trains an actor-critic policy whose reward is shaped by a learned control contraction metric, aiming to combine stability guarantees with long-horizon optimality for path tracking when dynamics are unknown. The paper reports competitive simulated and real-robot tracking results, but the central theoretical convergence proof has a gap.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 is unsupported: Eq. (23) applies a contraction bound assumed only for πc to the RL-optimal π*, and Lemma 1's geometric-series bound is negative due to a sign error; the advertised contraction certificate is not established.","rationale":"The paper's empirical contribution—CAC's MAUC results and real-world TurtleBot3 demonstration—is plausible and not what I would attack; the algorithm may be a useful heuristic, and the ablation with entropy regularization is informative. However, the abstract and Section 7 advertise a formal contraction certificate, and that claim rests entirely on Theorem 1. The proof has two independent, concrete defects. First, the only bound strong enough to pass through the infinite sum, Lemma 1, is proved only from Assumption 1's Eq. (11) for a contracting policy πc. In Eq. (23) the same bound is asserted for the RL-optimal π* without any argument that π* is contracting or satisfies that inequality. Optimality under the bounded reward (13) does not imply such a bound. Second, Lemma 1's own statement is arithmetically false: the denominator 1-e^{2λΔt} is negative, so the claimed upper bound is impossible. Both defects are internal, not merely disagreements with consensus, and they invalidate the central theoretical claim. The Limitations section candidly says the guarantees 'may not always hold' in practice, which is sensible but does not repair the proof. I agree with the reader's assessment and recommend no change to the REJECT verdict; my only adjustment would be to emphasize that the formal failure is twofold—a transfer fallacy plus a sign error—while the empirical findings remain useful heuristics. If the authors can supply a corrected Lemma 1 and a valid argument that π* satisfies the contraction bound, the theoretical component could be reconsidered, but as written the advertised certificate is unsupported.","tokens_in":13293,"tokens_out":6975,"duration_ms":73879,"concrete_test":"Analytically re-derive Eq. (23) from the stated assumptions: attempt to justify the last inequality for an arbitrary policy, and specifically for π*, without assuming that π* itself satisfies Eq. (11). As an arithmetic sanity check, set λ=1, Δt=1, and ||δx(t0)||^2_M=1 in Lemma 1; the stated bound becomes 1/(1-e^2), which is negative. Correct the denominator to 1-e^{-2λΔt}, then check whether the Cauchy argument in Eq. (25) can be completed for continuous time. If the last inequality in Eq. (23) cannot be derived for π*, or if the corrected Lemma 1 still does not imply continuous-time convergence, Theorem 1 as stated is not proven.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central formal claim—Theorem 1—says that the existence of one contracting policy πc implies that the RL-optimal policy π* from Eq. (14) converges asymptotically. The proof in Appendix A.3 reduces the claim to Lemma 1 via Eq. (23): Eπ[Σ γ^k C_k] ≤ Eπ[Σ C_k] ≤ Eπ[Σ ||δx(t_k)||^2_M] ≤ ||δx(t0)||^2_M / (1 - e^{2λΔt}). The final inequality uses Lemma 1, whose premise, Eq. (11), is assumed in Assumption 1 only for πc. No argument shows that π*, or any policy used in Eq. (23), satisfies Eq. (11); maximizing the bounded reward R = 1/(1+||δx||^2_M) does not imply the undiscounted L2 sum is finite or that π* is contracting. Separately, Lemma 1 is internally wrong: for λ>0 and Δt>0, e^{2λΔt}>1, so the stated denominator 1-e^{2λΔt} is negative and the claimed upper bound is negative. The correct geometric series gives 1-e^{-2λΔt}. Even after that fix, the Cauchy step in Eq. (25) only shows sampled terms tend to zero, not lim_{t→∞} ||δx(t)||^2_M = 0 without an additional sampling/continuity argument. These defects sit in the theorem that the abstract and conclusion rely on; the empirical results can still stand as heuristic evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes contraction actor-critic (CAC), an actor-critic variant in which a learned contraction metric generator (CMG) provides a reward signal based on a control contraction metric. The paper claims a theoretical guarantee (Theorem 1) that if at least one contracting policy exists, then the RL-optimal policy under the CAC reward converges asymptotically to the reference trajectory. Empirically, CAC is evaluated on four simulated platforms (Car, PVTOL, NeuralLander, Quadrotor) and on a real TurtleBot3, reporting lower modified AUC than C3M, PPO, SD-LQR, and LQR in several settings.","tokens_in":13642,"tokens_out":6681,"duration_ms":65332,"significance":"The empirical study is substantial and the paper is transparent about its limitations, including the online-interaction requirement and the gap between theoretical guarantees and practical condition satisfaction. If the convergence theorem were established, the paper would make a useful conceptual contribution by connecting CCM certificates to long-horizon RL objectives. The real-world robot experiments and the ablation of entropy regularization are valuable. However, the formal claim rests on a proof with a sign error, an unjustified transfer of a contraction bound to the RL-optimal policy, and a missing continuity argument; these issues are load-bearing rather than cosmetic. As it stands, the manuscript is best read as an empirical heuristic paper, not as a paper with a rigorous contraction certificate.","major_comments":[{"comment":"In Eq. (23), the proof applies Lemma 1's upper bound to Eπ[Σ γ^k C_k] for an arbitrary policy π, but Lemma 1's premise, Eq. (11), is stated in Assumption 1 only for the contracting policy π_c. No argument shows that the policy optimized by Eq. (14) is itself contracting or satisfies Eq. (11). The theorem's conclusion—that π* converges—is therefore imported from the assumption rather than derived; this is a circularity in the central claim.","section":"Appendix A.3, Eq. (23)"},{"comment":"The geometric-series evaluation in Eq. (16) is wrong in sign: since e^{-2λkΔt} with λ>0 and Δt>0, the sum is 1/(1-e^{-2λΔt}), not 1/(1-e^{2λΔt}). As written, the denominator is negative and Lemma 1 asserts a negative upper bound for a nonnegative quantity, which invalidates the bound used in Eq. (23).","section":"Lemma 1, Eq. (16)"},{"comment":"Eq. (24) only establishes that the infinite discounted cost sum is finite; the Cauchy convergence argument in Eq. (25) gives C_k → 0 along the sampled times t_k, and since C(x)=∥δx∥²_M/(1+∥δx∥²_M), it gives ∥δx(t_k)∥²_M → 0 along the sampling grid. To conclude lim_{t→∞}∥δx(t)∥²_M=0 in continuous time, the proof needs an additional argument (e.g., continuity of trajectories and a bound on inter-sample growth); none is provided.","section":"Appendix A.3, Eq. (25)"},{"comment":"The CMG loss in Eq. (9) evaluates the contraction and CCM conditions using the learned dynamics model f̂_ξ, B̂_ζ, and the Limitations section acknowledges that the guarantees may not hold during training or execution. The paper does not quantify how dynamics-model error propagates to the claimed contraction certificate, so the abstract's statement that CAC provides a contraction certificate is stronger than what is actually shown.","section":"Section 3.2, Eq. (9)"},{"comment":"Theorem 1 and Lemma 1 assume a fixed contraction metric M and reward R(x)=1/(1+δx^T M δx). In the actual algorithm, M is sampled from the CMG Mχ(x) and the CMG parameters χ are updated jointly with the policy (Algorithm 1, Eq. (9)). The objective in Eq. (14) is therefore non-stationary, and the theorem does not establish guarantees for the simultaneous-learning procedure described in the abstract.","section":"Section 3.2 vs Section 4"}],"minor_comments":[{"comment":"Replace 'scaler' with 'scalar' in Eq. (10) and Table 2.","section":"Eq. (10) and Table 2"},{"comment":"Theorem 1 states the contraction rate as α > 0, but Lemma 1 and the proof of Theorem 1 use λ; the notation should be unified.","section":"Theorem 1"},{"comment":"References [6] and [17] duplicate the same Singh et al. paper; the duplicate should be consolidated.","section":"References"},{"comment":"Lemma 1's statement uses J^{π_c}_T but its proof writes J^π_T; make the policy under which the expectation is taken explicit.","section":"Lemma 1"},{"comment":"The norm in Eq. (11) is defined through a geodesic integral, while Lemma 1 applies the bound at sampled instants; clarify how the continuous-time inequality transfers to the sampled norm.","section":"Eq. (11)"},{"comment":"The inference-time column does not state whether the CMG forward pass is included; specify this for reproducibility.","section":"Table 1"},{"comment":"The theorem analyzes the reward with βπ=0, while Algorithm 1 and Table 2 use βπ=1e-2; the implications for the theoretical guarantee are not discussed.","section":"Section 4 vs Algorithm 1"}],"recommendation":"reject","confidential_remarks":"The empirical portion is useful and the writing is generally clear, but the theoretical contribution advertised in the abstract and conclusion is not established: the proof of Theorem 1 contains a sign error, an unproven transfer from π_c to π*, and a missing sampling-to-continuity argument, and the theorem is not stated for the actual bi-level learning procedure. These are fundamental issues with the central claim rather than local presentation problems, so I recommend rejection. If the authors reposition the paper as an empirical method without the asymptotic guarantee, a revised submission could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my read. The interesting part of this paper is the combination: use a learned contraction metric generator to define a Lyapunov-style reward inside an actor-critic loop, with freeze-and-learn updates and reward-conditioned entropy. That specific design is not in [7] or [8], and it leads to a real empirical study: four simulated platforms plus a TurtleBot deployment, with baselines that are meaningfully compared. The performance story is plausible: CAC is best on PVTOL, NeuralLander, and Quadrotor, and comparable on Car, with far lower inference cost than SD-LQR. The authors are also honest that the learned dynamics are only used to evaluate the conditions and that real-world satisfaction is not guaranteed.\n\nThe paper's central formal claim, Theorem 1, does not hold as written. Lemma 1 has a sign error: for λ > 0 the denominator 1 - e^{2λΔt} is negative, so the claimed upper bound is negative. The correct geometric-series factor is 1 - e^{-2λΔt}. That invalidates the lemma as stated. More seriously, the proof of Theorem 1 applies Lemma 1 to an arbitrary policy π in Equation (23), but Lemma 1 is proved only for the assumed contracting policy πc. No argument connects the RL-optimal policy π* to that bound. The conclusion that a finite discounted sum implies lim_{t→∞} ||δx(t)||²_M = 0 also skips from sampled discrete terms to continuous time. The Limitations section concedes that conditions may not hold in practice, but the proof itself is the load-bearing issue and the paper doesn't flag it.\n\nThat said, the empirical method can still stand as a heuristic. The problem is that the abstract and conclusion sell the theorem as a guarantee, and the guarantee is not established. I would not rely on the theory in its current form, and I wouldn't cite it as a certificate.\n\nWho should read this? Researchers working on RL with stability certificates will find the algorithmic idea worth discussing, and the empirical results are useful evidence that contraction-motivated rewards can help. The paper is also a good teaching example of a proof gap. A serious editor should send this to review—the empirical and algorithmic contributions deserve referee time—but the reviewers should insist on either a corrected proof of the transfer from πc to π*, or a re-framing of the theoretical claim as a heuristic rationale.","headline":"Promising empirical RL+contraction method, but Theorem 1's proof is broken and the advertised certificate is unestablished.","tokens_in":14189,"tokens_out":3139,"would_cite":false,"duration_ms":30906,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training a contraction metric alongside an actor-critic policy yields a path tracker that minimizes cumulative error and asymptotically converges to the reference, provided some contracting policy exists.","keywords":["Contraction theory","Control contraction metrics","Reinforcement learning","Actor-critic","Path tracking","Incremental exponential stability","Learned dynamics models","Uncertain systems"],"falsifier":"Run CAC on a linear system whose optimal LQR controller is provably contracting, then measure the closed-loop squared metric distance $\\|\\delta x(t)\\|_M^2$ along a reference over many initial offsets. If the trained policy's error does not decay exponentially at a positive rate for all tested references, the theorem's conclusion is not realized by the implemented algorithm.","tokens_in":13053,"feed_emoji":"🤖","tokens_out":13365,"duration_ms":132422,"temperature":0.7,"pith_summary":"Contraction actor-critic (CAC) claims to give reinforcement-learned path-tracking policies a contraction certificate: a guarantee that trajectories initialized close to a reference converge to it. The route is to train, alongside the policy, a contraction metric generator (CMG) that outputs a positive-definite Riemannian metric, then use that metric to define the reward the policy maximizes. The paper proves that if at least one contracting policy exists, the optimal policy for this reward converges asymptotically, $\\lim_{t\\to\\infty} \\|\\delta x(t)\\|_M^2 = 0$. In simulation, CAC has the lowest cumulative tracking error on three of four benchmarks and near-best on the fourth, and a TurtleBot3 deployment transfers the learned behavior to hardware. The contribution is a way to fold a stability certificate from contraction theory into model-free RL without solving a convex feasibility problem.","feed_headline":"CAC learns a contraction metric to certify an RL path tracker","feed_subtitle":"A learned metric reward gives provable convergence and the lowest tracking error in 3 of 4 testbeds.","key_machinery":"The central object is the contraction metric $M(x) \\succ 0$, a positive-definite Riemannian metric under which the differential Lyapunov function $V = \\delta x^\\top M \\delta x$ decays at a prescribed rate. The contraction metric generator is a neural network trained by the loss in Equation (9) to output a distribution over such metrics whose samples approximately satisfy the contraction and CCM conditions in Equations (3)–(5), evaluated with a learned dynamics model from Equation (8). The reward $R(x) = 1/(1 + \\delta x^\\top M \\delta x) + \\beta_\\pi H(\\pi_\\theta(x))$ converts the metric into an RL signal, so maximizing it is equivalent to minimizing a bounded monotone function of the metric-weighted tracking error. A freeze-and-learn loop alternates CMG updates with $n$ policy updates, and the load-bearing estimate is the geometric-series bound of Lemma 1: $\\|\\delta x(t_k)\\|_M^2 \\le \\|\\delta x(t_0)\\|_M^2 e^{-2\\lambda k \\Delta t}$ under a contracting policy.","core_discovery":"The central claim is that simultaneously learning a CMG and an actor-critic policy yields a policy that minimizes cumulative tracking error while inheriting a contraction certificate. Formally, Theorem 1 states that under Assumption 1, which postulates at least one contracting policy $\\pi_c$ with rate $\\lambda > 0$, the optimal policy $\\pi^*$ obtained by maximizing the discounted reward $\\tilde{R}(x) = 1/(1 + \\|\\delta x\\|_M^2)$ satisfies $\\lim_{t\\to\\infty} \\|\\delta x(t)\\|_M^2 = 0$ for every initial condition. The proof chain is: Lemma 1 bounds the assumed contracting policy's cumulative metric distance by a geometric series, Lemma 2 equates reward maximization with cost minimization, and the resulting finite discounted cost implies convergence by Cauchy's criterion. Empirically, the paper reports that CAC achieves the lowest modified area-under-the-curve tracking error on PVTOL, NeuralLander, and Quadrotor, with Car comparable to the LQR baseline, and that a simulated-trained policy tracks a reference path on a real TurtleBot3 where C3M fails and PPO diverges.","pith_inferences":["Because the theorem's proof transfers a contraction bound from an assumed contracting policy to the optimal policy, the cleanest test of the theory is to measure the trained policy's own contraction rate under the learned metric; if that rate is non-positive, the practical gains should be attributed to reward shaping rather than to the certificate.","The paper defines the tracking displacement in practice as a Euclidean difference, while the theoretical argument is stated along geodesics; replacing the Euclidean term with a geodesic or neural approximation of the metric distance would align the implementation with the proven statement.","The freeze-and-learn alternation is a generic recipe: any certificate that can be turned into a bounded reward, such as a control barrier function for safety, could be co-trained with a policy in the same way."],"forward_implications":["End-to-end training can replace the infinite-dimensional convex feasibility search in CCM synthesis, provided the learned dynamics model is accurate enough for the CMG loss to certify contraction.","Policies trained this way retain the per-step cost of a neural network, so the certificate does not impose the real-time optimization overhead of LQR-based certified controllers.","The reward-conditioned entropy term in the CMG loss is part of the method: ablating it degrades Car and NeuralLander tracking, suggesting the alternating freeze-and-learn schedule is needed for stable co-training.","Because the dynamics model is used only to evaluate the CCM conditions, the policy itself can be trained model-free, which is what lets CAC transfer from simulation to a TurtleBot3 where C3M fails and PPO diverges."],"supporting_citations":[{"why":"Supplies the contraction and CCM conditions in Equations (3)-(5) that define the contracting policies on which Assumption 1 rests.","marker":"[1]"},{"why":"Foundational contraction-theory result behind the differential Lyapunov function and the exponential convergence used throughout the paper.","marker":"[2]"},{"why":"Tutorial treatment of contraction theory that frames the metric-based stability certification CAC builds on.","marker":"[4]"},{"why":"Provides the original learned CMG loss and the C3M baseline that CAC extends toward RL optimality.","marker":"[7]"},{"why":"Shows that learning a contracting policy with the C3M approach can fail under approximate dynamics and supplies the dynamics-model pretraining and SD-LQR baseline.","marker":"[8]"},{"why":"The PPO actor-critic algorithm used as CAC's policy-learning backbone and as a model-free baseline.","marker":"[13]"}],"fun_headline_variants":["CAC: RL with a contraction guarantee for path tracking","Contraction actor-critic: provable convergence for RL tracking","RL path tracking gets a contraction certificate","Learn a metric, track a path: CAC's dual objective","CAC: contraction-guided RL for robust path tracking"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof assumes that the policy that maximizes the metric-weighted reward is itself one of the contracting policies for which the exponential-decay inequality was assumed, and the paper does not prove that the optimal policy inherits contraction.","fun_headline_variants_meta":{"raw":{"variants":["CAC: RL with a contraction guarantee for path tracking","Contraction actor-critic: provable convergence for RL tracking","RL path tracking gets a contraction certificate","Learn a metric, track a path: CAC's dual objective","CAC: contraction-guided RL for robust path tracking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000935,"raw_usage":{"total_tokens":4051,"prompt_tokens":1044,"completion_tokens":3007,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":2929}},"tokens_in":660,"tokens_out":3007,"duration_ms":25072,"temperature":1.0,"reasoning_tokens":2929,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:57:49.865714+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CAC on a linear system whose optimal LQR controller is provably contracting, then measure the closed-loop squared metric distance $\\|\\delta x(t)\\|_M^2$ along a reference over many initial offsets. If the trained policy's error does not decay exponentially at a positive rate for all tested references, the theorem's conclusion is not realized by the implemented algorithm.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the contraction and CCM conditions in Equations (3)-(5) that define the contracting policies on which Assumption 1 rests."},{"cited_title":"Lohmiller and J.-J","cited_arxiv_id":null,"evidence_quote":"Foundational contraction-theory result behind the differential Lyapunov function and the exponential convergence used throughout the paper."},{"cited_title":"Tsukamoto, S.-J","cited_arxiv_id":null,"evidence_quote":"Tutorial treatment of contraction theory that frames the metric-based stability certification CAC builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the original learned CMG loss and the C3M baseline that CAC extends toward RL optimality."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that learning a contracting policy with the C3M approach can fail under approximate dynamics and supplies the dynamics-model pretraining and SD-LQR baseline."}],"review_version":1}