{"id":"ac38ee1b-045a-4576-98e7-5fc8e7bf5e01","arxiv_id":"2501.12989","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"This paper proposes an ADMM-based multi-agent Bayesian optimization algorithm to learn the parameters of distributed model predictive controllers under model mismatch, and claims convergence and optimality guarantees.","lead":"The paper presents a method to tune distributed model predictive controllers using multi-agent Bayesian optimization, where each agent's controller parameters are learned cooperatively via an ADMM-style consensus scheme, even when local models of the system are imperfect. The approach is relevant to engineers who want to deploy DMPC in real multi-robot or networked systems without needing exact dynamic models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Convergence proof is missing: Remark 5 defers ADMM-MABO convergence, and Theorem 2's proof invalidly interchanges min and expectation, so the claimed rigorous convergence/optimality is not established.","rationale":"Good faith reading: the design idea—coordinate local BO through ADMM to tune parameterized DMPC costs—is plausible and the simulations show qualitative improvements for the presented examples. Credit is due for making the parameterization explicit (Eq. 33) and for stating Remark 5 rather than hiding it. However, the abstract promises rigorous proofs of optimality and convergence. The convergence statement is not merely a missing technical detail: Algorithm 1's ADMM step minimizes a nonconvex EI-based acquisition function, and nonconvex ADMM convergence is well known to require additional conditions; none are supplied, and Remark 5 concedes this. The optimality proof has a specific invalid step: moving minimization across an expectation in Eqs. (20)-(22). This is not a matter of disagreement with community consensus; it is an internal logical error in the derivation. The concrete LQ test would settle whether the interchange can hold in a simple case; if it fails, Theorem 2 cannot support the claimed optimality. The reader's expressiveness concern (Section III-C) is real and separate; we only partially agree with the reader's weakest-assumption choice because the proof gap in Theorem 2/Remark 5 is at least as load-bearing and is directly checkable. Verdict remains REJECT.","tokens_in":18887,"tokens_out":8891,"duration_ms":90009,"concrete_test":"Numerically test Theorem 2 on a two-agent LQ problem, e.g. x'1 = x1 + u1 + 0.5 x2, x'2 = x2 + u2 + 0.5 x1, J_i = E Σ γ^k (x_{k,i}^2 + 0.1 u_{k,i}^2 + 0.5 x_{k,1} x_{k,2}), γ=0.9. Jointly solve the Bellman equation for the optimal joint policy and compare with the fixed point obtained by the local minimization in Eqs. (20)-(22), where each agent minimizes its own value function treating the neighbor as π⋆_j. If the two fixed points differ, the min/expectation interchange in Theorem 2 is invalid and the proof of convergence to π⋆_i fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is stated as 'rigorous proofs establishing the optimality and convergence of the underlying learning method.' That claim is undercut by internal gaps. (i) The convergence analysis of Algorithm 1 is explicitly deferred: Remark 5 says convergence analysis of non-convex ADMM is left to future work. Lemma 1 asserts Lipschitz continuity of ∇α_EI without bounding the GP posterior mean/standard-deviation derivatives or supplying L; Lemma 2 shows α_EI is decreasing in its two arguments, but this does not imply the augmented Lagrangian in (66)-(67) decreases monotonically. Thus no theorem in Section IV-C proves ζ^k → ζ* or even stationarity. (ii) Theorem 2, the optimality proof, is also not sound. Eq. (20) moves min_{π_j∈M_i} outside the expectation; Eq. (21) 'folds' that minimum into the transition model as π⋆_j; Eq. (22) then moves min_{π_i} inside the expectation. In a stochastic multi-agent system the neighboring policy affects the transition distribution itself, so min over policies of an expectation is not equal to the expectation of the minimum with a fixed neighbor policy. This invalidates the derivation of the Bellman equation (24) and the conclusion π⋆_i ∈ arg min in (25). Even granting the reader's expressiveness concern about the parameterized class (33), the claimed proof machinery that converts learned value functions into optimal policies is absent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a method for learning parameterized distributed model predictive control (DMPC) schemes using multi-agent Bayesian optimization (MABO). It first establishes a formal connection between DMPC and multi-agent Markov decision processes by modifying local terminal and stage cost functions, then introduces a parametric DMPC and an ADMM-based MABO algorithm for coordinated learning. The paper claims rigorous proofs of optimality and convergence of the learning method, and reports two numerical examples (a linear multi-agent system and a formation-control problem with wheeled mobile robots) showing improved closed-loop performance relative to non-learned or non-coordinated DMPC.","tokens_in":19222,"tokens_out":2539,"duration_ms":28131,"significance":"If the theoretical claims were sound, the paper would provide a useful bridge between DMPC and multi-agent Bayesian optimization, enabling data-driven tuning of distributed controllers under model mismatch. The algorithmic structure, particularly the ADMM-based coordinated BO formulation, is reasonable and the numerical examples are suggestive. However, the central theoretical contributions—the optimality of the modified DMPC and the convergence of the MABO algorithm—are not established by the proofs provided. The main value of the manuscript is therefore conditional on repairing the theory; in its current form, the claims substantially exceed what is demonstrated.","major_comments":[{"comment":"Theorem 1 is a definitional identity rather than a structural result. The proof chooses the modified terminal cost in (17a) and stage cost in (17b) as telescoping differences of the true value function V_i^pi, which forces Eq. (16) to hold by construction. The theorem therefore does not show that a DMPC scheme—whose costs are not chosen with oracle access to V_i^pi—can capture the true value function. To make the claim meaningful, the authors would need to characterize a class of cost functions for which the equality holds without constructing the costs from V_i^pi itself, or provide an approximation bound for using a surrogate value function.","section":"Section III-B, Theorem 1 (Eqs. 15-18)"},{"comment":"The proof of Theorem 2 contains an invalid exchange of minimization and expectation. In Eq. (20), the min over neighboring policies pi_j is moved outside the expectation; then Eq. (21) folds that minimum into the transition model as pi*_j, and Eq. (22) moves min over pi_i inside the expectation. In a stochastic multi-agent system, the neighboring policy affects the transition kernel P_i itself, so the minimax-type interchange requires additional assumptions such as a saddle-point property or a contracting Bellman operator on the relevant function space. Without such justification, the Bellman equation (24) and the conclusion pi*_i in arg min in (25) are not established.","section":"Section III-B, Theorem 2 (Eqs. 20-25)"},{"comment":"The convergence analysis does not deliver the claimed result. Lemma 1 asserts Lipschitz continuity of the gradient of alpha_EI but the proof does not bound the derivatives of the GP posterior mean and standard deviation; the statement that these are 'bounded due to the smoothness of GP posterior predictions' is not a proof, and no value of L is given. Lemma 2 shows monotonicity of alpha_EI with respect to two scalar arguments, but this does not imply monotonic decrease of the augmented Lagrangian (66), which also involves lambda_i and the consensus term. Most importantly, Remark 5 explicitly states that convergence analysis of the non-convex ADMM is left to future work, which directly contradicts the abstract's claim of 'rigorous proofs establishing the optimality and convergence of the underlying learning method.'","section":"Section IV-C, Lemmas 1-2 and Remark 5"},{"comment":"The paper assumes without proof that the parameterized DMPC scheme (33) is expressive enough to represent the optimal policy, i.e., that there exists theta* such that V_i^{theta*}_i approximates V*_i. This assumption is structurally distinct from Theorem 1, which constructs modified costs using the true value function, and it is not derived from any approximation theorem. Since the learning algorithm can only search within the parametric class {V_i^{theta_i}}, the optimality claims for the learned controller require either a representability result or an explicit approximation bound. Without such a result, the numerical successes demonstrate tuning benefits but not convergence to an optimal policy.","section":"Section III-C, parameterized DMPC (33)"},{"comment":"Corollary 1 passes from the expectation-based formulation of Theorem 1 to the deterministic DMPC value function by assuming the trajectories are 'approximately deterministic,' but no quantitative condition or error bound is given for this approximation. Since the stochasticity of the true system is central to the multi-agent MDP formulation, this step is load-bearing for the claim that the DMPC scheme (28) captures V_i^pi. The proof of the corollary should state the approximation error explicitly and identify the regime in which it vanishes.","section":"Corollary 1 and Eq. (29)"}],"minor_comments":[{"comment":"The concluding remarks refer to 'dual composition'; this should presumably read 'dual decomposition.'","section":"Section VI"},{"comment":"The notation 'pi_j in M_i' is not formally defined; it should specify that pi_j ranges over the policies of the neighboring agents in M_i.","section":"Section III-A, Eq. (9)"},{"comment":"The multiplier update appears with superscript I, but the stopping criteria in (7) are stated in terms of mu_i and the text switches between iteration indices; the notation could be made consistent.","section":"Section II-B, Eq. (6)"},{"comment":"The proof of Lemma 1 depends on properties of Phi and phi that are standard, but the statement 'both nabla mu and nabla sigma are bounded due to smoothness' should either be proven or replaced by an explicit assumption on the kernel and the fitted GP.","section":"Section IV-C, Lemma 1"},{"comment":"The caption of Fig. 2 lists colors in the order gray, green, cyan, red, but the order of appearance in the sentence does not match the order in the figure; please align the description.","section":"Section V, Example 1"},{"comment":"The acquisition function is written as alpha_EI(theta) = -((J* - mu(theta)) Phi(Z) + sigma(theta) phi(Z)); this is the negative expected improvement, so the text should explicitly note the sign convention used for the minimization.","section":"Section IV-A, Eq. (39)"}],"recommendation":"reject","confidential_remarks":"The manuscript has a clear narrative and the numerical illustrations are plausible, but the central theoretical contributions are not sound: Theorem 1 is a tautology, Theorem 2's proof has an invalid expectation-minimization exchange, and the convergence analysis is explicitly deferred in Remark 5. These are load-bearing for the abstract's claims. A revision that addresses the theory could eventually be reconsidered, but in the current form the gap between claims and proof is too large for publication. I would also flag that the manuscript repeatedly describes proofs as 'rigorous' while the main proof of convergence is left to future work; the abstract should be reconciled with the body of the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper's genuinely new piece is the combination: an ADMM-based multi-agent Bayesian optimization loop that tunes parameterized local MPC cost functions when the local models are imperfect and the agents' closed-loop costs are coupled. That is a real gap in the BO-MPC literature, and the paper's decomposition of the global tuning problem into local acquisition functions with a consensus constraint is a plausible way to approach it. The problem is practical and the framing is sensible.\n\nWhat the paper does well: it spells out why the local value function depends on neighbor policies through the multipliers, and the parameterized scheme in (33) with slack smoothing is a reasonable way to make the DMPC policy learnable. The examples suggest the learned controller beats both the unlearned DMPC and independent BO on the reported settings.\n\nThe soft spots are serious and largely where the reader puts them. Theorem 1 and Corollary 1 are not predictive theorems: the modified costs are chosen as telescoping differences of the true value function, so the equality is built in. That is a construction, not a derivation, and the paper should say so. Theorem 2's proof is not valid as written. Equations (20)-(22) move minima across an expectation over a transition distribution that depends on the neighbors' policies; the min of an expectation is not the expectation of the min with a fixed neighbor policy in a stochastic coupled system. The Bellman equation (24) and the optimality conclusion (25) do not follow. Lemma 1's Lipschitz proof is also incomplete: boundedness of the GP gradient is asserted, not shown, and even a valid Lipschitz gradient would not by itself give ADMM convergence for the non-convex EI objective. Remark 5 explicitly punts the convergence analysis of the non-convex ADMM to future work, which directly contradicts the abstract's 'rigorous proofs' claim. The numerics are qualitative—no code, no repeated trials, no quantitative metrics—so the empirical claim is weaker than the plots suggest.\n\nThat said, the core idea is salvageable. The authors need to either prove the min-expectation interchange under stated assumptions or drop the optimality theorem, provide an actual convergence result or clearly label the ADMM analysis as heuristic, and add quantitative experiments. A serious referee could help them do that. I would send it to review, but with the expectation of major revision.\n\nRecommendation: engage, but require the advertised proofs to be replaced by deliverable versions.","headline":"A useful algorithmic combination for DMPC tuning, but the claimed proofs of optimality and convergence are not delivered; worth major revision, not publication as is.","tokens_in":19756,"tokens_out":2639,"would_cite":false,"duration_ms":27678,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proves that a distributed model predictive controller can reproduce the optimal policy of a multi-agent system, even when every local model is wrong, by modifying the local costs and learning them with coordinated Bayesian…","keywords":["distributed model predictive control","multi-agent systems","Bayesian optimization","ADMM","Markov decision process","cost function learning","model mismatch","closed-loop performance"],"falsifier":"Take a two-agent linear system with a known quadratic optimal value function $V^\\star_i$. Implement the modified costs from Theorem 1 ($\\hat{V}^{f}_i = V^{\\pi}_i$ at the terminal state, $\\hat{L}_i = V^{\\pi}_i - \\gamma \\mathbb{E}[V^{\\pi}_i(\\text{next})]$) in the DMPC scheme (28) under a deliberately wrong model, simulate the closed loop, and compare the resulting $N$-step value function with $V^{\\pi}_i(\\tilde{s}^k)$ at every visited augmented state in $\\Xi$; any mismatch would falsify the claimed equality. Separately, on a non-convex closed-loop performance landscape with a known global optimum, run Algorithm 1 and record whether the iterates converge to that optimum; persistent convergence to a worse local optimum would show that the monotone-decrease guarantee is weaker than what the paper's convergence language suggests.","tokens_in":18655,"feed_emoji":"🤖","tokens_out":10985,"duration_ms":102696,"temperature":0.7,"pith_summary":"This paper claims that a distributed model predictive controller can be tuned, using only closed-loop data and not an accurate model, to reproduce the optimal policy of the true multi-agent Markov decision process. The central theorem shows that no matter how inaccurate the local prediction models are, one can modify each agent's terminal and stage costs so that the N-step value function of the local MPC exactly equals the true local value function. Because that modification is built from the unknown true value function, the paper then parameterizes the DMPC costs and learns the parameters with an ADMM-coordinated multi-agent Bayesian optimizer, proving monotone decrease of the associated augmented Lagrangian. A sympathetic reader should care because this gives a principled, decomposition-friendly route to optimal distributed control under model mismatch, exactly the regime where standard DMPC degrades.","feed_headline":"Modified DMPC costs can match the true multi-agent optimal policy","feed_subtitle":"Coordinated Bayesian optimization tunes each agent's costs from data despite wrong models","key_machinery":"The load-bearing object is the telescoping cost modification: choose the modified terminal cost as the true value function $V^{\\pi}_i(\\hat{s}^{k+N})$ and the modified stage cost as the temporal-difference-like difference $V^{\\pi}_i(\\hat{s}^{\\ell}) - \\gamma \\mathbb{E}[V^{\\pi}_i(\\hat{s}^{\\ell+1})]$. Summing these along an $N$-step trajectory cancels every intermediate term and leaves the current true value $V^{\\pi}_i(\\tilde{s}^k)$, regardless of the model used to generate the trajectory, provided the expectation is finite on $\\Xi$ as in Assumption 2. In the deterministic DMPC variant, the same cancellation is enforced by forcing the slack variable to start at the neighbor's influence $W_{ij}(s^k_j)$ and propagate through $\\delta^{\\ell}_i$. The coordinated learning part rests on the ADMM split of the global closed-loop objective into local Bayesian-optimization problems with a consensus constraint $\\zeta_i = \\bar{\\zeta}$, using the expected-improvement acquisition whose smoothness and monotonicity are proved in Lemmas 1 and 2.","core_discovery":"Under Assumption 2, that the expected value of the local value function stays finite on the relevant set of augmented states, Theorem 1 constructs, for any joint policy $\\pi$, modified local terminal and stage costs $\\hat{V}^{f}_i(\\hat{s}^{k+N}) = V^{\\pi}_i(\\hat{s}^{k+N})$ and $\\hat{L}_i(\\hat{s}^{\\ell},\\pi_i(\\hat{s}^{\\ell})) = V^{\\pi}_i(\\hat{s}^{\\ell}) - \\gamma \\mathbb{E}[V^{\\pi}_i(\\hat{s}^{\\ell+1})]$ such that the $N$-step value function of the modified DMPC telescopes to exactly $V^{\\pi}_i(\\tilde{s}^k)$, the true local value function of the multi-agent MDP, for any horizon $N$. Corollary 1 carries the same equality over to the deterministic DMPC scheme (28), in which the initial slack variable is forced to equal the neighbor-coupling term $W_{ij}(s^k_j)$ and the slack is propagated along the horizon as $\\bar{w}^{\\ell+1}_i = \\bar{w}^{\\ell}_i + \\delta^{\\ell}_i$; the telescoping cancellation then leaves $V^{\\pi}_i(\\tilde{s}^k)$ even though the trajectories are generated by the wrong model $\\hat{f}_i$. Since these constructions require the unknown $V^{\\pi}_i$, the paper parameterizes the local costs (and optionally dynamics and constraints) as $V^{\\theta_i}_i$, and proposes to learn the parameters $\\theta_i$ by ADMM-based multi-agent Bayesian optimization, minimizing the closed-loop performance $J^N_i(\\zeta)$ with consensus variables; Lemma 1 and Lemma 2 establish differentiability, gradient Lipschitzness, and monotonicity of the expected-improvement acquisition function, which yields monotone decrease of the augmented Lagrangian under the ADMM updates and stationarity conditions at a fixed point. Numerical examples on a three-agent linear system and on a wheeled-mobile-robot formation task show coupling constraints satisfied and formation achieved after learning, whereas the unlearned DMPC violates them.","pith_inferences":["Editorial inference: the paper's existence proof does not quantify how expressive the parametric class in (33) must be; if that class cannot approximate $V^\\star_i$ well, the learned DMPC may converge to a consistent but strictly suboptimal policy, and measuring this gap would be a direct way to judge practical utility.","Editorial inference: the monotone-decrease argument for the augmented Lagrangian is not a convergence guarantee for the non-convex BO objective; a natural testable extension is to benchmark ADMM-MABO against known global optima on non-convex closed-loop costs to see how often it finds the global rather than a local optimum.","Editorial inference: because the telescoping construction works through expectations, the same cost modification should extend to stochastic transition models (for example, GP-based MPC) by replacing the deterministic DMPC trajectories with their expectations; the paper restricts itself to deterministic local models.","Editorial inference: anchoring the slack propagation to $W_{ij}(s^k_j)$ suggests a lightweight variant where agents learn the propagation function $\\delta^{\\ell}_i$ itself rather than only the costs, which would let the MABO correct for model mismatch in the coupling dynamics too."],"forward_implications":["If Theorem 1 and Corollary 1 hold, then model mismatch in the local prediction models does not prevent the distributed MPC architecture from representing the exact optimal value function; only the cost (and possibly parameterized dynamics and constraints) must be adjusted.","The proposed ADMM-MABO algorithm gives a parallel, communication-based way to learn these adjustments from closed-loop data, and the monotone-decrease result for the augmented Lagrangian provides a stationarity guarantee at any fixed point of the learning loop.","The examples demonstrate the practical consequence: coupling constraints that are violated under an unlearned dual-decomposition DMPC are satisfied after coordinated learning, and the learned scheme outperforms running independent Bayesian optimizers per agent.","The non-myopic acquisition function of Proposition 1 can be substituted into the same MABO structure without changing the architecture, so the framework is not tied to the greedy expected-improvement choice."],"supporting_citations":[{"why":"Provides the dual-decomposition distributed optimization framework on which the local MPC problems are built.","marker":"[4]"},{"why":"Extends dual-decomposition DMPC to nonlinear settings and serves as the baseline local MPC formulation.","marker":"[5]"},{"why":"Introduced the MDP–MPC cost-modification approach that Theorem 1 and Corollary 1 adapt to distributed systems with imperfect models.","marker":"[8]–[12]"},{"why":"Supplies the linear value-decomposition used in Remark 2 to split the joint value function into per-agent local value functions.","marker":"[22]"},{"why":"Provides the ADMM-based multi-agent Bayesian optimization algorithm that the paper adapts for coordinated learning of local DMPC parameters.","marker":"[23]"},{"why":"Single-agent stability-informed BO for MPC cost learning is the closest prior work that the multi-agent coordinated scheme extends.","marker":"[20]"}],"fun_headline_variants":["Bayesian optimization tunes DMPC costs to match optimal policy","Bayesian tuning of DMPC costs hits multi-agent optimal policy","Learn DMPC costs that match the optimal multi-agent policy","BO-tuned DMPC costs replicate the optimal joint policy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the parameterized DMPC family (33) contains a parameter choice $\\theta^*$ whose value functions closely approximate the true optimal value functions; the paper assumes this expressivity without providing an approximation bound, and it is separate from the cost-modification theorem, which uses the true value function directly.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian optimization tunes DMPC costs to match optimal policy","Bayesian tuning of DMPC costs hits multi-agent optimal policy","Learn DMPC costs that match the optimal multi-agent policy","BO-tuned DMPC costs replicate the optimal joint policy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001254,"raw_usage":{"total_tokens":5227,"prompt_tokens":1122,"completion_tokens":4105,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":738,"completion_tokens_details":{"reasoning_tokens":4033}},"tokens_in":738,"tokens_out":4105,"duration_ms":26770,"temperature":1.0,"reasoning_tokens":4033,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:33:09.618230+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a two-agent linear system with a known quadratic optimal value function $V^\\star_i$. Implement the modified costs from Theorem 1 ($\\hat{V}^{f}_i = V^{\\pi}_i$ at the terminal state, $\\hat{L}_i = V^{\\pi}_i - \\gamma \\mathbb{E}[V^{\\pi}_i(\\text{next})]$) in the DMPC scheme (28) under a deliberately wrong model, simulate the closed loop, and compare the resulting $N$-step value function with $V^{\\pi}_i(\\tilde{s}^k)$ at every visited augmented state in $\\Xi$; any mismatch would falsify the claimed equality. Separately, on a non-convex closed-loop performance landscape with a known global optimum, run Algorithm 1 and record whether the iterates converge to that optimum; persistent convergence to a worse local optimum would show that the monotone-decrease guarantee is weaker than what the paper's convergence language suggests.","supporting_citations":[{"cited_title":"and Shames, I","cited_arxiv_id":null,"evidence_quote":"Provides the dual-decomposition distributed optimization framework on which the local MPC problems are built."},{"cited_title":"Nonlinear dist ributed model predictive control with dual decomposition and event -based communication approach,","cited_arxiv_id":null,"evidence_quote":"Extends dual-decomposition DMPC to nonlinear settings and serves as the baseline local MPC formulation."},{"cited_title":"Multi-agent blac k-box opti- mization using a bayesian approach to alternating directio n method of multipliers,","cited_arxiv_id":null,"evidence_quote":"Provides the ADMM-based multi-agent Bayesian optimization algorithm that the paper adapts for coordinated learning of local DMPC parameters."},{"cited_title":"S tability- informed bayesian optimization for mpc cost function learn ing,","cited_arxiv_id":null,"evidence_quote":"Single-agent stability-informed BO for MPC cost learning is the closest prior work that the multi-agent coordinated scheme extends."}],"review_version":1}