{"id":"81c97c47-124d-4207-9505-29e89d8ab969","arxiv_id":"2506.19160","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"AgenticControl tunes controllers on four simulated plants using six cooperating LLM agents, reporting performance near LQR and above MATLAB PIDTuner, without releasing code or complete data.","lead":"The authors built AgenticControl, a system in which six large language model agents cooperate through JSON messages to select controllers, tune gains, judge termination, and adjust search ranges, then tested it on four simulated control plants. The reported results are competitive with LQR and better than MATLAB PIDTuner, but no code or full data is provided, so independent verification is not possible.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (6) in Appendix B models a stable pendulum, not the unstable inverted pendulum claimed in Table 1; if used in simulation, the FSF-vs-LQR and PID-vs-PIDTuner comparisons are for the wrong plant.","rationale":"Both the abstract's headline and the reader's weakest_assumption point to the closed-loop simulations. I found a concrete flaw in the plant model: Eq. (6) has the wrong sign for gravity for an inverted pendulum, so either the simulations used a stable plant (undermining the 'competitive against LQR/PIDTuner' claims) or the appendix is inaccurate (defeating reproducibility). The 55% figure in the abstract adds a second checkable inconsistency with Table 3. The framework design itself—six LLM agents with JSON communication—is plausible, and the conversation logs provide some transparency, but the paper ships no code, so the load-bearing quantitative claims cannot be independently verified. The reader's conditional verdict remains appropriate; I would not shift to rejection because the sign error may be a typo and the underlying experiments could be correct. The condition should explicitly require release of the simulation code and a corrected, verifiable plant equation.","tokens_in":39052,"tokens_out":12837,"duration_ms":123986,"concrete_test":"Linearize Eq. (6) at θ=0 with Table 1 parameters and compute the open-loop eigenvalues; also recompute the closed-loop step response for the FSF gains reported in Table 3. If the open-loop eigenvalues are stable, the plant is not the claimed inverted pendulum and the case-study results must be re-run with the corrected sign (+L1 g m1 sinθ).","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix B Eq. (6) gives θ¨ = (τ − L1 g m1 sinθ − bθ˙)/(L1^2 m1). For an inverted pendulum, with θ measured from the upright, the gravity torque should enter with a positive sign (+m1 g L1 sinθ) because the upright equilibrium is unstable; as written, the term is stabilizing, and the open-loop linearization at θ=0 has poles with negative real part (approximately −2 ± j4.43 for the Table 1 parameters), contradicting Table 1's 'Unstable' entry. If the simulations used this sign, Tables 2–4 for the inverted pendulum describe a stable pendulum, so the claimed 'competitive performance against classical methods' does not address the stated control challenge. If the code used the corrected sign, then the manuscript's governing equations are wrong and the results cannot be reproduced from the paper as written. A second, independent inconsistency is that the abstract's '55% reduction' in PID tracking error versus PIDTuner is not supported by Table 3: the nominal row gives (0.4566−0.2694)/0.4566 ≈ 41%, the noise row ≈ 45%, and the uncertainty row ≈ 73%; no row yields 55%. This further indicates that the headline numbers are not tightly coupled to the tables.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"AgenticControl proposes a six-agent LLM framework (LLMSelector, LLMScenarist, LLMActor, LLMCritic, LLMTerminator, LLMJuror) that selects controller types, designs test scenarios, iteratively tunes parameters through an actor-critic loop, and decides when to terminate, reconsider parameter ranges, or switch controllers. The workflow is demonstrated on four plants: DC motor position control, ball and beam, inverted pendulum, and double inverted pendulum. The reported comparisons include full-state feedback versus LQR, PID versus MATLAB PIDTuner, and a comparison of five LLM backbones. The manuscript claims competitive performance against classical methods, a 55% reduction in PID tracking error relative to PIDTuner, and robust multi-scenario behavior. The authors also state that, for reproducibility, the controller list and scenarios were manually specified in the experiments while the corresponding agents remain available for fully automated operation.","tokens_in":39419,"tokens_out":5798,"duration_ms":55035,"significance":"If the results were fully supported, the framework would be a useful demonstration that structured multi-agent LLM collaboration can automate a meaningful portion of control design. The JSON-based inter-agent protocol, the actor-critic loop, and the juror-driven range reconsideration are concrete and potentially reusable ideas, and the comparisons to LQR and PIDTuner are explicit and falsifiable. However, the current manuscript does not ship code, data, or complete simulation details, and several headline claims are not backed by the tables. The significance is therefore prospective rather than established: the architecture is interesting, but the evidence as presented does not yet substantiate the stated performance gains.","major_comments":[{"comment":"The inverted-pendulum equation is written with a stabilizing gravity torque. As written, θ¨ = (τ − L1 g m1 sinθ − b θ˙)/(L1^2 m1), the term −L1 g m1 sinθ drives θ back to zero, so the open-loop linearization about θ = 0 has poles with negative real part (approximately −2 ± j4 with the Table 1 parameters), contradicting the 'Unstable' entry in Table 1. If this equation was used in the simulations, the inverted-pendulum results in Tables 2–4 are for a stable pendulum and do not address the claimed stabilization challenge. If the code used the opposite sign, the governing equation in the paper is wrong and the results cannot be reproduced from the manuscript.","section":"Appendix B, Eq. (6), Table 1"},{"comment":"The headline claim of 'reducing PID tracking error by 55%' is not supported by Table 3. Comparing the AgenticControl PID row with the PIDTuner row, the nominal scenario gives (0.4566 − 0.2694)/0.4566 ≈ 41.0%, the noise row gives ≈ 44.6%, and the uncertainty row gives ≈ 72.9%; no row yields 55%. The same conclusion holds if the FSF row is used instead of the PID row. The abstract's quantitative claim must be corrected to match a specific table entry or removed.","section":"Abstract and Table 3"},{"comment":"The framework declares success even when the stated targets are not met. In the DC motor case, the LLMTerminator returns TERMINATE_SUCCESS at iteration 27 while MSE = 0.9103 versus target 0.9, overshoot = 10.98% versus target 10%, and settling time = 3.76 s versus target 3.0 s. In the ball-and-beam Scenario 3, the text reports success although the overshoot (10.72% in the log and 12.78% in Table 2) far exceeds the 5% target. Because the termination decision is a central component of the framework, the paper needs a principled, externally checkable success criterion rather than allowing the LLM terminator to override unmet targets.","section":"Sections 4.2.1 and 4.2.2, Fig. 6, Table 2"},{"comment":"No simulation code, data, or random seeds are provided, and the implementation details needed to reproduce Tables 2–4 are missing. Table 1 lists actuator limits (Vmax = 24 V, τmax = 1 N m) but the manuscript does not state whether actuator saturation is enforced. It also does not specify the ODE solver, time step, Monte Carlo sampling distributions, or how measurement noise, actuator disturbances, and parametric uncertainties are realized in each scenario. Since every conclusion depends on the closed-loop simulations, these omissions prevent independent verification of the central claim.","section":"Sections 4.1 and Appendix B"},{"comment":"The text states that the best ball-and-beam parameters achieve overshoot ≤ 12.20% in all scenarios and that Scenario 3 obtains MSE = 0.0501, but Table 2 lists overshoot values of 11.87%, 13.89%, and 12.78% for the three scenarios and MSE = 0.0508 for the uncertain-parameters row. These inconsistencies make it unclear which numbers constitute the reported results. The text, the conversation log in Appendix C, and Table 2 need to be reconciled.","section":"Section 4.2.2 versus Table 2"}],"minor_comments":[{"comment":"The title contains 'A N AUTOMATED' and Fig. 1 contains 'Monte Catlo Simulations'; the abstract also uses 'significantly outperforming' where 'significantly outperforms' is required.","section":"Title page and Fig. 1"},{"comment":"The 'Stability' column lists 'Non-minimum Phase' as if it were a stability classification; non-minimum phase is a property of transmission zeros and should be listed separately.","section":"Table 1"},{"comment":"The lower panels share an x-axis extending to 120 iterations while the individual model subplots are truncated at 35–100 iterations; the axes should be unified or explicitly labeled.","section":"Fig. 9"},{"comment":"The model name is written inconsistently as 'GPT-4o mini' in the text and 'GPT-4omini' in Fig. 9 and Table 4; one consistent name should be used.","section":"Section 4.4, Fig. 9, Table 4"},{"comment":"The paper states that Supplementary Materials contain full conversation logs, but no supplementary file appears in the manuscript; the availability of these materials should be stated explicitly.","section":"Section 5 and Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about Eq. (6) is genuine on reading: the sign of the gravity term makes the inverted pendulum open-loop stable, which is incompatible with the paper's own Table 1. Combined with the unsupported 55% claim and the terminator declaring success despite unmet targets, the empirical case is not yet reliable. I would not reject outright because the framework idea is plausible and the fixes—rerunning simulations with the correct sign, correcting the abstract, shipping code and seeds, and enforcing a formal success criterion—are within the scope of a revision, but the revision must address these points before I could support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: plausible incremental extension of multi-agent LLM control design, but the empirical claims as written aren't reliable enough to build on. I'd send it to peer review, not desk reject, but I'd want major revisions and artifacts before citing anything.\n\nWhat's actually new: the six-agent topology with a dedicated Juror that resolves ambiguous terminations and can reconsider parameter ranges; the progression from nominal to noise/uncertainty scenarios; and the five-LLM comparison on the double inverted pendulum. The supplementary conversation logs are a real plus—they show actual agent behavior rather than a black box. The Juror's range-reconsideration mechanism is a genuine addition over ControlAgent and RobotGPT, and the DC-motor log shows the loop exploring and converging as advertised.\n\nSoft spots, in order of severity:\n\n1. Appendix B Eq. (6) for the inverted pendulum has gravity in the wrong direction. As written it models a stable hanging pendulum (the torque term −L1 g m1 sinθ stabilizes θ = 0), contradicting Table 1's \"Unstable\" entry. If the simulations used that equation, the FSF-vs-LQR and PID-vs-PIDTuner comparisons for the inverted pendulum were run on the wrong plant. If they used the correct sign, the paper cannot be reproduced from the manuscript. Either way, one of the two headline case studies is compromised.\n\n2. The Terminator declares TERMINATE_SUCCESS even when targets are not met: DC motor final MSE 0.9103 vs target 0.9, overshoot 10.98% vs 10%; ball-and-beam scenario 3 overshoot 10.72% vs target 5%. The termination criteria are not actually enforcing the stated targets.\n\n3. The abstract claims a 55% PID tracking-error reduction versus PIDTuner. Table 3 yields roughly 41% nominal, 45% noise, 73% uncertainty—no row gives 55%. The headline number doesn't match the table.\n\n4. No code or data. The Monte Carlo results can't be checked, LQR weights are hand-picked, and implementation details (saturation, integration method, sampling) are unspecified. The hand-set targets and ranges also mean the automation isn't fully autonomous, though that's a stated design choice and not fatal.\n\nThe framework's core idea is sound and the agent interactions in the logs look genuine. As an automation layer for routine PID/state-feedback tuning, it deserves circulation. But the pendulum equation must be fixed, the metric reporting reconciled, and code/scripts released before the results should be taken quantitatively.\n\nThis paper is for researchers in LLM/control automation, especially those comparing agent architectures for design workflows. With those revisions, a serious referee could get it into good shape.","headline":"Worth reading as an incremental LLM-agent tuning framework, but the inverted-pendulum equation error and mismatched headline numbers make the empirical claims unreliable as written.","tokens_in":39857,"tokens_out":3785,"would_cite":false,"duration_ms":35200,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A coordinated team of six pretrained LLM agents automates controller design, matching LQR on state feedback and beating MATLAB's PIDTuner by 55% in PID tracking error.","keywords":["multi-agent LLM framework","control system design","PID tuning","full-state feedback","actor-critic optimization","in-context learning","scenario-based robustness","LQR comparison"],"falsifier":"Re-run the four plants using the Appendix B equations with the exact gains in Tables 2-4 and check whether the MSE, settling time, and overshoot values reproduce; in particular, compute the PID tracking-error reduction from Table 3's numbers and verify the claimed 55 percent.","tokens_in":38898,"feed_emoji":"🤖","tokens_out":8770,"duration_ms":78235,"temperature":0.7,"pith_summary":"AgenticControl is a multi-agent framework that automates the controller-design loop with six large language model agents—a selector, scenarist, actor, critic, terminator, and juror—that exchange structured JSON messages. The paper tries to establish that this LLM team can choose a controller family, propose gains, evaluate simulations, and decide when to stop or switch, well enough to match classical methods on four benchmark plants: DC motor position, ball and beam, inverted pendulum, and double inverted pendulum. Its full-state feedback results land close to an LQR baseline, and its PID controller beats MATLAB's PIDTuner with a reported 55 percent reduction in tracking error. If true, the implication is that controller design for nonlinear or uncertain plants can be handed to pretrained LLMs with no fine-tuning, lowering the expertise threshold.","feed_headline":"Six LLM agents tune controllers, matching LQR and beating PIDTuner","feed_subtitle":"Actor-critic LLM loop matches LQR accuracy and cuts PID tracking error 55% versus MATLAB's tuner.","key_machinery":"The load-bearing mechanism is the actor–critic optimization loop mediated by the LLMJuror and the shared buffer. In each inner iteration the LLMActor proposes a parameter vector; the LLMCritic compares simulated metrics against targets and issues EXPLORE or EXPLOIT guidance; the LLMTerminator decides CONTINUE or TERMINATE; and the LLMJuror, when termination is disputed, can issue RECONSIDER_RANGE to shrink or shift the parameter search window or REDESIGN_APPROVED to switch controller type. This loop is embedded in an outer scenario progression that escalates from nominal conditions to noise and disturbances to parametric uncertainty, so robustness is obtained by construction rather than by a separate worst-case analysis.","core_discovery":"The central discovery is that a structured debate among pretrained LLM agents can perform control design end to end. In AgenticControl, the LLMSelector starts with the simplest plausible controller, the LLMScenarist escalates test conditions from nominal to measurement noise to parametric uncertainty, the LLMActor proposes parameter values, the LLMCritic scores them against target MSE, settling time, overshoot, and stability, the LLMTerminator decides whether to continue, and the LLMJuror handles ambiguous cases by narrowing parameter ranges or approving a controller switch. On the four validated plants this loop converges to stable controllers; for the inverted pendulum the FSF gains match the LQR solution almost exactly (MSE $0.2531$ vs. $0.2529$), and in the double pendulum the DeepSeek-V3 model reached target performance fastest. The paper frames these results as evidence that a general-purpose LLM scaffold, not a bespoke tuning algorithm, can automate robust control design.","pith_inferences":["The framework's success likely comes more from the structured critic, terminator, and juror feedback loop than from any control-theoretic insight in the LLMs; the same loop could tune observer gains, MPC weights, or reward coefficients with minimal changes.","A clean testable extension is to run the loop with the juror removed; if convergence degrades, the range-reconsideration mechanism is the active ingredient, which would isolate why the framework works.","The reported 55 percent reduction should be checked against a consistent metric definition, since Table 3's PID MSE (0.2694) and PIDTuner (0.4566) imply a 41 percent reduction under the usual percentage formula; the paper may be using a different denominator or a different scenario.","Because no code or simulation details ship with the paper, the strongest validation would be an independent re-implementation; until then the framework's practical value depends on the fidelity of the closed-loop simulator used during optimization."],"forward_implications":["A plant model plus a simulator becomes sufficient input for a tuned controller; the six-agent loop replaces manual gain scheduling and expert heuristics.","The same actor-critic scaffold should transfer to other controller families, since the Actor prompt is templated over arbitrary parameter schemas.","Because scenario levels escalate automatically, controllers are pushed toward robustness under noise and uncertainty as part of the design search.","The five-model comparison on the double pendulum indicates that convergence speed and variance depend on the LLM chosen, so the framework's cost and reliability can be traded by model selection.","The 55 percent PID tracking-error reduction over PIDTuner, if reproduced, would give non-expert users a concrete automation advantage on standard tuning tasks."],"supporting_citations":[{"why":"Classical PID tuning reference that defines the manual tuning paradigm AgenticControl automates and compares against.","marker":"[Åström and Hägglund, 1995]"},{"why":"Standard feedback-control text supplying the LQR and state-feedback baseline used in the benchmark comparisons.","marker":"[Franklin et al., 2010]"},{"why":"ControlAgent, the closest prior multi-agent LLM control-design system; AgenticControl extends it with critic, terminator, and juror roles.","marker":"[Guo et al., 2024]"},{"why":"Six-agent LLM framework for power-electronics control that motivates the multi-agent collaboration structure.","marker":"[Cui et al., 2024]"},{"why":"SmartControl, an LLM-agent PID design tool with evolutionary optimization, a direct alternative this paper's PID results are compared against in spirit.","marker":"[Tohma et al., 2025]"},{"why":"Shows GPT-4 can analyze and design DC motor controllers but struggles with tuning, motivating the iterative actor-critic feedback loop.","marker":"[Gashi et al., 2024]"},{"why":"LLM tuning assistant for PMSM drives that found LLMs alone do not guarantee optimality, supporting the need for the computational simulation loop.","marker":"[Tarczewski et al., 2025]"},{"why":"Offline RL co-design with three LLM agents, one of the multi-agent patterns the framework builds on.","marker":"[Chen et al., 2024]"}],"fun_headline_variants":["LLM agent swarm designs controllers, matches LQR, beats PIDTuner","LLM actor-critic loop automates control design, matches LQR","LLM agents cut PID error 55% and match LQR accuracy","Automated control via LLM debate: matches LQR, reduces PID error 55%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's comparisons to LQR and PIDTuner rest on the closed-loop simulations behind Tables 2 to 4 being implemented correctly, because no code is provided and the appendix leaves details like actuator saturation and Monte Carlo sampling unspecified.","fun_headline_variants_meta":{"raw":{"variants":["LLM agent swarm designs controllers, matches LQR, beats PIDTuner","LLM actor-critic loop automates control design, matches LQR","LLM agents cut PID error 55% and match LQR accuracy","Automated control via LLM debate: matches LQR, reduces PID error 55%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00084,"raw_usage":{"total_tokens":3682,"prompt_tokens":988,"completion_tokens":2694,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":604,"completion_tokens_details":{"reasoning_tokens":2610}},"tokens_in":604,"tokens_out":2694,"duration_ms":18734,"temperature":1.0,"reasoning_tokens":2610,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:35:30.100791+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the four plants using the Appendix B equations with the exact gains in Tables 2-4 and check whether the MSE, settling time, and overshoot values reproduce; in particular, compute the PID tracking-error reduction from Table 3's numbers and verify the claimed 55 percent.","supporting_citations":[],"review_version":1}