{"id":"8c6b7f7c-2cc9-4975-adb1-d3ee0857d72b","arxiv_id":"2507.17695","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Pairing LLMs with deterministic optimizers improves RAN control and SLA negotiation accuracy and lets small language models run in near-real-time loops.","lead":"This paper proposes pairing large language models with classic mathematical optimizers to make AI agents more accurate in running wireless networks. The authors show on a 5G testbed that the combination cuts decision errors versus standalone language-model agents and lets small models work in near-real-time loops.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Type II '95% CI' is not a statistical bound on the Pareto optimum, and Table 7's ground truth is the same optimizer that produces it; the SLA fivefold claim is partly by construction.","rationale":"The central architectural idea — optimizer-flanked LLM control — is coherent, and the Type I results are a fair comparison: the same intent target and channel trace are used for standalone LLMs and for the symbiotic P-control agent, with RMSE measured against the operator intent. The most load-bearing issue is concentrated in Type II. Eq. (6) is strictly concave, so the unique maximizer is a fixed point of the utility model; jittering the starting point x(0) and re-running GD in Sec. 3.3.1 cannot produce a valid 95% confidence interval for that maximizer. It produces at best a sensitivity analysis of the solver's convergence. Interpreting it as 'where the Pareto-optimal value lies with 95% confidence' (Sec. 3.3) is unsupported. The same optimizer defines the ground-truth consensus used for MAE in Table 7, and the guard-rail instruction in Sec. 3.3.1 tells every agent to bid inside the optimizer's own interval. The 8-15x improvement in SLA MAE is therefore at least partly a measurement artifact. An independent analytic ground truth, as proposed in the concrete test, would settle whether the symbiotic advantage survives. If it does, the paper's practical claim is intact and the statistical language can be corrected; if not, the central 'trustworthy bound' claim fails. I also note the fivefold headline is not uniform: Table 4 ratios range from about 2.8x to 10x depending on model, and the GPU comparison reports roughly 3500 GB VRAM for cloud gpt-4o, which is not a locally allocated resource. These are secondary. Section 7 limits the testbed to single-cell and Section 9 promises code only upon acceptance, which hinders independent verification but is not itself a correctness flaw. The reader's CONDITIONAL verdict is appropriate, and my analysis does not change it.","tokens_in":26298,"tokens_out":4614,"duration_ms":47708,"concrete_test":"Recompute Table 7's MAE for the symbiotic agents using an independent analytic ground-truth optimum for Eq. (6), which has a closed form because the objective is a sum of quadratics, instead of the algorithm's returned consensus. In the same pass, re-run the Eq. (8) interval procedure across many random game instances and check whether the interval contains the analytic optimum at roughly 95% frequency. If the symbiotic MAE rises above the claimed sub-1.3 Mbps, or the coverage is far from 95%, the confidence-interval construction and the evaluation are circular.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest link is the Type II '95% confidence interval' (Eq. 8) and its role as the trustworthiness guarantee. Eq. (6) is a deterministic, strictly concave program with a unique maximizer; the R=100 'jittered restarts' in Sec. 3.3.1 perturb only the starting point x(0), not the utility parameters or the data. All runs therefore converge to the same optimum up to numerical tolerance, so the spread s in Eq. (8) mostly reflects incomplete convergence or the arbitrary jitter scale, not statistical uncertainty about the true Pareto-optimal SLA. Worse, the guard-rail instruction in Sec. 3.3.1 forces every bid into [L,U], while Table 7's MAE ground truth is the same optimizer's consensus value. The fivefold error reduction in the SLA experiments is then substantially by construction: any model forced to bid near the optimizer's own answer will have low MAE against that answer. If the utility model in Eqs. (4)-(5) or the jitter scheme is mis-specified, the claimed 'bound' is not a bound, and the trustworthiness argument reduces to 'the prompt contains the answer.' The paper's own Section 7 concedes the testbed is single-cell and single-RIC, and Section 9 promises code only upon acceptance, so the CI computation cannot currently be independently reproduced.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 'symbiotic agents,' an agentic architecture that pairs LLMs/SLMs with deterministic optimizers to make network control and SLA negotiation more trustworthy. Two agent types are designed: Type I agents let an LLM act as a meta-optimizer that tunes the proportional gain of a P-controller for real-time RAN slicing, and Type II agents inject a confidence interval produced by a gradient-descent optimizer into the prompts of negotiating LLM agents to bound SLA bids. The authors evaluate on a 5G testbed with OpenAirInterface and FlexRIC, using channel traces from 78 moving vehicles, and report up to a 5x reduction in decision error, 82 ms near-real-time loops for SLMs, 99.9% GPU memory savings, and a 44% reduction in RAN over-utilization in an end-to-end AGI-RAN demonstration. The paper also proposes a Next-G Open/AI-RAN architecture and releases a demo video and (upon acceptance) partial open-source code.","tokens_in":26604,"tokens_out":6424,"duration_ms":61412,"significance":"If the central claims hold, the symbiotic-agent paradigm is a concrete and practically useful step toward deploying LLM-based decision-making in near-real-time network control, because it externalizes numeric guarantees to deterministic optimizers while preserving the semantic flexibility of LLMs. The Type I experiment is a strength: the 20 Mbps operator intent is external to the LLM and P-controller, the 78-vehicle trace is concrete, and the RMSE comparison against a hand-tuned P-controller is internally consistent. The paper also provides a useful multi-model benchmark showing that small open-weight SLMs can match large proprietary models, which supports edge deployment. The proposed architecture and live demo are valuable for the community. However, the Type II trustworthiness argument is weakened by a statistically questionable confidence interval and by an evaluation that measures MAE against the same optimizer that produces the injected guard-rail, so the claimed error reduction for SLA negotiation is partly by construction.","major_comments":[{"comment":"The '95% confidence interval' is computed from R jittered restarts of a deterministic, strictly concave optimization problem (Eqs. (6)-(7)). Because the objective is strictly concave with a unique maximizer, perturbing only the initial point x(0) cannot produce a distribution that reflects statistical uncertainty about the true Pareto-optimal SLA; any spread s in Eq. (8) mostly reflects incomplete convergence or the arbitrary jitter scale. The interval x* ± 1.96 s/sqrt(R) is a confidence interval for the mean of the restart outcomes, not a bound on the true optimum. The paper's statement that this interval 'serves as a statistical estimate of the true optimal SLA' is therefore unjustified and should be revised or replaced with a proper uncertainty quantification (e.g., bootstrapping over data or utility parameters, or a prediction interval).","section":"Sec. 3.3, Eq. (8)"},{"comment":"The MAE ground truth is the same optimizer's consensus value that produces the confidence interval injected into every LLM prompt, and the guard-rail instruction in Sec. 3.3.1 forces bids into [L,U]. Consequently, the large MAE reduction for symbiotic agents relative to standalone LLMs is substantially by construction: a model that obeys the guard-rail will have low MAE against the optimizer's own answer. This does not demonstrate that the LLM improves numerical accuracy; it demonstrates compliance with the injected interval. Please provide a control condition in which the LLM receives the interval but is not constrained by it, or evaluate against an independent held-out objective, and then report the marginal contribution of the LLM beyond the guard-rail.","section":"Sec. 5.2.3, Table 7"},{"comment":"The solution of Eq. (6) is a weighted sum of individual quadratic utilities and a mediator utility with hand-set weights alpha_i, gamma, beta and an unspecified lambda. This is a particular scalarization, not generally the Pareto-optimal SLA of the multi-agent game described in the text. The paper repeatedly refers to 'Pareto-optimal' (e.g., Figures 5, 6, 8 and Sec. 3.3). Please either compute an actual Pareto front (e.g., by varying lambda) or soften the language to 'the optimizer's preferred SLA under the chosen weights.' Also specify the lambda value used in the experiments and the distribution of the jitter in Sec. 3.3.1, which is currently not described.","section":"Sec. 3.3.2 and Eq. (6)"},{"comment":"The claim of 'fivefold' error reduction relative to standalone LLMs is not consistently supported by the tables. In Table 4, gpt-4o standalone RMSE is 12.8 Mbps vs. symbiotic 4.5 Mbps (~2.8x), while mistral-7b shows ~4.7x; in Table 7, gpt-4o shows a much larger reduction. The 'up to 5 times' phrasing in the abstract is technically defensible only if the ratio is computed per model and the best case is reported. Please report the exact per-model ratios and qualify the headline claim accordingly.","section":"Abstract and Sec. 8 (Conclusion)"}],"minor_comments":[{"comment":"Section 3.2.1 states that the LLM freely chooses K_new_p in (0, inf), but Listing 3 in Appendix B instructs the LLM to choose Kp between 0.5 and 1.5 with granularity 0.1. These conflicting descriptions should be reconciled, and the impact of the hard bounds on the claimed adaptivity should be discussed.","section":"Sec. 3.2.1 and Appendix B"},{"comment":"The caption reads 'Zoom on P-Cotrol' and the text mentions 'base line'; these typos should be corrected.","section":"Fig. 4 caption"},{"comment":"The column headers 'RMSE↑' and 'MAE↑' use an up arrow, which conventionally indicates that higher is better, but for these error metrics lower is better. The arrows should be reversed or removed.","section":"Tables 4 and 7"},{"comment":"The VRAM values for gpt-4o (about 3500 GB) are cited in the same units as local models, but gpt-4o is accessed via API and its exact runtime VRAM is not directly comparable. Please clarify whether these figures are model parameter sizes in GB, estimated deployment footprints, or something else.","section":"Tables 4 and 7"},{"comment":"In Table 5, the last column header 'Score↓' is odd because higher scores are better; please rename to 'Score' and explain that higher is better in the text.","section":"Sec. 5.2.1"},{"comment":"The introduction says 'We are the first to formalize an agent architecture' while the related work states that Agoran [36] is 'the first work to formally utilize and scale the symbiotic paradigm.' These claims should be aligned to avoid an apparent contradiction.","section":"Sec. 1 vs. Sec. 2"},{"comment":"The open-source repository is promised only upon acceptance; during review, reproducibility would be improved by providing an anonymized or partial version of the simulation framework and agent code, even if the full testbed code cannot be released.","section":"Sec. 9 (Data availability)"}],"recommendation":"major_revision","confidential_remarks":"The Type I contribution is solid and the architecture proposal is timely, but the Type II evaluation conflates compliance with a guard-rail with improved numerical accuracy, and the 'confidence interval' is not a valid statistical bound on the Pareto-optimal SLA. These issues are load-bearing for the trustworthiness claim, but they are fixable with additional control experiments and careful rephrasing, so a major revision is appropriate rather than outright rejection. The authors' own Section 7 limitation statement and the conditional code release should be taken into account in the revision requirements."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The Type I symbiotic agent is the real meat here: an LLM/SLM acting as a slow outer loop that tunes the Kp of a P-controller on a live 5G testbed with 78-vehicle CQI traces. The measurements are concrete and internally consistent, and the result that a 3B Llama model matches a hand-tuned P-controller (about 4.4 vs 4.5 Mbps RMSE) while fitting in ~2 GB VRAM is a useful, creditable data point for the near-RT RIC story. This part deserves a serious referee.\n\nThe Type II SLA negotiation work has a load-bearing statistical flaw. The objective in Eq. (6) is deterministic and strictly concave with a unique maximizer; the R=100 jittered restarts perturb only the starting point, so the spread feeding Eq. (8) reflects jitter scale and convergence tolerance, not statistical uncertainty about the Pareto optimum. Section 3.3.1 then instructs every agent to bid inside [L,U], and Table 7's MAE is computed against the same optimizer's consensus value. So the fivefold error reduction is substantially by construction, and the 'trustworthy bound' reduces to 'the prompt contains the answer.' That is a real problem, not a nitpick.\n\nSome smaller soft spots: the headline fivefold claim does not match the tables cleanly (reductions range from roughly 2.8x to 15x depending on model and metric), the GPU comparison conflates local VRAM for SLMs with whatever cloud API footprint gpt-4o is billed as (3500 GB is not a meaningful VRAM number), and several hand-set parameters (alpha, gamma, beta schedule, eta, threshold) get no sensitivity analysis. Section 7 honestly says the testbed is single-cell/single-RIC, and Section 9 promises code only upon acceptance, so independent reproduction is currently impossible.\n\nThat said, the overarching architecture is not nonsense: pairing a deterministic inner controller with an LLM meta-optimizer is a sensible division of labor, and the Type I results stand on their own. The fix for Type II is tractable: either drop the 'confidence interval' language and call the band a regularization or sensitivity range, or actually propagate utility-model uncertainty into a real interval.\n\nThis paper is for people working on LLM-in-the-loop RAN control and intent-based orchestration. I would send it to peer review, with a clear request to rework the Type II claims and tighten the headline numbers before acceptance.","headline":"Type I is a genuine empirical contribution worth reading; Type II's confidence-interval story is substantially circular and needs rework before the trustworthiness claims can stand.","tokens_in":27143,"tokens_out":2972,"would_cite":true,"duration_ms":33085,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Symbiotic agents pair LLMs with optimizers to cut network-control decision errors fivefold.","keywords":["symbiotic agents","large language models","small language models","trustworthy AI","RAN slicing","SLA negotiation","confidence interval guard-rail","AGI-driven networks"],"falsifier":"Run the Type II negotiation with a ground-truth Pareto frontier obtained by exhaustive grid search over SLA values for many random tenant intents, and count how often the paper's 95% confidence interval, computed from jittered gradient-descent restarts, contains the true optimum; if the coverage is below 95%, the interval is not a genuine confidence bound and the guard-rail is not providing the claimed trustworthiness.","tokens_in":26059,"feed_emoji":"📡","tokens_out":7653,"duration_ms":73289,"temperature":0.7,"pith_summary":"This paper argues that language models alone cannot be trusted for numerically precise, real-time network control, and it proposes a remedy it calls symbiotic agents: an LLM or small language model paired with a deterministic optimizer that handles the arithmetic while the model handles reasoning. It claims this pairing cuts decision errors fivefold relative to standalone LLM agents in two network tasks, RAN slicing and multi-tenant SLA negotiation, on a real 5G testbed with channel fluctuations from moving vehicles. It also claims that small models of 3 to 8 billion parameters match large ones in accuracy while using 99.9% less GPU memory and running in 82 ms loops, making edge deployment practical. If correct, the work offers a path to AGI-driven networks where flexibility comes from language models and guarantees come from optimizers.","feed_headline":"LLM plus optimizer cuts network control errors fivefold","feed_subtitle":"A 5G testbed shows small models match large ones with 99.9% less GPU in 82 ms loops.","key_machinery":"The machinery is the optimizer wrapped around the language model. On the input side, Oin pre-processes the prompt with a bounded uncertainty estimate: for Type II, a gradient-descent side-car runs R = 100 jittered restarts on a strictly concave utility model and returns a 95% confidence interval x* ± 1.96 s / sqrt(R) for the Pareto-optimal SLA, which is injected into every agent's prompt as a numeric guard-rail. On the output side, Oout is a proportional controller, a linear feedback rule that adjusts the resource allocation in proportion to the gap between the intent and the current state, with the LLM asynchronously retuning the gain Kp when a convergence KPI exceeds a threshold. The two-optimizer loop is what converts stochastic text generation into numerically certified actions.","core_discovery":"The central discovery is that attaching a tiny optimizer to a language model converts the model's semantic flexibility into numerically trustworthy network decisions. In the Type I design, the LLM acts as a meta-optimizer that periodically retunes the proportional gain Kp of a P-controller allocating Physical Resource Blocks to enforce a throughput intent, so the sub-millisecond control loop stays deterministic while the LLM adapts to channel variability. In the Type II design, a gradient-descent side-car computes a 95% confidence interval for the Pareto-optimal SLA from jittered restarts and injects it into every agent's prompt as a numeric guard-rail, requiring bids to stay inside the interval unless justified. On the testbed this yields a fivefold error reduction over standalone LLM agents, negotiation error below 1.3 Mbps, an 82 ms near-real-time loop for a small model with 99.9% less GPU footprint than a large model, and a 44% reduction in RAN over-utilization in the collaborative demonstration.","pith_inferences":["One could test the guard-rail's statistical claim directly by computing the same interval with different utility weights or a different optimizer and checking whether the true Pareto-optimal SLA falls inside it at the claimed 95% rate; the paper's weights are chosen by hand and the same optimizer defines the evaluation ground truth, so the bound is only as good as the utility model.","The Type I pattern generalizes beyond RAN: any domain where an LLM tunes a small set of hyperparameters of a fast, stable inner controller, such as video bitrate adaptation or power management, could inherit the same error reduction and latency guarantees.","The 82 ms near-real-time loop depends on a deliberately short memory window; as prompts grow with richer context or larger memory, small-model latency will climb, so prompt compression or cache reuse would be the next practical step.","The trustworthiness claim covers numeric decision accuracy and bounded outputs; adversarial robustness of the guard-rail itself, for example a tenant prompting the LLM to ignore the interval, is not evaluated and would be a natural red-team test."],"forward_implications":["If the central claim holds, standalone LLM agents should not be used for real-time RAN resource allocation; the symbiotic pairing becomes the natural architecture for such loops.","Small models of 3 to 8 billion parameters become viable substitutes for large ones in near-real-time network control, opening the door to edge GPUs with roughly 2 GB footprints.","The confidence-interval guard-rail improves not only numeric accuracy but also the alignment and fairness scores of negotiation dialogues, across model families and sizes.","The architecture enables automatic SLA renegotiation during channel degradation, cutting RAN over-utilization by roughly 44% compared with static SLA enforcement.","Because next-token sampling stays stochastic, even improved future LLMs will still need external optimizers to deliver deterministic numeric bounds."],"supporting_citations":[{"why":"Establishes that LLMs can act as black-box optimizers, the baseline capability this paper extends by pairing them with external optimizers.","marker":"[19]"},{"why":"Prior work that scales the symbiotic paradigm with a multi-objective optimizer producing Pareto-front SLA offers, positioning the present negotiation design.","marker":"[36]"},{"why":"Supplies the PID control framework used as the Type I inner loop that the LLM tunes.","marker":"[38]"},{"why":"Supplies the convex-optimization result guaranteeing a unique maximizer, which underpins the confidence-interval computation.","marker":"[42]"},{"why":"Supplies the geometric-convergence result for gradient descent, justifying the side-car optimizer's speed.","marker":"[43]"},{"why":"Supplies the open-source 5G RAN, core, and user-equipment platform used for the testbed.","marker":"[47]"},{"why":"Supplies the channel-quality traces of moving vehicles that drive the mobility scenarios.","marker":"[57]"},{"why":"Supplies the LLM-based annotation method used to evaluate negotiation dialogue quality.","marker":"[61]"}],"fun_headline_variants":["Symbiotic agents: LLM plus optimizer cuts errors fivefold","AGI networks with tiny optimizer achieve 5x fewer control errors","Optimizer sidecar makes LLM agents numerically trustworthy","Small LLM + optimizer matches large, with 99.9% less GPU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim stands or falls on whether the hand-built utility model used to compute the negotiation confidence interval actually captures the true Pareto-optimal SLA; if the weights or the jitter scheme are wrong, the 95% confidence guard-rail bounds nothing real and would steer agents toward a confidently wrong target.","fun_headline_variants_meta":{"raw":{"variants":["Symbiotic agents: LLM plus optimizer cuts errors fivefold","AGI networks with tiny optimizer achieve 5x fewer control errors","Optimizer sidecar makes LLM agents numerically trustworthy","Small LLM + optimizer matches large, with 99.9% less GPU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000518,"raw_usage":{"total_tokens":2575,"prompt_tokens":1076,"completion_tokens":1499,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":692,"completion_tokens_details":{"reasoning_tokens":1424}},"tokens_in":692,"tokens_out":1499,"duration_ms":11284,"temperature":1.0,"reasoning_tokens":1424,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:18:51.555847+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Type II negotiation with a ground-truth Pareto frontier obtained by exhaustive grid search over SLA values for many random tenant intents, and count how often the paper's 95% confidence interval, computed from jittered gradient-descent restarts, contains the true optimum; if the coverage is below 95%, the interval is not a genuine confidence bound and the guard-rail is not providing the claimed trustworthiness.","supporting_citations":[{"cited_title":"Agoran: An Agentic Open Marketplace for 6G RAN Automation","cited_arxiv_id":"2508.09159","evidence_quote":"Prior work that scales the symbiotic paradigm with a multi-objective optimizer producing Pareto-front SLA offers, positioning the present negotiation design."},{"cited_title":"Visioli, Practical PID control, Springer Science & Business Media, 2006","cited_arxiv_id":null,"evidence_quote":"Supplies the PID control framework used as the Type I inner loop that the LLM tunes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the convex-optimization result guaranteeing a unique maximizer, which underpins the confidence-interval computation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the geometric-convergence result for gradient descent, justifying the side-car optimizer's speed."},{"cited_title":"Nikaein, M","cited_arxiv_id":null,"evidence_quote":"Supplies the open-source 5G RAN, core, and user-equipment platform used for the testbed."},{"cited_title":"Tsourdinis, I","cited_arxiv_id":null,"evidence_quote":"Supplies the channel-quality traces of moving vehicles that drive the mobility scenarios."}],"review_version":1}