{"id":"22de9f6c-e8bf-4ade-9e50-e8550e49f6c4","arxiv_id":"2505.01834","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A Model Context Protocol pool of lightweight wireless experts raises LLM environment-classification accuracy on synthetic channel data from 45-59% to above 95%.","lead":"This paper proposes an architecture that lets large language models (LLMs) sense wireless environments by calling small, specialized classifiers through the Model Context Protocol. In tests on synthetic channel data, the setup lifts wireless attribute classification accuracy from roughly 50% to over 95%.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The decisive gap is external validity: both expert training and the 1000-sample test set come from a single unspecified synthetic channel simulator, so the reported >95% accuracy has not been shown to survive over-the-air channels.","rationale":"The reader's weakest-assumption analysis matches mine: the entire evaluation, from expert training to the 1000-sample test set, is synthetic and the simulator is undisclosed. I checked whether a more central internal flaw exists, such as the absence of an expert-only baseline making the LLM's contribution untested. That is a real secondary concern, since Section IV-B admits the MCP result 'approaches the classification accuracy ceiling determined by the experts themselves,' but it does not invalidate the architecture-level comparison against LLM-only agents. The most load-bearing issue remains external validity: if the synthetic channel generator does not reproduce measured multipath, Doppler, and noise characteristics, the >95% accuracy is an artifact of the simulation distribution. The paper itself flags over-the-air evaluation as future work, which is the strongest possible confirmation that this condition is untested. I would keep the CONDITIONAL verdict rather than escalate, because the simulation result is internally coherent and the architecture is clearly specified; the condition is that OTA validation and a released or fully described simulator must be supplied before the general 'wireless environment-aware LLM agent' claim can be accepted.","tokens_in":8387,"tokens_out":4728,"duration_ms":50331,"concrete_test":"Freeze the four trained expert classifiers and the MCP prompt pipeline, then run them without retraining on a public over-the-air channel dataset with ground-truth LoS/NLoS, Doppler, and fading labels (e.g., a Wi-Fi CSI dataset or USRP/SDR indoor measurement campaign). Report per-attribute accuracy for the standalone LLM, the MCP-augmented LLM, and a trivial rule that thresholds expert outputs directly. If MCP-augmented accuracy drops materially below the synthetic >95% (e.g., more than 10 points) or is indistinguishable from the trivial expert-only rule, the paper must rescope its claim to synthetic channels and clarify the LLM's incremental contribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim rests on Table I, but every number in Table I is generated inside one closed simulation loop. Section IV-A trains the experts on 'synthetic wireless traces,' and Section IV-B builds the 1000-sample test set with 'a randomized channel simulation pipeline' using 'a mixture of Rayleigh and Rician components, incorporating time-varying Doppler effects.' No simulator parameters, noise model, antenna or hardware impairments, or dataset release are given, so the reader cannot tell whether the test distribution is the same as the training distribution or whether it resembles any real channel. If the simulator's simplified channel model is easy to classify (e.g., well-separated K-factors and clean Doppler offsets), the near-ceiling expert accuracy in Fig. 6 will transfer neither to measured channels nor to the diverse environments the paper claims to address. The Conclusion itself lists 'evaluate the framework on over-the-air channel measurements' as future work, confirming that the decisive real-world condition has not been tested. This does not make the simulation result internally inconsistent; it makes the headline claim's scope unverified beyond synthetic channels.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an MCP-based Internet of Experts (IoX) framework that equips LLM agents with wireless environment awareness. Lightweight MLP experts are trained separately for each wireless attribute (LoS, high Doppler, Rayleigh fading, Rician fading with K=10) and exposed to the LLM as MCP tools. At inference, the LLM selects and queries the relevant experts and receives structured confidence scores, which are injected into its context for final decision-making. Experiments compare standalone LLMs with MCP-augmented agents on 1000 synthetic channel observations, reporting 45%-59% accuracy without MCP versus over 95% with MCP across eight LLMs. The conclusion frames this as a significant gain from structured wireless perception, with over-the-air validation deferred to future work.","tokens_in":8566,"tokens_out":3437,"duration_ms":36813,"significance":"If the reported results are representative, the architecture is a clean and timely contribution: it decouples physical-layer perception from LLM reasoning, avoids fine-tuning, and is modular. The evaluation across eight LLMs is a strength, as is the explicit comparison against a no-tool baseline. The paper is internally consistent, and there is no equation-level circularity: expert confidences, not ground-truth labels, are fed to the LLM, and final labels are compared to an external ground truth. However, the central quantitative claim rests entirely on one undisclosed synthetic channel simulator, so the paper's broader claim of 'wireless environment-aware' agents remains unvalidated for real over-the-air channels. The proof-of-concept is valuable, but the external-validity gap is the decisive limitation.","major_comments":[{"comment":"The end-to-end claim rests on a 1000-sample test set generated by a 'randomized channel simulation pipeline' whose parameters, noise model, antenna configuration, and train/test separation are not disclosed. Since the expert classifiers are trained on 'synthetic wireless traces' from the same family of processes, the reported >95% accuracy may reflect an easy, in-distribution test rather than general wireless perception. Please provide the full simulator configuration, specify how the test distribution differs from the training distribution, report error bars or per-run variance, and add an evaluation on measured channels or a standard benchmark (e.g., 3GPP channel models) before claiming general wireless-environment-aware performance.","section":"Section IV-B, Table I"},{"comment":"The expert learning results are presented only as training loss and test accuracy curves, without test-set sizes, final accuracy values, hyperparameters, confidence intervals, or train/test split details. The paper later states that end-to-end accuracy 'approaches the classification accuracy ceiling determined by the experts themselves,' so this expert ceiling must be quantified. Without these numbers, the reader cannot judge how robust the 95% end-to-end ceiling is or whether the binary expert classifiers are well calibrated for the confidence scores the LLM receives.","section":"Section IV-A, Fig. 6"},{"comment":"The comparison is LLM-with-MCP versus LLM-without-MCP, but there is no no-LLM rule-based baseline that thresholds the expert confidence scores (e.g., predict the attribute as present when the expert outputs p > 0.5). Such a baseline is necessary to isolate the marginal contribution of the LLM's planning and reasoning from the contribution of the expert classifiers themselves. If a simple thresholding baseline also achieves >95%, the current experiment does not support the claim that the LLM agent 'interprets' or 'reasons' over expert outputs.","section":"Section IV-B, end-to-end evaluation"},{"comment":"Tool-selection and invocation errors are mentioned qualitatively ('minor errors from MCP call formatting inconsistencies or borderline expert predictions') but are never measured. The framework's value depends on the LLM choosing the right experts and correctly reading their outputs. Please report per-LLM tool-call success rates, the number of expert invocations per query, and accuracy conditioned on successful MCP calls. This is needed to distinguish failures of the MCP orchestrator from failures of the underlying experts.","section":"Section III-B and Section IV-B"}],"minor_comments":[{"comment":"The caption says 'three representative expert classifiers,' but the figure contains four subplots; please correct the count.","section":"Fig. 6 caption"},{"comment":"The abstract's '40%-50% improvements' mixes percentage points and relative improvements; for example, 46.7% to 95.5% is a 48.8 percentage-point increase but a roughly 104% relative increase. Please state the metric unambiguously.","section":"Abstract and Table I"},{"comment":"The experts are trained for 4000 epochs with no mention of early stopping or regularization; please specify whether overfitting was monitored and report the final hyperparameters.","section":"Section IV-A"},{"comment":"The 1000-sample test set is generated with no seeds or reproducibility details; please provide the code or data release to allow independent replication.","section":"Section IV-B"},{"comment":"The term 'Internet of Experts' is used without a definition or reference; please clarify whether this is a new term or an existing concept.","section":"Section I"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible proof of concept, and the architecture is reasonable, but the evaluation is too closed to justify the headline claim of general wireless environment awareness. I recommend major revision rather than rejection because the missing validation—simulator disclosure, a thresholding baseline, expert accuracy quantification, and ideally an over-the-air or standard-channel-model experiment—is obtainable and would directly strengthen the central claim. I see no novelty-disclosure or citation-pattern concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a straightforward, well-scoped system study. The contribution is not a new learning method or theory; it's a practical orchestration pattern: MCP-mediated access to lightweight wireless attribute classifiers lifts LLM accuracy from roughly 45–59% to over 95% on a synthetic test set. That result is believable and consistent.\n\nWhat's actually new is the combination of MCP with a pool of independent expert classifiers for wireless physical-layer attributes. The architecture is clearly described, and the comparison across eight LLMs is a nice touch. The experts are simple MLPs, which is fine—the point is that the LLM can call them at inference time without retraining. The paper's internal logic holds up: if the experts are accurate, the LLM wrapper should be accurate, and the experiments confirm it.\n\nThe soft spots are real but not fatal. The evaluation is entirely synthetic; the channel simulator is undisclosed, there are no error bars, no code release, and no baseline that uses the expert outputs without the LLM. So we cannot tell whether the test distribution resembles the training distribution, or whether the >95% number survives real over-the-air channels. The authors themselves list over-the-air evaluation as future work, which is honest but confirms the external validity gap. Also, the input representation is just the magnitude of the channel impulse response—real baseband features (phase, complex taps) may be harder to classify. These are limitations, but they don't invalidate the claimed result as scoped to synthetic channels.\n\nThe citation pattern looks fine; no red flags. The paper is internally consistent and the authors are upfront about what is not tested.\n\nBottom line: a serious referee should spend time on this. It's a worthwhile proof-of-concept for tool-augmented LLMs in wireless, and the weaknesses are addressable. I'd ask for over-the-air measurements or a detailed simulator description, error bars, and a non-LLM baseline before publication. But the idea deserves to go through review rather than be desk rejected.","headline":"A clean, modest system paper whose synthetic-only evaluation is the main gap; worth sending to peer review with demands for over-the-air data and an ablation.","tokens_in":9132,"tokens_out":2272,"would_cite":true,"duration_ms":24182,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that giving LLMs a pool of MCP-accessible expert classifiers lifts wireless attribute classification accuracy from 45%–59% to over 95% without retraining the language model.","keywords":["Model Context Protocol","Internet of Experts","wireless environment awareness","LLM agents","channel impulse response classification","tool-augmented reasoning","line-of-sight detection","synthetic channel simulation"],"falsifier":"Run the same MCP-IoX pipeline on over-the-air channel impulse response measurements, with ground-truth scene labels from a channel sounder or ray-tracing, and compare the expert and end-to-end accuracy to the synthetic >95% result; a large drop would show the synthetic experts do not transfer.","tokens_in":8136,"feed_emoji":"📡","tokens_out":6157,"duration_ms":57664,"temperature":0.7,"pith_summary":"The paper proposes that the right way to give large language models awareness of wireless radio environments is not to retrain them or prompt them with raw numbers, but to let them call a pool of small, task-specific classifiers through the Model Context Protocol at inference time. Each classifier is a lightweight network trained on channel impulse responses to detect one physical attribute, such as line-of-sight propagation, high Doppler shift, or Rayleigh/Rician fading. At query time the LLM decides which experts are relevant, the MCP runtime fetches their confidence scores, and the model answers with those scores in its context. On a 1000-sample synthetic test set, the authors report that this arrangement raises average classification accuracy from 45%–59% for standalone LLMs to over 95% for all eight evaluated models, including weaker ones. The claim matters because it points to a modular way to give general-purpose LLMs reliable, physical perception without per-deployment fine-tuning.","feed_headline":"Expert wireless tools lift LLM classification accuracy from 45% to 95%","feed_subtitle":"A Model Context Protocol lets frozen LLMs query specialized classifiers at inference, no retraining needed.","key_machinery":"The load-bearing object is the MCP-based Internet of Experts: MCP provides a uniform JSON-RPC interface that lets an LLM discover, call, and receive structured replies from external tools, and the IoX is the pool of lightweight per-attribute MLP classifiers, each trained with binary cross-entropy on scene-specific positive and negative channel samples. The pipeline has six stages: offline expert registration, LLM-driven expert planning, JSON-RPC invocation, standardized responses with confidence scores, context augmentation, and final reasoning. This machinery decouples physical-layer interpretation from high-level reasoning, so adding a new environment condition means training and registering one new classifier without retouching the LLM or the other experts.","core_discovery":"The central claim is that structured wireless perception, not better language reasoning, is what LLM wireless agents lack. The authors build an Internet of Experts in which every wireless attribute is a separate binary classification problem solved by a compact MLP trained on the magnitude of complex channel impulse responses, and they connect the expert pool to the LLM through MCP's JSON-RPC tool layer. The LLM's job is reduced to planning which experts to call and interpreting their outputs; it never sees the raw channel vector as a reasoning target and never changes its weights. The result is that the agent's accuracy is bounded by the experts' own performance rather than by the LLM's ability to read numbers: the reported end-to-end accuracy exceeds 95% for all eight models, and weaker models match or beat stronger standalone models when the experts are available.","pith_inferences":["Real-world transfer is the open risk: the experts are trained on synthetic traces, and the paper lists over-the-air evaluation as future work; a drop in accuracy there would not contradict the synthetic result but would bound its practical value.","Beyond the tested wireless attributes, the same MCP-plus-expert-pool recipe likely transfers to other domains where LLMs must reason over raw high-dimensional signals, such as power-system telemetry, speech features, or sensor arrays.","The reported standalone baseline may understate what LLMs could do with better prompting or multimodal inputs; the headline gain therefore measures raw-number reasoning difficulty as much as MCP's contribution.","A testable next step is cost-aware expert selection: the paper notes latency and cost profiling as future work, and optimal selection would trade the number of expert calls against accuracy and budget."],"forward_implications":["Standalone LLM accuracy on wireless attribute prediction is bounded at 45%–59%; adding MCP expert querying raises every tested model above 95%, so the bottleneck is perception, not reasoning.","A frozen, general-purpose LLM can acquire wireless-environment awareness without parameter updates, avoiding costly fine-tuning and preserving language generalization.","The expert pool is modular and extensible: a new attribute only requires new positive samples and a newly trained lightweight classifier, with no retraining of the rest of the system.","Weaker LLMs paired with expert outputs can match or surpass stronger models operating alone, so the architecture shifts the performance ceiling to the expert classifiers.","Because experts output interpretable confidence scores rather than free text, the agent's wireless reasoning is grounded in verifiable physical states, which should reduce hallucinated channel descriptions."],"supporting_citations":[{"why":"Supplies the Model Context Protocol definition and its role in exposing external tools to language models.","marker":"[8]"},{"why":"The protocol specification from the MCP's creators that the runtime pipeline implements.","marker":"[9]"},{"why":"Shows the tool-invocation API design that the MCP client's JSON-RPC calls are modeled on.","marker":"[14]"},{"why":"Provides the binary cross-entropy loss used to train each expert classifier.","marker":"[13]"},{"why":"Defines the fading and interference scenarios that become the labeled wireless attributes the experts detect.","marker":"[12]"},{"why":"Supports the idea of embedding expert knowledge into model-aided wireless AI, which the IoX modularizes.","marker":"[11]"}],"fun_headline_variants":["MCP gives LLMs wireless senses, boosting accuracy to 95%","No retraining: LLMs query expert models for 95% wireless accuracy","Wireless-aware LLMs hit 95% accuracy via expert tool network","LLMs gain wireless perception with MCP, accuracy leaps to 95%","Frozen LLMs, expert tools: wireless classification at 95% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the synthetic channel simulator used to train the experts and label the 1000-sample test set captures real wireless environments closely enough; if real over-the-air channels differ in Doppler, multipath, or hardware effects, the reported >95% accuracy may not carry over.","fun_headline_variants_meta":{"raw":{"variants":["MCP gives LLMs wireless senses, boosting accuracy to 95%","No retraining: LLMs query expert models for 95% wireless accuracy","Wireless-aware LLMs hit 95% accuracy via expert tool network","LLMs gain wireless perception with MCP, accuracy leaps to 95%","Frozen LLMs, expert tools: wireless classification at 95% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000734,"raw_usage":{"total_tokens":3276,"prompt_tokens":930,"completion_tokens":2346,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":2247}},"tokens_in":546,"tokens_out":2346,"duration_ms":15683,"temperature":1.0,"reasoning_tokens":2247,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:09:15.253781+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same MCP-IoX pipeline on over-the-air channel impulse response measurements, with ground-truth scene labels from a channel sounder or ray-tracing, and compare the expert and end-to-end accuracy to the synthetic >95% result; a large drop would show the synthetic experts do not transfer.","supporting_citations":[{"cited_title":"Model context protocol,","cited_arxiv_id":null,"evidence_quote":"The protocol specification from the MCP's creators that the runtime pipeline implements."},{"cited_title":"ToolLLM: Facilitating large language models to master 16000+ real-world APIs,","cited_arxiv_id":null,"evidence_quote":"Shows the tool-invocation API design that the MCP client's JSON-RPC calls are modeled on."},{"cited_title":"Binary cross entropy with deep learning technique for image classification,","cited_arxiv_id":null,"evidence_quote":"Provides the binary cross-entropy loss used to train each expert classifier."},{"cited_title":"Panic, M","cited_arxiv_id":null,"evidence_quote":"Defines the fading and interference scenarios that become the labeled wireless attributes the experts detect."},{"cited_title":"Model-aided wireless artificial intelligence: Embedding expert knowl- edge in deep neural networks for wireless system optimization,","cited_arxiv_id":null,"evidence_quote":"Supports the idea of embedding expert knowledge into model-aided wireless AI, which the IoX modularizes."}],"review_version":1}