{"id":"d9518882-c0fe-468c-b816-5c738da20e3a","arxiv_id":"2606.20574","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"LLM fine-tuned on synthetic data corrects gNB misconfigurations with 92.7% accuracy on 480 unseen OAI testbed scenarios, up from 13.8% zero-shot.","lead":"The paper introduces an LLM-assisted framework that generates synthetic training data via a configuration-log-correction pipeline to fine-tune models for automatically correcting gNB parameter misconfigurations from error logs. A smart generalist might read it to see how AI techniques could automate complex configuration tasks in large-scale telecom infrastructure.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Synthetic training data may not match real RAN misconfiguration distributions, risking overstated generalization to testbed","rationale":"The reader's weakest_assumption directly names the load-bearing premise. No other internal inconsistency (e.g., in the reported accuracy numbers or RAG usage) appears more central once the synthetic-to-real transfer risk is acknowledged. The UNVERDICTED status is therefore appropriate pending the proposed distributional check.","tokens_in":1773,"tokens_out":347,"duration_ms":15041,"concrete_test":"Sample 50 synthetic training examples and 50 of the 480 testbed scenarios; have a domain expert (or automated parser) label the misconfigured parameters and error-log signatures; compute overlap in the multisets of (parameter, value-range, log-pattern) tuples. If overlap < 60% or if key tail events are missing from synthetic data, the generalization claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result (85.4% / 92.7% accuracy on 480 testbed scenarios) depends on the assumption that LLM-generated synthetic data (produced via the configuration-log-correction pipeline from technical references) has the same joint distribution over error logs, parameter interactions, and corrective actions as real OAI gNB deployments. The paper does not report any statistical comparison (e.g., KL divergence on log feature histograms, coverage of parameter ranges, or expert review of reasoning traces) between synthetic and real misconfigurations. If the synthetic set under-represents rare interactions or produces overly clean reasoning traces, the fine-tuning gains are explained by distribution match rather than robust inference, and performance on operational RANs would drop.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes an LLM-assisted framework for automatic gNB parameter configuration correction in radio access networks. It introduces a synthetic data generation pipeline that uses a commercial LLM to produce training examples via a configuration-log-correction workflow derived from technical references, fine-tunes an LLM on this data to map error logs to corrective actions, and evaluates the approach on an OpenAirInterface gNB testbed using 480 unseen misconfiguration scenarios. The central empirical claims are accuracy gains from 13.8% (zero-shot) to 85.4% (fine-tuned) and 92.7% (with RAG).","tokens_in":1934,"tokens_out":605,"duration_ms":20386,"significance":"If the results hold under scrutiny of the synthetic data assumption, the work could contribute to scalable autonomous RAN operations by reducing manual diagnosis of misconfigurations. The physical testbed validation provides a concrete empirical anchor that is stronger than purely simulated evaluations.","major_comments":[{"comment":"Synthetic data generation pipeline: The headline accuracy claims on the 480 testbed scenarios rest on the unverified premise that LLM-generated synthetic misconfigurations and reasoning traces have the same joint distribution over error logs, parameter interactions, and corrective actions as real OAI gNB deployments. No statistical comparison (KL divergence on log feature histograms, parameter range coverage, or expert review of traces) is reported; if the synthetic set under-represents rare interactions, the reported generalization is explained by distribution match rather than robust inference.","section":"synthetic data generation pipeline"},{"comment":"Evaluation section: The 85.4% and 92.7% accuracy figures on the 480 scenarios are presented without error bars, confidence intervals, or statistical significance tests relative to the 13.8% baseline. This omission makes it impossible to determine whether the observed gains are reliable or sensitive to scenario selection.","section":"evaluation on OAI testbed"},{"comment":"Scenario selection: The manuscript does not describe the procedure used to generate or select the 480 unseen misconfiguration scenarios for the testbed experiments (e.g., whether they were drawn from a documented distribution of real faults or constructed to match synthetic patterns). This detail is load-bearing for the claim that the fine-tuned model generalizes beyond the training distribution.","section":"evaluation on OAI testbed"}],"minor_comments":[{"comment":"The abstract and results text should explicitly state the base LLM used for both synthetic data generation and fine-tuning, as well as any hyperparameter choices for the fine-tuning stage.","section":null},{"comment":"Figure captions for any performance plots should include the exact number of scenarios per condition and whether the reported accuracies are macro- or micro-averaged.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address each major comment below and indicate planned revisions to improve the manuscript.","responses":[{"response":"We agree that a direct statistical comparison to real distributions would strengthen the claims. The synthetic pipeline is constructed from gNB technical references via the configuration-log-correction workflow to capture realistic parameter interactions. Comprehensive real-world misconfiguration logs are not publicly available due to operator privacy constraints, precluding KL divergence or similar metrics against operational data. In revision we will add quantitative coverage analysis of parameter ranges in the synthetic set, include expert review of a sampled subset of reasoning traces, and expand the limitations section to discuss the synthetic data assumption explicitly.","revision_made":"partial","referee_comment":"Synthetic data generation pipeline: The headline accuracy claims on the 480 testbed scenarios rest on the unverified premise that LLM-generated synthetic misconfigurations and reasoning traces have the same joint distribution over error logs, parameter interactions, and corrective actions as real OAI gNB deployments. No statistical comparison (KL divergence on log feature histograms, parameter range coverage, or expert review of traces) is reported; if the synthetic set under-represents rare interactions, the reported generalization is explained by distribution match rather than robust inference."},{"response":"We accept this criticism. The revised manuscript will report 95% bootstrap confidence intervals for all accuracy figures and include McNemar's test (or equivalent) to establish statistical significance of the gains over the zero-shot baseline.","revision_made":"yes","referee_comment":"Evaluation section: The 85.4% and 92.7% accuracy figures on the 480 scenarios are presented without error bars, confidence intervals, or statistical significance tests relative to the 13.8% baseline. This omission makes it impossible to determine whether the observed gains are reliable or sensitive to scenario selection."},{"response":"The 480 scenarios were generated by applying a documented set of misconfiguration patterns (distinct from those used in training) drawn from the same technical references. We will add a dedicated subsection detailing the pattern list, the systematic variation method, and the verification that each scenario is unseen during fine-tuning.","revision_made":"yes","referee_comment":"Scenario selection: The manuscript does not describe the procedure used to generate or select the 480 unseen misconfiguration scenarios for the testbed experiments (e.g., whether they were drawn from a documented distribution of real faults or constructed to match synthetic patterns). This detail is load-bearing for the claim that the fine-tuned model generalizes beyond the training distribution."}],"tokens_in":1494,"tokens_out":561,"duration_ms":34312,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"This paper reports strong accuracy gains on a real OAI testbed but leaves the synthetic data realism unverified.\n\nThey generate synthetic training pairs by feeding technical references into a commercial LLM to create modified configs and structured reasoning from the resulting logs, then fine-tune an LLM on those pairs. On 480 unseen misconfiguration scenarios run on actual OpenAirInterface hardware, zero-shot starts at 13.8 percent, fine-tuning reaches 85.4 percent, and adding RAG pushes it to 92.7 percent.\n\nThe testbed evaluation is the part that stands out. Most papers in this area stay in simulation, so seeing corrected parameters actually deployed and measured on physical gNB equipment gives the numbers more weight.\n\nThe soft spot is the one the stress-test note flags. The method description shows no statistical check or expert review comparing the synthetic error distributions and correction traces to real RAN logs. Without that, it is unclear whether the model learned robust rules or patterns specific to the generation process. That assumption carries a lot of the generalization claim.\n\nThis is for researchers and engineers already working on AI tools for network automation. Someone looking for a concrete pipeline and testbed numbers to compare against would find it useful.\n\nIt deserves a serious referee. The empirical results on hardware are concrete enough to warrant review, even if the data-fidelity question needs more attention.","headline":"This paper reports strong accuracy gains on a real OAI testbed but leaves the synthetic data realism unverified.","tokens_in":2445,"tokens_out":347,"would_cite":false,"duration_ms":27095,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Fine-tuning an LLM on synthetic data corrects gNB misconfigurations at 92.7 percent accuracy on testbed.","keywords":["LLM","gNB","RAN","parameter configuration","misconfiguration correction","synthetic data","fine-tuning","retrieval-augmented generation"],"falsifier":"Running the fine-tuned model on misconfigurations collected directly from a live commercial RAN deployment and measuring whether correction accuracy remains above 80 percent would test whether the synthetic distribution matches reality.","tokens_in":2689,"feed_emoji":"📡","tokens_out":654,"duration_ms":21658,"temperature":0.7,"pith_summary":"This paper develops an LLM-assisted framework to automatically correct gNB parameter misconfigurations in radio access networks. It starts with a workable configuration and gNB technical references, then uses a commercial LLM to generate modified configurations and structured reasoning traces from error logs, creating synthetic training data that maps network states to corrective actions. The resulting fine-tuned LLM generates valid configurations from logs at inference time. Validation on an OpenAirInterface gNB testbed with 480 unseen misconfiguration scenarios shows accuracy rising from 13.8 percent in zero-shot to 85.4 percent after fine-tuning and 92.7 percent with retrieval-augmented generation added. A sympathetic reader cares because manual analysis of complex logs does not scale with network size, so automation could reduce downtime and operator effort in RANs.","feed_headline":"LLM corrects 92.7% of gNB misconfigurations on testbed","feed_subtitle":"Synthetic data pipeline and fine-tuning raise accuracy from 13.8% zero-shot baseline across 480 unseen scenarios.","key_machinery":"The synthetic data generation pipeline that uses a commercial LLM to derive structured reasoning traces from gNB error logs and map network states to corrective actions.","core_discovery":"The paper establishes that a synthetic data generation pipeline following the configuration-log-correction workflow produces training examples sufficient to fine-tune an LLM that maps gNB error logs to deployable parameter configurations, reaching 85.4 percent correction accuracy on 480 unseen scenarios in an OAI testbed, with RAG raising the figure to 92.7 percent.","pith_inferences":["The same synthetic-data approach could be applied to misconfiguration diagnosis for other RAN components such as the core network.","Collecting a small number of real misconfiguration logs over time could be used to iteratively refine the training distribution.","Integration with existing network monitoring systems would allow the model to trigger corrections in near real time."],"forward_implications":["The framework enables automated recovery from gNB misconfigurations without manual intervention.","It supports scalable and autonomous RAN operation.","Fine-tuning substantially outperforms zero-shot prompting on this task.","Adding retrieval-augmented generation provides further accuracy gains beyond fine-tuning alone."],"fun_headline_variants":["Fine-tuned LLM corrects gNB at 85.4% on OAI testbed","RAG LLM reaches 92.7% gNB correction accuracy","Synthetic data fine-tunes LLM to 85.4% gNB correction","92.7% gNB misconfiguration correction by RAG-enhanced LLM"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The synthetic data generated by the commercial LLM faithfully represents the distribution of real misconfigurations and their corrective actions that would be encountered in operational RANs.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuned LLM corrects gNB at 85.4% on OAI testbed","RAG LLM reaches 92.7% gNB correction accuracy","Synthetic data fine-tunes LLM to 85.4% gNB correction","92.7% gNB misconfiguration correction by RAG-enhanced LLM"]},"model":"grok-4.3","cost_usd":0.01394,"raw_usage":{"total_tokens":6017,"prompt_tokens":667,"num_sources_used":0,"completion_tokens":81,"cost_in_usd_ticks":139399500,"prompt_tokens_details":{"text_tokens":667,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":5269,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":667,"tokens_out":81,"duration_ms":40788,"temperature":1.0,"reasoning_tokens":5269,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T07:51:19.489761+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the fine-tuned model on misconfigurations collected directly from a live commercial RAN deployment and measuring whether correction accuracy remains above 80 percent would test whether the synthetic distribution matches reality.","supporting_citations":[],"review_version":1}