{"id":"abfbca21-a97d-46a1-ab09-4d54a493500d","arxiv_id":"2507.21385","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A DQN agent on a contextual-bandit model, trained with a smooth QoS-threshold reward, selects Rel-18 cell DTX/DRX cycle and on-duration settings, achieving up to 45% simulated energy savings with under 1% average data-rate loss.","lead":"Intel researchers trained a deep reinforcement learning agent to pick 3GPP Release 18 cell DTX/DRX sleep-cycle settings for 5G base stations, reporting up to 45% energy savings with roughly 1% throughput loss in simulations. The work matters because cell DTX/DRX is a new 3GPP feature and operators need automated ways to tune it without sacrificing latency-sensitive traffic.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The contextual-bandit independence assumption in Section III.A is untested, and the paper's own evaluation design induces correlated states across consecutive inference steps, so the claim that the agent always selects the best configuration is not yet supported.","rationale":"The reader's weakest-assumption analysis correctly identifies the contextual-bandit independence assumption as the load-bearing point. I agree with that identification and add a concrete internal inconsistency: training uses one-step independent episodes, while inference uses 10-step episodes in the same deployment with persistent per-UE traffic parameters, so the i.i.d. assumption is not merely unverified but is violated by the paper's own evaluation protocol. This matters because the DTX/DRX action has delayed effects on queueing and the next step's state, which a one-step CB cannot represent. The paper gives no autocorrelation measurements, no comparison against an MDP-aware learner, and no optimality baseline, so the 'always best' claim overreaches. Secondary concerns include tuning reward constants on the same evaluation episodes and the absence of error bars, which further weaken the quantitative headline. These issues justify keeping the reader's CONDITIONAL verdict rather than accepting the paper as is; they do not warrant rejection, because the core idea—learning a smooth QoS-aware reward for cell DTX/DRX configuration—is plausible and the simulator study is a reasonable first step. The concrete autocorrelation and MDP-comparison test would settle whether the central concern actually lands.","tokens_in":11140,"tokens_out":5596,"duration_ms":72663,"concrete_test":"Run the existing SLS under a fixed DTX/DRX configuration for many deployments, recording the Section III state features per 1 s step over 10 consecutive steps per cell. Compute lag-1 autocorrelation of PRB utilization and traffic intensity and compare against the i.i.d. null using a permutation or Ljung-Box test. If |rho| > 0.1 for either feature, the Section III.A independence assumption fails; then retrain a DQN that stores (s, a, r, s') transitions and uses a target network, and compare its held-out energy/QoS trade-off against the CB agent. If the MDP-aware agent outperforms the CB agent, the CB-based results are conditional on an invalid assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the Section III.A assertion that consecutive ~1 s RAN observation states are 'approximately independent', justifying a contextual bandit and a one-step DQN. This assumption is not supported by any measurement, and the paper's own evaluation design works against it. Training episodes are independent 1-step SLS runs (Section IV), so training states are i.i.d. by construction. Inferencing episodes, however, consist of 10 consecutive 1 s steps in the same deployment, with per-UE packet size, inter-arrival time, and delay requirement drawn once at episode start (Table IV). The state features—traffic intensity, inter-arrival statistics, PRB utilization—therefore have persistent, cell-specific components across the 10 steps. Moreover, the DTX/DRX action at step t determines which packets are delayed into step t+1, so the action has a delayed effect on the next state and reward. A contextual bandit maximizing one-step reward ignores this temporal coupling, so the learned Q-values can be biased relative to the true multi-step control problem. The 'always selects the best possible configuration' claim is therefore not established. The quantitative headline is also fragile: the reported 45% / 1% numbers come from the same 10 inferencing episodes used to select the reward constants, with no confidence intervals, and Footnote 3 merely asserts that 210 cells are 'statistically representative enough'.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses configuration optimization for 3GPP Release 18 cell DTX/DRX, a time-domain network energy saving feature. The authors formulate the problem as a per-cell choice of DTX/DRX cycle length and on-duration, and propose a DQN trained on a contextual-bandit (CB) model. A smooth approximation of a QoS-threshold reward is designed to balance delivered data ratio and power consumption. The agent is intended to run as an O-RAN xApp. Using a Python system-level simulator with 7 sites and 21 cells, 500 training episodes, and 10 deployment episodes (210 cells), the paper reports up to about 45% energy saving in light load and at most about 1% data-rate loss compared with the no-DTX/DRX baseline. The central claims are that the agent 'always tries to select the best possible' configuration under any network and traffic conditions, and that the approximated QoS reward is more stable and more QoS-resilient than a linear reward.","tokens_in":11490,"tokens_out":5005,"duration_ms":63701,"significance":"If the claims were fully supported, the contribution would be practically valuable: an automated xApp that configures Rel-18 cell DTX/DRX based on RAN observations could reduce RAN energy consumption while protecting delay-sensitive traffic. The paper contains useful ingredients: a plausible formulation of the configuration problem, a smooth reward function that mitigates discontinuity-induced training instability, a realistic system-level simulator, and a comparison across light, medium, and heavy load categories. The authors also honestly acknowledge some limitations, such as ignoring SSB/paging overhead in Footnote 1 and noting the cost of longer simulation in Footnote 3. However, the quantitative headline results rest on an untested independence assumption, reward constants tuned on the same evaluation data, and a QoS measure that is reported through data rate rather than the delivered-data-ratio objective. These issues are fixable but are load-bearing for the paper's strongest claims, so the current evidence falls short of what the abstract and conclusions assert.","major_comments":[{"comment":"The contextual-bandit assumption is asserted without evidence and is internally tensioned by the evaluation protocol. The text claims that consecutive ~1 s RAN observation states are 'approximately independent', which justifies treating the problem as a CB. Training episodes are indeed independent one-step SLS runs, so training states are i.i.d. by construction. However, inferencing episodes consist of 10 consecutive 1 s steps in the same deployment, with per-UE packet size, inter-arrival time, and delay requirement drawn once at episode start (Table IV). The state features therefore contain persistent, cell-specific components across the 10 steps, and the DTX/DRX action at step t determines which packets are delayed into step t+1, creating temporal coupling between action, next state, and reward. A one-step greedy DQN does not account for this delayed effect, so the learned Q-values can be biased relative to the true multi-step control problem. The claim that the agent 'always tries to select the best possible cell DTX/DRX configuration' is not established. I recommend either an autocorrelation analysis of state features across consecutive inferencing steps, a comparison against a full MDP/DQN baseline, or an explicit redefinition of the claim to cover only the one-step setting.","section":"Section III.A and Section IV"},{"comment":"The reward constants c=0.75, y0=0.9, m=2, alpha=3 are selected by comparing performance on the same 10 inferencing episodes (210 cells) that generate the headline numbers, and no confidence intervals are reported. Section IV states: 'The best coefficient/constants we found from these comparisons are ...' and then uses those same deployments to report the ~45% energy saving and ≤1% data-rate loss. This is model selection on the evaluation set, so the headline point estimates are optimistically biased. Footnote 3 merely asserts that 210 cells are 'statistically representative enough' without supporting evidence. Please report results on a holdout set, provide confidence intervals or variability measures across the 10 episodes, and account for the number of constant combinations tried. Without this, the phrases 'always maintaining no more than ~1% QoS degradation' and 'up to ~45% energy saving' are not statistically supported.","section":"Section IV, reward tuning and statistical evaluation"},{"comment":"There is a mismatch between the QoS metric used in the reward and the QoS metric reported in the results. The reward and the problem formulation use the delivered data ratio y (Eq. 2), which counts data in packets received within their delay requirement. The reported QoS result, however, is 'achieved data rate' (Fig. 7). Data rate can remain high even when packets miss their latency deadlines, because delayed packets may still be transmitted and counted in throughput. The conclusion states that the agent maintains 'no more than ~1% degradation of the achieved data rate on average', which is not the same as protecting the delay-sensitive QoS that the whole reward design is built around. Please report the delivered data ratio y and/or the delay-violation ratio for the same experiments, and qualify the 'QoS degradation' claim accordingly.","section":"Section IV, Figs. 6-7 and Eq. (2)"},{"comment":"The energy-saving numbers are computed under a power model that ignores the mandatory residual signaling during non-active periods, namely SSB transmission, random access, paging, and system information broadcast. Footnote 1 justifies this by saying the periodicity considered is much less than 160 ms, but the simulator does not report the actual non-active durations or the power cost of these residual transmissions. Since the paper's headline claims are about absolute energy saving percentages, this omission could materially overstate the gains. I ask the authors to either incorporate a minimal model of residual signaling into the power consumption calculation, or to report a sensitivity analysis showing the impact on the projected savings for representative SSB/paging periodicities.","section":"Footnote 1 and Section II.B"}],"minor_comments":[{"comment":"Equation (6) is introduced with the example constants y0=0.9, m=2, alpha=2, but Section IV reports the best constants as y0=0.9, m=2, alpha=3. Please clarify whether the example in Section III.B is superseded by the tuned value, and state whether alpha=2 was also tried.","section":"Section III.B"},{"comment":"The action space is defined only through the constraints 'cycle length smaller than the minimum latency requirement' and 'on-duration smaller than cycle length'. Because Table IV gives delay requirements of 50/75/100 ms, the effective set of cycle lengths is much smaller than the full Table I. Please list the actual (cycle length, on-duration) pairs used in training and inference so the reader can assess the search space and the meaning of the Q-network output.","section":"Section III.A"},{"comment":"The manuscript contains several OCR-style symbol corruptions, for example the author email 'hosein.nikopour@intel.co m', the subscripts in Eqs. (1)-(3), and the subscripted reward notation r_QoS_approx. These make the mathematical definitions harder to read and should be cleaned up in the final version.","section":"General presentation"},{"comment":"The convergence plot shows mean Q-values only. Adding a plot of the training reward (or loss) over episodes, and also reporting the number of training steps per episode, would make the convergence claim easier to verify.","section":"Section IV, Fig. 5"},{"comment":"The conclusion says the agent maintains 'no more than ~1% degradation of the achieved data rate on average'. The word 'average' is in tension with 'always' in the same sentence and with the abstract's 'always maintaining'. Please choose precise quantifiers and use them consistently.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a networking journal and the problem is timely, but the evaluation protocol currently supports a weaker claim than the abstract makes. The main risks are the untested contextual-bandit independence assumption, reward tuning on the evaluation set without confidence intervals, and the reported QoS metric being data rate rather than delay-aware delivered ratio. These are all addressable with additional experiments and a more careful statistical presentation, so I view this as a major revision rather than a rejection. I would also encourage the editor to consider whether the journal expects code or detailed hyperparameter tables; the current manuscript provides enough detail to reproduce the setup in principle but stops short of full reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth a look if you work on RAN energy efficiency. It is one of the first to tackle configuration of Rel-18 cell DTX/DRX with RL, and it has a genuinely useful idea: a smooth approximation of the QoS-threshold reward that keeps QoS first while remaining trainable. The simulations show that the approach can save around 45% energy in light load with little throughput loss, and the comparison between linear and approximated QoS rewards is informative. The O-RAN xApp framing is practical, not buzzwordy.\n\nThat said, the strength of the quantitative claims is overrated. The reward constants are tuned by comparing performance on the same ten evaluation episodes used for the final numbers. That is selection on the test set, and it means the headline percentages are optimistic. There are no confidence intervals, and the footnote calling 210 cells 'statistically representative enough' is not a statistical argument. Also, the reported QoS metric is achieved data rate, while the reward function uses delivered data ratio; the connection between the two is not discussed.\n\nThe stress-test note gets one real issue right: Section III.A asserts that consecutive one-second states are approximately independent, and that assumption is doing real work. Training uses one-step episodes, so the agent never sees the temporal coupling that appears in inference, where ten steps share a deployment and the chosen DTX/DRX action delays packets into the next step. That makes the learned Q-values potentially biased for the actual control problem. This is not a fatal flaw for a first study, but the paper should acknowledge it and test whether a full MDP formulation changes the results.\n\nThe paper is otherwise honest. It flags the SSB/paging simplification, makes its problem formulation clearly, and does not overclaim novelty. The simulations are plausible, and the method is reproducible in principle, though no code or data is shipped.\n\nBottom line: this is an applied engineering contribution with a useful reward design and a realistic feature under study. It deserves peer review, but a referee should demand a separate validation set, confidence intervals, and a more careful treatment of the temporal coupling. If those are fixed, it would be a solid venue paper.","headline":"A practical RL method for Rel-18 cell DTX/DRX with a good reward idea, but the quantitative claims lean on on-evaluation-set tuning and an untested independence assumption.","tokens_in":11994,"tokens_out":5574,"would_cite":true,"duration_ms":61864,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A learned agent picks per-cell sleep-cycle settings, saving up to 45% of radio energy while keeping QoS loss near 1%.","keywords":["cell DTX/DRX","network energy saving","deep reinforcement learning","DQN","contextual bandit","5G RAN","near-real-time controller application","QoS-delay trade-off"],"falsifier":"Measure lag-1 autocorrelation of per-cell PRB utilization across successive 1-second windows in a real or high-fidelity trace; if the autocorrelation is non-negligible (say above 0.1-0.2), the bandit decomposition is violated and the agent's chosen configuration will deviate from the brute-force optimum. Alternatively, run the trained agent on a trace with 10-60 second bursts (e.g., video-conference sessions) and compare its average reward against an oracle that knows the future traffic; a systematic gap larger than a few percent would refute the 'always selects the best configuration' claim.","tokens_in":10970,"feed_emoji":"🔋","tokens_out":6701,"duration_ms":66738,"temperature":0.7,"pith_summary":"3GPP Release 18 lets a 5G cell periodically go silent between bursts of transmission, but choosing how long the silent gap should be trades energy savings against packet delay. The paper claims that a deep reinforcement learning agent, trained as a one-step decision maker (a contextual bandit) with a carefully smoothed reward, can pick the best cycle length and active duration for each cell under changing traffic. In a 21-cell system-level simulation, the agent achieves up to about 45% energy saving in light load and at least 22% in medium load, while keeping the average data-rate loss at or below 1%. The practical stake is that the same agent could run as a near-real-time controller application, cutting radio access network energy consumption without violating delay-sensitive service guarantees.","feed_headline":"AI cuts 5G cell energy by 45% with near-zero quality loss","feed_subtitle":"Agent trained with deep Q-learning chooses Rel-18 sleep cycles per cell, keeping average QoS loss at or below 1%.","key_machinery":"The load-bearing design is the contextual-bandit decomposition: because consecutive observation windows of about one second are treated as nearly independent, the reinforcement-learning problem collapses to a one-step expected-reward estimate, and a deep Q-network with experience replay learns the mapping from RAN observations to the best cell DTX/DRX action. The second load-bearing piece is the approximated QoS reward, a smooth version of a discontinuous threshold reward. The threshold reward is $-ax$ when the delivered-data ratio $y$ meets the target $y_0$, and $y-b$ otherwise; the smooth approximation is $r_{\\mathrm{QoS\\_approx}} = -\\frac{u[1+(\\alpha-1)(1-y)]+x}{u+1}$ with $u=\\left(\\frac{1-y}{(1-y_0)(1-x)}\\right)^m$, which behaves like a QoS-only reward when the failed-data ratio is large and like an energy-only reward when QoS is satisfied. Together with the state vector of traffic and transmission-condition measurements, these two mechanisms carry the argument.","core_discovery":"On the paper's own terms, the discovery is that the cell DTX/DRX configuration problem—choosing cycle length and on-duration for each cell—can be posed as a contextual bandit and solved by a deep Q-network, provided the reward is a smooth approximation of a discontinuous 'QoS first, then energy' reward. The agent observes per-cell traffic intensity, inter-arrival statistics, packet-size statistics, delay requirements, and transmission capability, and outputs one of a finite set of RRC configurations, including the always-on choice. The smooth reward function makes training stable, and the simulation results show the trained agent keeps data-rate loss within roughly 1% on average while saving up to about 45% energy under light load and 22% under medium load, compared with no cell DTX/DRX. The paper presents the agent as one that 'always tries to select the best possible configuration under any network and traffic conditions,' which is the core claim to be tested.","pith_inferences":["Editorial inference: The bandit assumption is testable and fragile; if real deployments show bursty traffic at 10-second or longer timescales, the one-step DQN would need to be replaced by an MDP with longer memory, since the paper's own justification rests on weak correlations between consecutive 1-second observations.","Editorial inference: Because the simulations ignore SSB, paging, and system-information broadcast during non-active periods, the claimed savings are an upper bound; accounting for these transmissions would reduce the net energy gain, especially at very light load where fixed signaling cost is a larger fraction.","Editorial inference: The start offset could become a learning target rather than a randomizer; a multi-cell extension that jointly optimizes offsets to coordinate interference would likely improve both QoS and energy over the single-cell policy.","Editorial inference: The reward formulation suggests a general recipe for other delay-sensitive energy-saving features: use a smooth approximation of a lexicographic 'QoS first, then energy' objective to stabilize training, and evaluate against an average QoS degradation budget rather than a hard threshold."],"forward_implications":["A near-real-time controller application could deploy the trained agent, periodically reconfiguring each cell's DTX/DRX parameters from RAN measurements over the E2 interface.","Operators can expect the largest energy savings (about 45%) at light load, where the sleep gaps are longest, and still save about a fifth of energy at medium load.","The smoothed reward makes the approach robust to measurement noise, so the agent does not need per-scenario tuning of the QoS-versus-energy trade-off coefficient.","Setting a random start offset per cell in inference mode staggers the active periods of neighboring cells, reducing inter-cell interference compared with synchronized cycles.","The same framework should extend to other time-domain energy-saving parameters, provided the observation window is long enough for nearly independent states."],"supporting_citations":[{"why":"This work supplies the power-consumption and sleep-mode transition model, including the relative power values and the scaling expression used to compute the power metric.","marker":"[16]"},{"why":"This work reports the empirical observation that RAN measurement randomness requires observation periods near one second, which underlies the contextual-bandit decomposition.","marker":"[10]"},{"why":"This specification defines the Rel-18 cell DTX/DRX RRC parameters and their allowed values that form the action space.","marker":"[12]"},{"why":"This book supplies the deep Q-network, experience replay, and epsilon-greedy exploration machinery used to train the agent.","marker":"[9]"},{"why":"This 3GPP contribution reports the 47.8-71.3% energy-saving potential of cell DTX/DRX that motivates the paper's targets.","marker":"[14]"},{"why":"This work provides the open RAN architecture and E2 interface that allow the trained agent to run as a controller application.","marker":"[13]"}],"fun_headline_variants":["Deep RL tunes 5G sleep modes for 45% energy cut with 1% QoS hit","AI agent balances energy and delay: 45% cell savings, <1% QoS loss","5G cell DTX/DRX optimized by deep Q-network: 45% energy saving, 1% QoS cost","Reinforcement learning picks sleep cycles to save 45% energy with minimal delay","Contextual bandit agent cuts 5G energy by 45% while protecting QoS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole scheme stands on treating consecutive ~1-second RAN observations as nearly independent, so the next best configuration depends only on the current window; if real traffic has longer-range autocorrelation, the learned Q-values will be biased, and the reported savings also ignore SSB, paging, and system-information overhead during silent periods.","fun_headline_variants_meta":{"raw":{"variants":["Deep RL tunes 5G sleep modes for 45% energy cut with 1% QoS hit","AI agent balances energy and delay: 45% cell savings, <1% QoS loss","5G cell DTX/DRX optimized by deep Q-network: 45% energy saving, 1% QoS cost","Reinforcement learning picks sleep cycles to save 45% energy with minimal delay","Contextual bandit agent cuts 5G energy by 45% while protecting QoS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000774,"raw_usage":{"total_tokens":3481,"prompt_tokens":1054,"completion_tokens":2427,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":670,"completion_tokens_details":{"reasoning_tokens":2305}},"tokens_in":670,"tokens_out":2427,"duration_ms":19564,"temperature":1.0,"reasoning_tokens":2305,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:48:21.417819+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure lag-1 autocorrelation of per-cell PRB utilization across successive 1-second windows in a real or high-fidelity trace; if the autocorrelation is non-negligible (say above 0.1-0.2), the bandit decomposition is violated and the agent's chosen configuration will deviate from the brute-force optimum. Alternatively, run the trained agent on a trace with 10-60 second bursts (e.g., video-conference sessions) and compare its average reward against an oracle that knows the future traffic; a systematic gap larger than a few percent would refute the 'always selects the best configuration' claim.","supporting_citations":[{"cited_title":"Enabling Network Power Savings in 5G- Advanced and Beyond,","cited_arxiv_id":null,"evidence_quote":"This work supplies the power-consumption and sleep-mode transition model, including the relative power values and the scaling expression used to compute the power metric."},{"cited_title":"Machine Learning Based C-DRX Configuration Optimization for 5G,","cited_arxiv_id":null,"evidence_quote":"This work reports the empirical observation that RAN measurement randomness requires observation periods near one second, which underlies the contextual-bandit decomposition."},{"cited_title":"Radio Resource Control (RRC) protocol specifications,","cited_arxiv_id":null,"evidence_quote":"This specification defines the Rel-18 cell DTX/DRX RRC parameters and their allowed values that form the action space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This book supplies the deep Q-network, experience replay, and epsilon-greedy exploration machinery used to train the agent."},{"cited_title":"Network energy saving techniques in time, frequency and spatial domain,","cited_arxiv_id":null,"evidence_quote":"This 3GPP contribution reports the 47.8-71.3% energy-saving potential of cell DTX/DRX that motivates the paper's targets."},{"cited_title":"O-RAN: Towards an Open and Smart RAN,","cited_arxiv_id":null,"evidence_quote":"This work provides the open RAN architecture and E2 interface that allow the trained agent to run as a controller application."}],"review_version":1}