REVIEW 4 major objections 5 minor 1 cited by
Graph Reinforcement Learning for QoS-Aware Load Balancing in Open Radio Access Networks
T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read This paper claims that a graph reinforcement learning load balancer, trained as a GNN-based dueling DQN, reduces QoS dissatisfaction by more than 53% and quadruples the 5th-percentile goodput of best-effort users in a simulated multi-band…
desk verdict Solid extension of the authors' GRL load balancer to mixed GBR/BE O-RAN, but the BE coverage reward proxy is unvalidated and the evaluation lacks statistical rigor. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the RAN graph representation $\mathcal{G}=(\mathcal{N},\mathcal{E},X_u,X_c)$ with UEs and cells as heterogeneous nodes and two edge types (UE-to-cell access links and cell-to-cell load-balancing links). The argument is carried by a GNN-based dueling DQN: two graph convolutional layers propagate node embeddings over the graph, a mean pooling layer aggregates cell embeddings, and the dueling architecture splits the Q-value into state value $V(s)$ and advantage $A(s,a)$ via $Q(s,a)=V(s)+A(s,a)-\frac{1}{|\mathcal{A}|}\sum_{a'}A(s,a')$. The reward is the improvement of a QoS-aware objective $f(\mathcal{G})$ that sums GBR QoS metrics and the minimum average BE rate after each handover, and a subgraph extraction heuristic keeps the graph tractable for large deployments.
What would settle it
Run the same training pipeline in the simulator but compute the reward from a quantile estimate of the BE rate (e.g., the 5th percentile over a long sliding window) instead of the minimum average rate, holding all other settings fixed; if the learned policy no longer delivers the reported fourfold increase in the 5th-percentile goodput, the claimed coverage gain depends on the reward proxy rather than on the graph RL mechanism.
Extended reading notes
Core claim
The paper claims that a load balancing policy trained with graph reinforcement learning can jointly improve GBR QoS satisfaction and BE coverage under realistic interference, non-full-buffer traffic, and mixed bands. The central discovery is that representing the RAN state as a heterogeneous graph — with UE and cell features, UE-to-cell access links, and cell-to-cell load-balancing opportunity edges — and learning Q-values with a two-layer graph convolutional network inside a dueling DQN yields a policy that moves cell-edge UEs to underutilized cells across bands, reducing QoS violation rate by over 53% and multiplying the BE 5th percentile goodput by roughly four compared to max-RSRP and max-SINR. The learned policy is permutation invariant, scales to different network sizes via subgraph extraction, and accounts for spatial dependencies among cells.
Load-bearing premise
The training reward replaces the coverage metric it aims to maximize — the 5th percentile best-effort rate — with the minimum average best-effort rate over the current window, and assumes that minimum faithfully tracks true edge-user coverage; if a single persistently unlucky user drives the minimum, the agent's learning signal becomes noise.
Editorial extensions
If this is right
- The learned LB policy can be hosted as an xApp and re-run as the RAN graph evolves, with no need to fix the input size beforehand.
- The approach generalizes to different numbers of UEs and cells because GCN processing and mean pooling are permutation invariant and the subgraph extraction keeps complexity manageable.
- Jointly optimizing GBR QoS and BE coverage is feasible in a single RL reward, without separating the two traffic classes into independent controllers.
- The observed load distribution across bands indicates the GRL agent can move traffic to underused lower bands even when SINR favors higher bands, balancing utilization more evenly.
Reading between the lines
- Because the reward uses the minimum average BE rate rather than the true 5th percentile, the reported fourfold coverage gain is contingent on that proxy; a robust quantile estimate or long-horizon evaluation would test whether the gain reflects genuine edge-user improvement.
- The same graph formulation could be extended to other O-RAN controls, such as cell sleep modes or spectrum allocation, by redefining actions and reward terms while reusing the GNN-based dueling DQN.
- The subgraph extraction heuristic suggests a natural testable extension: compare full-graph vs. subgraph training on larger topologies to see whether the segmentation degrades policy quality.
- Because the policy is trained on deployments with random UE counts and traffic parameters, a direct next step is out-of-distribution generalization testing with UE counts or band configurations outside the training range.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a graph reinforcement learning (GRL) approach to quality-of-service (QoS)-aware load balancing in a multi-band Open RAN. The RAN is represented as a heterogeneous graph with UE and cell nodes, the load-balancing problem is cast as an MDP, and a GNN-based dueling DQN is trained to select handover actions for cell-edge UEs. The reward is constructed from the QoS objective for GBR traffic and a proxy for BE coverage. In a system-level simulator, the proposed method is compared against max-RSRP and max-SINR baselines, reporting a 53% reduction in QoS dissatisfaction rate and a fourfold increase in the 5th percentile BE goodput.
Significance. If validated, the paper would make a useful contribution by extending GNN-based DRL load balancing from BE-only scenarios to mixed GBR/BE traffic with per-UE QoS constraints, while exploiting permutation invariance and variable-size graph inputs. The problem formulation is coherent, the simulation environment is relatively detailed, and the algorithmic idea of processing subgraphs for scalability is sensible. However, the central performance claims currently rest on an unvalidated reward proxy and on an evaluation that lacks variance characterization and held-out test splits, so the significance is conditional on additional evidence.
major comments (4)
- [Section III-B, Eq. (5) and Fig. 3] The reward substitutes the minimum average BE rate, r_min,BE, for the coverage metric cBE defined in Eq. (3), with the justification that the 5th percentile cannot be estimated from limited samples. With U in {35, ..., 70} and a 3:1 GBR:BE traffic ratio, each deployment contains only about 9 to 17 BE UEs, so r_min,BE is the most extreme order statistic and can be dominated by a single UE with a poor average rate. The manuscript provides no evidence that maximizing this minimum aligns with the 5th-percentile objective in Eq. (4a); the fourfold coverage improvement in Fig. 3 could therefore be an artifact of optimizing a different, more brittle quantity. Please validate the proxy, for example by comparing against a reward based on a lower quantile estimated over a longer window, and report sensitivity of the learned policy to individual outlier BE UEs.
- [Section IV, Figs. 2-4] All performance results are shown as single CDF curves without confidence intervals, multiple-seed statistics, or an explicit train/test separation. The text states that the model is trained over deployments with random UE counts, locations, and traffic parameters, but it does not report how many deployments were used, whether the evaluation deployments were held out from training, or how stable the reported averages (0.12 versus 0.29 and 0.26 in Fig. 2) are. The headline numbers in the abstract are point estimates; error bars and a clearly described evaluation protocol are needed to support them.
- [Section IV] The comparison is limited to max-RSRP and max-SINR baselines. Because neither baseline is learning-based, the experiments do not isolate whether the gains come from the graph representation, the dueling DQN, or reinforcement learning in general. Adding a learning-based baseline, such as a DQN with a fixed-size MLP or the earlier GRL method in [12], would be necessary to support the stronger claim that the proposed architecture itself is responsible for the gains and to position the method relative to existing work.
- [Section III-C and Section IV] The description of the Q-network is incomplete and several training details are missing. After the mean pooling over cell nodes, the text does not specify how the advantage A(s,a) is computed for each feasible action; Eq. (12) assumes per-action advantages, but the action representation and the action branch of the network in Fig. 1 are not described. In addition, the value of alpha in the reward (5) is never reported, and the target-network update schedule ('after every 10 episodes') is not tied to a concrete number of gradient steps or environment interactions. These omissions prevent reproduction of the method and should be addressed.
minor comments (5)
- [Abstract] The phrase 'QoS consideration are integrated' should be 'QoS considerations are integrated'.
- [Eq. (4a) and Eq. (5)] The set-difference notation 'U_GBR / U_ce' is ambiguous; use the standard backslash notation such as '\mathcal{U}^{\mathrm{GBR}} \setminus \mathcal{U}^{\mathrm{ce}}'.
- [Reference [15]] The 3GPP reference is given as 'TS 23.50'; the correct specification number appears to be TS 23.501, and the year/version should be completed.
- [Algorithm 1, line 4] The loop condition 'while episode ends do' is misleading; it should read 'while the episode has not ended do' or 'while not terminal do'.
- [Fig. 4] The average bandwidth utilization per cell is reported as a single bar for each band; a fairness index or per-cell utilization distribution would make the load-balancing comparison more informative.
Circularity Check
No significant circularity: the GRL agent optimizes a reward derived from the paper's stated QoS/coverage objectives, and the reported gains are measured against external simulation baselines rather than being forced by construction.
full rationale
The derivation chain is self-contained. The GNN-based dueling DQN is trained to maximize the cumulative reward built from f(G) in Eq. (5), which is an explicit surrogate for the optimization objective in Eq. (4a): the first two terms use the QoS metric q_{u,k} from Eq. (2), and the third uses the minimum average BE rate as a stand-in for the 5th-percentile coverage cBE from Eq. (3). This is ordinary RL design—the agent is trained on a reward that tracks the stated objective—rather than a hidden circular fit: no parameter is fitted to the reported QoS dissatisfaction rate or to the 5th-percentile BE goodput, and the evaluation against max-RSRP and max-SINR baselines is external to the trained policy. The substitution of r_min,BE for cBE is an acknowledged approximation whose statistical alignment with the 5th percentile is not validated; that is a correctness/robustness risk, not circularity. The self-citation to the authors' prior GRL work [12] is contextual, and the method's load-bearing building blocks (GCN propagation, dueling DQN, Bellman update) are standard external results [16,17]; no load-bearing uniqueness or reduction to prior work is imported. Therefore no step of the claimed derivation reduces by definition to its own inputs.
Assumptions & free parameters
free parameters (2)
- alpha (BE coverage weight)
- rho (GFBR/MFBR ratio per 5QI) =
0.6, 0.9, 0.8 for 5QI 2, 3, 67
assumptions (3)
- domain assumption The RAN graph state G=(N,E,Xu,Xc) is a sufficient statistic for the MDP (Markov property).
- domain assumption The Python system-level simulator faithfully represents real O-RAN behavior.
- domain assumption The reward difference Ri=f(G(i))-f(G(i-1)) with discount beta=0.999 drives the agent toward the original objective.
Cite this review
Pith. "Pith review of Graph Reinforcement Learning for QoS-Aware Load Balancing in Open Radio Access Networks." pith.science (2026). https://pith.science/paper/7W6Y5FTA
@misc{pith2026250419499,
author = {Pith},
title = {Pith review of: Graph Reinforcement Learning for QoS-Aware Load Balancing in Open Radio Access Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/7W6Y5FTA}},
note = {Machine review of arXiv:2504.19499}
}
abstract
Next-generation wireless cellular networks are expected to provide unparalleled Quality-of-Service (QoS) for emerging wireless applications, necessitating strict performance guarantees, e.g., in terms of link-level data rates. A critical challenge in meeting these QoS requirements is the prevention of cell congestion, which involves balancing the load to ensure sufficient radio resources are available for each cell to serve its designated User Equipments (UEs). In this work, a novel QoS-aware Load Balancing (LB) approach is developed to optimize the performance of Guaranteed Bit Rate (GBR) and Best Effort (BE) traffic in a multi-band Open Radio Access Network (O-RAN) under QoS and resource constraints. The proposed solution builds on Graph Reinforcement Learning (GRL), a powerful framework at the intersection of Graph Neural Network (GNN) and RL. The QoS-aware LB is modeled as a Markov Decision Process, with states represented as graphs. QoS consideration are integrated into both state representations and reward signal design. The LB agent is then trained using an off-policy dueling Deep Q Network (DQN) that leverages a GNN-based architecture. This design ensures the LB policy is invariant to the ordering of nodes (UE or cell), flexible in handling various network sizes, and capable of accounting for spatial node dependencies in LB decisions. Performance of the GRL-based solution is compared with two baseline methods. Results show substantial performance gains, including a $53\%$ reduction in QoS violations and a fourfold increase in the 5th percentile rate for BE traffic.
Figures
Forward citations
Cited by 1 Pith paper
-
Deep Reinforcement Learning-based Cell DTX/DRX Configuration for Network Energy Saving
A DQN agent on a contextual-bandit model, trained with a smooth QoS-threshold reward, selects Rel-18 cell DTX/DRX cycle and on-duration settings, achieving up to 45% simulated energy savings with under 1% average data...
Reference graph
Works this paper leans on
-
[11]
Qos-aware load balancing in wireless networks using clipped double q-learning,
P. E. Iturria-Rivera and M. Erol-Kantarci, “Qos-aware load balancing in wireless networks using clipped double q-learning,” in IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems , 2021
work page 2021
-
[12]
O. Orhan, V . N. Swamy, T. Tetzlaff, M. Nassar, H. Nikopour, and S. Talwar, “Connection Management xAPP for O-RAN RIC: A Graph Neural Network and Reinforcement Learning Approach,” in 20th IEEE International Conference on Machine Learning and Applications , 2021
work page 2021
-
[1]
A vision of 6G wireless systems: Applications, trends, technologies, and open research problems,
W. Saad, M. Bennis, and M. Chen, “A vision of 6G wireless systems: Applications, trends, technologies, and open research problems,” IEEE Network, vol. 34, no. 3, pp. 134–142, 2020
2020
-
[2]
Adaptive Mobility Load Balancing Algorithm for LTE Small-Cell Networks,
M. M. Hasan, S. Kwon, and J.-H. Na, “Adaptive Mobility Load Balancing Algorithm for LTE Small-Cell Networks,”IEEE Transactions on Wireless Communications, vol. 17, no. 4, pp. 2205–2217, 2018
work page 2018
-
[3]
Y .-J. Cho, H.-M. Yoo, K.-S. Kim, J. Na, and E.-K. Hong, “Practical Load Balancing Algorithm for 5G Small Cell Networks Based on Real- World 5G Traffic and O-RAN Architecture,” IEEE Access , vol. 12, pp. 121 947–121 957, 2024
work page 2024
-
[4]
Cell breathing algorithms for load bal- ancing in Wi-Fi/cellular heterogeneous networks,
˙Ilhan Demirci and ¨Omer Korc ¸ak, “Cell breathing algorithms for load bal- ancing in Wi-Fi/cellular heterogeneous networks,” Computer Networks , vol. 134, pp. 140–151, 2018
work page 2018
-
[5]
Load Balancing With Traffic Splitting for QoS Enhancement in 5G HetNets,
A. Manan, S. M. Shahid, S. Kim, and S. Kwon, “Load Balancing With Traffic Splitting for QoS Enhancement in 5G HetNets,” IEEE Transactions on Network Science and Engineering , vol. 11, no. 6, pp. 6272–6284, 2024
work page 2024
-
[6]
Probabilistic Mobility Load Balancing for Multi-Band 5G and Beyond Networks,
S. A. Lahham, D. Wu, E. Hossain, X. Liu, and G. Dudek, “Probabilistic Mobility Load Balancing for Multi-Band 5G and Beyond Networks,” in 2024 IEEE ICC Workshops , 2024, pp. 1673–1678
work page 2024
Show all 17 references
-
[7]
Ma- chine learning-based load balancing algorithms in future heterogeneous networks: A survey,
E. Gures, I. Shayea, M. Ergen, M. H. Azmi, and A. A. El-Saleh, “Ma- chine learning-based load balancing algorithms in future heterogeneous networks: A survey,” IEEE Access, vol. 10, pp. 37 689–37 717, 2022
2022
-
[8]
Lstm based proactive access point selection and mobility load balancing for ultra- dense networks,
M. Shabbir, S. Kandeepan, A. Al-Hourani, and W. Rowe, “Lstm based proactive access point selection and mobility load balancing for ultra- dense networks,” in 2024 International Conference on Artificial Intelli- gence in Information and Communication (ICAIIC) , 2024, pp. 452–458
2024
-
[9]
Load balancing for ultradense networks: A deep reinforcement learning-based approach,
Y . Xu, W. Xu, Z. Wang, J. Lin, and S. Cui, “Load balancing for ultradense networks: A deep reinforcement learning-based approach,” IEEE Internet of Things Journal , vol. 6, no. 6, pp. 9399–9412, 2019
2019
-
[10]
Deep reinforcement learning-based cio and energy control for lte mobility load balancing,
G. Alsuhli, H. A. Ismail, K. Alansary, M. Rumman, M. Mohamed, and K. G. Seddik, “Deep reinforcement learning-based cio and energy control for lte mobility load balancing,” in 2021 IEEE 18th Annual Consumer Communications & Networking Conference , 2021, pp. 1–6
2021
-
[13]
Artificial Intelligence in 3GPP 5G-Advanced: A Survey,
X. Lin, “Artificial Intelligence in 3GPP 5G-Advanced: A Survey,” 2023. [Online]. Available: https://arxiv.org/abs/2305.05092
2023 arXiv
-
[14]
Graph reinforcement learning for radio resource allocation,
J. Zhao and C. Yang, “Graph reinforcement learning for radio resource allocation,” 2023. [Online]. Available: https://arxiv.org/abs/2203.03906
2023 arXiv
-
[15]
System architecture for the 5G System (5GS),
3GPP, “System architecture for the 5G System (5GS),” TS 23.50, 2020
2020
-
[16]
Semi-Supervised Classification with Graph Convolutional Networks,
T. N. Kipf and M. Welling, “Semi-Supervised Classification with Graph Convolutional Networks,” in International Conference on Learning Rep- resentations, 2017
2017
-
[17]
Dueling Network Architectures for Deep Reinforcement Learning,
Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot, and N. de Freitas, “Dueling Network Architectures for Deep Reinforcement Learning,” 2016. [Online]. Available: https://arxiv.org/abs/1511.06581
2016 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.