Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models

Jiadong Yu; Xinren Zhang

REVIEW 5 major objections 6 minor 2 cited by

Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models

T0 review · 5 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that D2RL, a generative-diffusion-enhanced DRL framework, shortens training time to convergence for wireless resource allocation—by over 58% in the best reported reward-design case—while preserving policy performance.

desk verdict D2RL extends GDM-based action exploration to reward and state spaces for wireless DRL, but the reward objective looks circular and the single-run experiments don't support the headline 58% efficiency gain. read the letter →

arxiv 2502.07211 v1 pith:6C67FSBQ submitted 2025-02-11 cs.LG

Xinren Zhang , Jiadong Yu This is my paper · ORCID

classification cs.LG

keywords deepreinforcementlearninggenerativediffusionmodelswirelessresourceallocationtrainingefficiencyfull-duplexcommunicationstateaugmentationrewarddesignactionexploration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dynamic wireless networks force periodic retraining of deep reinforcement learning policies, and the cost of that retraining can be prohibitive. This paper argues that three DRL ingredients—state data, action exploration, and reward design—are the bottlenecks, and proposes D2RL, which uses generative diffusion models to improve all three. The central claim is that this speeds convergence and lowers total GPU time without sacrificing policy quality, with the best reported case cutting total GPU time by more than 58% relative to an expert-designed reward baseline. A careful reader would care because if true, D2RL offers a practical way to make DRL retraining affordable in resource-constrained wireless systems.

What carries the argument

The load-bearing mechanism is the trained noise-prediction network (NPNN) of a diffusion model, used in two modes. In Mode I the reverse chain alone generates actions and rewards, with the action predictor turning Gaussian noise into actions conditioned on the state and the reward predictor turning noise into rewards conditioned on state-action pairs. In Mode II a forward noising step $X_{p}\sim\mathcal{N}(\sqrt{\bar{\alpha}_p}X_0,(1-\bar{\alpha}_p)I)$ followed by reverse denoising synthesizes states $\hat{S}_0$; the mode-II predictor minimizes $\mathbb{E}\|\epsilon-\epsilon_{\mathrm{ModII}}\|^2$, and the synthetic state is used with probability $\chi=\min(\chi+\eta,M)$ once the synthesis loss falls below threshold $T$. The policy-gradient identity $\nabla J(\pi)=\mathbb{E}_\pi[\sum_t \nabla\log\pi(A_t|S_t)\,G_t]$ is the stated reason these three spaces matter: fluctuations in state, action, and reward distributions directly shape gradient updates and hence training efficiency.

What would settle it

The premise would be tested by running Mode II with replacement probability at its maximum from the start, so that nearly every recorded state is synthetic while next states and rewards stay true; if the sum rate collapses or diverges, the synthetic-state tuple is not a valid sample of the environment, and if it learns steadily, the premise survives.

Watch

Extended reading notes

Core claim

The paper's central claim is that a generative diffusion model can be attached to a DRL agent to explore all three learning spaces simultaneously, and that doing so makes training converge faster and cheaper. Mode I, used when no original dataset exists, runs only the reverse diffusion process: an Action Exploration NPNN turns noise into actions conditioned on the current state, and a Reward Exploration NPNN turns noise into rewards conditioned on state-action pairs. Mode II, used when original state data exist, adds noise through a forward process and then denoises to synthesize new states, which replace true states with probability $\chi$ that ramps up as the synthesis loss improves. The reported experiments in a full-duplex system show that GDM action exploration cuts total GPU time by about 8.3% despite more than doubling per-epoch cost; that the expert-designed reward combined with GDM reward exploration cuts total GPU time by more than 58% compared with the expert reward alone while reaching a higher sum rate; and that GDM state exploration accelerates convergence for suboptimal reward designs but slows it when the reward is already near-optimal.

Load-bearing premise

The load-bearing premise is that a training tuple in which the received state has been replaced by a diffusion-generated synthetic state, while the next state and reward still come from the true environment responding to the true state, remains a valid experience sample for the agent.

Editorial extensions

If this is right

If the headline result generalizes, operators can retrain DRL policies in dynamic wireless networks using substantially less GPU time, with the largest reported gain coming from pairing expert-designed rewards with GDM-based reward exploration.
Because D2RL is proposed as algorithm-agnostic over the base DRL agent, the same three GDM exploration networks could be attached to other actor-critic and policy-gradient methods without changing the framework.
The reported dual effect of state exploration implies that state augmentation should be switched on or off according to reward quality: it helps weak reward designs and can hurt near-optimal ones, so the gain is conditional rather than universal.
The gradient analysis predicts that interventions smoothing the state, action, or reward distributions will show up in gradient weight and bias statistics, giving a diagnostic for when a GDM mode is actually accelerating learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension the paper does not report is to isolate Mode II by comparing Bellman residuals on true-state versus synthetic-state transitions; larger residuals on synthetic transitions would indicate the tuple-validity assumption is failing.
The 58% GPU-time reduction is measured in one full-duplex simulator with one DRL backbone, so extrapolating it to other wireless settings or other base algorithms is an inference, not a result of this paper.
If GDM-generated rewards are indeed more discriminative, reward exploration of this kind could be lifted into offline reinforcement learning, where reward labels are scarce; the paper points toward this connection but does not demonstrate it.
The dual effect of state exploration is consistent with viewing Mode II as a regularizer rather than a true expansion of the state distribution; under that view the optimal replacement probability should track how much generated states overlap the true support.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper extends the diffusion-for-DRL action-exploration idea from [17] to reward and state spaces in a full-duplex wireless resource allocation setting. That's a legitimate extension, and the paper is clearly written and honest about where state exploration helps and hurts. But the central efficiency claim rests on a reward-exploration objective that looks self-referential, and the experiments are single-run with per-case hyperparameter selection, so I don't trust the 58% GPU-time reduction.

What's new: Mode I reward exploration (training a diffusion network to produce rewards that maximize the critic's Q) and Mode II state replacement (using a diffusion denoiser to synthesize states). The full-duplex model with joint beamforming and power control is nontrivial, and the evaluation covers five reward designs plus a hyperparameter sweep. The authors deserve credit for reporting that state exploration degrades performance when the reward is already good — that's a useful negative result.

Where it falls down. First, Eq. (21) trains the reward network to maximize Q, while Q in Eq. (23) is trained on rewards generated by that same network. That's a closed loop with no anchor to the true sum-rate objective or the expert-designed reward. The reported speedup could be reward inflation rather than faster convergence to a good policy. The paper gives no correlation analysis of learned reward vs. sum rate, no reward magnitude stats, and no constraint on the reward network's output. Second, Algorithm 1 stores transitions (\hat{S}_0, A_t, S_{t+1}, R_t) where the next state and reward come from the true state, not the synthetic one; that violates MDP consistency. The authors' own finding that state exploration can hurt suggests they know it's not universally helpful, but the mechanism is still questionable. Third, the experiments are single-run, no error bars, and the hyperparameters M and eta are tuned per reward design to maximize performance — with no held-out evaluation, that's a multiple-comparisons problem. The 'theoretical analysis' in Section IV-B is just the policy gradient theorem restated; it doesn't quantify anything.

Who gets value: a reader working on GDM-based DRL or wireless resource allocation might get ideas about reward/state exploration, but the paper shouldn't be used as a reliable benchmark.

Recommendation: I would not trust the efficiency claims as-is. That said, the idea is worth a serious referee, not a desk reject — a good reviewer could push the authors to add multi-seed runs, fix the reward objective with an anchor to the true objective, and validate the learned reward. If those checks pass, the 58% number might be real; right now it's not supported.

Referee Report

5 major / 6 minor

Summary. The paper proposes D2RL, a framework that uses generative diffusion models (GDMs) to improve the training efficiency of deep reinforcement learning (DRL) for resource allocation in a full-duplex wireless system. The framework targets three DRL components: action exploration (via a diffusion-based action network), reward exploration (via a diffusion-based reward network trained to maximize the critic's Q-value), and state exploration (via a diffusion-based state generator that replaces observed states with synthetic ones). Experiments on a simulated full-duplex system compare several reward designs with and without GDM-based action, reward, and state exploration. The central empirical claim is that the Designed+GDM reward variant reduces total GPU time by over 58% compared to the Designed reward baseline while improving or maintaining sum-rate performance.

Significance. If the claims were validated, D2RL would be a practically relevant contribution: reducing retraining cost is an important problem for DRL in dynamic wireless networks. The paper has a reasonable system model, a standard DDPG-style base algorithm, and a clear application scenario. However, the main efficiency claim rests on the reward-exploration mechanism, whose training objective is self-referential, and the experimental evaluation lacks controlled comparisons (single runs, no error bars, and per-case hyperparameter tuning). The state-exploration mechanism also violates MDP transition assumptions. Therefore the significance is conditional on a substantial revision that addresses these load-bearing issues.

major comments (5)

[Section IV-C2, Algorithm 1] The reward-exploration mechanism is circular. The RENPNN is trained with the objective arg min L = -E[Q] (Eq. (21)), while the critic Q is trained on rewards generated by that same RENPNN through Eq. (23). There is no term anchoring the generated reward to the true sum-rate objective C or to the designed reward C - R'. This allows the reward model to inflate its output or converge to a degenerate reward that maximizes Q without improving the true policy objective. Section V-B2 attributes the over-58% total-GPU-time reduction (Fig. 6) specifically to the Designed+GDM R. variant; if the learned reward does not track C, the comparison may measure reward over-optimization rather than faster convergence to a good policy. The paper provides no statistics on generated reward values, no constraint on reward magnitude, and no evidence that the learned reward correlates with sum rate. This must be fixed, e.g., by anchoring the reward network to the true or designed reward and reporting reward-quality diagnostics.
[Section V, Figs. 2-9] The state-exploration mechanism replaces the observed state S_t with a GDM-generated synthetic state \hat{S}_0, then records the tuple (\hat{S}_0, A_t, S_{t+1}, R_t) and trains the agent on it. However, S_{t+1} and R_t are generated by the true environment from the true underlying state, not from \hat{S}_0. The tuple is therefore not a valid MDP sample under the transition distribution P(S_{t+1}|S_t,A_t). This can bias the critic and policy. The authors' own results in Section V-B3 show that state exploration degrades performance in the Designed+GDM R. case, which is consistent with this concern. The paper should either provide a theoretical justification for why this substitution is valid, or validate the synthetic states against the true state distribution (e.g., statistical distance or downstream policy evaluation) and report the effect on final policy performance.
[Section V-B1, Figs. 3, 6, 9] The experimental evaluation is not controlled. All comparisons appear to be based on a single run with no random seeds, no error bars, and no statistical tests. In Section V-B3, the hyperparameters M and \eta are selected per reward design by comparing against the baseline ('we selected the best hyperparameter combination'), which is a form of tuning on the test setting and makes cross-method comparisons unfair. The paper should report results over multiple seeds with confidence intervals, fix the hyperparameter-selection protocol before evaluation, and state the number of runs used for each reported GPU-time number.
[Section IV-B] The central efficiency metric 'total GPU time for convergence' is never given a precise definition. There is no stated convergence criterion (e.g., a threshold on moving-average sum rate over a window), and the x-axis scales differ across experiments (Fig. 2 uses 2e4 epochs while Figs. 4 and 8 use 8000 epochs). Without a reproducible convergence definition, the reported GPU-time reductions cannot be independently verified. Define the criterion and apply it identically to all methods.
[Algorithm 2 and Section IV-C4] The theoretical analysis in Section IV-B does not support the contribution claim that the paper 'quantifies the impact' of the three spaces on DRL training efficiency. Equations (16)-(20) are the standard score-function estimator derivation of the policy gradient, and the subsequent statements about gradient magnitude and fluctuations are qualitative. No quantitative bound or new theoretical result connects state, action, or reward characteristics to convergence speed. Either provide a concrete analysis with measurable quantities, or revise the contribution claim to describe the gradient analysis as a motivation rather than a quantitative result.

minor comments (6)

[Algorithm 2 and Section IV-C4] Algorithm 2 states 'reward R_t based on Mode II', but Section IV-C4 and Fig. 1 clearly assign reward exploration to Mode I. This inconsistency should be corrected.
[Section V-B3] The method names are inconsistent: the text uses 'Designed R. + GDM' in some places and 'Designed+GDM R.' in others; please standardize.
[Fig. 5] The text in Section V-B2 says gradient bias values reach up to 0.85 in Layer 2, but the y-axis of Fig. 5(b) only goes to 0.1; clarify which value is correct.
[Section II and Section III-B] The symbol P is used both for the number of diffusion steps (Eq. (1)) and for the environment transition kernel (Eq. (15)); please use distinct symbols.
[Section V-A3] The 'Designed R.' reward is defined as R = C - R', where R' is called a 'theoretical upper bound' and citation [17] is a general diffusion-model tutorial. The paper should specify what R' is for this system or provide a direct reference for the upper bound.
[Table II] The discount factor is set to 1 in Table II, while Eq. (14) defines \gamma in (0,1]. If \gamma=1 is used intentionally, explain why the finite-horizon setting makes this safe; otherwise adjust the value.

Circularity Check

1 steps flagged · score 7.0 of 10

Mode I reward exploration is a closed loop: the RENPNN is trained to maximize Q, and Q is trained on rewards generated by that same RENPNN; the headline 58% GPU-time saving is attributed to this component.

self definitional [Section IV-C1 (Eq. 21) and Section IV-C4 (Eq. 23)]
"The objective of the NPNN for Mode I is mathematically formulated as: arg min L =−E[Q], which aims to optimize the Q-value. Specifically, Mode I is used to explore the action and reward spaces. ... The reward generation process involves training a Reward Exploration NPNN (RENPNN) ... the RENPNN can generate rewards that consider both the current context and exploratory variations. ... Q(S_j,A_j)= E[R+γ·Q(S_{j+1},A_{j+1})]."

The RENPNN is trained to maximize the critic's Q via Eq. (21). The critic's Q, in Eq. (23), is defined as the expected return of rewards R that were generated by that same RENPNN and stored in the replay buffer. No term in the paper anchors the generated R to the true sum-rate C or to the designed reward C−R'. The reward signal is therefore its own training target: the network can increase Q by inflating generated rewards without improving the true objective. The paper's key efficiency claim (Designed+GDM R. reducing total GPU time by over 58% versus Designed R., Fig. 6) is attributed precisely to this reward-exploration component, so the comparison is partially by construction rather than an independent test of training efficiency.

full rationale

The main circularity is in the reward-exploration loop. Eq. (21) defines the Mode I NPNN objective as minimizing −E[Q], and Section IV-C4 applies Mode I to train the RENPNN that produces rewards R_t. Eq. (23) then trains the critic Q on exactly those generated rewards. This creates a closed, self-referential loop: the reward model is rewarded for maximizing Q, the actor is rewarded for maximizing Q, and Q is fit to rewards produced by the same reward model. Unless an external anchor such as the designed reward C−R' enters the loop in a way the paper does not specify, the reported computational-efficiency gains for reward exploration can reflect reward over-optimization rather than faster convergence to the true resource-allocation objective. The paper does separately plot sum rate, which provides some external evidence that the learned policy improves, so this is partial circularity rather than a complete collapse of the derivation. I do not find load-bearing self-citation: the GDM action-exploration baseline and reward-design reference [17] is an external tutorial, and the self-citations ([38], [41]) are for standard replay-buffer/soft-update and MA-reward definitions, not for the central claim. Mode II state replacement raises a genuine MDP-consistency concern, but that is a correctness issue, not circularity; the authors themselves report that state exploration can degrade performance when reward design is already good (Section V-B3). The policy-gradient analysis in Section IV-B is standard and does not smuggle in the D2RL components; it is merely motivation. Overall, the reward-loop circularity affects the headline efficiency claim, justifying a score of 7.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central efficiency claim rests on three ad hoc components: synthetic state substitution, Q-maximizing reward generation, and dataset-free reverse diffusion exploration. None of these are derived from the policy gradient analysis, and the tuning of M and eta per case adds free parameters. Standard simulation assumptions are also domain-specific.

free parameters (3)

M (maximum state-replacement probability) = 0.9 for Designed R.; 0.3 for Designed R.+MLP; 1.0 for Designed R.+GDM; 0.4 for GDM R.
Tuned per reward design by comparing against the baseline (Section V-B3); this is post-hoc selection on the evaluation metric.
eta (state-exploration update rate) = 0.001 for Designed R. and Designed R.+GDM; 0.01 for Designed R.+MLP and GDM R.
Same tuning procedure as M; values chosen to maximize reported performance.
T (SENPNN loss threshold) = 5e-4
Set in Table II; controls when state-replacement probability increases. Listed as a constant, but arbitrary.

assumptions (4)

ad hoc to paper A GDM-generated state \hat{S}_0 can be substituted for the true environment state S_t in the agent's observation stream without breaking the MDP transition model (Algorithm 1).
The paper asserts this replacement directly, but S_{t+1} and R_t are still produced by the true environment from the true state, so the stored transition may not be a valid sample.
ad hoc to paper The reward network RENPNN trained to maximize Q (Eq. 21) produces a reward signal aligned with the true sum-rate objective C (Eq. 13a).
No proof or external validation; only single-run empirical curves are offered, and Q itself is defined from the generated rewards.
domain assumption Mode I can explore action and reward spaces without any original dataset, using only reverse diffusion.
This is the defining assumption of Mode I (Section IV-C1); whether it yields useful actions and rewards in wireless systems is not theoretically grounded.
domain assumption The line-of-sight channel model with fixed interferers (Section V-A) is representative enough that conclusions transfer to real full-duplex networks.
Simulation parameters from Table I are standard for ISAC full-duplex studies, but no real-world validation is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models." pith.science (2026). https://pith.science/paper/6C67FSBQ

@misc{pith2026250207211,
  author       = {Pith},
  title        = {Pith review of: Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6C67FSBQ}},
  note         = {Machine review of arXiv:2502.07211}
}

read the original abstract

Dynamic resource allocation in mobile wireless networks involves complex, time-varying optimization problems, motivating the adoption of deep reinforcement learning (DRL). However, most existing works rely on pre-trained policies, overlooking dynamic environmental changes that rapidly invalidate the policies. Periodic retraining becomes inevitable but incurs prohibitive computational costs and energy consumption-critical concerns for resource-constrained wireless systems. We identify three root causes of inefficient retraining: high-dimensional state spaces, suboptimal action spaces exploration-exploitation trade-offs, and reward design limitations. To overcome these limitations, we propose Diffusion-based Deep Reinforcement Learning (D2RL), which leverages generative diffusion models (GDMs) to holistically enhance all three DRL components. Iterative refinement process and distribution modelling of GDMs enable (1) the generation of diverse state samples to improve environmental understanding, (2) balanced action space exploration to escape local optima, and (3) the design of discriminative reward functions that better evaluate action quality. Our framework operates in two modes: Mode I leverages GDMs to explore reward spaces and design discriminative reward functions that rigorously evaluate action quality, while Mode II synthesizes diverse state samples to enhance environmental understanding and generalization. Extensive experiments demonstrate that D2RL achieves faster convergence and reduced computational costs over conventional DRL methods for resource allocation in wireless communications while maintaining competitive policy performance. This work underscores the transformative potential of GDMs in overcoming fundamental DRL training bottlenecks for wireless networks, paving the way for practical, real-time deployments.

Figures

Figures reproduced from arXiv: 2502.07211 by the authors.

**Figure 1.** Illustration of D2RL Framework. 3) Challenges for Reward Function Design: Designing effective reward functions for wireless communications is challenging due to competing objectives like throughput, energy efficiency, and latency. Encapsulating diverse requirements into a single function is complex. While expert knowledge can provide valuable insights into reward function design, an overreliance on predetermined r… view at source ↗

**Figure 2.** Comparison of (a) the sum rate achieved by GDM-based action [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

**Figure 3.** Comparison of (a) total GPU time for convergence achieved by GDM [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 6.** Figure 6: Comparison of (a) total GPU time for convergence and (b) GPU time [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 4.** Figure 4: Comparison of (a) the sum rate and (b) MA reward for different [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

**Figure 5.** Figure 5: Comparison of (a) the mean gradient weight and (b) the mean gradient [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

**Figure 7.** Figure 7: Comparison of gradient loss for (a) the weight sum of Designed R., (b) the bias sum of Designed R., (c) the weight sum of Designed R.+MLP, (d) [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

**Figure 8.** Figure 8: Comparison of (a) the sum rate and (b) MA reward for different [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

**Figure 9.** Figure 9: Comparison of (a) total GPU time for convergence and (b) GPU time [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Sustainable Air-Ground Integrated Coverage Networks: ISCC Architecture, Technologies, and Testbed
cs.IT 2026-07 conditional novelty 4.0 of 10

A closed-loop ISCC control framework for sustainable air-ground networks is shown on a 17-base-station testbed to cut power about 20% while keeping coverage above 90%.
Energy-Efficient RSMA-enabled Low-altitude MEC Optimization Via Generative AI-enhanced Deep Reinforcement Learning
cs.NI 2025-07 conditional novelty 3.0 of 10

A diffusion-enhanced SAC agent jointly optimizes UAV trajectory, offloading, and RSMA power allocation, reporting higher energy efficiency than NOMA/FDMA and DRL baselines in a simulated low-altitude MEC system.

Reference graph

Works this paper leans on

41 extracted references · 35 canonical work pages · cited by 2 Pith papers

[17]

Enhancing deep reinforcement learning: A tutorial on generative diffusion models in network optimization,

H. Du, R. Zhang, Y . Liu, J. Wang, Y . Lin, Z. Li, D. Niyato, J. Kang, Z. Xiong, S. Cui et al. , “Enhancing deep reinforcement learning: A tutorial on generative diffusion models in network optimization,” IEEE Commun. Surveys Tutorials , vol. 26, no. 4, pp. 2611–2646, 2024

work page 2024
[1]

Optimal power allocation for rate splitting communications with deep reinforcement learning,

N. Q. Hieu, D. T. Hoang, D. Niyato, and D. I. Kim, “Optimal power allocation for rate splitting communications with deep reinforcement learning,” IEEE Wireless Commun. Lett., vol. 10, no. 12, pp. 2820–2823, 2021

work page 2021
[2]

UA V- assisted MEC system with mobile ground terminals: DRL-based joint terminal scheduling and UA V 3D trajectory design,

Y . Gao, X. Yuan, D. Yang, Y . Hu, Y . Cao, and A. Schmeink, “UA V- assisted MEC system with mobile ground terminals: DRL-based joint terminal scheduling and UA V 3D trajectory design,” IEEE Trans. V eh. Technol., vol. 73, no. 7, pp. 10 164–10 180, 2024

work page 2024
[3]

Full-duplex communication for ISAC: Joint beamforming and power optimization,

Z. He, W. Xu, H. Shen et al. , “Full-duplex communication for ISAC: Joint beamforming and power optimization,” IEEE J. Sel. Areas Com- mun., vol. 41, no. 9, pp. 2920–2936, 2023

work page 2023
[4]

Full-duplex cooperative NOMA with signal space diversity: Minimizing SIC operations,

A. Chauhan, A. Jaiswal, and C. Tellambura, “Full-duplex cooperative NOMA with signal space diversity: Minimizing SIC operations,” IEEE Wireless Commun. Lett. , 2024, doi: 10.1109/LWC.2024.3429236

work page arXiv 2024
[5]

A DRL approach for RIS-assisted full-duplex UL and DL transmission: Beamforming, phase shift and power optimization,

N. Nayak, S. Kalyani, and H. A. Suraweera, “A DRL approach for RIS-assisted full-duplex UL and DL transmission: Beamforming, phase shift and power optimization,” IEEE Trans. Wireless Commun. , vol. 23, no. 10, pp. 14 652–14 666, 2024

work page 2024
[6]

Conservative q-learning for offline reinforcement learning,

A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,” Adv. Neural Inf. Process. Syst. , vol. 33, pp. 1179–1191, 2020

work page 2020
[7]

An optimistic perspec- tive on offline reinforcement learning,

R. Agarwal, D. Schuurmans, and M. Norouzi, “An optimistic perspec- tive on offline reinforcement learning,” in International Conference on Machine Learning . PMLR, 2020, pp. 104–114

work page 2020

Show all 41 references

[8]

Deep-reinforcement-learning-based sustainable energy distribution for wireless communication,

G. Muhammad and M. S. Hossain, “Deep-reinforcement-learning-based sustainable energy distribution for wireless communication,” IEEE Wire- less Commun. , vol. 28, no. 6, pp. 42–48, 2021

2021
[9]

VWP: An efficient DRL- based autonomous driving model,

Y .-L. Jin, Z.-Y . Ji, D. Zeng, and X.-P. Zhang, “VWP: An efficient DRL- based autonomous driving model,” IEEE Trans. Multimedia, vol. 26, pp. 2096–2108, 2022

2022
[10]

Offline DRL for price-based demand response: Learning from suboptimal data and beyond,

T. Qian, Z. Liang, C. Shao, H. Zhang, Q. Hu, and Z. Wu, “Offline DRL for price-based demand response: Learning from suboptimal data and beyond,” IEEE Trans. Smart Grid , vol. 15, no. 5, pp. 4618–4635, 2024

2024
[11]

Provably efficient causal reinforcement learning with confounded observational data,

L. Wang, Z. Yang, and Z. Wang, “Provably efficient causal reinforcement learning with confounded observational data,” Adv. Neural Inf. Process. Syst., vol. 34, pp. 21 164–21 175, 2021

2021
[12]

Vrl3: A data-driven framework for visual deep reinforcement learning,

C. Wang, X. Luo, K. Ross, and D. Li, “Vrl3: A data-driven framework for visual deep reinforcement learning,” Adv. Neural Inf. Process. Syst. , vol. 35, pp. 32 974–32 988, 2022

2022
[13]

Optimizing deep reinforcement learning in data-scarce domains: A cross-domain evaluation of double DQN and dueling DQN,

N. Mohi Ud Din, A. Assad, S. Ul Sabha, and M. Rasool, “Optimizing deep reinforcement learning in data-scarce domains: A cross-domain evaluation of double DQN and dueling DQN,” International Journal of System Assurance Engineering and Management , pp. 1–12, 2024

2024
[14]

Structure-enhanced DRL for optimal transmission scheduling,

J. Chen, W. Liu, D. E. Quevedo, S. R. Khosravirad, Y . Li, and B. Vucetic, “Structure-enhanced DRL for optimal transmission scheduling,” IEEE Trans. Wireless Commun. , vol. 23, no. 1, pp. 379–393, 2023

2023
[15]

Understanding learned reward functions,

E. J. Michaud, A. Gleave, and S. Russell, “Understanding learned reward functions,” arXiv preprint arXiv:2012.05862 , 2020

2012 arXiv
[16]

Movement and communication co-design in multi-UA V enabled wireless systems via DRL,

Z. Lyu, C. Ren, and L. Qiu, “Movement and communication co-design in multi-UA V enabled wireless systems via DRL,” in 2020 IEEE 6th International Conference on Computer and Communications (ICCC) . IEEE, 2020, pp. 220–226

2020
[18]

Deep unsupervised learning using nonequilibrium thermodynamics,

J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International Conference on Machine Learning . PMLR, 2015, pp. 2256–2265

2015
[19]

Deep reinforcement learning-based resource allocation for D2D communications in hetero- geneous cellular networks,

Y . Zhi, J. Tian, X. Deng, J. Qiao, and D. Lu, “Deep reinforcement learning-based resource allocation for D2D communications in hetero- geneous cellular networks,” Digit. Commun. Netw. , vol. 8, no. 5, pp. 834–842, 2022

2022
[20]

Deep reinforcement learning-based resource allocation in cooperative UA V-assisted wireless networks,

P. Luong, F. Gagnon, L.-N. Tran, and F. Labeau, “Deep reinforcement learning-based resource allocation in cooperative UA V-assisted wireless networks,” IEEE Trans. Wireless Commun. , vol. 20, no. 11, pp. 7610– 7625, 2021

2021
[21]

Hybrid hierarchical DRL enabled resource allocation for secure transmission in multi-IRS-assisted sensing-enhanced spectrum sharing networks,

L. Wang, W. Wu, F. Zhou, Q. Wu, O. A. Dobre, and T. Q. Quek, “Hybrid hierarchical DRL enabled resource allocation for secure transmission in multi-IRS-assisted sensing-enhanced spectrum sharing networks,” IEEE Trans. Wireless Commun. , vol. 23, no. 6, pp. 6330–6346, 2024

2024
[22]

DRL-based energy-efficient resource allocation frameworks for uplink NOMA sys- tems,

X. Wang, Y . Zhang, R. Shen, Y . Xu, and F.-C. Zheng, “DRL-based energy-efficient resource allocation frameworks for uplink NOMA sys- tems,” IEEE Internet Things J. , vol. 7, no. 8, pp. 7279–7294, 2020

2020
[23]

DRL-driven dynamic resource allocation for task-oriented semantic communication,

H. Zhang, H. Wang, Y . Li, K. Long, and A. Nallanathan, “DRL-driven dynamic resource allocation for task-oriented semantic communication,” IEEE Trans. Commun. , vol. 71, no. 7, pp. 3992–4004, 2023

2023
[24]

Multi-agent DRL approach for energy-efficient resource allocation in URLLC-enabled grant-free NOMA systems,

D.-D. Tran, S. K. Sharma, V . N. Ha, S. Chatzinotas, and I. Woungang, “Multi-agent DRL approach for energy-efficient resource allocation in URLLC-enabled grant-free NOMA systems,” IEEE Open J. Commun. Soc., vol. 4, pp. 1470–1486, 2023

2023
[25]

Planning with diffu- sion for flexible behavior synthesis,

M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine, “Planning with diffu- sion for flexible behavior synthesis,” arXiv preprint arXiv:2205.09991 , 2022

2022 arXiv
[26]

Hybrid-generative diffusion models for attack-oriented twin migration in vehicular metaverses,

Y . Kang, J. Wen, J. Kang, T. Zhang, H. Du, D. Niyato, R. Yu, and S. Xie, “Hybrid-generative diffusion models for attack-oriented twin migration in vehicular metaverses,” arXiv preprint arXiv:2407.11036 , 2024

2024 arXiv
[27]

Diffusion-based reinforcement learning for edge-enabled AI-generated content services,

H. Du, Z. Li, D. Niyato, J. Kang, Z. Xiong, H. Huang, and S. Mao, “Diffusion-based reinforcement learning for edge-enabled AI-generated content services,” IEEE Trans. Mobile Comput., vol. 23, no. 9, pp. 8902– 8918, 2024

2024
[28]

Diffusiongpt: LLM-driven text-to-image generation system,

J. Qin, J. Wu, W. Chen, Y . Ren, H. Li, H. Wu, X. Xiao, R. Wang, and S. Wen, “Diffusiongpt: LLM-driven text-to-image generation system,” arXiv preprint arXiv:2401.10061 , 2024

2024
[29]

Multi- agent DRL for task offloading and resource allocation in multi-UA V enabled iot edge network,

A. M. Seid, G. O. Boateng, B. Mareri, G. Sun, and W. Jiang, “Multi- agent DRL for task offloading and resource allocation in multi-UA V enabled iot edge network,” IEEE Trans. Netw. Service Manag. , vol. 18, no. 4, pp. 4531–4547, 2021

2021
[30]

DRL-based partial offloading for maximizing sum computation rate of wireless pow- ered mobile edge computing network,

S. Zhang, H. Gu, K. Chi, L. Huang, K. Yu, and S. Mumtaz, “DRL-based partial offloading for maximizing sum computation rate of wireless pow- ered mobile edge computing network,” IEEE Trans. Wireless Commun. , vol. 21, no. 12, pp. 10 934–10 948, 2022

2022
[31]

Delay-oriented scheduling in 5G downlink wireless networks based on reinforcement learning with partial observations,

Y . Hao, F. Li, C. Zhao, and S. Yang, “Delay-oriented scheduling in 5G downlink wireless networks based on reinforcement learning with partial observations,” IEEE ACM Trans. Netw. , vol. 31, no. 1, pp. 380– 394, 2022

2022
[32]

D- RAN: A DRL-based demand-driven elastic user-centric RAN optimiza- tion for 6G & beyond,

S. K. Kasi, U. S. Hashmi, S. Ekin, A. Abu-Dayya, and A. Imran, “D- RAN: A DRL-based demand-driven elastic user-centric RAN optimiza- tion for 6G & beyond,” IEEE Trans. Cogn. Commun. Netw., vol. 9, no. 1, pp. 130–145, 2022

2022
[33]

QoS- DRAMA: Quality of service aware drl-based adaptive mid-level resource allocation scheme,

E. Swistak, M. Roshdi, R. German, and M. Harounabadi, “QoS- DRAMA: Quality of service aware drl-based adaptive mid-level resource allocation scheme,” in IEEE INFOCOM 2024-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) . IEEE, 2024, pp. 1–6

2024
[34]

Self-organizing mmwave MIMO cell-free networks with hybrid beamforming: A hierarchical DRL-based design,

Y . Al-Eryani and E. Hossain, “Self-organizing mmwave MIMO cell-free networks with hybrid beamforming: A hierarchical DRL-based design,” IEEE Trans. Commun. , vol. 70, no. 5, pp. 3169–3185, 2022

2022
[35]

Reinforcement learning based down- link OFDMA scheduling for time-sensitive wifi networks,

J. He, B. Tan, and Y . Gao, “Reinforcement learning based down- link OFDMA scheduling for time-sensitive wifi networks,” in 2023 IEEE/CIC International Conference on Communications in China (ICCC). IEEE, 2023, pp. 1–6

2023
[36]

DRL-based dynamic resource allocation for multi-beam satellite systems,

R. Chai, G. Yang, L. Liu, and Q. Chen, “DRL-based dynamic resource allocation for multi-beam satellite systems,” IEEE Trans. Netw. Serv. Manag., vol. 21, no. 4, pp. 3829–3845, 2024

2024
[37]

Denoising diffusion implicit models,

J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502 , 2020

2010 arXiv
[38]

Attention- based QoE-aware digital twin empowered edge computing for immersive virtual reality,

J. Yu, A. Alhilal, T. Zhou, P. Hui, and D. H. Tsang, “Attention- based QoE-aware digital twin empowered edge computing for immersive virtual reality,” IEEE Trans. Wireless Commun. , vol. 23, no. 9, pp. 11 276–11 290, 2024

2024
[39]

Intelligent resource allocation for edge-cloud collaborative networks: A hybrid DDPG-D3QN approach,

H. Hu, D. Wu, F. Zhou, X. Zhu, R. Q. Hu, and H. Zhu, “Intelligent resource allocation for edge-cloud collaborative networks: A hybrid DDPG-D3QN approach,” IEEE Trans. V eh. Technol. , vol. 72, no. 8, pp. 10 696–10 709, 2023

2023
[40]

Utility-oriented optimization for video streaming in UA V-aided MEC network: A DRL approach,

J. Miao, S. Bai, S. Mumtaz, Q. Zhang, and J. Mu, “Utility-oriented optimization for video streaming in UA V-aided MEC network: A DRL approach,” IEEE Trans. Green Commun. Netw. , vol. 8, no. 2, pp. 878– 889, 2024

2024
[41]

Federated prompt- based decision transformer for customized VR services in mobile edge computing system,

T. Zhou, J. Yu, J. Zhang, and D. H. Tsang, “Federated prompt- based decision transformer for customized VR services in mobile edge computing system,” arXiv preprint arXiv:2402.09729 , 2024

2024 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [17]

Enhancing deep reinforcement learning: A tutorial on generative diffusion models in network optimization,

H. Du, R. Zhang, Y . Liu, J. Wang, Y . Lin, Z. Li, D. Niyato, J. Kang, Z. Xiong, S. Cui et al. , “Enhancing deep reinforcement learning: A tutorial on generative diffusion models in network optimization,” IEEE Commun. Surveys Tutorials , vol. 26, no. 4, pp. 2611–2646, 2024

work page 2024

[2] [1]

Optimal power allocation for rate splitting communications with deep reinforcement learning,

N. Q. Hieu, D. T. Hoang, D. Niyato, and D. I. Kim, “Optimal power allocation for rate splitting communications with deep reinforcement learning,” IEEE Wireless Commun. Lett., vol. 10, no. 12, pp. 2820–2823, 2021

work page 2021

[3] [2]

UA V- assisted MEC system with mobile ground terminals: DRL-based joint terminal scheduling and UA V 3D trajectory design,

Y . Gao, X. Yuan, D. Yang, Y . Hu, Y . Cao, and A. Schmeink, “UA V- assisted MEC system with mobile ground terminals: DRL-based joint terminal scheduling and UA V 3D trajectory design,” IEEE Trans. V eh. Technol., vol. 73, no. 7, pp. 10 164–10 180, 2024

work page 2024

[4] [3]

Full-duplex communication for ISAC: Joint beamforming and power optimization,

Z. He, W. Xu, H. Shen et al. , “Full-duplex communication for ISAC: Joint beamforming and power optimization,” IEEE J. Sel. Areas Com- mun., vol. 41, no. 9, pp. 2920–2936, 2023

work page 2023

[5] [4]

Full-duplex cooperative NOMA with signal space diversity: Minimizing SIC operations,

A. Chauhan, A. Jaiswal, and C. Tellambura, “Full-duplex cooperative NOMA with signal space diversity: Minimizing SIC operations,” IEEE Wireless Commun. Lett. , 2024, doi: 10.1109/LWC.2024.3429236

work page arXiv 2024

[6] [5]

A DRL approach for RIS-assisted full-duplex UL and DL transmission: Beamforming, phase shift and power optimization,

N. Nayak, S. Kalyani, and H. A. Suraweera, “A DRL approach for RIS-assisted full-duplex UL and DL transmission: Beamforming, phase shift and power optimization,” IEEE Trans. Wireless Commun. , vol. 23, no. 10, pp. 14 652–14 666, 2024

work page 2024

[7] [6]

Conservative q-learning for offline reinforcement learning,

A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,” Adv. Neural Inf. Process. Syst. , vol. 33, pp. 1179–1191, 2020

work page 2020

[8] [7]

An optimistic perspec- tive on offline reinforcement learning,

R. Agarwal, D. Schuurmans, and M. Norouzi, “An optimistic perspec- tive on offline reinforcement learning,” in International Conference on Machine Learning . PMLR, 2020, pp. 104–114

work page 2020