Resilient UAV Trajectory Planning via Few-Shot Meta-Offline Reinforcement Learning

Eslam Eldeeb; Hirley Alves

REVIEW 6 major objections 8 minor 45 references

Resilient UAV Trajectory Planning via Few-Shot Meta-Offline Reinforcement Learning

T0 review · 6 major / 8 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A meta-offline RL algorithm adapts UAV trajectory planning to new environments in under 40 training epochs, using only offline data.

desk verdict A plausible CQL+MAML integration for offline UAV trajectory optimization, but the printed reward is non-causal and the optimality claims outrun the evidence. read the letter →

arxiv 2502.01268 v1 pith:K7RXKDLA submitted 2025-02-03 cs.RO cs.AI

Eslam Eldeeb , Hirley Alves This is my paper · ORCID

classification cs.ROcs.AI

keywords age-of-informationmeta-learningofflinereinforcementlearningpreciseagricultureresilienceunmannedaerialvehiclesconservativeQ-learningfew-shot

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a few-shot meta-offline reinforcement learning algorithm that combines conservative Q-learning (CQL) with model-agnostic meta-learning (MAML) to plan a UAV's trajectory and scheduling policy from a static offline dataset, without any online interaction with the environment. The authors aim to show that meta-learning provides an initialization that lets CQL adapt to a new environment, defined by a new trade-off weight $\lambda$ between age-of-information (AoI) and transmission power, in under 40 training epochs, whereas plain CQL needs more than 100 epochs. They report that the method is the only offline scheme that reaches a joint optimum in their tests, achieving AoI of 6 and transmission power of 1.6 mW versus AoI of 28 and 6 mW for CQL, and that it also makes the UAV avoid a sudden heavy-rain outage area with near-zero outages after about 100 epochs. If correct, this would make offline-trained RL practical for UAV networks in smart agriculture and other settings where online training is costly, unsafe, or impossible.

What carries the argument

The load-bearing mechanism is the meta-learned initial weight vector $w_0$ of the Q-network, produced by an outer loop that minimizes the sum of CQL losses over meta-training tasks after each task's inner-loop CQL updates on support samples. Inside each task, CQL's conservative regularization $\alpha \hat{E}[\log \sum_{\tilde a} \exp(Q(s,\tilde a)) - Q(s,a)]$ prevents out-of-distribution actions from being overestimated, which is what makes offline training possible at all; the meta-loop then turns that offline learner into a fast adapter by finding an initialization from which a few gradient steps on a new task's data suffice. The same mechanism is what the paper credits for resilience: the initialization encodes how to balance AoI and power and where safe regions are, so a new task with an outage zone can be learned quickly.

What would settle it

Run Algorithm 4 exactly as written with the reward in Eq. (6) and check whether the temporal-difference loss can even be computed at each step; if the algorithm does not converge to the reported AoI of 6 and power of 1.6 mW at $\lambda = 300$, the claimed results cannot come from the described reward. Alternatively, with the intended per-step reward, compare CQL-MAML against CQL over many random seeds and verify that the meta-learned initialization converges in under 40 epochs while CQL needs more than 100; a single seed or a different initialization should not flip the gap.

Watch

Extended reading notes

Core claim

The central claim is that CQL-MAML, an algorithm that pretrains a Q-network's initial weights with MAML over several tasks (each with a different $\lambda$) and then runs a few CQL updates on a new task's offline support set, converges to the optimal policy where standard offline and online baselines do not. In the authors' experiments, the meta-learned initialization converges in fewer than 40 epochs on a new unseen task, while CQL with random initialization requires more than 100 epochs, and DQN and DQN-MAML fail entirely because of distributional shift. On the $\lambda = 300$ test, the converged policy achieves an average AoI of 6 and transmission power of 1.6 mW, compared with AoI of 28 and power of 6 mW for CQL and AoI of 7 and 2.2 mW for an online DQN agent. The authors further claim that after meta-training on outage-free tasks, the same initialization lets the UAV adapt to five unseen tasks with unpredictable outage regions, avoiding the outage zone almost completely after about 100 training epochs, whereas CQL continues to incur roughly 20 outages. The paper presents this as the first combination of meta-learning and offline RL in the wireless domain.

Load-bearing premise

The load-bearing premise is that the reward in Eq. (6) is a typo: as printed, it assigns the full episode cost to every time step, which is non-causal and would make the temporal-difference targets in Eqs. (8) and (9) ill-defined, so the reported results depend on the unstated assumption that the implementation used a per-step reward depending only on the current state and action.

Editorial extensions

If this is right

Offline datasets from prior UAV operations can replace online interaction for trajectory and scheduling optimization, removing safety and cost barriers.
When the service objective changes (a new $\lambda$), the policy can be adapted from a few hundred offline data points in well under 100 epochs, rather than retraining from scratch.
The meta-initialized policy can learn to avoid sudden outage regions (e.g., heavy rain) without explicit online exploration, converging to near-zero outages after about 100 epochs.
Larger offline datasets and more meta-training tasks both speed convergence, and even two meta-tasks beat random-initialization CQL.
The same framework is positioned to extend beyond agriculture to other UAV verticals such as disaster response, where online training is often infeasible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the printed reward in Eq. (6) is a typo and the implementation actually uses a per-step reward, then the paper's headline result is really about that causal variant, and an explicit per-step reward would strengthen reproducibility.
The meta-initialization appears to encode a general notion of moving toward devices with stale AoI while avoiding high transmission power, which could transfer to different numbers of devices or different channel models than those seen in meta-training—an empirical question the paper does not test directly.
The resilience result suggests the Q-network learns a spatial risk map of outage areas during meta-training; a testable extension would be to place the outage region at an entirely new location and measure how quickly the meta-initialized policy avoids it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the short version: the paper is a straightforward integration of CQL and MAML, applied to UAV trajectory/scheduling for AoI and power minimization. The results are plausibly real, but the manuscript has one genuinely load-bearing presentation error—the printed reward in Eq. (6) is non-causal—and a few smaller sloppiness items. I'd send it to review, but only after the authors fix the reward definition and release code or a detailed implementation description.

What's new: CQL-MAML for offline UAV trajectory optimization with few-shot adaptation. That is new in this specific wireless setting, though not conceptually surprising: MAML finds initial weights, CQL trains offline. The value is in the demonstration. The paper does a few things well: it defines tasks via the λ trade-off, tests dataset size and number of meta-tasks, and includes a resilience experiment with a shifting outage area. The reported convergence advantage (under 40 epochs vs. more than 100 for CQL) is consistent with the expected effect of a good initialization, and the outage-avoidance result is a sensible empirical claim.

Now the soft spots. Eq. (6) defines r(t) as the full-episode average cost repeated at every time step. If taken literally, the TD target in Eqs. (8)–(9) contains future AoI and power values, so the Bellman update is invalid and the learned Q-function cannot represent the causal objective. The simulations in Figs. 5–9 could only have been produced by a per-step reward, probably r(t) = -Σ_k[δ_k A_k(t) + (λ/K)P_k(t)], which is not what the paper says. That is not cosmetic; every number depends on which reward was coded. The dataset constraint (5c) is printed as |D| ≤ Dconst while the text says the dataset should be at least Dconst; one of them is wrong. There are no error bars or multiple seeds, so I cannot tell how robust the 40-epoch claim is. And “optimal” is used loosely—there is no optimality certificate or comparison to an exhaustive or optimal baseline.

The self-citation pattern does not bother me much; the authors' prior offline-RL and meta-RL papers are the natural building blocks, and the new content is the comparison itself, not a derivation from thin air.

Bottom line: this is a plausible incremental contribution for the wireless-RL community. With Eq. (6) fixed, the constraint clarified, and code/data released, it could be a solid workshop or journal paper. As posted, it should not be accepted without correction. I would still send it to peer review—a good referee will catch exactly these issues, and the direction is worth engaging.

Referee Report

6 major / 8 minor

Summary. The paper proposes CQL-MAML, a combination of conservative Q-learning (CQL) with model-agnostic meta-learning (MAML), for offline few-shot training of a UAV trajectory and scheduling policy that minimizes the age-of-information (AoI) and transmission power of IoT devices. The authors formulate a joint optimization problem P1, define an MDP with state/action/reward, and present Algorithms 1-4 for DQN, CQL, MAML, and the proposed meta-offline method. Numerical experiments in Section V compare CQL-MAML against CQL, DQN, and DQN-MAML in adaptive-objective and resilience (rain outage) scenarios, reporting faster convergence, lower AoI and power, and outage avoidance.

Significance. If the reported results are reproducible, the combination of offline RL with MAML is a practical contribution to wireless UAV trajectory optimization, because it addresses two real concerns: avoiding online interaction and adapting to new tasks or channel disruptions with limited data. The paper also evaluates a concrete resilience scenario (changing rain-outage areas), which is valuable for deployments in smart agriculture. The work is empirical; there is no theoretical optimality proof, and the 'optimal' claims are not backed by certificates. Nonetheless, the proposed architecture and the resilience experiment are of interest to the community.

major comments (6)

[§III-A, Eq. (6)] The reward in Eq. (6) is non-causal as written: r(t) is defined as the negative of (1/T) times the sum over all time steps t' of the AoI and power costs, so the same full-episode cost is assigned to every time step. Substituting this r(t) into the DQN loss (8) or the CQL loss (9) makes the TD target r + γ max_a' Q(s',a') depend on future states' AoI and power values that are unavailable at decision time t. Thus the Bellman bootstrap is invalid and the simulations in Figs. 5-9 could not have been produced by the described algorithm unless the implementation used a per-step reward such as r(t) = -Σ_k [δ_k A_k(t) + (λ/K) P_k(t)]. The authors must correct Eq. (6), state precisely which reward was used in the code, and provide code or a sufficiently detailed implementation description to verify causality of the TD updates.
[§II-A, Eq. (5c)] Constraint (5c) is written |D| ≤ Dconst, while the sentence immediately below it says the constraint bounds the offline dataset size to be 'greater than or equal to a certain threshold size Dconst'. These directions are contradictory. Moreover, (5c) does not constrain the optimization variables v(t) and w(t), so it is not a constraint of P1 in the usual sense. Please correct the inequality direction and clarify whether this is a problem constraint or a data-availability assumption.
[§V-B, Figs. 5-8] The central quantitative claims—CQL-MAML converges in fewer than 40 epochs, achieves AoI=6 and power=1.6 mW versus AoI=28 and power=6 mW for CQL, and avoids outages—are presented without error bars, multiple seeds, or confidence intervals. DQN/CQL training is stochastic, and the figures appear to show single runs (or a single aggregation). At least 5-10 independent training runs with mean and standard deviation should be reported before these comparative claims can be supported.
[§V-B and §VI] The abstract and conclusion claim that the algorithm 'achieves optimal joint AoI and transmission power' and is 'the only algorithm' that does so. No optimality certificate, lower bound, exhaustive search, or comparison against a known optimal policy is provided; the experiments only compare against DQN, DQN-MAML, and CQL baselines. These statements should be softened to claims of superiority over the considered baselines unless an optimal benchmark is added.
[§I-D and §IV] The contribution list in Section I-D describes a 'meta-offline MARL framework,' and the abstract uses 'few-shot meta-offline RL,' but Algorithm 4 and the experiments describe a single-agent setup with one UAV and no multi-agent interactions. If MARL is intended, the formulation in Section II and the algorithm in Section IV are missing the multi-agent components (agent index, joint actions, shared/private rewards, or communication). The terminology should be corrected to avoid overclaiming novelty relative to prior multi-agent offline RL works.
[§IV, Algorithm 4] The 'few-shot' aspect is never made precise: Algorithm 4 samples 'k shots' in the support set, but the value of k is not specified in Table I or the experiments, and Section V uses dataset sizes of 100, 300, and 500 experiences, which are not the typical 'few-shot' regime in meta-learning benchmarks. The authors should state the exact k for support and query sets and discuss why the reported dataset sizes constitute 'few-shot' offline learning.

minor comments (8)

[Throughout] The term 'UA V' is inconsistently spaced throughout the manuscript; it should be 'UAV' for uniformity.
[§I-A] In the introduction, 'receives an award' should be 'receives a reward'; the same typo appears in the abstract and in Algorithm 1 comments.
[§IV, Algorithm 4] Algorithm 4 line 12 says 'Return model converged initial weights w0'; this should read 'Return the converged initial weights w0.'
[§V] The name 'meta-CQL' is used interchangeably with 'CQL-MAML' in Section V; consistency in naming would improve clarity.
[§V.A] The neural network architecture is described only as '2 hidden layers'; the number of neurons per layer, activation functions, batch size, and exploration schedule (epsilon decay) are not given, making reproduction difficult.
[Eq. (3)] The noise power is denoted σ2 in Eq. (3) and in the text, but Table I gives σ2 = -100 dBm; the units and the symbol should be made consistent (e.g., sigma^2 vs sigma).
[References] Reference [3] is incomplete: the standard citation for Sutton and Barto's book lacks the publisher and year; please complete all bibliographic entries.
[§V.C] 'monte-Carlo loops' should be capitalized as 'Monte Carlo loops' and the sentence should be completed with a period.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the convergence and AoI/power claims are empirical measurements, and Eq. (6) is a correctness issue, not a circular reduction.

full rationale

I find no circularity. The paper's central claims that CQL-MAML converges in under 40 epochs, reaches AoI=6 and power=1.6 mW at lambda=300, and avoids outage areas are empirical results obtained by running Algorithm 4 against DQN, CQL, DQN-MAML, and an online DQN; they are measured simulation outcomes, not quantities derived algebraically from the problem definition. The CQL and MAML components are standard external algorithms (Kumar et al. 2020; Finn et al. 2017), and the meta-testing protocol trains on tasks with different lambda values or outage-free conditions before testing on unseen lambda/outage tasks, so the fast-adaptation and resilience results are not forced by construction. The self-citations (e.g., [6], [22], [32], [33], [35], [36], [38], [44]) are related-work and baseline references; none supplies a uniqueness theorem or an ansatz that the paper then treats as a prediction. The one substantive flaw is Eq. (6), where r(t) is printed as the full-episode normalized cost repeated at every time step, which makes the TD targets in (8)-(9) non-causal; that is a correctness/reproducibility defect (likely a typo for a per-step reward), not a circular step, because it does not make the reported outputs equivalent to the inputs by construction, it instead means the algorithm as written could not produce the reported numbers as stated.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities. The method rests on standard RL machinery (CQL, MAML, DQN) plus hand-chosen hyperparameters. The most fragile element is the reward implementation: the printed Eq. (6) is non-causal, and the reported results depend on a per-step interpretation. All performance claims are empirical, with no theoretical guarantees.

free parameters (6)

CQL conservative parameter α = 1
Chosen by hand in Table I; controls the strength of the OOD penalty in Eq. (9). The reported results depend on this value and no sensitivity analysis is provided.
Inner learning rate η_inner = 10^-2
Chosen in Table I for the MAML inner update in Eq. (13); affects the few-shot adaptation speed.
Outer learning rate η_outer = 10^-3
Chosen in Table I for the MAML outer update in Eq. (15); affects meta-initialization quality.
Objective trade-off weight λ = 300 (Fig. 8); task distribution over multiple λ values
Defines each meta-task in Eq. (5a). The final evaluation uses λ=300, and the claim of optimality is for that specific weight.
Offline dataset size |D| = 500 (default); varied {100, 300, 500}
The 'few-shot' claim is demonstrated with 100-500 transitions; the results in Fig. 6 depend on this size.
Number of meta-training tasks = 8 (default); varied {2, 4, 6, 8, 10}
Fig. 7 shows that at least 8 tasks are needed to surpass online DQN, so the default choice is load-bearing.

assumptions (6)

domain assumption The device-UAV link is always line-of-sight with channel gain g_k,u(t) = g0 / (h^2 + ||l_k - l_u(t)||^2).
Invoked in Eq. (2); ignores shadowing, small-scale fading, and any non-LoS links, which is optimistic for the claimed real-world precision agriculture scenario.
domain assumption The offline dataset D is collected from an online DQN agent's replay buffer and is treated as a fixed static dataset for offline RL.
Section V-A states datasets are sampled from the replay buffer of an online DQN agent; this assumes the buffer contents are representative and diverse enough for CQL to train without the original behavioral policy.
ad hoc to paper The reward r(t) in Eq. (6) is interpreted as a per-step reward despite the printed equation summing over the entire episode.
As printed, r(t) equals the negative total episode cost divided by T, which would make the reward non-causal and the TD target in Eqs. (8)-(9) inconsistent. The reported results are only reproducible if the equation is a typo for a per-step reward.
domain assumption A task distribution over λ values shares enough structure that MAML can learn initial weights that transfer to a new λ.
The MAML formulation in Section IV assumes tasks with different λ are related; no analysis is given for when this assumption breaks.
standard math Standard Q-learning convergence assumptions apply (function approximation with neural networks, experience replay).
The DQN background in Section III-A assumes the usual deep RL setup; no convergence guarantees are established for the combined CQL-MAML loss.
domain assumption The rain attenuation model with R=12.5 mm/h and fitting parameters from [42] adequately captures link failures.
Section II adopts the ITU-R model; the simulation treats outage as entering a static 'heavy rain area', which may not reflect the 'quickly changes' claim in Section V-C.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Resilient UAV Trajectory Planning via Few-Shot Meta-Offline Reinforcement Learning." pith.science (2026). https://pith.science/paper/K7RXKDLA

@misc{pith2026250201268,
  author       = {Pith},
  title        = {Pith review of: Resilient UAV Trajectory Planning via Few-Shot Meta-Offline Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K7RXKDLA}},
  note         = {Machine review of arXiv:2502.01268}
}

read the original abstract

Reinforcement learning (RL) has been a promising essence in future 5G-beyond and 6G systems. Its main advantage lies in its robust model-free decision-making in complex and large-dimension wireless environments. However, most existing RL frameworks rely on online interaction with the environment, which might not be feasible due to safety and cost concerns. Another problem with online RL is the lack of scalability of the designed algorithm with dynamic or new environments. This work proposes a novel, resilient, few-shot meta-offline RL algorithm combining offline RL using conservative Q-learning (CQL) and meta-learning using model-agnostic meta-learning (MAML). The proposed algorithm can train RL models using static offline datasets without any online interaction with the environments. In addition, with the aid of MAML, the proposed model can be scaled up to new unseen environments. We showcase the proposed algorithm for optimizing an unmanned aerial vehicle (UAV) 's trajectory and scheduling policy to minimize the age-of-information (AoI) and transmission power of limited-power devices. Numerical results show that the proposed few-shot meta-offline RL algorithm converges faster than baseline schemes, such as deep Q-networks and CQL. In addition, it is the only algorithm that can achieve optimal joint AoI and transmission power using an offline dataset with few shots of data points and is resilient to network failures due to unprecedented environmental changes.

Figures

Figures reproduced from arXiv: 2502.01268 by the authors.

**Figure 2.** Illustration of Offline RL, which involves two phases: data collection [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Illustration of the proposed CQL-MAML algorithm, composed of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: An illustration of the meta-training performance of the proposed CQL [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: The meta-testing convergence of the proposed CQL-MAML algorithm [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: A visualization of the effect of the size of the offline dataset [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 8.** Figure 8: The AoI and transmission power achieved after meta-testing using [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

**Figure 9.** Figure 9: Performance evaluation of the proposed algorithm in detecting outages [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 37 canonical work pages

[1]

Machine learning for large-scale optimization in 6G wireless networks,

Y . Shi, L. Lian, Y . Shi, Z. Wang, Y . Zhou, L. Fu, L. Bai, J. Zhang, and W. Zhang, “Machine learning for large-scale optimization in 6G wireless networks,” IEEE Communications Surveys & Tutorials , vol. 25, no. 4, pp. 2088–2132, 2023

2023
[2]

Artificial intelligence enabled wireless networking for 5G and beyond: Recent advances and future challenges,

C.-X. Wang, M. Di Renzo, S. Stanczak, S. Wang, and E. G. Larsson, “Artificial intelligence enabled wireless networking for 5G and beyond: Recent advances and future challenges,” IEEE Wireless Communica- tions, vol. 27, no. 1, pp. 16–23, 2020

work page 2020
[3]

Reinforcement learning: An introduction,

R. S. Sutton, “Reinforcement learning: An introduction,” A Bradford Book, 2018

2018
[4]

Self- organization in small cell networks: A reinforcement learning approach,

M. Bennis, S. M. Perlaza, P. Blasco, Z. Han, and H. V . Poor, “Self- organization in small cell networks: A reinforcement learning approach,” IEEE Transactions on Wireless Communications , vol. 12, no. 7, pp. 3202–3212, 2013

work page 2013
[5]

Deep reinforcement learning for resource management in network slicing,

R. Li, Z. Zhao, Q. Sun, C.-L. I, C. Yang, X. Chen, M. Zhao, and H. Zhang, “Deep reinforcement learning for resource management in network slicing,” IEEE Access, vol. 6, pp. 74 429–74 441, 2018

work page 2018
[6]

Multi-UA V path learning for age and power optimization in IoT with UA V battery recharge,

E. Eldeeb, J. M. de Souza Sant’Ana, D. E. P ´erez, M. Shehab, N. H. Mahmood, and H. Alves, “Multi-UA V path learning for age and power optimization in IoT with UA V battery recharge,” IEEE Transactions on Vehicular Technology, vol. 72, no. 4, pp. 5356–5360, 2022

work page 2022
[7]

Multi-agent deep reinforcement learning to manage connected autonomous vehicles at tomorrow’s inter- sections,

G.-P. Antonio and C. Maria-Dolores, “Multi-agent deep reinforcement learning to manage connected autonomous vehicles at tomorrow’s inter- sections,” IEEE Transactions on Vehicular Technology , vol. 71, no. 7, pp. 7033–7043, 2022

work page 2022
[8]

Applications of deep reinforcement learning in communications and networking: A survey,

N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang, Y .-C. Liang, and D. I. Kim, “Applications of deep reinforcement learning in communications and networking: A survey,” IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3133–3174, 2019

work page 2019

Show all 45 references

[9]

Deep reinforcement learning for internet of things: A comprehensive survey,

W. Chen, X. Qiu, T. Cai, H.-N. Dai, Z. Zheng, and Y . Zhang, “Deep reinforcement learning for internet of things: A comprehensive survey,” IEEE Communications Surveys & Tutorials , vol. 23, no. 3, pp. 1659– 1692, 2021

2021
[10]

Human-level control through deep reinforce- ment learning,

V . Mnih, K. Kavukcuoglu, D. Silver, A. Rusu, J. Veness, M. Bellemare, A. Graves, M. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforc...

2015
[11]

Exploring the YOLO-FT deep learning algorithm for UA V- based smart agriculture detection in communication networks,

B. Cui, L. Liang, B. Ji, L. Zhang, L. Zhao, K. Zhang, F. Shi, and J.-C. Cr ˜A©put, “Exploring the YOLO-FT deep learning algorithm for UA V- based smart agriculture detection in communication networks,” IEEE Transactions on Network and Service Management , vol. 21, no. 5, pp. 5...

2024
[12]

Unmanned aerial vehicles in smart agriculture: Applications, requirements, and challenges,

P. K. Reddy Maddikunta, S. Hakak, M. Alazab, S. Bhattacharya, T. R. Gadekallu, W. Z. Khan, and Q.-V . Pham, “Unmanned aerial vehicles in smart agriculture: Applications, requirements, and challenges,” IEEE Sensors Journal, vol. 21, no. 16, pp. 17 608–17 619, 2021

2021
[13]

Leveraging precision agriculture techniques using UA Vs and emerging disruptive technologies,

M. Raj, H. N B, S. Gupta, M. Atiquzzaman, O. Rawlley, and L. Goel, “Leveraging precision agriculture techniques using UA Vs and emerging disruptive technologies,” Energy Nexus, vol. 14, p. 100300, 2024

2024
[14]

AoI- Aware Energy-Efficient SFC in UA V-Aided Smart Agriculture Using Asynchronous Federated Learning,

M. Akbari, A. Syed, W. S. Kennedy, and M. Erol-Kantarci, “AoI- Aware Energy-Efficient SFC in UA V-Aided Smart Agriculture Using Asynchronous Federated Learning,” IEEE Open Journal of the Commu- nications Society, vol. 5, pp. 1222–1242, 2024

2024
[15]

Crop yield prediction using deep reinforcement learning model for sustainable agrarian applications,

D. Elavarasan and P. M. D. Vincent, “Crop yield prediction using deep reinforcement learning model for sustainable agrarian applications,” IEEE Access, vol. 8, pp. 86 886–86 901, 2020

2020
[16]

IoT-aerial base station task offloading with risk-sensitive reinforcement learning for smart agriculture,

T. Pamuklu, A. C. Nguyen, A. Syed, W. S. Kennedy, and M. Erol- Kantarci, “IoT-aerial base station task offloading with risk-sensitive reinforcement learning for smart agriculture,” IEEE Transactions on Green Communications and Networking , vol. 7, no. 1, 2023

2023
[17]

Offline reinforcement learning: Tutorial, review, and perspectives on open problems,

S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” arXiv preprint arXiv:2005.01643, 2020

2005 arXiv
[18]

Distributed learning methodologies for massive machine type commu- nication,

M. V . da Silva, E. Eldeeb, M. Shehab, H. Alves, and R. D. Souza, “Distributed learning methodologies for massive machine type commu- nication,” IEEE Internet of Things Magazine , vol. 8, no. 1, 2025

2025
[19]

Comeback kid: Resilience for mixed-critical wireless network resource management,

R.-J. Reifert, S. Roth, A. A. Ahmad, and A. Sezgin, “Comeback kid: Resilience for mixed-critical wireless network resource management,” IEEE Transactions on Vehicular Technology, vol. 72, no. 12, pp. 16 177– 16 194, 2023

2023
[20]

Edge-IoT-UA V Adaptation To- ward Precision Agriculture Using 3D-LiDAR Point Clouds,

A. K. Sangaiah, J. Anandakrishnan, V . Meenakshisundaram, M. A. A. Rahman, P. Arumugam, and M. Das, “Edge-IoT-UA V Adaptation To- ward Precision Agriculture Using 3D-LiDAR Point Clouds,” IEEE Internet of Things Magazine , vol. 8, no. 1, pp. 19–25, 2025

2025
[21]

Conservative Q-learning for offline reinforcement learning,

A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative Q-learning for offline reinforcement learning,” in NeurIPS, vol. 33, 2020

2020
[22]

Offline and distributional reinforcement learn- ing for wireless communications,

E. Eldeeb and H. Alves, “Offline and distributional reinforcement learn- ing for wireless communications,” IEEE Communications Magazine, pp. 1–7, 2025

2025
[23]

Model-agnostic meta-learning for fast adaptation of deep networks,

C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proceedings of the 34th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, vol. 70. PMLR, 06–11 Aug 2017

2017
[24]

3D UA V trajectory and data collection optimisation via deep reinforcement learning,

K. K. Nguyen, T. Q. Duong, T. Do-Duy, H. Claussen, and L. Hanzo, “3D UA V trajectory and data collection optimisation via deep reinforcement learning,” IEEE Transactions on Communications , vol. 70, no. 4, pp. 2358–2371, 2022

2022
[25]

Deep reinforce- ment learning based resource allocation and trajectory planning in inte- grated sensing and communications UA V network,

Y . Qin, Z. Zhang, X. Li, W. Huangfu, and H. Zhang, “Deep reinforce- ment learning based resource allocation and trajectory planning in inte- grated sensing and communications UA V network,” IEEE Transactions on Wireless Communications, vol. 22, no. 11, pp. 8158–8169, 2023

2023
[26]

Deep reinforcement learning and NOMA-based multi-objective RIS-assisted IS-UA V-TNs: Trajectory optimization and beamforming design,

K. Guo, M. Wu, X. Li, H. Song, and N. Kumar, “Deep reinforcement learning and NOMA-based multi-objective RIS-assisted IS-UA V-TNs: Trajectory optimization and beamforming design,” IEEE Transactions on Intelligent Transportation Systems , vol. 24, no. 9, 2023

2023
[27]

Meta-learning to communicate: Fast end-to-end training for fading channels,

S. Park, O. Simeone, and J. Kang, “Meta-learning to communicate: Fast end-to-end training for fading channels,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 5075–5079

2020
[28]

Transfer learning and meta learning-based fast downlink beamforming adapta- tion,

Y . Yuan, G. Zheng, K.-K. Wong, B. Ottersten, and Z.-Q. Luo, “Transfer learning and meta learning-based fast downlink beamforming adapta- tion,” IEEE Transactions on Wireless Communications , vol. 20, no. 3, 2021

2021
[29]

MetaGraphLoc: A graph-based meta-learning scheme for indoor localization via sensor fusion,

Y . Etiabi, E. Eldeeb, M. Shehab, W. Njima, H. Alves, M.-S. Alouini, and E. M. Amhoud, “MetaGraphLoc: A graph-based meta-learning scheme for indoor localization via sensor fusion,” 2024. [Online]. Available: https://arxiv.org/abs/2411.17781

2024 arXiv
[30]

Meta-reinforcement learning based resource allocation for dynamic V2X communications,

Y . Yuan, G. Zheng, K.-K. Wong, and K. B. Letaief, “Meta-reinforcement learning based resource allocation for dynamic V2X communications,” IEEE Transactions on Vehicular Technology, vol. 70, no. 9, 2021

2021
[31]

Distributed multi- agent meta learning for trajectory design in wireless drone networks,

Y . Hu, M. Chen, W. Saad, H. V . Poor, and S. Cui, “Distributed multi- agent meta learning for trajectory design in wireless drone networks,” IEEE Journal on Selected Areas in Communications , vol. 39, no. 10, pp. 3177–3192, 2021

2021
[32]

Age and power minimization via meta-deep reinforcement learning in UA V networks,

S. Sarathchandra, E. Eldeeb, M. Shehab, H. Alves, K. Mikhaylov, and M.-S. Alouini, “Age and power minimization via meta-deep reinforcement learning in UA V networks,” 2025. [Online]. Available: https://arxiv.org/abs/2501.14603

2025 arXiv
[33]

Conservative and risk-aware offline multi-agent reinforcement learning for digital twins,

E. Eldeeb, H. Sifaou, O. Simeone, M. Shehab, and H. Alves, “Conservative and risk-aware offline multi-agent reinforcement learning for digital twins,” 2024. [Online]. Available: https://arxiv.org/abs/2402. 08421 11

2024
[34]

Offline reinforcement learning for wireless network optimization with mixture datasets,

K. Yang, C. Shi, C. Shen, J. Yang, S.-p. Yeh, and J. J. Sydir, “Offline reinforcement learning for wireless network optimization with mixture datasets,” IEEE Transactions on Wireless Communications , 2024

2024
[35]

Offline and distributional reinforcement learning for radio resource management,

E. Eldeeb and H. Alves, “Offline and distributional reinforcement learning for radio resource management,” 2024. [Online]. Available: https://arxiv.org/abs/2409.16764

2024 arXiv
[36]

An offline multi-agent reinforcement learning framework for radio resource management,

——, “An offline multi-agent reinforcement learning framework for radio resource management,” 2025. [Online]. Available: https: //arxiv.org/abs/2501.12991

2025 arXiv
[37]

Deep reinforcement learning for fresh data collection in UA V-assisted IoT networks,

M. Yi, X. Wang, J. Liu, Y . Zhang, and B. Bai, “Deep reinforcement learning for fresh data collection in UA V-assisted IoT networks,” in IEEE INFOCOM Workshops 2020 , 2020, pp. 716–721

2020
[38]

Traffic learning and proactive UA V trajectory planning for data uplink in markovian IoT models,

E. Eldeeb, M. Shehab, and H. Alves, “Traffic learning and proactive UA V trajectory planning for data uplink in markovian IoT models,”IEEE Internet of Things Journal , vol. 11, no. 8, pp. 13 496–13 508, 2024

2024
[39]

Path loss models for outdoor environment-with a focus on rain attenuation impact on short-range millimeter-wave links,

A. A. Budalal and M. R. Islam, “Path loss models for outdoor environment-with a focus on rain attenuation impact on short-range millimeter-wave links,” e-Prime-Advances in Electrical Engineering, Electronics and Energy, vol. 3, p. 100106, 2023

2023
[40]

Impact of UA V failure and severe weather conditions in mmWave and terahertz signals for aerial edge computing,

R. W. Ibrahim, T. K. Rodrigues, and N. Kato, “Impact of UA V failure and severe weather conditions in mmWave and terahertz signals for aerial edge computing,” in 2023 IEEE 98th Vehicular Technology Conference (VTC2023-Fall), 2023, pp. 1–7

2023
[41]

Specific attenuation model for rain for use in prediction methods,

I. R. P. Series, “Specific attenuation model for rain for use in prediction methods,” Recommendation ITU-R, pp. 838–3, 2005

2005
[42]

Meteorologically introduced impacts on aerial channels and UA V communications,

M. Song, Y . Huo, T. Lu, X. Dong, and Z. Liang, “Meteorologically introduced impacts on aerial channels and UA V communications,” in 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall) , 2020, pp. 1–5

2020
[43]

Implicit quantile networks for distributional reinforcement learning,

W. Dabney, G. Ostrovski, D. Silver, and R. Munos, “Implicit quantile networks for distributional reinforcement learning,” in International conference on machine learning . PMLR, 2018, pp. 1096–1105

2018
[44]

Semantic meta-split learning: A tinyml scheme for few-shot wireless image classification,

E. Eldeeb, M. Shehab, H. Alves, and M.-S. Alouini, “Semantic meta-split learning: A tinyml scheme for few-shot wireless image classification,” 2024. [Online]. Available: https://arxiv.org/abs/2409. 12978

2024
[45]

Pytorch: An imperative style, high- performance deep learning library,

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high- p...

2019

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Machine learning for large-scale optimization in 6G wireless networks,

Y . Shi, L. Lian, Y . Shi, Z. Wang, Y . Zhou, L. Fu, L. Bai, J. Zhang, and W. Zhang, “Machine learning for large-scale optimization in 6G wireless networks,” IEEE Communications Surveys & Tutorials , vol. 25, no. 4, pp. 2088–2132, 2023

2023

[2] [2]

Artificial intelligence enabled wireless networking for 5G and beyond: Recent advances and future challenges,

C.-X. Wang, M. Di Renzo, S. Stanczak, S. Wang, and E. G. Larsson, “Artificial intelligence enabled wireless networking for 5G and beyond: Recent advances and future challenges,” IEEE Wireless Communica- tions, vol. 27, no. 1, pp. 16–23, 2020

work page 2020

[3] [3]

Reinforcement learning: An introduction,

R. S. Sutton, “Reinforcement learning: An introduction,” A Bradford Book, 2018

2018

[4] [4]

Self- organization in small cell networks: A reinforcement learning approach,

M. Bennis, S. M. Perlaza, P. Blasco, Z. Han, and H. V . Poor, “Self- organization in small cell networks: A reinforcement learning approach,” IEEE Transactions on Wireless Communications , vol. 12, no. 7, pp. 3202–3212, 2013

work page 2013

[5] [5]

Deep reinforcement learning for resource management in network slicing,

R. Li, Z. Zhao, Q. Sun, C.-L. I, C. Yang, X. Chen, M. Zhao, and H. Zhang, “Deep reinforcement learning for resource management in network slicing,” IEEE Access, vol. 6, pp. 74 429–74 441, 2018

work page 2018

[6] [6]

Multi-UA V path learning for age and power optimization in IoT with UA V battery recharge,

E. Eldeeb, J. M. de Souza Sant’Ana, D. E. P ´erez, M. Shehab, N. H. Mahmood, and H. Alves, “Multi-UA V path learning for age and power optimization in IoT with UA V battery recharge,” IEEE Transactions on Vehicular Technology, vol. 72, no. 4, pp. 5356–5360, 2022

work page 2022

[7] [7]

Multi-agent deep reinforcement learning to manage connected autonomous vehicles at tomorrow’s inter- sections,

G.-P. Antonio and C. Maria-Dolores, “Multi-agent deep reinforcement learning to manage connected autonomous vehicles at tomorrow’s inter- sections,” IEEE Transactions on Vehicular Technology , vol. 71, no. 7, pp. 7033–7043, 2022

work page 2022

[8] [8]

Applications of deep reinforcement learning in communications and networking: A survey,

N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang, Y .-C. Liang, and D. I. Kim, “Applications of deep reinforcement learning in communications and networking: A survey,” IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3133–3174, 2019

work page 2019