{"id":"d8c12337-4e65-418d-9582-299a851ed1d4","arxiv_id":"2502.09038","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A deep reinforcement learning algorithm with sequence memory and attention reduces age-of-information and energy use in a simulated UAV-assisted IoT forwarding scenario.","lead":"This paper proposes a UAV-relay system where drones use distributed beamforming to keep sensor data fresh, and trains a modified reinforcement learning agent to trade off data age against drone energy. A reader might care because age-of-information is a key bottleneck in remote monitoring, so any practical improvement in UAV scheduling has operational value.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (5) defines an age proxy, not AoI; the reported AoI gains may not reflect actual data freshness.","rationale":"The paper's central claim is empirical: SAC-TLA achieves lower AoI than benchmarks. The simulation's objective and evaluation both use Eq. (5). If Eq. (5) is not a faithful AoI model, the comparison is on a mislabeled metric. This is an internal correctness issue rather than a physical-layer idealization, so it is the most load-bearing. I considered the VAA phase-alignment assumption, but that idealization is shared by all benchmarks and is an explicitly stated modeling choice; it would affect the absolute feasibility but not necessarily the relative algorithm ranking. The AoI update, by contrast, is the definitional target of the optimization and is mathematically suspect: it resets to 0 on full forwarding and scales ages fractionally under partial forwarding, which has no counterpart in the standard age-of-information definition. Since the paper claims to minimize AoI, a wrong AoI model directly undermines the headline result. The reader's verdict already asks for revisions; our concern adds a specific, testable requirement.","tokens_in":8577,"tokens_out":8912,"duration_ms":91064,"concrete_test":"Write a discrete-event packet-level simulator for the same scenario (or a single-sensor reduction) that tracks the exact AoI at the BS as the age of the most recent successfully forwarded packet, using the same channel and forwarding decisions from a logged SAC-TLA run. Recompute the time-averaged AoI under this exact definition and compare it with the values produced by Eq. (5) and reported in Fig. 4(b). If the ranking among SAC-TLA, TD3, PPO, TQC, and SAC changes, or if the absolute AoI values shift by more than 20%, the empirical claim is not about AoI. If exact AoI logging is unavailable, replace Eq. (5) in the simulator with the standard update (increment on failure, reset to 1 on successful forwarding of at least one packet from that sensor) and rerun the benchmark comparisons.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SAC-TLA minimizes AoI, but the AoI dynamics in Eq. (5) do not model the age of information. Standard discrete-time AoI at the receiver increments by 1 when no new packet is delivered and resets to the age of the delivered packet upon successful delivery. Eq. (5) instead applies a global multiplicative factor (1-Q(t)) to every sensor whenever it is served: when Q(t)=1, AoI resets to 0 (rather than 1 or the packet age), and when Q(t)<1, AoI scales down fractionally even if no fresh data from that sensor is actually forwarded. For a Bernoulli forwarding process with success probability Q, the stationary mean of the standard AoI is 1/Q (with reset to 1), whereas the fixed point of Eq. (5) is (1-Q)/Q; the two differ by a factor of 2 when Q=0.5. Thus the quantity minimized in Eq. (7) and reported in Fig. 4(b) is a fluid proxy, not the AoI claimed in the title and abstract. Neither the paper nor the DPAM description specifies how partial data delivery maps to per-sensor freshness, so the objective may reward schedules that would have poor true AoI. This is load-bearing because the headline performance comparison is made on this metric.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies a UAV-assisted IoT data-forwarding system in which a swarm of UAVs collects sensor data, broadcasts it among themselves, and relays it to a remote base station using distributed beamforming. The authors formulate a joint optimization problem that minimizes time-averaged age of information (AoI) and UAV energy consumption by choosing UAV trajectories and communication schedules. Because the problem is non-convex and dynamic, they reformulate it as an MDP and propose SAC-TLA, a modified soft actor-critic algorithm that combines temporal sequence input, layer-normalized gated recurrent units, and a global soft attention mechanism. The paper reports simulation results comparing SAC-TLA with TD3, PPO, TQC, and SAC, and claims that SAC-TLA achieves the lowest AoI while maintaining competitive energy consumption.","tokens_in":8924,"tokens_out":3884,"duration_ms":41258,"significance":"The paper addresses a practically relevant problem, and the proposed algorithmic combination (SAC plus temporal sequence input, LNGRU, and attention) is a plausible way to handle partially observed dynamics in UAV trajectory optimization. The DPAM simplification of discrete scheduling decisions is a useful idea for avoiding mixed discrete-continuous action spaces. However, the central quantitative claims rest on a nonstandard AoI model and on an MDP simplification that is not shown to be equivalent to the original optimization problem; the experimental section also lacks the parameter settings and statistical reporting needed for reproducibility. If the AoI model and system assumptions were validated, the proposed SAC-TLA would be a reasonable baseline for trajectory-only resource allocation in UAV-assisted IoT, but as written the paper does not establish that the reported gains correspond to actual data freshness.","major_comments":[{"comment":"The quantity called AoI is not the age of information. Standard discrete-time AoI increments by one when no new packet is delivered and resets to the age of the delivered packet (at least one) on success. Eq. (5) instead applies the global forwarding fraction Q(t) as a multiplicative factor to the age of every served sensor: when Q(t)=1 the AoI resets to 0, and when Q(t)<1 it scales down fractionally even if no fresh packet from that sensor is delivered. For a Bernoulli forwarding process with success probability Q, the fixed point of Eq. (5) is (1-Q)/Q, whereas the stationary mean of standard AoI is 1/Q; the two differ by a factor of two when Q=0.5. Because Eq. (7) and Fig. 4(b) are evaluated with this proxy, the headline claim that SAC-TLA minimizes AoI is not established, and the reported optimization may reward schedules that would perform poorly under a faithful AoI metric.","section":"Section II-C, Eq. (5)"},{"comment":"The MDP action space contains only the continuous movements {a_x^j(t), a_y^j(t)}, and the DPAM mechanism deterministically maps trajectory choices to communication decisions by checking whether sensors are within the communication radius. This removes the binary scheduling variables beta_{i,j}(t) from the learning problem, so the MDP no longer optimizes the problem in Eq. (7), which includes scheduling constraints (7b) and (7d). The paper does not show that DPAM preserves feasibility or optimality with respect to the original problem, nor does it discuss how the trajectory-only policy relates to the required schedule. Consequently, the simulation results validate a different, simplified problem from the one formulated in Section II.","section":"Section III-A2 and III-A3"},{"comment":"The simulation section does not report any parameter values: environment constants a and b, channel parameters rho0 and alpha, noise power sigma^2, bandwidths B and B_j, transmit powers, communication radius used by DPAM, UAV propulsion and speed parameters, reward weights rho1, rho2, rho3, penalty p_j, training hyperparameters, episode length, or sequence length n. No error bars, confidence intervals, or number of random seeds are given, and Fig. 4(a) appears to show single training curves without a legend. As a result, the claims of faster convergence, better stability, and a superior AoI-energy trade-off are not reproducible, and the statement that SAC-TLA maintains 'competitive energy consumption' is not quantified.","section":"Section IV"},{"comment":"The model assumes that all data collected from sensors can be reliably broadcast among the UAVs within each time slot and that the UAVs form a perfectly phase-aligned virtual antenna array whose SNR is given by Eq. (4) using the formula from reference [5]. These assumptions are asserted without physical-layer derivation, simulation at waveform level, or sensitivity analysis for phase errors and channel estimation errors. Since these assumptions directly determine the forwarding fraction Q(t) and the claimed communication-range extension, the quantitative AoI improvements could be optimistic.","section":"Section II-B2 and II-B3"}],"minor_comments":[{"comment":"The symbol Q is overloaded: in Eq. (7), Q denotes the set of UAV positions, while in Eq. (5) and the text, Q(t) denotes the data forwarding fraction. Please use distinct symbols to avoid confusion.","section":"Notation"},{"comment":"The benchmark algorithms TD3, PPO, TQC, and SAC are all cited to a survey [8]; the original papers for these algorithms should be cited instead.","section":"References"},{"comment":"The sentence 'the actor network combines LNGRU and the attention mechanism' appears twice consecutively; one occurrence should be removed.","section":"Section III-C4"},{"comment":"Line 11 says 'Select action a_t with exploration noise,' but SAC normally samples actions from a stochastic policy rather than adding exploration noise to a deterministic action; please clarify the action-selection procedure.","section":"Algorithm 1"},{"comment":"There are several typographical issues, including 'UA V' with an internal space in the author line and throughout, 'muti-access' in Section II-A, 'characteristize' in Section II, and 'ro reduce' in Section IV.","section":"Typos"},{"comment":"Fig. 4(a) appears to lack a legend identifying the curves, and neither panel reports error bars or confidence intervals; please add these to support the stability and convergence claims.","section":"Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript relies heavily on the authors' own prior models in references [4] and [5] for the virtual-antenna-array beamforming rate and on reference [7] for DPAM; this is not by itself grounds for rejection, but the lack of independent validation or a clear statement of the modeling assumptions should be addressed. The paper is within the scope of a communications or networking journal, but the experimental reporting is currently below the standard needed to support the central claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible but thinly supported DRL recipe paper. The model combines distributed beamforming with UAV trajectory planning, which is a sensible way to reduce flight delays, and SAC-TLA (temporal sequence + LNGRU + attention) is a reasonable engineering extension of SAC. The paper is readable and the problem setup is clear.\n\nThe main problem is the AoI model. Eq. (5) is not age of information. Standard AoI resets to the age of the delivered packet on a successful update; here it multiplies every sensor's age by (1-Q(t)) whenever the sensor is served, with Q(t) a global fraction of data forwarded. When Q=0.5, the fixed point is 0.5, while true Bernoulli-delivery AoI has mean 2. So the quantity minimized in Eq. (7) and plotted in Fig. 4(b) is a fluid proxy, not the AoI promised in the title and abstract. That is load-bearing: the headline comparison is on this metric.\n\nSecond, the MDP simplification changes the problem. DPAM removes the binary schedule variables from the action space, so the agent only controls UAV movement. The paper never reformulates the optimization problem (7) accordingly. This might be fine as a heuristic, but it leaves a mismatch between the stated problem and what is actually solved.\n\nThird, the evaluation is under-reported: no parameter values, no error bars, no code, and the beamforming rate relies on the authors' own prior models without independent validation. The training curves and scatter plot cannot be independently checked.\n\nWhat the paper does well: it identifies a real issue (flight-induced AoI), and the architectural modifications to SAC are clearly motivated. If the AoI model were corrected to actual packet-level freshness, and the experiments were made reproducible, this could be a useful incremental contribution. As it stands, the central performance claim is not supported by the evidence presented.\n\nI would send this to peer review rather than desk reject, because the underlying idea is plausible and the flaws are fixable. But a referee should demand a corrected metric, a revised problem statement, and full experimental details. I would not cite it in its current form.","headline":"Plausible DRL recipe undermined by an AoI metric that is really a fluid proxy; empirical claims cannot be assessed without code and parameters.","tokens_in":9421,"tokens_out":2073,"would_cite":false,"duration_ms":21161,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a modified SAC algorithm with temporal sequence input, LNGRU, and attention achieves the lowest AoI and competitive energy consumption in a distributed-beamforming UAV relay network.","keywords":["age of information","UAV-assisted IoT","distributed beamforming","virtual antenna array","deep reinforcement learning","soft actor-critic","trajectory optimization","data forwarding"],"falsifier":"Re-run the same five algorithms with a physical-layer model that includes per-UAV carrier phase offsets or a finite A2A broadcast capacity, so the forwarded fraction is not the idealized $\\min\\{S_{\\mathrm{G2A}},S_{\\mathrm{A2G}}\\}/S_c$; if SAC-TLA no longer holds the lowest time-averaged AoI, the reported advantage depends on the idealized distributed-beamforming assumptions rather than on the learning architecture alone.","tokens_in":8405,"feed_emoji":"📡","tokens_out":9144,"duration_ms":83916,"temperature":0.7,"pith_summary":"This paper tries to establish that a UAV-assisted IoT relay can keep sensor data fresh without excessive flight energy by forwarding data from hovering UAVs with distributed beamforming, instead of flying back and forth to the base station. To make that work, the authors write a joint optimization over UAV trajectories and communication schedules that minimizes time-averaged age of information (AoI) plus UAV energy, note that the problem is non-convex and highly dynamic, and solve it with a modified soft actor-critic algorithm called SAC-TLA. SAC-TLA feeds a temporal sequence of states into a layer-normalized gated recurrent unit with a global soft attention mechanism, and the simulations compare it with TD3, PPO, TQC, and standard SAC. The reported result is that SAC-TLA converges fastest and reaches the lowest AoI while keeping energy consumption competitive. If true, this gives a practical recipe for AoI-sensitive UAV relaying in remote IoT settings.","feed_headline":"Modified SAC nets lowest data age in UAV relay simulations","feed_subtitle":"A sequence-memory and attention upgrade over standard SAC cuts age of information while holding down UAV energy use.","key_machinery":"The machinery is SAC-TLA, a modified soft actor-critic whose policy network processes a window of past states through LNGRU cells — gated recurrent units with layer normalization applied before the gated computations — and then applies a global soft attention mechanism $\\alpha_t=\\exp(h_t^\\top W_a h_c)/\\sum_{t'}\\exp(h_{t'}^\\top W_a h_c)$ to the hidden states, focusing the agent on the sensors with the highest AoI. Dynamic proximity-based action mapping (DPAM) removes discrete sensor-association decisions by making them deterministic functions of which sensors lie inside each UAV's communication radius, so the learner only outputs continuous UAV movements. The physical layer that makes the scenario work is the virtual antenna array: distributed beamforming lets the UAVs coherently combine signals at the base station, giving the A2G rate $R_{\\mathrm{BS}}(t)=B\\log_2(1+\\gamma_{\\mathrm{SNR}}(t))$, and this extended range is what lets the UAVs relay from hover rather than flying frequently.","core_discovery":"On its own terms, the paper establishes that combining distributed-beamforming relays with an architecture-enhanced off-policy reinforcement learner gives the best AoI-energy balance among the tested methods in a simulated UAV-assisted IoT network. The algorithmic claim is that SAC-TLA — soft actor-critic with temporal sequence state input, layer-normalized gated recurrent units (LNGRU), and a global soft attention mechanism — achieves lower time-averaged AoI than TD3, PPO, TQC, and standard SAC, with competitive energy consumption, faster convergence, and greater training stability. The system-level claim underneath is that the UAVs, acting as a virtual antenna array through distributed beamforming, can reach the remote base station from their hovering positions, which reduces the flight frequency that would otherwise inflate AoI. The evaluation rests on the AoI update $A_i(t+1)=(1-Q(t))(A_i(t)+1)$ for a served sensor and $A_i(t)+1$ otherwise, where $Q(t)=\\min\\{S_{\\mathrm{G2A}},S_{\\mathrm{A2G}}\\}/S_c$ is the fraction of all collected data the array can forward to the base station in a time slot.","pith_inferences":["A direct extension the authors leave implicit is that the temporal-sequence, LNGRU, and attention modifications are generic and could be grafted onto TD3, PPO, or TQC; an ablation that swaps the underlying algorithm while keeping the three components would quantify how much of the gain comes from the architecture rather than from SAC itself.","The global forwarding fraction $Q(t)$ compresses every sensor's success into a single number, so average AoI can hide large asymmetries between sensors; a per-sensor or tail-AoI metric would test whether the policy is also fair.","Because the VAA downlink rate depends on coherently adding amplitudes, phase misalignment is the most direct threat to the modeled AoI advantage; simulating with nonzero phase-error variance would turn the qualitative robustness claim into a quantitative curve."],"forward_implications":["Under the modeled assumptions, distributed beamforming lets a UAV swarm relay to the base station while hovering, so UAVs fly less often and the age of collected data grows more slowly than in fly-back-and-forth designs.","The joint trajectory-and-communication problem can be solved in real time by a single DRL policy, with DPAM converting sensor-association choices into deterministic functions of UAV positions.","The AoI-energy tradeoff has an adjustable operating point: SAC-TLA sits at the low-AoI end and TD3 at the low-energy end, so a network operator can choose the policy whose balance fits the application.","Faster convergence and flatter training curves imply SAC-TLA needs fewer episodes to reach a usable policy, reducing the computational cost of adapting when the IoT deployment changes."],"supporting_citations":[{"why":"Supplies the AoI-aware trajectory-and-scheduling formulation and the non-convexity argument that motivates DRL.","marker":"[2]"},{"why":"Supplies the virtual-antenna-array distributed beamforming model that lets the UAV swarm relay from hover.","marker":"[4]"},{"why":"Supplies the VAA-to-BS SNR expression used in the A2G rate and the forwarded-data fraction.","marker":"[5]"},{"why":"Supplies the rotary-wing UAV propulsion and hovering energy model that forms the energy objective.","marker":"[6]"},{"why":"Supplies the proximity-based action mapping (DPAM) that removes discrete communication actions from the learning problem.","marker":"[7]"},{"why":"Supplies the DRL background and the TD3, PPO, TQC, and SAC baselines against which SAC-TLA is compared.","marker":"[8]"},{"why":"Supplies layer normalization, the ingredient of the LNGRU cells used to stabilize training.","marker":"[9]"},{"why":"Supplies the global soft attention mechanism used to weight hidden states toward high-AoI sensors.","marker":"[10]"}],"fun_headline_variants":["Modified SAC minimizes data age in UAV-assisted IoT","Beamforming UAVs achieve fresher data with DRL","SAC-TLA optimizes AoI and energy in UAV networks","Distributed beamforming plus DRL yields lowest AoI","Attention-enhanced SAC bests data age benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the A2A broadcast reliably gives every UAV all data collected by every other UAV and that the UAVs form a perfectly phase-aligned virtual antenna array, so that each time slot the system forwards exactly the smaller of the collected data volume and the array's downlink capacity; if either piece fails, the AoI update rule used to train the policy no longer describes the physical network.","fun_headline_variants_meta":{"raw":{"variants":["Modified SAC minimizes data age in UAV-assisted IoT","Beamforming UAVs achieve fresher data with DRL","SAC-TLA optimizes AoI and energy in UAV networks","Distributed beamforming plus DRL yields lowest AoI","Attention-enhanced SAC bests data age benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000526,"raw_usage":{"total_tokens":2533,"prompt_tokens":935,"completion_tokens":1598,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":1519}},"tokens_in":551,"tokens_out":1598,"duration_ms":11189,"temperature":1.0,"reasoning_tokens":1519,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T22:49:53.830567+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same five algorithms with a physical-layer model that includes per-UAV carrier phase offsets or a finite A2A broadcast capacity, so the forwarded fraction is not the idealized $\\min\\{S_{\\mathrm{G2A}},S_{\\mathrm{A2G}}\\}/S_c$; if SAC-TLA no longer holds the lowest time-averaged AoI, the reported advantage depends on the idealized distributed-beamforming assumptions rather than on the learning architecture alone.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the AoI-aware trajectory-and-scheduling formulation and the non-convexity argument that motivates DRL."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the virtual-antenna-array distributed beamforming model that lets the UAV swarm relay from hover."},{"cited_title":"Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning","cited_arxiv_id":"2404.07450","evidence_quote":"Supplies the VAA-to-BS SNR expression used in the A2G rate and the forwarded-data fraction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the rotary-wing UAV propulsion and hovering energy model that forms the energy objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the proximity-based action mapping (DPAM) that removes discrete communication actions from the learning problem."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DRL background and the TD3, PPO, TQC, and SAC baselines against which SAC-TLA is compared."},{"cited_title":"Lei Ba, J","cited_arxiv_id":null,"evidence_quote":"Supplies layer normalization, the ingredient of the LNGRU cells used to stabilize training."},{"cited_title":"Brauwers and F","cited_arxiv_id":null,"evidence_quote":"Supplies the global soft attention mechanism used to weight hidden states toward high-AoI sensors."}],"review_version":1}