Pith. sign in

REVIEW 4 major objections 6 minor 45 references

Continual Reinforcement Learning for Digital Twin Synchronization Optimization

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A continual reinforcement learning scheduler keeps a factory digital twin accurate while using the same wireless resource budget, reducing state-estimation error by up to 55.2%.

desk verdict A plausible simulation result with a real integration gap: the resource budget M never enters the state, so the continual-learning adaptation claim in Fig. 4 is not supported as written. read the letter →

arxiv 2501.08045 v2 pith:C46FHEQI submitted 2025-01-14 cs.NI

Haonan Tong , Mingzhe Chen , Jun Zhao , Ye Hu , Zhaohui Yang , Yuchen Liu , Changchuan Yin This is my paper · ORCID
classification cs.NI
keywords digitaltwinconstrainedMarkovdecisionprocesscontinualreinforcementlearningdeviceschedulingresourceblockallocationsoftactor-criticmulti-timescalereplayinvariantriskminimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a base station can keep a factory digital twin closely matched to physical devices by selecting only a few devices to upload data at each time, and that a continual reinforcement learning algorithm can learn this selection without retraining when network capacity changes. A sympathetic reader would care because wireless resources are too scarce to transmit all sensor data continuously; a scheduler that learns which mismatches matter could cut signaling load while preserving twin fidelity. The paper supports this with a constrained Markov decision process formulation, solves it through a state-wise Lagrangian dual, and simulates with real sensor data, showing up to 55.2% lower normalized root mean square error between physical and virtual states for the same number of resource blocks.

What carries the argument

The central object is the multi-timescale replay (MTR) buffer, a cascade of FIFO sub-buffers plus an overflow buffer that stores experiences with a power-law age distribution, so both fresh and old experiences remain available for training. It is combined with a soft actor-critic (SAC) agent, a Lagrange multiplier network that enforces the state-wise resource-block constraint, and an invariant risk minimization (IRM) penalty in the actor loss that stabilizes the policy across changing environments. The state for each device is $[\Phi_{n,t}, Y_{n,t}, \gamma_t]$: time since the last correctly received packet, the last reported mismatch, and the data reception indicator. This machinery carries the argument because it prevents catastrophic forgetting when the resource budget changes, letting the policy re-converge quickly.

What would settle it

Run the trained MTR-SAC policy in a simulator where the true mismatch of an unscheduled device evolves quickly between transmissions while its last received mismatch and elapsed time stay unchanged; if the scheduler keeps skipping that device and NRMSE climbs toward the polling baseline, the proxy state is not a sufficient statistic. A cleaner check is to give the agent the true current mismatch vector as state in the same simulator and compare NRMSE; a large gap would indicate the partial-observability assumption is the limiting factor.

Watch

Extended reading notes

Core claim

The central discovery is that mismatch minimization for digital twin synchronization can be posed as a state-wise constrained Markov decision process, and its Lagrangian dual can be learned by an SAC variant augmented with a multi-timescale replay buffer and an invariant risk minimization penalty. The paper shows in simulation that the resulting MTR-SAC policy adapts to abrupt changes in the number of available resource blocks (from 30 to 10 to 26), converges faster than standard SAC after each change, and reduces normalized root mean square error by up to 55.2% relative to DP and polling methods using the same number of resource blocks. The state-wise constraint formulation also keeps instantaneous resource consumption at or below the budget, unlike the expectation-constraint formulation, which temporarily exceeds the budget in resource-scarce settings.

Load-bearing premise

The paper assumes that a device's time since its last successful upload and its most recently reported mismatch are enough information for the base station to make near-optimal scheduling decisions, even though the reward is defined using the full current mismatch of all devices.

Editorial extensions

If this is right

  • At the same resource-block budget, the MTR-SAC scheduler reduces NRMSE by up to 55.21% compared with DP and 68.42% compared with polling at M=15.
  • To achieve a target NRMSE of 0.02, the proposed method consumes up to 55.56% fewer resource blocks than DP and 61.54% fewer than polling, making fidelity gains translate directly into spectrum savings.
  • When the resource budget abruptly changes, MTR-SAC converges faster than standard SAC because the IRM penalty and multi-timescale replay retain common knowledge across budgets.
  • Under scarce resources (M=9 or 18), the state-wise constraint keeps instantaneous resource consumption at or below the budget, whereas the expectation-constraint formulation exceeds it.
  • As device count grows with fixed resource blocks, MTR-SAC degrades more slowly than DP and polling, indicating scalability to denser sensor deployments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same MTR-SAC structure could be applied to other constrained scheduling problems with changing budgets, such as age-of-information or semantic-value scheduling, because the framework separates the reward from the constraint and stabilizes policy learning across budgets; this is an extension the paper does not test.
  • The reported 55.2% gain depends on physical-state dynamics that are smooth enough that the last-received mismatch and elapsed time remain informative; on faster or more erratic dynamics, the proxy state may lose information and the advantage would likely shrink.
  • The interaction between the IRM penalty and the Lagrange multiplier network is not analyzed separately; testing that interaction could identify when invariant policy learning and constraint satisfaction conflict.
  • A direct ablation comparing the proxy state with an oracle state containing the true current mismatch vector would isolate how much of the gain comes from the scheduling policy versus how much is lost to partial observability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies adaptive device scheduling and resource-block allocation for digital twin (DT) synchronization over wireless networks. A base station selectively schedules devices to transmit sensing data, and the problem is formulated as a constrained Markov decision process (CMDP) minimizing long-term weighted mismatch between physical and virtual states under an average RB constraint. The authors transform the constraint into a state-wise constraint, introduce a Lagrangian dual problem, and propose a continual reinforcement learning algorithm (MTR-SAC) that combines a multi-timescale replay buffer, soft actor-critic, and invariant risk minimization. Simulations with Intel Berkeley sensor data and indoor positioning data report that the proposed algorithm reduces NRMSE by up to 55.2% against DP and polling baselines using the same number of RBs, and converges faster than standard SAC when the RB budget changes.

Significance. If the empirical claims hold, the paper offers a practical scheme for resource-efficient DT synchronization in dynamic networks, with comparisons against polling and DP baselines on real-world datasets. The strengths are the clear system model, the inclusion of packet errors and mismatch losses, and the evaluation of state-wise constraint satisfaction. However, the theoretical support for the dual transform and the convergence guarantee is incomplete, and the state representation omits the resource budget M that changes in the key continual-learning experiment. The central contribution is a plausible integration of known components (MTR, SAC, feasible actor-critic, IRM) rather than a fundamentally new method.

major comments (4)
  1. [Section III-B, State definition and Eq. (15)] The state S_t is defined as [s_{1,t}, ..., s_{N,t}] with s_{n,t} = [Phi_{n,t}, Y_{n,t}, gamma_t], and no component encodes the current RB budget M. Yet the cost in Eq. (15) is M when b^T a_t <= M and b^T a_t otherwise, and the multiplier lambda_zeta(S_t) and cost critic Q_phi_C(S_t, a_t) take only S_t as input. When M changes from 30 to 10 to 26 in Fig. 4, the same state-action pair receives conflicting cost labels and conflicting constraint thresholds, so a single function of S_t cannot represent the required mapping. The claimed quick adaptation to network capacity changes is therefore not supported by the printed formulation; the state should be augmented with M, and M should be an input to the actor, cost critic, and multiplier.
  2. [Theorem 1 and Appendix A] Theorem 1 states that if the Lagrangian dual problem of L_stw has an optimal scheduling vector sequence u* and multiplier lambda*, then u* is also optimal for the state-wise constrained problem P2. The proof in Appendix A only rewrites the constraint using d0(Z) and shows that L'_stw(u,lambda) is the Lagrangian of the reformulated problem (29). It does not prove strong duality, zero duality gap, or that u* is feasible for the state-wise constraint (10a). Without these steps, the equivalence between P3 and P2 is not established, and the claim that the dual transform refines the RB constraint impact is not rigorously justified.
  3. [Lemma 1 and Section III-D1] The convergence proof of Lemma 1 does not establish the claimed guarantee. The argument 'we can always choose pi_new = pi_old in the invariant distribution' only gives a trivial equality, not the required monotonic improvement Q^{pi_new} >= Q^{pi_old}. Moreover, the reduction using Lemma 2 in [40] ignores that the actor loss in Eqs. (22)-(23) contains the IRM term and the multiplier term lambda_zeta(S_t)(Q_phi_C - M), which are not part of the standard SAC loss for which [40] provides convergence conditions. The authors should either prove convergence under explicit conditions for the modified loss or state clearly that convergence is an empirical observation rather than a guarantee.
  4. [Section III-B, Reward and State] The reward in Eq. (14) is defined on the full mismatch vector Z_t, while the agent's state uses only the last received mismatch Y_{n,t} and the time since last successful reception Phi_{n,t}. The paper does not justify that this proxy is a sufficient statistic for near-optimal scheduling in the CMDP. Since the BS cannot observe the current mismatch of unscheduled devices, the learned policy may not track the true objective in deployment; the discrepancy between the full-information reward used in training and the partial-information state used at execution should be analyzed or at least discussed as an approximation.
minor comments (6)
  1. [Eq. (16)] The expression for \bL_stw has an extra closing parenthesis: it should read lambda_zeta(S_t)(c(S_t,a_t) - M) rather than lambda_zeta(S_t)(c(S_t,a_t)) - M).
  2. [Eq. (20)] The stochastic gradient for the cost critic uses Q_phi_C(S_t, a_t) in the target term, but the loss in Eq. (19) defines the target with Q_phi_C(S_{t+1}, a_{t+1}); the gradient expression should be corrected for consistency.
  3. [Algorithm 1] Line 8 writes 'phi <- phi - beta_Q grad ... for i in {1,2,C}', which mixes the reward critics phi_1, phi_2 and the cost critic phi_C; the update rule should distinguish these parameters explicitly.
  4. [Figures 10 and 11] The subfigure labels are duplicated and inconsistent (for example, '(a)M=9 (a)M=36 (a)M=27 (a)M=18' appears in both figures); the labels should be corrected to a-d with the corresponding M values.
  5. [Section II-A] The notation bX_{n,t} is used where the hat is presumably meant; please use a consistent notation such as \hat{X}_{n,t} for the virtual state.
  6. [Section III-A] The text says 'seek a prime-dual solution'; this should be 'primal-dual solution'.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the 55.2% NRMSE reduction is a simulated measurement against external baselines, not an identity derived from the paper's own definitions.

full rationale

The paper's central claim is the simulated NRMSE reduction of up to 55.2% (Section IV, Fig. 8) achieved by MTR-SAC relative to polling and DP baselines under the same RB budget. That number is produced by running the stated SAC/MTR/IRM losses (Eqs. 17–26) on the Intel Berkeley and positioning datasets; no constant is fitted to the baseline curves and then renamed a prediction. The Lagrangian and state-wise-constraint material (Eqs. 9–13, Theorem 1) is a standard reformulation: Theorem 1 asserts that an optimal solution of the sampled dual is optimal for P2, and the appendix establishes the equivalence of the sampled expectation to the original state-wise Lagrangian via the support condition on d0(Z). That equivalence is definitional of the Lagrangian rather than a back-door insertion of the target empirical result. The MTR buffer and IRM term come from external works [36], [42], and the state-wise constraint technique from [37]; the paper's own earlier works appear mainly as related work or minor design shortcuts (e.g., [43] for the multiplier loss), and none is load-bearing for the measured improvement. The 55.2% figure is self-contained against external benchmarks, so the honest finding is no significant circularity. The skeptical concern that the state St omits the current RB budget M while Eq. (15) defines cost using M is a correctness/falsifiability issue, not a case of a prediction reducing by construction to its inputs; the same applies to any gap in the proof of Theorem 1 concerning strong duality.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several unstated or weakly proved assumptions: strong duality of the statewise Lagrangian, applicability of SAC convergence to the modified actor, sufficiency of a stale-mismatch state, and transfer of IRM invariance to unseen M. None is demonstrated by a proof or a dedicated experiment, and no code is provided to verify the simulations. The listed hyperparameters are hand-chosen and could affect the headline result.

free parameters (5)
  • βmtr = 0.8
    Cascade probability in the multi-timescale replay buffer; controls how many old experiences survive and directly shapes the claimed continual-learning benefit. No sensitivity analysis is reported.
  • λIRM = 0.01
    Weight of the IRM penalty in the actor loss (23); this is the mechanism claimed to reduce catastrophic forgetting. No ablation or sensitivity study is shown.
  • nb (sub-buffer count) = 4
    Number of cascaded FIFO sub-buffers; sets the timescale granularity of retained experiences. Chosen by hand without analysis.
  • Nb (total replay capacity) = 5e3
    Replay buffer size; affects sample distribution and training stability. No study of its effect on the reported NRMSE.
  • ξth (mismatch threshold) = 1e-2
    Threshold in loss functions (6) and (7); decides when a physical state change is significant enough to trigger scheduling. Directly influences scheduling behavior and is set without sensitivity analysis.
assumptions (4)
  • ad hoc to paper Strong duality and feasibility of the Lagrangian dual optimum for the statewise constrained CMDP
    Theorem 1 assumes an optimal dual pair (u*, λ*) exists and concludes u* is optimal for P2; Appendix A only performs a constraint reformulation and never proves zero duality gap or feasibility of u*. This underlies the algorithm in Section III-A.
  • ad hoc to paper Soft policy improvement and convergence conditions from SAC apply to the IRM-augmented MTR-SAC updates
    Lemma 1 says convergence follows from [40, Theorem 1], but the proof's key step is 'we can always choose πnew=πold', which does not demonstrate monotone improvement. The IRM loss and multiplier update change the policy objective, so the cited theorem does not directly apply.
  • domain assumption The proxy state defined from last received mismatch and time since last successful update is sufficient for optimal scheduling
    The state s_n,t = [Φ_n,t, Y_n,t, γ_t] in Section III-B does not include current mismatches of unscheduled devices, yet the reward in Eq. (14) is defined on the complete mismatch vector. No argument is provided that the proxy state is Markovian or sufficient.
  • ad hoc to paper IRM invariance learned over historical M values transfers to unseen network capacities
    Section III-C2 claims the IRM loss and MTR buffer yield quick adaptation to unseen M. This is inherited from [36] and [42]; no proof or dedicated out-of-distribution experiment is provided beyond the M=30/10/26 sequence in Figure 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Continual Reinforcement Learning for Digital Twin Synchronization Optimization." pith.science (2026). https://pith.science/paper/C46FHEQI

@misc{pith2026250108045,
  author       = {Pith},
  title        = {Pith review of: Continual Reinforcement Learning for Digital Twin Synchronization Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C46FHEQI}},
  note         = {Machine review of arXiv:2501.08045}
}
read the original abstract

This article investigates the adaptive resource allocation scheme for digital twin (DT) synchronization optimization over dynamic wireless networks. In our considered model, a base station (BS) continuously collects factory physical object state data from wireless devices to build a real-time virtual DT system for factory event analysis. Due to continuous data transmission, maintaining DT synchronization must use extensive wireless resources. To address this issue, a subset of devices is selected to transmit their sensing data, and resource block (RB) allocation is optimized. This problem is formulated as a constrained Markov process (CMDP) problem that minimizes the long-term mismatch between the physical and virtual systems. To solve this CMDP, we first transform the problem into a dual problem that refines RB constraint impacts on device scheduling strategies. We then propose a continual reinforcement learning (CRL) algorithm to solve the dual problem. The CRL algorithm learns a stable policy across historical experiences for quick adaptation to dynamics in physical states and network capacity. Simulation results show that the CRL can adapt quickly to network capacity changes and reduce normalized root mean square error (NRMSE) between physical and virtual states by up to 55.2%, using the same RB number as traditional methods.

Figures

Figures reproduced from arXiv: 2501.08045 by the authors.

Figure 1
Figure 1. DT enabled smart factory architecture. to support continuous simulation and prediction of object behaviors. Notice that, DT mapping mismatch, the difference between the physical states and virtual states, inevitably exists because of the transmission delay and errors. Meanwhile, due to the limited wireless resources, the BS can only selectively schedule a subset of the devices to collect effective sensing data that … view at source ↗
Figure 2
Figure 2. Architecture of the proposed MTR-SAC method. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 2
Figure 2. • MTR buffer: The MTR buffer stores the experiences at different timescales and consists of nb cascaded first in first out (FIFO) sub-buffers and a separate over￾flow buffer, as shown in [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Convergence of the MTR-SAC method with variable numbers of [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Device scheduling vectors of the proposed CRL algorithm. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Estimated virtual state signals and the physical state signals. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 9
Figure 9. Figure 9: NRMSE as number of devices N increases, the number of RB constraint is M = 10. while the NRMSE of location in [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 8
Figure 8. Figure 8: NRMSE of devices as number of RBs M increases. thereby maintaining fairness in device scheduling. This is because the CRL algorithm uses action entropy maximization loss in (22) to generate stochastic actions [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 10
Figure 10. Figure 10: Instant consumed RBs by the CRL algorithm with different constraints, [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Distributions of instant consumed RBs, M = 9, 18, 27, 36. This is because that the on-demand synchronization of the frequent and dramatic physical state changes cannot be fully supported by limited number of RBs, leading to an increase of NRMSE at DT. The slower NRMSE…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 41 canonical work pages

  1. [40]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” in Proc. Int. Conf. on Mach. Learning (ICML) , vol. 80, Jul. 2018

    work page 2018

  2. [1]

    Digital twin networks: A survey,

    Y . Wu, K. Zhang, and Y . Zhang, “Digital twin networks: A survey,” IEEE Internet of Things Journal , vol. 8, no. 18, pp. 13 789–13 804, May 2021

    work page 2021

  3. [2]

    Meta- verse for wireless systems: Vision, enablers, architecture, and future directions,

    L. U. Khan, Z. Han, D. Niyato, M. Guizani, and C. S. Hong, “Meta- verse for wireless systems: Vision, enablers, architecture, and future directions,” IEEE Wireless Communications, pp. 1–7, Aug. 2024

    work page 2024

  4. [3]

    Digital twin for networking: A data-driven performance modeling perspective,

    L. Hui, M. Wang, L. Zhang, L. Lu, and Y . Cui, “Digital twin for networking: A data-driven performance modeling perspective,” IEEE Network, vol. 37, no. 3, pp. 202–209, Jun. 2023

    work page 2023

  5. [4]

    Mobility-aware service provisioning in edge computing via digital twin replica placements,

    Y . Zhang, W. Liang, Z. Xu, and X. Jia, “Mobility-aware service provisioning in edge computing via digital twin replica placements,” IEEE Transactions on Mobile Computing , pp. 1–16, to appear, Apr. 2024

    work page 2024

  6. [5]

    Toward communication-efficient digital twin via ai-powered transmission and reconstruction,

    M. Li, C. Chen, X. Yang, J. T. Zhou, T. Zhang, and Y . Li, “Toward communication-efficient digital twin via ai-powered transmission and reconstruction,” IEEE Journal on Selected Areas in Communications , vol. 41, no. 11, pp. 3624–3635, Nov. 2023

    work page 2023

  7. [6]

    End-to-end network sla quality assurance for c-ran: A closed-loop management method based on digital twin network,

    Y . Ren, S. Guo, B. Cao, and X. Qiu, “End-to-end network sla quality assurance for c-ran: A closed-loop management method based on digital twin network,” IEEE Transactions on Mobile Computing, vol. 23, no. 5, pp. 4405–4422, Jun. 2024

    work page 2024

  8. [8]

    Digital-twin-enabled intelligent dis- tributed clock synchronization in industrial IoT systems,

    P. Jia, X. Wang, and X. Shen, “Digital-twin-enabled intelligent dis- tributed clock synchronization in industrial IoT systems,” IEEE Internet of Things Journal , vol. 8, no. 6, pp. 4548–4559, Oct. 2021

    work page 2021

Show all 45 references
  1. [9]

    Digital twin-empowered network planning for multi-tier computing,

    C. Zhou, J. Gao, M. Li, X. Sherman Shen, and W. Zhuang, “Digital twin-empowered network planning for multi-tier computing,” Journal of Communications and Information Networks , vol. 7, no. 3, pp. 221– 238, Sept. 2022

    2022

  2. [10]

    A federated digital twin framework for uavs-based mobile scenarios,

    L. Zhou, S. Leng, and Q. Wang, “A federated digital twin framework for uavs-based mobile scenarios,” IEEE Transactions on Mobile Com- puting, vol. 23, no. 6, pp. 7377–7393, Jun. 2024

    2024

  3. [11]

    Cybertwin assisted wire- less asynchronous federated learning mechanism for edge computing,

    Y . Xu, H. Zhou, J. Chen, T. Ma, and S. Shen, “Cybertwin assisted wire- less asynchronous federated learning mechanism for edge computing,” in Proc. IEEE Global Communications Conference (GLOBECOM) , Dec. 2021

    2021

  4. [12]

    Adaptive federated learning and digital twin for industrial internet of things,

    W. Sun, S. Lei, L. Wang, Z. Liu, and Y . Zhang, “Adaptive federated learning and digital twin for industrial internet of things,” IEEE Trans- actions on Industrial Informatics , vol. 17, no. 8, pp. 5605–5614, Aug. 2021

    2021

  5. [13]

    Adaptive digital twin for vehicular edge computing and networks,

    Y . Dai and Y . Zhang, “Adaptive digital twin for vehicular edge computing and networks,” Journal of Communications and Information Networks, vol. 7, no. 1, pp. 48–59, Mar. 2022

    2022

  6. [14]

    Digital twin-enhanced deep reinforcement learning for resource management in networks slicing,

    Z. Zhang, Y . Huang, C. Zhang, Q. Zheng, L. Yang, and X. You, “Digital twin-enhanced deep reinforcement learning for resource management in networks slicing,” IEEE Transactions on Communications , pp. 1–1, May 2024. 14

    2024

  7. [15]

    Digital twin-driven collaborative scheduling for heterogeneous task and edge-end resource via multi-agent deep reinforcement learning,

    C. Xu, Z. Tang, H. Yu, P. Zeng, and L. Kong, “Digital twin-driven collaborative scheduling for heterogeneous task and edge-end resource via multi-agent deep reinforcement learning,” IEEE Journal on Selected Areas in Communications , vol. 41, no. 10, pp. 3056–3069, 2023

    2023

  8. [16]

    Blockchain- aided digital twin offloading mechanism in space-air-ground networks,

    Y . Gong, H. Yao, Z. Xiong, C. L. P. Chen, and D. Niyato, “Blockchain- aided digital twin offloading mechanism in space-air-ground networks,” IEEE Transactions on Mobile Computing , vol. 24, no. 1, pp. 183–197, 2025

    2025

  9. [17]

    A joint communication and computation framework for digital twin over wireless networks,

    Z. Yang, M. Chen, Y . Liu, and Z. Zhang, “A joint communication and computation framework for digital twin over wireless networks,” IEEE Journal of Selected Topics in Signal Processing , vol. 18, no. 1, pp. 6–17, Dec. 2024

    2024

  10. [18]

    A dynamic hierarchical framework for IoT-assisted digital twin synchronization in the metaverse,

    Y . Han, D. Niyato, C. Leung, D. I. Kim, K. Zhu, S. Feng, X. Shen, and C. Miao, “A dynamic hierarchical framework for IoT-assisted digital twin synchronization in the metaverse,” IEEE Internet of Things Journal, vol. 10, no. 1, pp. 268–284, Aug. 2023

    2023

  11. [19]

    Optimizing synchronization delay for digital twin over wireless networks,

    Z. Yang, M. Chen, Y . Liu, and Z. Zhang, “Optimizing synchronization delay for digital twin over wireless networks,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr. 2024, pp. 9106–9110

    2024

  12. [20]

    Uav- assisted digital twin synchronization with tiny machine learning-based semantic communications,

    J. Tang, J. Nie, J. Bai, J. Xu, S. Li, Y . Zhang, and Y . Yuan, “Uav- assisted digital twin synchronization with tiny machine learning-based semantic communications,” IEEE Internet of Things Journal, to appear, May, 2024

    2024

  13. [21]

    Data synchronization in vehicular digital twin network: A game theoretic approach,

    J. Zheng, T. H. Luan, Y . Zhang, R. Li, Y . Hui, L. Gao, and M. Dong, “Data synchronization in vehicular digital twin network: A game theoretic approach,” IEEE Transactions on Wireless Communications , vol. 22, no. 11, pp. 7635–7647, Mar. 2023

    2023

  14. [22]

    Deep reinforcement learning for downlink scheduling in 5G and beyond networks: A review,

    M. Seguin, A. Omer, M. Koosha, F. Malandra, and N. Mastronarde, “Deep reinforcement learning for downlink scheduling in 5G and beyond networks: A review,” in 2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Sept. 2023, pp. 1–6

    2023

  15. [23]

    Deep reinforcement learning for dynamic uplink/downlink resource allocation in high mobility 5G hetnet,

    F. Tang, Y . Zhou, and N. Kato, “Deep reinforcement learning for dynamic uplink/downlink resource allocation in high mobility 5G hetnet,” IEEE Journal on Selected Areas in Communications , vol. 38, no. 12, pp. 2773–2782, Jun. 2020

    2020

  16. [24]

    Uplink power control framework based on reinforcement learning for 5G networks,

    F. H. Costa Neto, D. C. Ara ´ujo, M. P. Mota, T. F. Maciel, and A. L. F. de Almeida, “Uplink power control framework based on reinforcement learning for 5G networks,” IEEE Transactions on Vehicular Technology, vol. 70, no. 6, pp. 5734–5748, Apr. 2021

    2021

  17. [25]

    Deep reinforcement learning for resource demand prediction and virtual function network migration in digital twin network,

    Q. Liu, L. Tang, T. Wu, and Q. Chen, “Deep reinforcement learning for resource demand prediction and virtual function network migration in digital twin network,” IEEE Internet of Things Journal , vol. 10, no. 21, pp. 19 102–19 116, Jun. 2023

    2023

  18. [26]

    Adaptive edge association for wireless digital twin networks in 6G,

    Y . Lu, S. Maharjan, and Y . Zhang, “Adaptive edge association for wireless digital twin networks in 6G,” IEEE Internet of Things Journal, vol. 8, no. 22, pp. 16 219–16 230, July 2021

    2021

  19. [27]

    Adaptive digital twin and multi- agent deep reinforcement learning for vehicular edge computing and networks,

    K. Zhang, J. Cao, and Y . Zhang, “Adaptive digital twin and multi- agent deep reinforcement learning for vehicular edge computing and networks,” IEEE Transactions on Industrial Informatics , vol. 18, no. 2, pp. 1405–1413, Feb. 2022

    2022

  20. [28]

    Digital-twin- assisted task assignment in multi-uav systems: A deep reinforcement learning approach,

    X. Tang, X. Li, R. Yu, Y . Wu, J. Ye, F. Tang, and Q. Chen, “Digital-twin- assisted task assignment in multi-uav systems: A deep reinforcement learning approach,” IEEE Internet of Things Journal , vol. 10, no. 17, pp. 15 362–15 375, Mar. 2023

    2023

  21. [29]

    3GPP TS 23.501: Sys- tem Architecture for the 5G System (5GS),

    3rd Generation Partnership Project (3GPP), “3GPP TS 23.501: Sys- tem Architecture for the 5G System (5GS),” https://www.3gpp.org/ dynareport/23501.htm, Dec. 2017

    2017

  22. [30]

    A general upper bound to evaluate packet error rate over quasi-static fading channels,

    Y . Xi, A. Burr, J. Wei, and D. Grace, “A general upper bound to evaluate packet error rate over quasi-static fading channels,” IEEE Transactions on Wireless Communications, vol. 10, no. 5, pp. 1373–1377, May 2011

    2011

  23. [31]

    The logarithmic nature of qoe and the role of the weber-fechner law in QoE assessment,

    P. Reichl, S. Egger, R. Schatz, and A. D’Alconzo, “The logarithmic nature of qoe and the role of the weber-fechner law in QoE assessment,” in International Conference on Communications , Cape Town, South Africa, May 2010

    2010

  24. [32]

    Multistate constraint multipath-assisted positioning and mismatch alleviation,

    X. Xu, A. Peng, X. Hong, Y . Zhang, and X.-P. Zhang, “Multistate constraint multipath-assisted positioning and mismatch alleviation,” IEEE Internet of Things Journal , vol. 11, no. 7, pp. 11 271–11 286, Nov. 2024

    2024

  25. [33]

    Altman, Constrained Markov decision processes

    E. Altman, Constrained Markov decision processes . Routledge, Dec. 2021

    2021

  26. [34]

    Performance optimization for digital internet-of-things twins over wireless networks,

    Y . Zhu, S. Wang, H. Tong, and C. Yin, “Performance optimization for digital internet-of-things twins over wireless networks,” in 2023 IEEE International Conference on Communications Workshops (ICC Workshops), May 2023

    2023

  27. [35]

    Toward enhanced reinforcement learning-based resource management via dig- ital twin: Opportunities, applications, and challenges,

    N. Cheng, X. Wang, Z. Li, Z. Yin, T. Luan, and X. S. Shen, “Toward enhanced reinforcement learning-based resource management via dig- ital twin: Opportunities, applications, and challenges,” IEEE Network , vol. 1, no. 1, pp. 1–7, Aug. 2024

    2024

  28. [36]

    Continual reinforcement learning with multi-timescale replay,

    C. Kaplanis, C. Clopath, and M. Shanahan, “Continual reinforcement learning with multi-timescale replay,” arXiv:2004.07530, Apr. 2020

    2004 arXiv

  29. [37]

    Feasible actor-critic: Constrained reinforcement learning for ensuring statewise safety,

    H. Ma, Y . Guan, S. E. Li, X. Zhang, S. Zheng, and J. Chen, “Feasible actor-critic: Constrained reinforcement learning for ensuring statewise safety,” arXiv:2105.10682, May 2021

    2021 arXiv

  30. [38]

    The age of incorrect in- formation: an enabler of semantics-empowered communication,

    A. Maatouk, M. Assaad, and A. Ephremides, “The age of incorrect in- formation: an enabler of semantics-empowered communication,” IEEE Transactions on Wireless Communications , to appear, 2022

    2022

  31. [39]

    Last-iterate convergent policy gradient primal-dual methods for constrained MDPs,

    D. Ding, C.-Y . Wei, K. Zhang, and A. Ribeiro, “Last-iterate convergent policy gradient primal-dual methods for constrained MDPs,” in Ad- vances in Neural Information Processing Systems (NeurIPS) , vol. 36. Curran Associates, Inc., Dec. 2023

    2023

  32. [41]

    Soft actor-critic algorithms and applications,

    T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V . Kumar, H. Zhu, A. Gupta, P. Abbeel et al. , “Soft actor-critic algorithms and applications,” arXiv:1812.05905, Dec. 2018

    2018 arXiv

  33. [42]

    Invariant risk minimization,

    M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz, “Invariant risk minimization,” arXiv:1907.02893, Mar. 2020

    1907 arXiv

  34. [43]

    Semantic- aware remote state estimation in digital twin with minimizing age of incorrect information,

    H. Tong, S. Wang, Z. Yang, J. Zhao, M. Bennis, and C. Yin, “Semantic- aware remote state estimation in digital twin with minimizing age of incorrect information,” in IEEE Global Communications Conference (Globecom), Kuala Lumpur, Malaysia, Dec. 2023

    2023

  35. [44]

    Federated learning based audio semantic communication over wireless networks,

    H. Tong, Z. Yang, S. Wang, Y . Hu, W. Saad, and C. Yin, “Federated learning based audio semantic communication over wireless networks,” in Proc. IEEE Global Communications Conference (GLOBECOM) , Madrid, Spain, Dec. 2021

    2021

  36. [45]

    Intel berkeley research lab sensor data,

    P. Bodik, W. Hong, C. Guestrin, S. Madden, M. Paskin, and R. Thibaux, “Intel berkeley research lab sensor data,” Downloaded from http://db. csail.mit.edu/labdata/labdata.html, Jun. 2004

    2004

  37. [46]

    Indoor received signal strength data generated from ray- tracing,

    S. Sun, “Indoor received signal strength data generated from ray- tracing,” Dec. 2020. [Online]. Available: https://dx.doi.org/10.21227/ r9n4-xz17

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.