Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

ReinFog: A Deep Reinforcement Learning Empowered Framework for Resource Management in Edge and Cloud Computing Environments

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ReinFog brings centralized and distributed deep reinforcement learning to IoT scheduling across edge, fog, and cloud, reporting 45% lower response time, 39% lower energy, and 37% lower weighted cost than a genetic-algorithm baseline.

desk verdict ReinFog is a genuine, well-architected systems contribution for DRL-based scheduling, but the headline 45/39/37% gains are not yet supported because the OHNSGA baseline extension is unspecified and no variance is reported. read the letter →

arxiv 2411.13121 v3 pith:VD7W3FWL submitted 2024-11-20 cs.DC

Zhiyu Wang , Mohammad Goudarzi , Rajkumar Buyya This is my paper · ORCID
classification cs.DC
keywords InternetofThingsEdgeComputingFogCloudDeepReinforcementLearningDistributedSoftwareSystemsIoTApplicationSchedulingResourceManagement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ReinFog is a containerized, modular framework for scheduling IoT applications across heterogeneous edge, fog, and cloud nodes, with deep reinforcement learning at the core of the scheduling decisions. The paper's central claim is that a single framework can host both centralized and distributed DRL techniques, as native implementations or as imported libraries, and that these DRL schedulers beat traditional meta-heuristic scheduling on several objectives at once. Measured against OHNSGA, a genetic-algorithm scheduler from the FogBus2 framework that ReinFog extends, the DRL techniques reduce response time by 45%, energy consumption by 39%, and weighted cost by 37%, while keeping per-decision overhead in the 8–28 ms range. The paper also introduces MADCP, a memetic algorithm that places DRL Learners and Workers across nodes, which it reports accelerates training convergence by up to 38%.

What carries the argument

The Worker/Learner split is the load-bearing architectural pattern: DRL Workers carry the Environment Engine (state processing, action conversion, reward generation) and emit scheduling decisions, while DRL Learners carry the DRL Core (exploration engine, network architecture hub, DRL technique repository, policy constructor) and update the global policy. The DRL Technique Repository is the integration point that matters, offering a base class for native technique implementations and an adapter interface for external libraries. MADCP is the proposed memetic placement algorithm, combining genetic-algorithm operations, firefly movement, and particle-swarm velocity updates to solve the NP-hard assignment of DRL components to nodes under CPU, memory, and deadline constraints, with overall complexity $O(G \times P^2 \times M)$. The scheduling problem itself models each IoT application as a directed acyclic graph of dependent tasks, and the reward function penalizes response time, energy consumption, and their weighted combination.

What would settle it

Reproduce the evaluation on the same four IoT applications and testbed with OHNSGA replaced by a comparably tuned standard multi-objective optimizer such as NSGA-II with identical objectives, and report the spread across the paper's 10 runs; if the 45%, 39%, and 37% gaps shrink to within run-to-run variability, the central performance claim would not survive.

Watch

Extended reading notes

Core claim

ReinFog's core claim is that a modular, containerized framework can make deep reinforcement learning practical for IoT application scheduling in edge/fog/cloud environments without sacrificing the flexibility researchers need. The design separates the learning system into DRL Workers, which interact with the environment and produce scheduling decisions, and DRL Learners, which aggregate experience and optimize the global policy, an arrangement that spans the spectrum from a single centralized agent (DQN) to distributed actor-learner architectures (A3C, IMPALA). The same modular core lets users implement DRL techniques natively or import mature libraries, with R2D2 and SAC from the Ray library serving as the integration examples. On a testbed mixing cloud VMs, laptops, and a Raspberry Pi, the DRL schedulers converge to substantially better schedules than the GA-based OHNSGA baseline across response time, energy, and a weighted combined cost, with linear scaling as Workers grow from 1 to 30. The framework's placement engine, MADCP, treats the placement of DRL components as a constrained combinatorial optimization problem and shows up to 38% faster convergence than random placement.

Load-bearing premise

The reported 45%, 39%, and 37% gains are measured against a single genetic-algorithm scheduler that the authors extended themselves, so the comparison assumes that extended baseline is a fair and representative opponent.

Editorial extensions

If this is right

  • The same framework hosts centralized agents (DQN, PPO) and distributed actor-learner systems (A3C, IMPALA), so researchers can compare scheduling techniques without rebuilding environment interfaces for each one.
  • DRL scheduling is light enough for real deployments: decisions cost 8–28 ms, and growing from 1 to 30 DRL Workers adds only about 0.3 seconds of startup time and roughly 60 MB of RAM.
  • Across all three objectives, the DRL techniques beat the GA-based OHNSGA scheduler, with IMPALA the strongest in convergence speed and scalability.
  • Where the learning components sit matters: MADCP placement accelerates convergence by up to 38% and roughly halves the per-decision overhead of random placement.
  • The trained schedulers generalize to workloads they did not train on, as gains persist when evaluation video resolution drops from 480p to 240p.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stronger test than OHNSGA alone would pit the framework's DRL schedulers against well-tuned NSGA-II variants and against other DRL schedulers; if the gap shrinks, the headline numbers would read as 'DRL versus one GA baseline' rather than 'DRL versus meta-heuristics in general.'
  • Because the paper does not ablate the framework from the algorithms, running the same DRL techniques standalone would separate the contribution of the reward design and component placement from the contribution of the algorithms themselves.
  • The Worker/Learner interfaces are general enough to host techniques the paper does not try, such as multi-agent reinforcement learning, offline RL from logged schedules, or privacy-preserving federated policy sharing — the last being exactly the security direction the paper lists as future work.
  • The three-region CO2 analysis implies an operational consequence worth testing in practice: DRL-based scheduling cuts the carbon footprint of IoT processing most where the electricity mix rewards the energy savings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes ReinFog, a containerized and modular framework for IoT application scheduling in edge/fog and cloud environments, built on top of FogBus2. It supports centralized and distributed DRL techniques, both natively implemented and imported from external libraries (e.g., Ray), with customizable deployment of DRL Learners and Workers. It also presents MADCP, a memetic algorithm combining genetic algorithm, firefly algorithm, and particle swarm optimization for placing DRL components. Experiments compare six DRL techniques (IMPALA, A3C, PPO, DQN, R2D2, SAC) against a GA-based heuristic OHNSGA, reporting reductions of up to 45% in response time, 39% in energy consumption, and 37% in weighted cost, as well as low startup/RAM overhead and good scalability. The paper claims that ReinFog is the first framework comprehensively integrating centralized and distributed DRL techniques for IoT application scheduling, and that MADCP accelerates DRL convergence by up to 38%.

Significance. If its quantitative claims were substantiated, ReinFog would be a valuable contribution as a flexible testbed for DRL-based resource management research. The modular architecture—separating DRL Learners from Workers, providing both native and library-based integration, and adding a placement optimization component—addresses a real gap, since most existing frameworks are heuristic-based or support only a single RL technique. The implementation of six DRL techniques within one framework is a substantial engineering effort, and the scalability measurements (RAM and startup time) are concrete and worth reporting. However, the empirical evidence does not currently support the headline performance claims: the only baseline is an underspecified extension of the authors' own OHNSGA algorithm, no variance or statistical tests are reported, and no independent DRL-based scheduler is compared. No code or artifact is released, limiting reproducibility. With a properly specified baseline, variance reporting, and an independent comparison, the framework contribution could be significant; as it stands, the central performance claims are not yet established.

major comments (5)
  1. [§6.1, §6.5.1] The OHNSGA baseline is not sufficiently specified: the paper states in §6.1 that the authors 'extended this technique to support multi-objective optimization' but never defines the extension's fitness function, the multi-objective handling method (e.g., weighted sum versus Pareto dominance), how a single solution is selected from a Pareto front, or how the normalization in Eq. (23) interacts with this baseline. Consequently, the headline reductions of up to 45/39/37% reported in §6.5.1 are not interpretable as a valid measure of ReinFog's advantage; they could reflect an unfavorable or ad hoc baseline construction rather than a genuine improvement.
  2. [§6, §6.5.1] The paper reports only averages over 10 repeated experiments with no standard deviations, confidence intervals, or statistical significance tests. In the convergence comparisons (Figs. 10 and 11) and the scalability analysis (Fig. 13), the absence of variance information makes it impossible to determine whether the observed gaps between techniques are stable or within run-to-run noise; the claim of 'significant improvements' is therefore not statistically supported.
  3. [§6.5, Table 1] No comparison is made against any existing DRL-based scheduling framework or technique, despite the paper's claim of being 'the first framework that comprehensively integrates both centralized and distributed DRL techniques.' The qualitative novelty argument in Table 1 is a self-assessed feature matrix, and the quantitative evaluation compares only against OHNSGA, a baseline from the authors' own prior FogBus2 work; without an independent DRL scheduler comparison, the claims of superiority over prior DRL-based approaches are not supported.
  4. [§6.3, Eq. (23)] The weighted cost metric in Eq. (23) uses a normalization function Norm(x) 'relative to a predefined baseline,' but the baseline is never specified anywhere in the manuscript. Since weighted cost is used as a primary metric in the convergence, scalability, and overhead analyses, the reported numeric values (e.g., 0.14–0.25 in Fig. 13) cannot be reproduced or interpreted without knowing what that baseline is.
  5. [§6.6.1] The claim that MADCP 'accelerates the convergence rate by up to 38%' compared to random placement is not backed by a precise definition of convergence rate; neither the number of iterations to stabilization nor a threshold-based definition is given. Figure 15 shows curves, but the quantification is informal and no variance or statistical test accompanies the claim.
minor comments (5)
  1. [§7] The concluding paragraph contains a grammatical error: 'It offers capabilities to supporting centralized and distributed DRL techniqueintegrate both native and library-based DRL techniques, whiles.' This should be rephrased for clarity.
  2. [§6.4.3] The CO2 emission values (e.g., 0.99g per hour for Australia) lack context; please clarify whether these figures are per application or for the entire environment, and state the underlying electricity consumption and emission-factor calculations.
  3. [§6.5.2] In the overhead comparison, DQN and OHNSGA are both around 8 ms, yet the text says 'OHNSGA exhibits the lowest overhead'; this should be qualified as 'lowest among all techniques' or 'tied with DQN' to avoid an inaccurate impression.
  4. [§6.2] Table 2 lists hyperparameters for DRL techniques but omits the OHNSGA parameters (population size and generations) that are discussed in the text; including them in the table would improve completeness.
  5. [Figures 10, 11, 13] The figure captions do not indicate the number of runs or the error-bar representation; please add a note that all curves are averages over 10 runs and state where the variance information is (or is not) shown.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ReinFog's claims are empirical and self-contained, with performance measured against an underspecified but external baseline rather than derived from its own assumptions.

full rationale

This paper is an empirical systems/engineering contribution, not a derivation whose conclusion is fixed by its premises. ReinFog's central claims—that it is the first framework to integrate centralized and distributed DRL techniques, that MADCP is an effective component-placement heuristic, and that the DRL schedulers reduce response time, energy consumption, and weighted cost—are supported by an implementation and by comparative experiments rather than by equations that reduce to their own inputs. The 45%/39%/37% figures are measured against OHNSGA in Section 6.5.1; OHNSGA is a baseline from the authors' own FogBus2 work and its multi-objective extension is underspecified in Section 6.1, but that is an experimental-fairness and reproducibility concern, not circularity, because the numbers are empirical outcomes rather than fitted constraints. The self-citations to FogBus2 and prior DRL scheduling work are used as foundation, baseline, and related work, not as evidence that ReinFog's DRL mechanisms work. No fitted parameter is renamed as a prediction, no uniqueness theorem from the authors is imported into the argument, and no ansatz is smuggled in via citation. Accordingly, no circular steps are present.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest on tuned hyperparameters for each DRL technique and placement algorithm, on the DAG-based application model, and on the defined reward function. No new physical or theoretical entities are introduced.

free parameters (3)
  • DRL technique hyperparameters = learning rate 0.0001-0.01, discount 0.8-0.99, hidden units [256,256,128], etc.
    Optimal hyperparameters were selected via grid search per technique (Section 6.2). The claimed performance improvements depend on these tuned settings.
  • MADCP parameters = population size 200, generations 100, crossover 0.8, gamma 0.5, w 0.7 (for IMPALA); w 0.8 for R2D2
    Grid-searched in Section 6.2 for MADCP when paired with different DRL techniques. The convergence acceleration claim depends on these settings.
  • Weighted cost weights w1, w2 = 0.5, 0.5
    Set by hand in Section 6.3 to give equal priority to response time and energy. The weighted cost metric and the reported 37% improvement depend on this choice.
assumptions (3)
  • domain assumption IoT applications can be modeled as DAGs of interdependent tasks
    Sections 3.1.3 and 3.2.1: the scheduling problem, state representation, and DAG construction assume this structure.
  • domain assumption The reward function is the negative of the optimization metrics (response time, energy, weighted cost)
    Section 6.3: DRL training optimizes this reward, and the reported improvements are measured against these metrics.
  • standard math Standard DRL algorithms (IMPALA, A3C, PPO, DQN, R2D2, SAC) behave as described in their cited references
    The paper relies on the correctness of these published algorithms and their implementations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReinFog: A Deep Reinforcement Learning Empowered Framework for Resource Management in Edge and Cloud Computing Environments." pith.science (2026). https://pith.science/paper/VD7W3FWL

@misc{pith2026241113121,
  author       = {Pith},
  title        = {Pith review of: ReinFog: A Deep Reinforcement Learning Empowered Framework for Resource Management in Edge and Cloud Computing Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VD7W3FWL}},
  note         = {Machine review of arXiv:2411.13121}
}
read the original abstract

The growing IoT landscape requires effective server deployment strategies to meet demands including real-time processing and energy efficiency. This is complicated by heterogeneous, dynamic applications and servers. To address these challenges, we propose ReinFog, a modular distributed software empowered with Deep Reinforcement Learning (DRL) for adaptive resource management across edge/fog and cloud environments. ReinFog enables the practical development/deployment of various centralized and distributed DRL techniques for resource management in edge/fog and cloud computing environments. It also supports integrating native and library-based DRL techniques for diverse IoT application scheduling objectives. Additionally, ReinFog allows for customizing deployment configurations for different DRL techniques, including the number and placement of DRL Learners and DRL Workers in large-scale distributed systems. Besides, we propose a novel Memetic Algorithm for DRL Component (e.g., DRL Learners and DRL Workers) Placement in ReinFog named MADCP, which combines the strengths of Genetic Algorithm, Firefly Algorithm, and Particle Swarm Optimization. Experiments reveal that the DRL mechanisms developed within ReinFog have significantly enhanced both centralized and distributed DRL techniques implementation. These advancements have resulted in notable improvements in IoT application performance, reducing response time by 45%, energy consumption by 39%, and weighted cost by 37%, while maintaining minimal scheduling overhead. Additionally, ReinFog exhibits remarkable scalability, with a rise in DRL Workers from 1 to 30 causing only a 0.3-second increase in startup time and around 2 MB more RAM per Worker. The proposed MADCP for DRL component placement further accelerates the convergence rate of DRL techniques by up to 38%.

Figures

Figures reproduced from arXiv: 2411.13121 by the authors.

Figure 1
Figure 1. Heterogeneous multi-layered hardware environment for ReinFog 3.1.1. Cloud Layer The Cloud Layer represents the highest tier of computing resources in the ReinFog hardware environment. It consists of high-performance servers provided by different cloud service providers such as Amazon Web Services (AWS), Microsoft Azure, and Nectar. These cloud environments offer scalable computing power and storage capabilities, hig… view at source ↗
Figure 2
Figure 2. High-level software architecture of ReinFog supports multiple Master instances for scalability and fault tolerance. • Actor: Distributed across various nodes in the system, Actors are responsible for executing assigned tasks and managing local resources. They implement the actual task execution, monitor local system perfor￾mance, and handle data transfer between tasks when necessary. Actors continuously report resou… view at source ↗
Figure 3
Figure 3. ReinFog design overview [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Distributed DRL components design via the DRL Policy Constructor, enabling it to access and utilize current technique states and network archi￾tectures. This API-driven design enhances modularity and facilitates easy integration of new exploration strategies. 2. Networ…
Figure 5
Figure 5. Figure 5: Extended FogBus2 scheduler module Task Scheduling Decision Generator: The Task Schedul￾ing Decision Generator communicates with DRL com￾ponents to obtain IoT task scheduling decisions. It then manages the deployment of these tasks based on the received decisions. This …
Figure 6
Figure 6. Figure 6: Startup time comparison between ReinFog and FogBus2 components the lightweight nature of our framework despite its DRL capabilities. Next, we assess the environmental impact by evaluating carbon dioxide (CO2 ) emissions across the elec￾tricity generation patterns in di…
Figure 8
Figure 8. Figure 8: , ReinFog consistently produces lower CO2 emissions than FogBus2 across all regions. Specifically, in Australia, ReinFog emits 0.99g compared to FogBus2’s 1.24g. The difference is more pronounced in the USA, where ReinFog generates 1.35g versus FogBus2’s 1.67g. Germany…
Figure 9
Figure 9. Figure 9: Impact of increasing DRL Workers on ReinFog RAM usage and startup time 6.4.4. Framework Scalability Analysis In this experiment, we evaluate the scalability of Rein￾Fog by systematically varying the number of DRL Workers from 1 to 30. As illustrated in [PITH_FULL_IMAG…
Figure 10
Figure 10. Figure 10: Convergence performance comparison of scheduling techniques during training phase we assess the environmental impact of different techniques by analyzing the CO2 emissions. 6.5.1. Scheduling Techniques Convergence Analysis In this experiment, we evaluate and compare t…
Figure 11
Figure 11. Figure 11: Convergence performance comparison of scheduling techniques during evaluation phase [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 14
Figure 14. Figure 14: Hourly CO2 emissions comparison of scheduling techniques across different regions environmental sustainability compared to traditional meta￾heuristic methods. 6.6. DRL Component Placement Algorithm Analysis In this section, we evaluate the performance of our pro￾posed…
Figure 13
Figure 13. Figure 13: Impact of number of nodes on scheduling perfor￾mance across different scheduling techniques all DRL techniques demonstrate lower CO2 emissions com￾pared to OHNSGA. Among the DRL techniques, IMPALA shows the lowest emissions across all regions (0.84g in Australia, 1.14…
Figure 15
Figure 15. Figure 15: Impact of different DRL component placement algorithms on IMPALA’s convergence performance [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: Impact of different DRL component placement algorithms on IMPALA’s average scheduling overhead after 90 iterations. Notably, MADCP accelerates the con￾vergence rate by up to 38% compared to random placement method. These results highlight MADCP’s effectiveness in prov…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Resilience Evaluation of Kubernetes in Cloud-Edge Environments via Failure Injection

    cs.DC 2025-07 reject novelty 5.0 of 10

    Under network delay and partition faults, cloud-edge Kubernetes deployments show tighter response-time distributions than cloud-only deployments, while cloud deployments stay more stable under bandwidth throttling and...

Reference graph

Works this paper leans on

50 extracted references · 48 canonical work pages · cited by 1 Pith paper

  1. [1]

    Jamil, H

    B. Jamil, H. Ijaz, M. Shojafar, K. Munir, R. Buyya, Resource allo- cation and task scheduling in fog computing and internet of every- thingenvironments:Ataxonomy,review,andfuturedirections,ACM Computing Surveys 54 (11s) (2022) 1–38

    work page 2022

  2. [2]

    L. M. Meruje Ferreira, F. Coelho, J. Pereira, Databases in edge and fogenvironments:Asurvey,ACMComputingSurveys56(11)(2024) 1–40

    work page 2024

  3. [3]

    Y.Mansouri,M.A.Babar,Areviewofedgecomputing:Featuresand resourcevirtualization,JournalofParallelandDistributedComputing 150 (2021) 155–183

    work page 2021

  4. [4]

    Jeyaraj, A

    R. Jeyaraj, A. Balasubramaniam, A. K. MA, N. Guizani, A. Paul, Resource management in cloud and cloud-influenced technologies for internet of things applications, ACM Computing Surveys 55 (12) (2023) 1–37

    work page 2023

  5. [5]

    J.Jin,K.Yu,J.Kua,N.Zhang,Z.Pang,Q.-L.Han,Cloud-fogautoma- tion: Vision, enabling technologies, and future research directions, IEEE Transactions on Industrial Informatics 20 (2) (2023) 1039– 1054

    work page 2023

  6. [6]

    Wang, Goudarzi, Buyya Page 22 of 24 ReinFog: A DRL Empowered Framework for Resource Management in Edge and Cloud Computing Environments

    M.Goudarzi,M.Palaniswami,R.Buyya,Schedulingiotapplications in edge and fog computing environments: A taxonomy and future directions, ACM Computing Surveys 55 (7) (2022) 1–41. Wang, Goudarzi, Buyya Page 22 of 24 ReinFog: A DRL Empowered Framework for Resource Management in Edge and Cloud Computing Environments

    work page 2022

  7. [7]

    Buyya, S

    R. Buyya, S. N. Srirama, R. Mahmud, M. Goudarzi, L. Ismail, V. Kostakos, Quality of service (qos)-driven edge computing and smarthospitals:avision,architecturalelements,andfuturedirections, in: International Conference on Communication, Electronics and Digital Technology, Springer, 2023, pp. 1–23

    work page 2023

  8. [8]

    Goudarzi, M

    M. Goudarzi, M. A. Rodriguez, M. Sarvi, R. Buyya,𝜇-ddrl: A qos- aware distributed deep reinforcement learning technique for service offloading in fog computing environments, IEEE Transactions on Services Computing 17 (1) (2024) 47–59

    work page 2024

Show all 50 references
  1. [9]

    Sharif, M

    S. Sharif, M. H. Y. Moghaddam, S. A. H. Seno, A hybrid bi-level management framework for caching and communication in edge- ai enabled iot, Journal of Network and Computer Applications 232 (2024) 104000

    2024

  2. [10]

    E.S.Ali,R.A.Saeed,I.K.Eltahir,O.O.Khalifa,Asystematicreview onenergyefficiencyintheinternetofunderwaterthings(iout):Recent approaches and research gaps, Journal of Network and Computer Applications 213 (2023) 103594

    2023

  3. [11]

    W. Chen, X. Qiu, T. Cai, H.-N. Dai, Z. Zheng, Y. Zhang, Deep re- inforcement learning for internet of things: A comprehensive survey, IEEECommunicationsSurveys&Tutorials23(3)(2021)1659–1692

    2021

  4. [12]

    Z. Wang, M. Goudarzi, M. Gong, R. Buyya, Deep reinforcement learning-based scheduling for optimizing system load and response time in edge and fog computing environments, Future Generation Computer Systems 152 (2024) 55–69

    2024

  5. [13]

    Huang, C

    J. Huang, C. Yang, S. Zhang, F. Yang, O. Alfarraj, V. Frascolla, S. Mumtaz, K. Yu, Reinforcement learning based resource manage- ment for 6g-enabled miot with hypergraph interference model, IEEE Transactions on Communications 72 (7) (2024) 4179–4192

    2024

  6. [14]

    Z.Wang,M.Goudarzi,J.Aryal,R.Buyya,Containerorchestrationin edge and fog computing environments for real-time iot applications, in: Proceedings of the International Conference on Computational IntelligenceandDataAnalytics(ICCIDA),Springer,2022,pp.1–21

    2022

  7. [15]

    G. Zhou, R. Wen, W. Tian, R. Buyya, Deep reinforcement learning- basedalgorithmsselectorsfortheresourceschedulinginhierarchical cloudcomputing,JournalofNetworkandComputerApplications208 (2022) 103520

    2022

  8. [16]

    C. Wu, W. Li, L. Wang, A. Y. Zomaya, Hybrid evolutionary schedul- ing for energy-efficient fog-enhanced internet of things, IEEE Trans- actions on Cloud Computing 9 (2) (2018) 641–653

    2018

  9. [17]

    Hoseiny, S

    F. Hoseiny, S. Azizi, M. Shojafar, F. Ahmadiazar, R. Tafazolli, Pga: a priority-aware genetic algorithm for task scheduling in heteroge- neous fog-cloud computing, in: Proceedings of the IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), IEEE, 2021, pp. 1–6

    2021

  10. [18]

    I. M. Ali, K. M. Sallam, N. Moustafa, R. Chakraborty, M. Ryan, K.-K. R. Choo, An automated task scheduling model using non- dominated sorting genetic algorithm ii for fog-cloud systems, IEEE Transactions on Cloud Computing 10 (4) (2020) 2294–2308

    2020

  11. [19]

    J. Zou, T. Hao, C. Yu, H. Jin, A3C-DO: A regional resource schedul- ingframeworkbasedondeepreinforcementlearninginedgescenario, IEEE Transactions on Computers 70 (2) (2020) 228–239

    2020

  12. [20]

    Siyadatzadeh, F

    R. Siyadatzadeh, F. Mehrafrooz, M. Ansari, B. Safaei, M. Shafique, J.Henkel,A.Ejlali,Relief:Areinforcement-learning-basedreal-time task assignment strategy in emerging fault-tolerant fog computing, IEEE Internet of Things Journal 10 (12) (2023) 10752–10763

    2023

  13. [21]

    L. Liu, J. Feng, X. Mu, Q. Pei, D. Lan, M. Xiao, Asynchronous deep reinforcement learning for collaborative task computing and on-demand resource allocation in vehicular edge computing, IEEE Transactions on Intelligent Transportation Systems 24 (12) (2023) 15513–15526

    2023

  14. [22]

    Y.Zheng,H.Zhou,R.Chen,K.Jiang,Y.Cao,Sac-basedcomputation offloading and resource allocation in vehicular edge computing, in: Proceedings of the IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), IEEE, 2022, pp. 1–6

    2022

  15. [23]

    Huang, X

    L. Huang, X. Feng, C. Zhang, L. Qian, Y. Wu, Deep reinforce- ment learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing, Digital Communications and Networks 5 (1) (2019) 10–17

    2019

  16. [24]

    Yigitoglu, M

    E. Yigitoglu, M. Mohamed, L. Liu, H. Ludwig, Foggy: A framework for continuous automated iot application deployment in fog comput- ing, in: Proceedings of the IEEE International Conference on AI & Mobile Services (AIMS), IEEE, 2017, pp. 38–45

    2017

  17. [25]

    Merlino, R

    G. Merlino, R. Dautov, S. Distefano, D. Bruneo, Enabling workload engineering in edge, fog, and cloud computing through openstack- based middleware, ACM Transactions on Internet Technology 19 (2) (2019) 1–22

    2019

  18. [26]

    Zhang, W

    A.Yousefpour,A.Patil,G.Ishigaki,I.Kim,X.Wang,H.C.Cankaya, Q. Zhang, W. Xie, J. P. Jue, Fogplan: A lightweight qos-aware dynamicfogserviceprovisioningframework,IEEEInternetofThings Journal 6 (3) (2019) 5080–5096

    2019

  19. [27]

    S.Ghosh,A.Mukherjee,S.K.Ghosh,R.Buyya,Mobi-iost:mobility- aware cloud-fog-edge-iot collaborative framework for time-critical applications,IEEETransactionsonNetworkScienceandEngineering 7 (4) (2019) 2271–2285

    2019

  20. [28]

    Q. Deng, M. Goudarzi, R. Buyya, Fogbus2: a lightweight and dis- tributedcontainer-basedframeworkforintegrationofiot-enabledsys- tems with edge and cloud computing, in: Proceedings of the Interna- tionalWorkshoponBigDatainEmergentDistributedEnvironments, 2021, pp. 1–8

    2021

  21. [29]

    Pallewatta, V

    S. Pallewatta, V. Kostakos, R. Buyya, Microfog: a framework for scalable placement of microservices-based iot applications in feder- ated fog environments, Journal of Systems and Software 209 (2024) 111910

    2024

  22. [30]

    Tari, Greenfog: A framework for sustainable fog computing, in: Proceedings of the International Conference on Service-Oriented Computing, Springer, 2022, pp

    A.N.Toosi,C.Agarwal,L.Mashayekhy,S.K.Moghaddam,R.Mah- mud, Z. Tari, Greenfog: A framework for sustainable fog computing, in: Proceedings of the International Conference on Service-Oriented Computing, Springer, 2022, pp. 540–549

    2022

  23. [31]

    Nkenyereye, K.-J

    L. Nkenyereye, K.-J. Baeg, W. Chung, Deep reinforcement learning forcontainerizededgeintelligenceinferencerequestprocessinginiot edge computing, IEEE Transactions on Services Computing 16 (6) (2023) 4328–4344

    2023

  24. [32]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proxi- malpolicyoptimizationalgorithms,arXivpreprintarXiv:1707.06347 (2017)

    2017 arXiv

  25. [33]

    Espeholt, H

    L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y.Doron,V.Firoiu,T.Harley,I.Dunning,etal.,Impala:Scalabledis- tributeddeep-rlwithimportanceweightedactor-learnerarchitectures, in:ProceedingsoftheInternationalConferenceonMachineLearning, PMLR, 2018, pp. 1407–1416

    2018

  26. [34]

    V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., Human-level control through deep reinforcement learning, Nature 518 (7540) (2015) 529–533

    2015

  27. [35]

    Arulkumaran, M

    K. Arulkumaran, M. P. Deisenroth, M. Brundage, A. A. Bharath, Deepreinforcementlearning:Abriefsurvey,IEEESignalProcessing Magazine 34 (6) (2017) 26–38

    2017

  28. [36]

    Kapturowski, G

    S. Kapturowski, G. Ostrovski, J. Quan, R. Munos, W. Dabney, Re- current experience replay in distributed reinforcement learning, in: Proceedings of the International Conference on Learning Represen- tations, 2018

    2018

  29. [37]

    J.-M.Renders,S.P.Flasse,Hybridmethodsusinggeneticalgorithms for global optimization, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 26 (2) (1996) 243–258

    1996

  30. [38]

    Wu, Y.-G

    J. Wu, Y.-G. Wang, K. Burrage, Y.-C. Tian, B. Lawson, Z. Ding, An improved firefly algorithm for global continuous optimization problems, Expert Systems with Applications 149 (2020) 113340

    2020

  31. [39]

    P.Moradi,M.Gholampour,Ahybridparticleswarmoptimizationfor feature subset selection by integrating a novel local search strategy, Applied Soft Computing 43 (2016) 117–130

    2016

  32. [40]

    V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, K. Kavukcuoglu, Asynchronous methods for deep rein- forcement learning, in: Proceedings of the International Conference on Machine Learning, PMLR, 2016, pp. 1928–1937

    2016

  33. [41]

    Y. Tang, S. Agrawal, Discretizing continuous action space for on- policy optimization, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 2020, pp. 5981–5988. Wang, Goudarzi, Buyya Page 23 of 24 ReinFog: A DRL Empowered Framework for Resource Managemen...

    2020

  34. [42]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, Soft actor-critic: Off- policymaximumentropydeepreinforcementlearningwithastochas- ticactor,in:ProceedingsoftheInternationalConferenceonMachine Learning, PMLR, 2018, pp. 1861–1870

    2018

  35. [43]

    Glorot, Y

    X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the International ConferenceonArtificialIntelligenceandStatistics,JMLRWorkshop and Conference Proceedings, 2010, pp. 249–256

    2010

  36. [44]

    1441–1450

    F.Sun,J.Liu,J.Wu,C.Pei,X.Lin,W.Ou,P.Jiang,Bert4rec:Sequen- tial recommendation with bidirectional encoder representations from transformer,in:ProceedingsoftheACMInternationalConferenceon Information and Knowledge Management, 2019, pp. 1441–1450

    2019

  37. [45]

    Hessel, J

    M. Hessel, J. Modayil, H. Van Hasselt, T. Schaul, G. Ostrovski, W. Dabney, D. Horgan, B. Piot, M. Azar, D. Silver, Rainbow: Com- biningimprovementsindeepreinforcementlearning,in:Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32, 2018

    2018

  38. [46]

    K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjectivegeneticalgorithm:Nsga-ii,IEEETransactionsonEvo- lutionary Computation 6 (2) (2002) 182–197

    2002

  39. [47]

    K. Deb, H. Jain, An evolutionary many-objective optimization algo- rithm using reference-point-based nondominated sorting approach, part i: solving problems with box constraints, IEEE Transactions on Evolutionary Computation 18 (4) (2013) 577–601

    2013

  40. [48]

    J.D.Schaffer,R.Caruana,L.J.Eshelman,R.Das,Astudyofcontrol parameters affecting online performance of genetic algorithms for function optimization, in: Proceedings of the International Confer- ence on Genetic Algorithms, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, ...

    1989

  41. [49]

    X.-S. Yang, X. He, Firefly algorithm: recent advances and applica- tions,InternationalJournalofSwarmIntelligence1(1)(2013)36–50

    2013

  42. [50]

    Van den Bergh, A

    F. Van den Bergh, A. P. Engelbrecht, A study of particle swarm op- timization particle trajectories, Information Sciences 176 (8) (2006) 937–971. Wang, Goudarzi, Buyya Page 24 of 24

    2006

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.