Pith. sign in

REVIEW 5 major objections 5 minor 28 references

Diffusion-Modeled Reinforcement Learning for Carbon and Risk-Aware Microgrid Optimization

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DiffCarl claims a diffusion-modeled RL controller cuts microgrid cost, carbon, and tail risk together.

desk verdict DiffCarl is a competent engineering application of diffusion policies to microgrid scheduling, but the headline carbon claim is arithmetically wrong and the risk-aware critic update is undefined as written. read the letter →

arxiv 2507.16867 v1 pith:U7ZIK2QU submitted 2025-07-22 cs.LG cs.AI

Yunyi Zhao , Wei Zhang , Cheng Xiang , Hongyang Du , Dusit Niyato , Shuhua Gao This is my paper · ORCID
classification cs.LGcs.AI
keywords diffusionmodelsreinforcementlearningmicrogridenergyschedulingcarbon-awareoptimizationrisk-sensitiveRLCVaRgenerativeAIsoftactor-critic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a reinforcement-learning agent whose policy is a diffusion model can schedule multiple microgrids better than standard deep-RL and optimization baselines on cost, carbon emissions, and worst-case risk at once. The agent, DiffCarl, replaces the stochastic actor of a soft actor-critic controller with a denoising diffusion process that turns Gaussian noise into an action vector conditioned on the current state, and it adds a Conditional Value at Risk penalty to the reward objective. The authors argue this combination gives the policy enough expressiveness to handle renewable, load, and price uncertainty while explicitly trading off economy, emissions, and volatility. If correct, the result would let a single learned controller approximate offline-optimal scheduling in real time without forecasts, while cutting carbon relative to carbon-unaware policies. The paper reports 2.3–30.1% lower operating cost than several baselines, 28.7% lower carbon than its carbon-unaware variant, and lower cost variability under risk-averse settings.

What carries the argument

The load-bearing object is a Denoising Diffusion Probabilistic Model (DDPM) used as the actor: a generative model that learns to reverse a Markov chain which gradually adds Gaussian noise, so that starting from pure noise and conditioning on the current state it produces a sample from the action distribution. In DiffCarl this actor replaces the standard stochastic policy network of soft actor-critic; the paper also uses double critics, target networks, and soft updates from that framework, and adds a risk-adjusted Q-function whose temporal-difference target applies CVaR to the min over the two critics' target values. The diffusion actor is what carries the expressiveness argument; the CVaR target is what carries the risk argument.

What would settle it

The decisive experiment is a controlled ablation: train DiffCarl on the two-microgrid case with the diffusion actor replaced by a Gaussian stochastic actor while keeping the CVaR-based critic, reward, and all hyper-parameters identical, then compare cost, carbon, and the empirical 5% CVaR of daily costs; if the reported gains vanish, the diffusion-modeled policy is the load-bearing component, and if they persist, the risk-aware critic alone may explain the results.

Watch

Extended reading notes

Core claim

On its own terms, DiffCarl is a claim that the policy class matters more than the optimization recipe: replacing a conventional Gaussian policy with a diffusion-modeled policy inside soft actor-critic improves scheduling enough to beat day-ahead optimization, myopic greedy, MPC, DQN, SAC, and DDPG on operating cost across a two-microgrid community and the IEEE 15-bus and 33-bus feeders, while coming within about 2% of an offline optimal schedule on the two-microgrid case. The same framework also reports 28.7% lower carbon emissions than its carbon-unaware counterpart, and risk-averse settings ($\lambda = 1$) reduce the standard deviation of daily cost by 21% relative to the risk-neutral policy at a moderate cost premium. In the paper's formulation, carbon is priced into the reward, and the risk term is the Conditional Value at Risk of the cost distribution, so the policy is trained to avoid the worst 5% of days as well as to minimize average cost. The authors read these results as evidence that generative modeling of action distributions is a practical route to sustainable, resilient microgrid operation.

Load-bearing premise

The paper assumes, without showing a derivation, that its risk-adjusted Q-learning update really minimizes the stated objective of average cost plus a penalty on the worst 5% of days; the risk-awareness results rest entirely on that assumption.

Editorial extensions

If this is right

  • DiffCarl's reported 2.3% to 30.1% cost reduction over DQN, SAC, DDPG, day-ahead, myopic, and MPC baselines would mean a learned controller can beat both optimization and prior RL baselines without needing accurate forecasts.
  • The 28.7% carbon reduction versus the carbon-unaware variant implies that pricing carbon into the reward changes dispatch behavior substantially, shifting purchases away from carbon-intensive sources.
  • The 98.2% approximation of the offline-optimal schedule on the two-microgrid case suggests the model can approach the theoretical cost bound in real time.
  • Risk-averse settings cutting cost standard deviation by up to 21% at a modest premium would give operators a tunable knob for worst-case budgeting.
  • Consistent improvements on IEEE 15-bus and 33-bus feeders suggest the approach scales with system size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct ablation replacing the diffusion actor with a Gaussian stochastic actor, keeping the same CVaR critic and reward, would isolate whether the reported gains come from diffusion-model expressiveness or from other framework details; the paper does not include this ablation.
  • The risk-sensitive update in Eq. (19) is not derived from a distributional RL objective; until it is, the risk results should be read as empirical tuning behavior rather than a proof that the controller minimizes CVaR.
  • If the diffusion actor is the source of the advantage, the same module could transfer to other continuous-action energy tasks such as battery storage arbitrage or smart-manufacturing scheduling, since only state, action, and reward need redefining.
  • The 28.7% carbon figure is relative to the carbon-unaware variant of the same algorithm, so absolute emissions reductions in deployment would depend on carbon prices and grid carbon intensity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes DIFFCARL, an actor-critic algorithm in which the actor is a denoising diffusion model and the critic is trained with a CVaR-modified target, for scheduling ESS, CDG, and load shedding in a multi-microgrid community. The reward includes an operating-cost term with a carbon price, and the policy is regularized with entropy and risk terms. The authors compare against DAY-AHEAD, MYOPIC, MPC, DQN, SAC, and DDPG on a 2-microgrid case and IEEE 15/33-bus systems, reporting lower cost and carbon, and study risk sensitivity by varying lambda. The central claims are that DIFFCARL reduces cost by 2.3-30.1%, carbon by 28.7% versus its carbon-unaware variant, and performance variability.

Significance. Addressing an important application domain, and combining diffusion-modeled policies with carbon- and risk-aware objectives is a timely idea. The experimental setup is reasonably broad (three test systems, real and synthetic data), and the case study provides a useful qualitative check. However, the paper does not provide a well-defined risk-sensitive Q-target, does not define a valid entropy/log-probability for the continuous diffusion policy, and contains an arithmetic inconsistency in its headline carbon claim. No code, seeds, or error bars are reported. As a result, the significance of the contribution cannot be established on the evidence presented; the manuscript is not ready for publication. The idea may be salvageable with a correct distributional/risk formulation and rigorous experiments, but those are substantial missing elements.

major comments (5)
  1. [§II.C.2, Eq. (19); Algorithm 1 line 17] The CVaR-based target yCVaR = r + γ·CVaR_α(min_i Q̂(s',a') − α_ent log π̂(s')) applies CVaR to a quantity that, for a single transition, is a scalar; no return distribution over which the CVaR is computed is defined. If the CVaR is estimated across the batch D, then the target for one state-action pair is contaminated by transitions from other states, so the update is not a Bellman-style target for any per-state action-value function; if it is estimated from repeated action samples, it still lacks a derivation connecting it to the day-level CVaR objective in Eq. (3). The risk-awareness and variability-reduction claims in Section III.D.2 are therefore unsupported as written. A correct implementation requires an explicit distributional RL or augmented-state CVaR formulation, together with a derivation that the update optimizes Eq. (3).
  2. [Abstract; §III.D.1; §IV] The headline claim of 28.7% lower carbon emissions is contradicted by the paper's own numbers: C-DIFFCARL at 59.3 kg/h versus U-DIFFCARL at 64.7 kg/h is a reduction of about 8.3% (or 9.1% relative to the lower value), not 28.7%. This error appears in the abstract, in Section III.D.1, and in the conclusion, and it changes the paper's principal quantitative claim; it must be corrected and the remaining results re-evaluated.
  3. [§II.C.1, Eqs. (16)-(17); Algorithm 1 line 11] The actor objective maximizes πθ^T Q^risk + α_ent H with H(πθ) = −πθ^T log πθ, but the paper never defines how the probability density or differential entropy of the continuous diffusion policy is computed. In Algorithm 1, line 11, πθ(s) is said to be calculated from x0 via a softmax function; this is not meaningful for a continuous action vector. A valid diffusion policy update requires either the change-of-variables formula for log πθ over the denoising chain or a reparameterized sample-based objective; without this, Eq. (17) is not a well-defined policy-gradient update.
  4. [Table IV; Figs. 3-5; §III.C.2] All performance comparisons are presented as single numbers or single distributions, with no number of random seeds, no error bars, and no significance tests. The claims of stable convergence in Fig. 3 and reduced variability in Fig. 5 cannot be assessed from a single run; the 'reduces performance variability' part of the contribution is not statistically supported.
  5. [§III.D.2] The authors vary the risk coefficient λ and then declare that λ=0.1 is optimal based on the same test results shown in Fig. 5; no hold-out or cross-validation procedure is described. This is a test-set selection step, so the conclusion that a moderate risk preference is optimal is not a predictive statement about generalization.
minor comments (5)
  1. [Eq. (21)] The relative cost metric is defined as (C_DIFFCARL − C_COMP)/C_DIFFCARL × 100% with negative values meaning DIFFCARL is better, but the table heading 'Improv.' with negative numbers is confusing; please clarify the sign convention or relabel the column as 'Relative cost difference'.
  2. [Algorithm 1, line 17] The word 'retrun' should be 'return'.
  3. [Fig. 2b] The legend uses 'DiffCarl' and 'Ideal' rather than 'DIFFCARL' and 'OFFLINE'; please align notation with the rest of the paper.
  4. [Eq. (12)] The tanh squashing of the noise prediction bounds reconstructed actions to [-1,1], but no explicit action normalization or rescaling to the physical power limits in Table I is described; please clarify.
  5. [Section II.A.4, Eq. (4)] The CVaR risk term is defined on Cost, while the RL objective in Eq. (3) is written as a maximization over rewards; please state the sign convention relating cost and reward to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DiffCarl's cost, carbon, and risk results are empirical simulation outcomes, and the paper's few self-citations are background only; the under-specified CVaR update in Eq. (19) is an omitted justification, not a circular reduction.

full rationale

DiffCarl does not derive its headline results from its own assumptions in a way that makes them true by construction. Table IV and Figures 3-5 compare policies by simulation on a held-out test period, with DIFFCARL trained on separate data; no fitted parameter is renamed as a prediction. The carbon-aware versus carbon-unaware comparison is an ablation: Table I adds a carbon price (ρCarbon = 0.025 S$/kg) and carbon densities to the reward, so lower emissions in C-DiffCarl are aligned with the optimization objective, but the algorithm still has to learn the trade-off, and all carbon-aware baselines are compared under the same reward; this is not an identity such as Eq. X = Eq. Y. The risk-sensitivity analysis varies λ and reports the resulting cost distributions; Eq. (19) applies CVaR to a scalar TD target, and Algorithm 1 says to 'Estimate the αCVaR-quantile of the target retrun distribution for each (s,a)' without defining that distribution, so the link between Eq. (19) and the CVaR objective in Eq. (3) is asserted rather than derived. That is an omitted proof or design gap, but the paper does not claim Eq. (19) is implied by Eq. (3) in a self-referential way; it is an empirical claim about the trained policies. Self-citations ([12], [13], [20]) appear only in the introduction as motivation and background and are not load-bearing; no uniqueness theorem is imported from the authors' prior work. Separately, the abstract's 28.7% carbon claim is inconsistent with the quoted 59.3 and 64.7 kg/h values (about 8.4% reduction), which is a correctness and reproducibility concern, not a circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-set scenario parameters (carbon price and carbon density), a non-standard tanh modification to the DDPM reconstruction, and an unproven CVaR TD target. The empirical results depend on a private simulator and selected hyperparameters, with no code or data released.

free parameters (6)
  • λrisk (risk sensitivity coefficient) = 0.1
    Hand-set in Table III and later declared optimal from the test results in Section III.D.2; directly weights the CVaR risk term in Eq (3).
  • αCVaR (CVaR confidence level) = 0.95
    Set in Table III; defines the worst-case 5% tail used in Eq (4) and Eq (19).
  • αent (entropy temperature) = 0.05
    Set in Table III; controls exploration in the policy update Eq (16).
  • K (denoising steps) = 10
    Set in Table III; length of the diffusion chain in the actor.
  • ρCarbon (carbon price) = 0.025 S$/kg CO2
    Hand-set environmental parameter in Table I; determines the strength of carbon penalty and therefore the reported carbon emissions.
  • Carbon densities [ωCDG, ωGrid] = [0.9, 0.412] kg CO2/kWh
    Hand-set in Table I; these factors directly convert generated/purchased energy into the carbon emission numbers reported in Section III.D.1.
assumptions (4)
  • ad hoc to paper The tanh-modified reverse reconstruction in Eq (12) yields valid samples from the optimal action distribution.
    Introduced in Eqs (12)-(13) without derivation; deviates from the standard DDPM reconstruction x0 = (x_k - sqrt(1-alpha_bar_k) epsilon)/sqrt(alpha_bar_k) given in [27].
  • ad hoc to paper The CVaR target in Eq (19) correctly estimates the Q-function for the risk-sensitive objective in Eq (3).
    The paper states this design in Section II.C.2 but provides no proof that the TD update optimizes the CVaR of cumulative cost, and no details of how the CVaR of the target return distribution is estimated.
  • domain assumption The microgrid environment in Section III.A (cost coefficients, ramp rates, carbon parameters in Table I) is representative enough to support general performance claims.
    The parameters are drawn from prior studies but the results are only shown on a synthetic 2MG and two IEEE feeders; no sensitivity analysis is given.
  • standard math Standard DDPM theory and soft actor-critic convergence properties apply to the proposed architecture.
    The paper relies on [22] and [27] without re-deriving their assumptions for the modified tanh actor.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion-Modeled Reinforcement Learning for Carbon and Risk-Aware Microgrid Optimization." pith.science (2026). https://pith.science/paper/U7ZIK2QU

@misc{pith2026250716867,
  author       = {Pith},
  title        = {Pith review of: Diffusion-Modeled Reinforcement Learning for Carbon and Risk-Aware Microgrid Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U7ZIK2QU}},
  note         = {Machine review of arXiv:2507.16867}
}
read the original abstract

This paper introduces DiffCarl, a diffusion-modeled carbon- and risk-aware reinforcement learning algorithm for intelligent operation of multi-microgrid systems. With the growing integration of renewables and increasing system complexity, microgrid communities face significant challenges in real-time energy scheduling and optimization under uncertainty. DiffCarl integrates a diffusion model into a deep reinforcement learning (DRL) framework to enable adaptive energy scheduling under uncertainty and explicitly account for carbon emissions and operational risk. By learning action distributions through a denoising generation process, DiffCarl enhances DRL policy expressiveness and enables carbon- and risk-aware scheduling in dynamic and uncertain microgrid environments. Extensive experimental studies demonstrate that it outperforms classic algorithms and state-of-the-art DRL solutions, with 2.3-30.1% lower operational cost. It also achieves 28.7% lower carbon emissions than those of its carbon-unaware variant and reduces performance variability. These results highlight DiffCarl as a practical and forward-looking solution. Its flexible design allows efficient adaptation to different system configurations and objectives to support real-world deployment in evolving energy systems.

Figures

Figures reproduced from arXiv: 2507.16867 by the authors.

Figure 1
Figure 1. The framework of DIFFCARL with diffusion-modeled actor network. of renewable, loads, and the market. 3) We apply carbon- and risk-awareness to DIFFCARL algorithm, which effectively minimizes carbon emission and reduces the risk of high operational cost, thereby satisfying the user’s need of balancing cost efficiency, sustainability and manageable risk exposure. 4) We conduct extensive experiments to evaluate the per… view at source ↗
Figure 2
Figure 2. Case study results over a 24-hour test day. (a) shows the electrical power balance including contributions from photovoltaic (PV), wind [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. A comparison of learning curves under different RL policies. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Carbon emission distributions under carbon-aware (C-) and [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Cost distributions under DIFFCARL policies with varying risk￾sensitivity parameters. λ > 0 indicates risk-averse behavior with tighter cost distributions, while λ < 0 indicates risk-seeking behavior with broader cost distributions. DIFFCARL enables grid operators to sc…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 23 canonical work pages

  1. [1]

    Research and prospect of multi- microgrid control strategies,

    N. Zhi, X. Xiao, P. Tian, and H. Zhang, “Research and prospect of multi- microgrid control strategies,” Electric Power Automation Equipment , vol. 36, pp. 107–115, 04 2016

    work page 2016

  2. [2]

    Optimal coordinated energy dispatch of a multi-energy microgrid in grid-connected and islanded modes,

    Z. Li and Y . Xu, “Optimal coordinated energy dispatch of a multi-energy microgrid in grid-connected and islanded modes,” Applied Energy , vol. 210, pp. 974–986, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0306261917312230

    work page 2018

  3. [3]

    Minlp probabilistic scheduling model for demand response programs integrated energy hubs,

    M. Alipour, K. Zare, and M. Abapour, “Minlp probabilistic scheduling model for demand response programs integrated energy hubs,” IEEE Transactions on Industrial Informatics , vol. 14, no. 1, pp. 79–88, 2017

    work page 2017

  4. [4]

    Robust optimal power management system for a hybrid ac/dc micro-grid,

    M. Hosseinzadeh and F. R. Salmasi, “Robust optimal power management system for a hybrid ac/dc micro-grid,” IEEE Transactions on Sustainable Energy, vol. 6, no. 3, pp. 675–687, 2015

    work page 2015

  5. [5]

    Optimal operation of an energy management system for a grid-connected smart building considering photovoltaics’ uncertainty and stochastic electric vehicles’ driving schedule,

    D. Thomas, O. Deblecker, and C. S. Ioakimidis, “Optimal operation of an energy management system for a grid-connected smart building considering photovoltaics’ uncertainty and stochastic electric vehicles’ driving schedule,” Applied Energy, vol. 210, pp. 1188–1206, 2018

    work page 2018

  6. [6]

    A model predictive control approach to microgrid operation optimization,

    A. Parisio, E. Rikos, and L. Glielmo, “A model predictive control approach to microgrid operation optimization,” IEEE Transactions on Control Systems Technology, vol. 22, no. 5, pp. 1813–1827, 2014

    work page 2014

  7. [7]

    Real- time integration of optimal generation scheduling with mpc for the energy management of a renewable hydrogen-based microgrid,

    M. Petrollese, L. Valverde, D. Cocco, G. Cau, and J. Guerra, “Real- time integration of optimal generation scheduling with mpc for the energy management of a renewable hydrogen-based microgrid,” Applied Energy, vol. 166, pp. 96–106, 2016

    work page 2016

  8. [8]

    A hierarchical energy management system based on hierarchical optimization for microgrid community economic operation,

    P. Tian, X. Xiao, K. Wang, and R. Ding, “A hierarchical energy management system based on hierarchical optimization for microgrid community economic operation,” IEEE Transactions on Smart Grid , vol. 7, no. 5, pp. 2230–2241, 2015

    work page 2015

Show all 28 references
  1. [9]

    Distributed optimal energy management of a microgrid community,

    Y . Liu, S. Gao, C. Xiang, M. Yu, and K. T. Tan, “Distributed optimal energy management of a microgrid community,” in 2022 IEEE 17th International Conference on Control & Automation (ICCA) . IEEE, 2022, pp. 832–837

    2022

  2. [10]

    Double deep q-learning- based distributed operation of battery energy storage system considering uncertainties,

    V .-H. Bui, A. Hussain, and H.-M. Kim, “Double deep q-learning- based distributed operation of battery energy storage system considering uncertainties,” IEEE Transactions on Smart Grid , vol. 11, no. 1, pp. 457–469, 2019

    2019

  3. [11]

    Drl-hems: Deep reinforcement learning agent for demand response in home energy management systems considering customers and operators perspectives,

    A. A. Amer, K. Shaban, and A. M. Massoud, “Drl-hems: Deep reinforcement learning agent for demand response in home energy management systems considering customers and operators perspectives,” IEEE Transactions on Smart Grid , vol. 14, no. 1, pp. 239–250, 2022

    2022

  4. [12]

    Online optimal power scheduling of a microgrid via imitation learning,

    S. Gao, C. Xiang, M. Yu, K. T. Tan, and T. H. Lee, “Online optimal power scheduling of a microgrid via imitation learning,” IEEE Transac- tions on Smart Grid , vol. 13, no. 2, pp. 861–876, 2021

    2021

  5. [13]

    Online cooper- ative optimal power scheduling of multiple microgrids via hierarchical imitation learning,

    Y . Zhao, S. Gao, J. Wang, C. Li, M. Yu, and C. Xiang, “Online cooper- ative optimal power scheduling of multiple microgrids via hierarchical imitation learning,” in 2024 IEEE 18th International Conference on Control & Automation (ICCA) . IEEE, 2024, pp. 30–35

    2024

  6. [14]

    Multi-agent imitation learning based energy management of a microgrid with hybrid energy storage and real-time pricing,

    S. Gao, Y . Xu, Z. Zhang, Z. Wang, X. Zhou, and J. Wang, “Multi-agent imitation learning based energy management of a microgrid with hybrid energy storage and real-time pricing,” IEEE Internet of Things Journal , 2025

    2025

  7. [15]

    Imitation learning via off- policy distribution matching,

    I. Kostrikov, O. Nachum, and J. Tompson, “Imitation learning via off- policy distribution matching,” arXiv preprint arXiv:1912.05032 , 2019

    1912 arXiv

  8. [16]

    A novel model-free deep reinforcement learning framework for energy manage- ment of a pv integrated energy hub,

    A. Dolatabadi, H. Abdeltawab, and Y . A.-R. I. Mohamed, “A novel model-free deep reinforcement learning framework for energy manage- ment of a pv integrated energy hub,” IEEE Transactions on Power Systems, vol. 38, no. 5, pp. 4840–4852, 2022

    2022

  9. [17]

    A data-driven drl-based home energy management system optimization framework considering uncertain household parameters,

    K. Ren, J. Liu, Z. Wu, X. Liu, Y . Nie, and H. Xu, “A data-driven drl-based home energy management system optimization framework considering uncertain household parameters,” Applied Energy, vol. 355, p. 122258, 2024

    2024

  10. [18]

    Coordinated operation of active distribution network, networked microgrids, and electric vehicle: A multi-agent ppo optimization method,

    W. Shi, D. Zhang, X. Han, X. Wang, T. Pu, and W. Chen, “Coordinated operation of active distribution network, networked microgrids, and electric vehicle: A multi-agent ppo optimization method,” CSEE Journal of Power and Energy Systems , 2023

    2023

  11. [19]

    Scalable and privacy-preserving distributed energy management for multimicrogrid,

    Y . Zhang, J. Hu, G. Min, and X. Chen, “Scalable and privacy-preserving distributed energy management for multimicrogrid,” IEEE Transactions on Industrial Informatics , 2024

    2024

  12. [20]

    Diffusion-based Reinforcement Learning for Edge-enabled AI-Generated Content Services,

    H. Du, Z. Li, D. Niyato, J. Kang, Z. Xiong, H. Huang, and S. Mao, “Diffusion-based Reinforcement Learning for Edge-enabled AI-Generated Content Services,” Mar. 2023. [Online]. Available: https://arxiv.org/abs/2303.13052v3

    2023 arXiv

  13. [21]

    Two-Step Diffusion Policy Deep Reinforcement Learning Method for Low-Carbon Multi-Energy Microgrid Energy Management,

    Y . Zhang, Z. Mei, X. Wu, H. Jiang, J. Zhang, and W. Gao, “Two-Step Diffusion Policy Deep Reinforcement Learning Method for Low-Carbon Multi-Energy Microgrid Energy Management,” IEEE Transactions on Smart Grid , vol. 15, no. 5, pp. 4576–4588, Sep. 2024, conference Name: IEEE T...

    2024

  14. [22]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” in International conference on machine learning . PMLR, 2018, pp. 1861–1870

    2018

  15. [23]

    Risk-sensitive reinforcement learning,

    Y . Shen, M. J. Tobia, T. Sommer, and K. Obermayer, “Risk-sensitive reinforcement learning,” Neural computation, vol. 26, no. 7, pp. 1298– 1328, 2014

    2014

  16. [24]

    Risk-sensitive soft actor-critic for robust deep reinforcement learning under distribution shifts,

    T. Enders, J. Harrison, and M. Schiffer, “Risk-sensitive soft actor-critic for robust deep reinforcement learning under distribution shifts,” arXiv preprint arXiv:2402.09992, 2024

    2024 arXiv

  17. [25]

    Epistemic risk-sensitive reinforce- ment learning,

    H. Eriksson and C. Dimitrakakis, “Epistemic risk-sensitive reinforce- ment learning,” arXiv preprint arXiv:1906.06273 , 2019

    1906 arXiv

  18. [26]

    Dsac: Distributional soft actor-critic for risk-sensitive reinforcement learning,

    X. Ma, J. Chen, L. Xia, J. Yang, Q. Zhao, and Z. Zhou, “Dsac: Distributional soft actor-critic for risk-sensitive reinforcement learning,” Journal of Artificial Intelligence Research , vol. 83, 2025

    2025

  19. [27]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

    2020

  20. [28]

    Deep-reinforcement-learning-based capacity scheduling for pv-battery storage system,

    B. Huang and J. Wang, “Deep-reinforcement-learning-based capacity scheduling for pv-battery storage system,” IEEE Transactions on Smart Grid, vol. 12, no. 3, pp. 2272–2283, 2020

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.