Pith. sign in

REVIEW 3 major objections 6 minor 34 references

Hierarchical Constrained Reinforcement Learning with Dynamic Boundary for Spatio-Temporal Vehicle-to-Grid Scheduling

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HPC-RL enforces hard grid constraints and 100 percent EV demand satisfaction by splitting scheduling into a physics-solving upper layer and a boundary-driven lower layer.

desk verdict A useful hierarchical RL architecture for V2G scheduling with real speed gains, but the advertised hard guarantee on charging demand is only conditional and the abstract overclaims the MPC comparison. read the letter →

arxiv 2608.03409 v1 pith:JWMYZXJA submitted 2026-08-04 cs.CE

Haoyu Yan , Shutong Ding , Jiebao Zhang , Xi Yao , Yu Liu , Haoyu Wang , Chenchi Luo , Ye Shi This is my paper · ORCID
classification cs.CE
keywords constrainedreinforcementlearningvehicle-to-gridoptimalpowerflowhierarchicalpolicydynamicboundaryspatio-temporalconstraintsEVchargingscheduling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a hierarchical reinforcement-learning framework, HPC-RL, for vehicle-to-grid (V2G) charging scheduling that is fast enough for real-time use and strict enough to satisfy both grid physics and every EV's charging deadline. It claims that by separating the problem into a grid-level layer that solves power-flow equations inside the policy and a station-level layer that converts long-horizon charging obligations into per-step power bounds, the method achieves near-optimal costs while keeping constraint violations near zero. On IEEE 14-, 30-, and 141-bus test systems, the authors report objective gaps of about 10.2%, 13.1%, and 7.2% relative to model predictive control, with runtime reductions of 52x, 67x, and 319x and 100% demand satisfaction. If correct, this makes large-scale V2G coordination computationally tractable in online settings where traditional solvers take hours.

What carries the argument

Two mechanisms carry the argument. First, GRG-SAC: the policy network outputs basic actions (generator active power and voltage magnitudes), and non-basic variables are obtained by Newton iteration on the power-balance equations, with gradients propagated through the implicit mapping via the inverse Jacobian, so each action respects AC power flow by construction. Second, the dynamic boundary strategy: for each EV $i$ at time $t$, the lower layer enforces $P_{\min}^{i,\mathrm{ch}}(t) = \max\{0, (SOC_i^d - SOC_i(t)) - d_{\mathrm{re}}(t) \cdot P_{\mathrm{phys}}^{i,\mathrm{ch}}\}$ and an upper bound capped by battery headroom, which turns the departure-time SOC requirement into an instantaneous feasible interval that the station's priority-based allocation then fills.

What would settle it

Run HPC-RL on a test case where at some time step the sum of EVs' minimum required power at one station exceeds the active-power headroom the upper-level grid-feasible dispatch can provide, such as a congested feeder during a low-price period with many urgent EVs; if any EV departs below target state of charge, the claimed 100 percent demand satisfaction is refuted. A simpler check is to verify, step by step, that the lower-bound requirement from Eq. (16) never exceeds the station's available capacity.

Watch

Extended reading notes

Core claim

The central claim is that a two-layer architecture can resolve the trade-off that prior safe-RL and optimization approaches could not: strict satisfaction of both spatial (grid-level) and temporal (per-EV charging) constraints at scale. The upper layer uses the Generalized Reduced Gradient method inside a soft actor-critic agent, so the policy outputs only independent grid variables and the dependent voltage and angle variables are computed by solving the AC power-flow equalities, keeping every action on the feasible manifold. The lower layer computes a dynamic feasible interval for each EV at every step and guarantees, by construction, that any charging rate inside that interval keeps the EV on track to reach its target state of charge before departure. The paper reports that this design achieves near-zero constraint violations and 100 percent demand satisfaction across all tested cases, while matching the economic performance of MPC to within roughly 7-13 percent at a fraction of the runtime.

Load-bearing premise

The guarantee that every EV reaches its target charge assumes each EV can always charge at its physical maximum in every remaining hour, and that the sum of all minimum requirements at a station never exceeds the power the grid dispatch can actually deliver; if either fails, the 100 percent demand-satisfaction claim can break.

Editorial extensions

If this is right

  • V2G scheduling at the 141-bus scale moves from hours of solver time to seconds of inference, making real-time dispatch of large fleets feasible.
  • Hard physical constraints can be enforced inside RL policies by equation solving, so grid operators need not rely on soft penalties that allow violations.
  • The dynamic boundary turns a long-horizon charging obligation into per-step constraints, so the method handles variable numbers of EVs without retraining.
  • The 7-13 percent objective gap against MPC is the explicit price of speed; the paper positions HPC-RL as the feasible real-time alternative, not as a solver replacement.
  • Near-zero equality violations and 100 percent demand satisfaction, if they hold under varied conditions, make the approach viable for market participation where penalties for infeasibility are large.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same boundary logic should transfer to one-way smart charging or stationary battery storage, since it only needs a per-device energy target and a maximum power; a testable extension is applying Eq. (16) with V2G disabled.
  • The 100 percent demand guarantee rests on a worst-case future-capacity assumption; adding a station-level aggregate feasibility check before committing the upper-level action would close the gap between the two layers and is a natural hardening step.
  • Under stochastic renewables, the upper-level GRG solver could treat renewable injections as time-varying parameters; the hierarchy likely preserves its speed advantage because the per-EV boundaries are independent of generation uncertainty.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes HPC-RL, a hierarchical reinforcement learning framework for spatio-temporally coupled V2G scheduling. The upper level uses a GRG-SAC policy that embeds AC power-flow equations into action generation to enforce grid-level constraints, while the lower level computes dynamic per-EV charging power bounds (Eq. 16) to meet departure SOC requirements and prioritizes remaining station capacity by urgency. Experiments on IEEE 14-, 30-, and modified 141-bus systems compare HPC-RL with MPC and safe-RL baselines (CPO, CUP, DDPGLA, SACLA), reporting near-zero constraint violations, 100% demand satisfaction, and runtime reductions of 52x–319x versus MPC at the cost of modest objective gaps (7–13%).

Significance. If the claims hold, HPC-RL is a meaningful step toward real-time large-scale V2G scheduling: it demonstrates that hierarchical decoupling plus a physics-embedded upper-level policy can produce feasible AC-OPF-compatible actions orders of magnitude faster than numerical MPC. The experimental design is reasonably thorough, with multiple baseline algorithms, three network scales, and explicit metrics for objective, runtime, violations, and demand satisfaction. However, the paper's central theoretical guarantee for temporal demand satisfaction is incomplete, and the abstract overstates the comparison with MPC. These issues are fixable but require substantial revision.

major comments (3)
  1. [§4.2, Eq. (16)] The dynamic boundary strategy is claimed to 'mathematically guarantee' that all EVs meet their charging requirements, but the guarantee is only per-EV and conditional on the lower-level allocation being able to deliver P_min_i,ch(t) to every EV simultaneously. The OPF formulation (1)–(11) contains no constraint coupling the upper-level aggregate action P_ch(t) to L(t) = Σ_i P_min_i,ch(t), nor any station-capacity constraint, so nothing prevents P_ch(t) < L(t). In that case no feasible lower-level allocation exists and 100% demand satisfaction fails. The lower-level rule 'first ensures that all EVs receive their minimum required power, enforced through boundary constraints' is therefore not always realizable. The reported experiments (10 EVs per station, P_max=0.2, T=24) likely operate in a regime where capacity is non-binding, so they do not substantiate the unconditional guarantee. Please either prove an aggregate feasibility condition, add it to the OPF constraints, or rephrase the guarantee as conditional on P_ch(t) ≥ L(t).
  2. [Abstract and §5.2 (Table 3)] The abstract states that HPC-RL 'outperforms Model Predictive Control and state-of-the-art safe RL baselines across all metrics,' but Table 3 shows MPC achieves lower objective values in all three cases (34.21 vs 37.70 for Case 14; 7.11 vs 8.04 for Case 30; 5.12 vs 5.49 for Case 141). The paper's own §5.2 correctly describes HPC-RL as 'a much faster feasible alternative' with an objective gap, which contradicts the abstract's blanket 'outperforms' claim. The abstract should be corrected to reflect the actual trade-off: HPC-RL is faster than MPC but has higher objective cost, while it outperforms the safe-RL baselines on constraint satisfaction and demand fulfillment.
  3. [§4.1 (Projection for inequality constraints)] The paper states that inequality constraints are handled by 'a projection mechanism that maps the action generated by the policy back to the feasible set whenever an inequality violation occurs,' and that Proposition 1 guarantees physical feasibility. However, Proposition 1 proves only that the GRG adjustment preserves the tangent space of the equality constraints; it does not analyze how the subsequent inequality projection affects equality feasibility. If projection is applied independently of the power-flow equations, the equality manifold can be disturbed. The numerical results in Table 3 show small but nonzero equality violations (up to 0.4138 in Case 141), so the equality constraint is not strictly enforced to zero. Please clarify the projection algorithm and its interaction with the Newton correction, or weaken the claim of strict hard-constraint enforcement.
minor comments (6)
  1. [Tables 1 and 2] Tables 1 and 2 appear to be near-duplicate comparisons with overlapping entries; consider merging them into one table to avoid redundancy.
  2. [Figure 2 caption] The caption reads 'Modify Case 141'; this should be 'Modified Case 141'.
  3. [§3.2, Eq. (12)] The reward weights w_grid and w_ev are never specified; please state how they are chosen and whether the results are sensitive to them.
  4. [§4.2, Eq. (17)] The demand embedding s_demand_k = Σ_{τ=k}^T d_τ is not fully defined: please clarify whether d_τ is the per-EV demand at time τ or the aggregate station demand, and how this vector is incorporated into the state.
  5. [§5.3, Table 4] The text says the EV numbers in Table 4 are 'instantaneous active-set sizes,' but the experimental setup in §5.1 describes a fixed 8-hour dwell time with one arrival per time step; please clarify how the 1-EV and 50-EV scenarios are generated within that setup.
  6. [Overall] There are occasional formatting artifacts such as 'T able 1' and 'T able 3' in the displayed text; these should be fixed in the final version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dynamic boundary is a constructive derivation from battery dynamics, and the reported results are validated against independent MPC and safe-RL baselines.

full rationale

The paper's central mechanism, the dynamic boundary strategy in Eq. (16), is not a fitted value or a renamed prediction. It is a constructive sufficient condition derived from the SOC dynamics and hardware limits: setting P_min_i,ch(t) to the current deficit minus the maximum energy chargeable in the remaining intervals ensures that, if that lower bound is delivered, the departure SOC target can be reached. This is a forward-invariance argument from the model equations, not an equivalence to its inputs. The upper-level GRG-SAC layer analytically embeds the AC power-flow equalities using the implicit function theorem and Newton correction, which is standard mathematics rather than a self-citation or ansatz. The experimental comparisons use independent baselines: MPC and established safe-RL algorithms (CPO, CUP, DDPGLA, SACLA), and the objective gaps, runtimes, and constraint violations are measured quantities, not predictions derived from fitted constants. The only self-citations appear as background ([4]) or as baselines/comparison entries ([27], [28]); they do not supply any load-bearing theorem, uniqueness argument, or fitted parameter in this paper. The '100% demand satisfaction' figure is an enforcement property of the lower-level allocation rule, not a statistical prediction; while one can question whether the aggregate station power P_ch(t) will always be at least the sum of individual minimum requirements, that is a feasibility/correctness gap rather than a circular derivation. Overall, the claimed results are not forced by definition or by self-citation, so no significant circularity is present.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities; the dynamic boundary and demand embedding are algorithmic constructs. The main unstated input is the compatibility between the upper-level aggregate power and the lower-level per-EV minimum requirements, plus the linear battery model used to derive the bounds. Reward weights w_grid and w_ev are free parameters that are not reported.

free parameters (1)
  • Reward weights w_grid and w_ev = not stated
    Equation (12) balances grid generation cost against EV charging cost using these weights. Their values are not reported, which affects the objective trade-off and the comparison with MPC.
assumptions (5)
  • domain assumption The AC power-flow equations (Eq. 2) exactly model the grid.
    All spatial hard-constraint enforcement relies on the validity of the AC OPF model; approximation error would change feasibility.
  • standard math The Implicit Function Theorem and Newton's method provide a locally exact mapping a_N = phi(a_B) from basic to non-basic power-flow variables.
    Used in Section 4.1 to propagate gradients (Eq. 13) and recover equality feasibility; requires J_N nonsingular and convergence to tolerance.
  • domain assumption EV battery dynamics follow a linear SOC update with fixed efficiency eta = 0.98 (Eq. 8).
    The dynamic boundary strategy is derived from this linear model; real batteries have losses, temperature effects, and nonlinear charging curves.
  • domain assumption Each EV can charge at its maximum physical rate P_phys for every remaining interval.
    The lower bound in Eq. (16) uses d_re * P_phys as available future charging capacity; future grid constraints or connector sharing may reduce this.
  • domain assumption The sum of per-EV minimum power requirements is always compatible with the upper-level grid-feasible aggregate dispatch.
    No proof is given that the lower-level allocation is feasible within the station aggregate power chosen by the upper level; the experiments use fixed profiles where this compatibility apparently holds.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hierarchical Constrained Reinforcement Learning with Dynamic Boundary for Spatio-Temporal Vehicle-to-Grid Scheduling." pith.science (2026). https://pith.science/paper/JWMYZXJA

@misc{pith2026260803409,
  author       = {Pith},
  title        = {Pith review of: Hierarchical Constrained Reinforcement Learning with Dynamic Boundary for Spatio-Temporal Vehicle-to-Grid Scheduling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JWMYZXJA}},
  note         = {Machine review of arXiv:2608.03409}
}
read the original abstract

The rapid proliferation of Electric Vehicles (EVs) introduces significant spatio-temporal uncertainties into power grids, while Vehicle-to-Grid (V2G) technology offers critical flexibility through bidirectional power flow. However, integrating large-scale EVs into the Optimal Power Flow framework presents substantial challenges due to computational bottlenecks arising from solver complexity and coupled spatio-temporal constraints. Existing Reinforcement Learning (RL) methods often struggle to balance strict constraint satisfaction with scalability in highly dynamic EV fleet environments. To address these challenges, this paper proposes a Hierarchical Policy for Constrained Reinforcement Learning (HPC-RL) framework for spatially and temporally coupled V2G scheduling. The framework adopts a two-layer architecture: the upper level utilizes a RL algorithm based on the Generalized Reduced Gradient method to strictly enforce spatial grid-level hard constraints; the lower level implements a novel dynamic boundary strategy to compute real-time feasible charging power bounds for individual EVs, thereby ensuring the satisfaction of temporal charging demands. This integrated design not only enables the simultaneous handling of spatially and temporally coupled constraints during the RL optimization process but also significantly enhances generalization capabilities for large-scale fleets through hierarchical decoupling. Extensive experiments on IEEE 14, 30, and modified 141-bus systems demonstrate that HPC-RL outperforms Model Predictive Control and state-of-the-art safe RL baselines across all metrics. The proposed method achieves near-optimal scheduling strategies and drastically reduces online computation time in large-scale scenarios from hours to minutes, while maintaining a near-zero constraint violation rate and nearly 100\% charging demand satisfaction.

Figures

Figures reproduced from arXiv: 2608.03409 by the authors.

Figure 1
Figure 1. The HPC-RL pipeline consists of an upper layer solving the OPF-EV using RL with GRG constraints, and a lower layer managing EV charging allocations via dynamic boundaries and priority. To address the OPF-EV problem, we propose a hierarchical policy frame￾work utilizing a boundary strategy to decouple scheduling tasks across time and space which shown in Fig.1. The upper level employs a reinforcement learning network… view at source ↗
Figure 2
Figure 2. This figure shows the learning curves of five different safe reinforcement learning methods (HPC-RL, CPO, CUP, DDPGLA, and SACLA) in three different scenarios (Case 14, Case 30, and Modify Case 141). The x-axis represents the number of train￾ing epochs. The y-axis represents the episodic reward (first row), and the maximum instantaneous constraint violation (second row). and reduces runtime by factors of 52×, 67×, a… view at source ↗
Figure 3
Figure 3. Relationship between electricity price and charging amount. tion significantly improves learning efficiency. The demand-aware model not only converges to higher rewards but also exhibits reduced volatility (narrower stan￾dard deviation bands) and faster suppression of constraint violations compared to the baseline. 0 1 2 3 4 5 6 Epoch 1e4 −46 −44 −42 −40 −38 −36 −34 −32 Episodic Reward 0 1 2 3 4 5 6 Epoch 1e4 10 −5 … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The influence of demand information embedding. Evaluation of Lower-Level Resource Allocation [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Comparison between Priority-Based Allocation and Average Allocation. 6 CONCLUSION In this paper, we propose the HPC-RL framework, a physics-integrated hierarchi￾cal solution designed to address the complex coupling of spatial grid constraints and temporal EV charging d…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 31 canonical work pages

  1. [1]

    A large-scale high- resolution geographic analysis of impacts of electric vehicle charging on low-voltage grids,

    E. Hartvigsson, M. Taljegard, M. Odenberger, and P. Chen, “A large-scale high- resolution geographic analysis of impacts of electric vehicle charging on low-voltage grids,”Energy, vol. 261, p. 125180, 2022

    work page 2022

  2. [2]

    An investigation of plug-in electric vehicle charging impact on power systems scheduling and energy costs,

    S. I. Vagropoulos, G. A. Balaskas, and A. G. Bakirtzis, “An investigation of plug-in electric vehicle charging impact on power systems scheduling and energy costs,” IEEE Transactions on power systems, vol. 32, no. 3, pp. 1902–1912, 2016

    work page 1902

  3. [3]

    A non-iterative de- coupled solution of the coordinated robust opf in transmission and distribution networks with variable generating units,

    C. Lin, W. Wu, M. Shahidehpour, Y. Guo, and B. Wang, “A non-iterative de- coupled solution of the coordinated robust opf in transmission and distribution networks with variable generating units,”IEEE Transactions on Sustainable En- ergy, vol. 11, no. 3, pp. 1579–1588, 2019

    work page 2019

  4. [4]

    Two-stage optimization for efficient v2g coordination in distribution power system,

    P. Tian, S. Yan, B. Pan, and Y. Shi, “Two-stage optimization for efficient v2g coordination in distribution power system,” in2024 IEEE International Confer- ence on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm). IEEE, 2024, pp. 245–251

    work page 2024

  5. [5]

    Complexity of branch- and-bound and cutting planes in mixed-integer optimization—ii,

    A. Basu, M. Conforti, M. Di Summa, and H. Jiang, “Complexity of branch- and-bound and cutting planes in mixed-integer optimization—ii,”Combinatorica, vol. 42, no. Suppl 1, pp. 971–996, 2022

    work page 2022

  6. [6]

    Route planning for a fleet of electric vehicles with waiting times at charging stations,

    B. Li, S. S. Jha, and H. C. Lau, “Route planning for a fleet of electric vehicles with waiting times at charging stations,” inEvolutionary Computation in Combinatorial Optimization: 19th European Conference, EvoCOP 2019, Held as Part of EvoStar 2019, Leipzig, Germany, April 24–26, 2019, Proceedings 19. Springer, 2019, pp. 66–82

    work page 2019

  7. [7]

    Optimized in- tegration of electric vehicles in low voltage distribution grids,

    M. Spitzer, J. Schlund, E. Apostolaki-Iosifidou, and M. Pruckner, “Optimized in- tegration of electric vehicles in low voltage distribution grids,”Energies, vol. 12, no. 21, p. 4059, 2019

    work page 2019

  8. [8]

    Model predic- tive control for smart grids with multiple electric-vehicle charging stations,

    Y. Shi, H. D. Tuan, A. V. Savkin, T. Q. Duong, and H. V. Poor, “Model predic- tive control for smart grids with multiple electric-vehicle charging stations,”IEEE Transactions on Smart Grid, vol. 10, no. 2, pp. 2127–2136, 2018

    work page 2018

Show all 34 references
  1. [9]

    Deepopf: deep neural networks for optimal power flow,

    X. Pan, “Deepopf: deep neural networks for optimal power flow,” inProceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, 2021, pp. 250–251

    2021

  2. [10]

    Physics-informed reinforcement learning for real-time optimal power flow with renewable energy resources,

    Z. Wu, M. Zhang, S. Gao, Z.-G. Wu, and X. Guan, “Physics-informed reinforcement learning for real-time optimal power flow with renewable energy resources,”IEEE Transactions on Sustainable Energy, 2024

    2024

  3. [11]

    Physics-informed graphical representation-enabled deep reinforcement learning for robust distribu- tion system voltage control,

    D. Cao, J. Zhao, J. Hu, Y. Pei, Q. Huang, Z. Chen, and W. Hu, “Physics-informed graphical representation-enabled deep reinforcement learning for robust distribu- tion system voltage control,”IEEE Transactions on Smart Grid, vol. 15, no. 1, pp. 233–246, 2023

    2023

  4. [12]

    Learning to operate an electric vehicle charging station considering vehicle-grid integration,

    Z. Ye, Y. Gao, and N. Yu, “Learning to operate an electric vehicle charging station considering vehicle-grid integration,”IEEE transactions on smart grid, vol. 13, no. 4, pp. 3038–3048, 2022

    2022

  5. [13]

    Definition and evaluation of model-free coordination of electrical vehicle charging with reinforcement learning,

    N. Sadeghianpourhamami, J. Deleu, and C. Develder, “Definition and evaluation of model-free coordination of electrical vehicle charging with reinforcement learning,” IEEE Transactions on Smart Grid, vol. 11, no. 1, pp. 203–214, 2019

    2019

  6. [14]

    Safe off-policy deep reinforcement learning algorithm for volt-var control in power distribution systems,

    W. Wang, N. Yu, Y. Gao, and J. Shi, “Safe off-policy deep reinforcement learning algorithm for volt-var control in power distribution systems,”IEEE Transactions on Smart Grid, vol. 11, no. 4, pp. 3008–3018, 2019. 16 Haoyu Yan Author et al

    2019

  7. [15]

    A hybrid data-driven method for fast solution of security- constrained optimal power flow,

    Z. Yan and Y. Xu, “A hybrid data-driven method for fast solution of security- constrained optimal power flow,”IEEE Transactions on Power Systems, 2022

    2022

  8. [16]

    Safe deep reinforcement learning- based constrained optimal control scheme for active distribution networks,

    P. Kou, D. Liang, C. Wang, Z. Wu, and L. Gao, “Safe deep reinforcement learning- based constrained optimal control scheme for active distribution networks,”Applied energy, vol. 264, p. 114772, 2020

    2020

  9. [17]

    Model-augmented safe reinforcement learning for volt-var control in power distribution networks,

    Y. Gao and N. Yu, “Model-augmented safe reinforcement learning for volt-var control in power distribution networks,”Applied Energy, vol. 313, p. 118762, 2022

    2022

  10. [18]

    A safe policy learning-based method for decentralized and economic frequency control in isolated networked-microgrid systems,

    Y. Xia, Y. Xu, Y. Wang, S. Mondal, S. Dasgupta, A. K. Gupta, and G. M. Gupta, “A safe policy learning-based method for decentralized and economic frequency control in isolated networked-microgrid systems,”IEEE Transactions on Sustain- able Energy, vol. 13, no. 4, pp. 1982–1993, 2022

    1982

  11. [19]

    Deepopf: A deep neural network ap- proach for security-constrained dc optimal power flow,

    X. Pan, T. Zhao, M. Chen, and S. Zhang, “Deepopf: A deep neural network ap- proach for security-constrained dc optimal power flow,”IEEE Transactions on Power Systems, vol. 36, no. 3, pp. 1725–1735, 2020

    2020

  12. [20]

    Dc3: A learning method for optimization with hard constraints,

    P. L. Donti, D. Rolnick, and J. Z. Kolter, “Dc3: A learning method for optimization with hard constraints,” inInternational Conference on Learning Representations, 2020

    2020

  13. [21]

    Electric vehicle charging management based on deep reinforcement learning,

    S. Li, W. Hu, D. Cao, T. Dragiˇ cevi´ c, Q. Huang, Z. Chen, and F. Blaabjerg, “Electric vehicle charging management based on deep reinforcement learning,”Journal of Modern Power Systems and Clean Energy, vol. 10, no. 3, pp. 719–730, 2021

    2021

  14. [22]

    A deep reinforcement learning-based charging schedul- ing approach with augmented lagrangian for electric vehicle,

    G. Chen and X. Shi, “A deep reinforcement learning-based charging schedul- ing approach with augmented lagrangian for electric vehicle,”arXiv preprint arXiv:2209.09772, 2022

    2022 arXiv

  15. [23]

    Constrained ev charging scheduling based on safe deep reinforcement learning,

    H. Li, Z. Wan, and H. He, “Constrained ev charging scheduling based on safe deep reinforcement learning,”IEEE Transactions on Smart Grid, vol. 11, no. 3, pp. 2427–2439, 2019

    2019

  16. [24]

    Network-constrained reinforcement learning for optimal ev charging control,

    T. Wu, A. Scaglione, A. P. Surani, D. Arnold, and S. Peisert, “Network-constrained reinforcement learning for optimal ev charging control,” in2023 IEEE Interna- tional Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm). IEEE, 2023...

    2023

  17. [25]

    Uncertainty-aware critic augmentation for hierarchical multi-agent ev charging control,

    L. P.-Y. Ting, A. S ¸enol, H.-Y. Wang, H.-C. Lai, K.-T. Chuang, and H. Liu, “Uncertainty-aware critic augmentation for hierarchical multi-agent ev charging control,”arXiv preprint arXiv:2412.18047, 2024

    2024

  18. [26]

    Ev charging command fast allocation approach based on deep reinforcement learning with safety modules,

    J. Zhang, Y. Guan, L. Che, and M. Shahidehpour, “Ev charging command fast allocation approach based on deep reinforcement learning with safety modules,” IEEE Transactions on Smart Grid, vol. 15, no. 1, pp. 757–769, 2023

    2023

  19. [27]

    Distributionally robust optimiza- tion for vehicle-to-grid with uncertain renewable energy,

    Q. Li, P. Tian, Y. Shi, Y. Shi, and H. D. Tuan, “Distributionally robust optimiza- tion for vehicle-to-grid with uncertain renewable energy,” in2022 11th Interna- tional Conference on Control, Automation and Information Sciences (ICCAIS). IEEE, 2022, pp. 462–467

    2022

  20. [28]

    Reduced policy optimization for contin- uous control with hard constraints,

    S. Ding, J. Wang, Y. Du, and Y. Shi, “Reduced policy optimization for contin- uous control with hard constraints,”Advances in Neural Information Processing Systems, vol. 36, pp. 38 642–38 667, 2023

    2023

  21. [29]

    Constrained large-scale real-time ev scheduling based on recurrent deep reinforcement learning,

    H. Li, G. Li, T. T. Lie, X. Li, K. Wang, B. Han, and J. Xu, “Constrained large-scale real-time ev scheduling based on recurrent deep reinforcement learning,”Interna- tional Journal of Electrical Power & Energy Systems, vol. 144, p. 108603, 2023

    2023

  22. [30]

    Discovering temporal structure: An overview of hierarchical reinforcement learn- ing,

    M. Klissarov, A. Bagaria, Z. Luo, G. Konidaris, D. Precup, and M. C. Machado, “Discovering temporal structure: An overview of hierarchical reinforcement learn- ing,”arXiv preprint arXiv:2506.14045, 2025

    2025 arXiv

  23. [31]

    Constrained policy optimization,

    J. Achiam, D. Held, A. Tamar, and P. Abbeel, “Constrained policy optimization,” inInternational conference on machine learning. PMLR, 2017, pp. 22–31. HPC-RL 17

    2017

  24. [32]

    Cup: A conservative update policy algorithm for safe reinforcement learning,

    L. Yang, J. Ji, J. Dai, Y. Zhang, P. Li, and G. Pan, “Cup: A conservative update policy algorithm for safe reinforcement learning,”arXiv preprint arXiv:2202.07565, 2022

    2022 arXiv

  25. [33]

    De- terministic policy gradient algorithms,

    D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “De- terministic policy gradient algorithms,” inInternational conference on machine learning. Pmlr, 2014, pp. 387–395

    2014

  26. [34]

    Soft actor-critic: Off-policy maxi- mum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maxi- mum entropy deep reinforcement learning with a stochastic actor,” inInternational conference on machine learning. PMLR, 2018, pp. 1861–1870

    2018

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.