Pith. sign in

REVIEW 4 major objections 4 minor 47 references

Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that splitting power-grid topology control into regional proposers plus a learned coordinator—a centrally coordinated multi-agent design—improves both sample efficiency and final performance, with the greedy-proposer…

desk verdict The architecture is a reasonable incremental variant and the experiments are thorough, but the headline claim rests on a selective seed subset, and the paper's own test table contradicts the abstract. read the letter →

arxiv 2502.08681 v2 pith:EUP344SR submitted 2025-02-12 cs.MA cs.AIcs.LG

Barbera de Mol , Davide Barbieri , Jan Viebahn , Davide Grossi This is my paper · ORCID
classification cs.MAcs.AIcs.LG
keywords multi-agentreinforcementlearninghierarchicalactionspacefactorizationpowergridtopologycontrolL2RPNProximalPolicyOptimizationsampleefficiencyN-1secure
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Power grid operators face a combinatorial action space: each substation can be reconfigured in many topologies, and choosing one action from the full set is what makes learning slow. The paper's central claim is that this problem is easier if the decision is factored: regional agents each propose a reconfiguration for their own substation, and a coordinating agent then picks the single best proposal using the global state. This centrally coordinated multi-agent (CCMA) architecture is meant to replace one large action space with several small ones plus a small coordinator problem, so sample complexity grows with the sum rather than the product of sub-problem sizes. Across 5-bus and 14-bus Grid2Op simulations, with and without random line outages, the paper reports that CCMA variants—especially Greedy-RL, which pairs rule-based greedy proposers with a learned coordinator—outperform single-agent and rule-based baselines in both sample efficiency and final timesteps survived. The significance, if true, is that costly grid topology decisions could be handled by an architecture whose parts are trained more cheaply and that scales more gracefully to larger networks.

What carries the argument

The load-bearing mechanism is the propose-then-select hierarchy: regional agents generate candidate topological reconfigurations concurrently under a shared observation, and a coordinating agent selects the final action from that proposal list, optionally augmented with the regional agents' action-value estimates. The hierarchy is embedded in a feedback-control gate that only calls the agent when the maximum line loading exceeds a threshold, so in calm periods the system executes "do nothing" and accumulates reward into a semi-MDP transition. Because the coordinator chooses among a small number of regional proposals rather than among the primitive topology space, the factored MDP's sample complexity is argued to scale with the sum of the sub-action-space sizes instead of their product.

What would settle it

Run the Greedy-RL and RL-RL architectures on the 14-bus-with-opponent benchmark to completion for all ten seeds, and compare per-seed and median timesteps survived on held-out scenarios. If Greedy-RL's median does not exceed the single-RL baseline's and RL-RL's, the paper's headline claim of superior final performance and sample efficiency in the hardest setting is refuted.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes a new division of labor for topology control: instead of one policy choosing among the full set of substation configurations, every regional agent proposes an action for its region at once, and a coordinator chooses which proposal to execute. Through experiments in 5-bus and 14-bus networks with and without an opponent that randomly disconnects lines, the paper argues that the fully learned RL-RL and RL-Action Value RL variants beat the single-agent PPO baseline in the harder 14-bus settings, and that Greedy-RL—greedy regional proposers plus an RL-trained coordinator—exhibits the best sample efficiency and asymptotic performance, including in the hardest 14-bus-with-opponent setting. The paper also finds that rule-based coordinators (CAPA and value-softmax) are adequate on the small 5-bus network but no longer competitive once the network grows or stochastic outages appear, and that passing regional critics' action-values to the learned coordinator adds no measurable benefit.

Load-bearing premise

The reported advantage of Greedy-RL in the hardest 14-bus-with-opponent environment rests on treating the two seeds that finished training—and the high-performing seed subset used for the mean learning curve—as representative, even though the paper's own test table reports a higher mean survival for RL-RL (1122.4 vs 923.7).

Editorial extensions

If this is right

  • The factored design means each regional proposer can be trained as a small, independent policy, and the coordinator as a separate policy over a much smaller action set.
  • Greedy-RL's warm start—frozen greedy proposers that always supply valuable actions—makes the coordinator trainable with far fewer environment interactions than a single agent over the full action space.
  • Rule-based coordination is sufficient only when the grid is small and deterministic; larger or stochastic grids require a learned coordinator with longer-term reasoning.
  • On a preliminary 36-bus experiment, the same Greedy-RL pattern yields better best validation performance than both the single-agent and greedy baselines, though overfitting remains.
  • In the hardest setting, the learned-coordinator architectures (Greedy-RL and RL-RL) outperform the single-agent RL baseline both asymptotically and in sample efficiency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper's advantage hinges on seeds that finished training and on a high-performing-seed mean, an all-seed completion of the 14-bus-with-opponent runs could either confirm or overturn the headline result; until then, the reported Greedy-RL margin should be treated as provisional.
  • The bimodal seed behavior hints that rare contingency types drive the jump in performance; if that is right, prioritizing replay of low-survival scenarios or targeted reward shaping could stabilize training across seeds.
  • The null result for action-value inputs suggests local critic values do not help coordination, but a single shared critic or centralized value function—which the paper leaves to future work—might behave differently.
  • The architecture's gains come partly from freezing the proposers; co-training both levels introduces non-stationarity, so an imitation-learned surrogate for the greedy proposers could preserve the warm-start benefit at a fraction of Greedy-RL's recorded per-timestep cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a centrally coordinated multi-agent (CCMA) architecture for Grid2Op/L2RPN power grid topology control. Regional agents propose topological reconfigurations and a coordinating agent selects among the proposals. Several instantiations are compared: Greedy-RL, RL-RL, RL-Action Value RL, RL-CAPA, and RL-Value Softmax, against single-agent PPO, greedy, Greedy-CAPA, and do-nothing baselines on 5-bus and 14-bus networks, with and without an opponent. The central claim, stated in the abstract, is that the CCMA architecture exhibits higher sample efficiency and superior final performance than the baselines. The paper reports learning curves during training and held-out test results in Table 9, and includes extensive appendices on hyperparameters, per-seed curves, a preliminary 36-bus experiment, and computational costs.

Significance. If the empirical claims were fully supported, the CCMA architecture would be a useful contribution: the propose-then-coordinate design is a natural way to factor the topology-control action space, and the paper is careful to include multiple seeds, hyperparameter grid searches, per-seed appendix curves, and compute-time reporting. The comparison with established L2RPN baselines is valuable, and the explicit discussion of bimodal seed behavior is honest. However, the headline claims as stated are not supported by the evidence. The strongest result, Greedy-RL in the 14-bus-with-opponent environment, rests on only two completed seeds and a selected high-performing-seed mean, while the paper's own held-out test table shows a different ranking. Because the abstract and Section 6 make broad claims that the data do not consistently back, the current manuscript cannot be accepted as is.

major comments (4)
  1. [§6, Fig. 4 (lower-right), App. C, Fig. 5] The flagship result—Greedy-RL exhibiting 'extraordinary sample efficiency and performance' in the 14-bus environment with opponent—is not supported by a representative experimental sample. Appendix C states that training completed on only two of ten seeds for Greedy-RL, and Section 6 explains that the lower-right panel of Fig. 4 averages only 'high-performing seeds' because the per-seed curves are bimodal. A mean over a selected subset of seeds is not an estimator of expected behavior, and Fig. 5 shows that the subset choice can change the qualitative conclusion. This directly undermines the abstract's claim of 'superior final performance' and the stability discussion in Section 7 that refers to 'ten different random seeds.' The authors must either complete the full seed sweep, report all seeds individually with appropriate quantiles, or substantially weaken the claims.
  2. [Table 9 vs. §6] The held-out test results contradict the claimed final-performance superiority in the 14-bus-with-opponent environment. Table 9 shows RL-RL with a higher mean number of survived timesteps (1122.4) than Greedy-RL (923.7), with standard deviations around 1000 for both. Section 6 dismisses these test results ('the model evaluation during training is more relevant than final test results'), but the abstract explicitly claims 'superior final performance,' making the dismissal internally inconsistent. Moreover, Table 9 compares the best-validation-seed model for each architecture, so Greedy-RL is the best of 2 completed seeds while RL-RL is the best of 10; with these sample sizes and variances, no significant difference is established. The claims need to be aligned with a statistically sound comparison of final test performance.
  3. [§5.3, Fig. 4, Table 9] The blanket statement that 'The CCMA architecture exhibits higher sample efficiency and superior final performance than the baseline approaches' is contradicted by the paper's own results for several CCMA variants. RL-CAPA and RL-Value Softmax underperform the single RL baseline in the 5-bus-with-opponent and 14-bus-without-opponent settings (Fig. 4 and Table 9), and they were not even trained in the hardest 14-bus-with-opponent environment. The contribution bullet in Section 1 is more careful ('some of its variants surpass all baselines'), and the abstract, conclusion, and Section 6 discussion should be narrowed accordingly.
  4. [Table 11 and §6] The sample-efficiency comparisons are reported in environment interactions, but the paper also reports that Greedy-RL costs about 5.089 seconds per environment interaction versus 0.307 seconds for single RL, roughly an order of magnitude more compute per step. As a result, the 'sample efficiency' advantage of Greedy-RL does not translate into computational efficiency, and the abstract's unqualified efficiency claim is misleading for practical grid-operation settings. The paper should either present compute-adjusted comparisons or explicitly restrict the efficiency claim to environment interactions and discuss the compute trade-off.
minor comments (4)
  1. [Keywords] The keyword 'Reinforcment' is a typo and should read 'Reinforcement.'
  2. [Footnote 3, §6] The footnote stating that the 14-bus-without-opponent training curves have not fully converged and were halted at 100,000 timesteps should be moved into the main text, since it affects the interpretation of the asymptotic performance comparisons in that setting.
  3. [Appendix E, Table 10] Table 10 contains duplicate rows for Greedy-RL (0.00005, 1024, 3072, 5 appears twice) and for Single-RL (0.00005, 512, 3072, 10 appears twice); this should be corrected if the rows are intended to represent distinct hyperparameter configurations.
  4. [Eq. (2)-(3), §4.2] The treatment of the case n'' = 1 in the N-1 secure action-space formula is unclear; the statement that such cases 'are always kept in the base topology' should be formalized so the reader can verify that Eq. (2) is applied consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the CCMA comparison is an independent empirical benchmark against Grid2Op, and no claimed result is defined by its own inputs.

full rationale

The paper's central claim is an empirical comparison of agent architectures on Grid2Op environments. None of the reported quantities (timesteps survived, learning curves, sample efficiency) is constructed from a fitted parameter that is then renamed as a prediction. The action-space counting formulas (Eqs. 1-3) are combinatorial identities with no circular dependency on the RL results. The CCMA design is motivated by prior HRL/MARL work including co-authored papers [25, 16, 4] and [38], but those citations supply context and baseline algorithms rather than the measured outcomes; the comparisons are run against the public Grid2Op simulator and L2RPN baselines. The main caveats are statistical, not circular: Section 6 averages only 'high-performing seeds' for Greedy-RL in the 14-bus-with-opponent environment, and Appendix C states that training completed on only two of ten seeds for Greedy-RL there, while Table 9 (best validation seed on test scenarios) ranks RL-RL above Greedy-RL (1122.4 vs 923.7). These points affect how representative and internally consistent the 'extraordinary' Greedy-RL claim is, but they do not show that any derived result reduces by definition to its inputs. No circular step can be exhibited, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper relies on standard RL training choices and domain-specific thresholds. The most consequential choices are the gate threshold, the opponent parameters, and the per-architecture hyperparameter tuning; these affect the comparison but are not fitted to derive a result. No new physical entities are introduced.

free parameters (4)
  • Gate threshold rho_tilde = 0.95
    Line-loading threshold that determines when the agent is activated (Section 4.2). Chosen by hand, common in L2RPN studies; affects when actions are taken.
  • Opponent attack parameters = duration 48 timesteps, cooldown 144 timesteps
    Random line outage parameters (Table 1), taken from Manczak et al. [25]; these shape the stochastic environment and the difficulty of the task.
  • Scenario priority constant = 2
    Factor in the priority logit formula (Eq. 4), chosen by hand to scale between -1 and 1.
  • Per-architecture PPO hyperparameters = learning rate, batch size, minibatch size, iterations, clip, etc. (Tables 5-8, 10)
    Each architecture was tuned via a two-stage grid search on validation scenarios (Section 5.4.2), so the reported comparisons are between tuned configurations, not a single shared setting.
assumptions (5)
  • domain assumption The Grid2Op simulator with the specified opponent and reward function is a valid model of power network control.
    All claims about performance are measured inside this simulator; real-world transfer is speculative.
  • standard math Factored MDP sample complexity scales as the sum of sub-problem sizes rather than the product (Osband and Van Roy [29]).
    The motivation for the architecture; the paper does not re-derive this.
  • domain assumption The structural N-1 secure action space (Eq. 2) excludes only configurations that would cause immediate blackouts under a single line outage, and does not remove actions needed for optimal control.
    The action space restriction is justified by a counting argument, not by a proof of optimality preservation.
  • domain assumption The gate threshold rho_tilde = 0.95 and the L2RPN reward function adequately encode the goal of safe grid operation.
    These choices define the task; the paper does not validate them against other thresholds.
  • domain assumption Splitting scenarios into two-day segments and the priority sampling (Eq. 4-5) do not introduce distribution shift that biases architecture comparisons.
    Used to speed training; the effect on relative performance is not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control." pith.science (2026). https://pith.science/paper/EUP344SR

@misc{pith2026250208681,
  author       = {Pith},
  title        = {Pith review of: Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUP344SR}},
  note         = {Machine review of arXiv:2502.08681}
}
read the original abstract

Power grid operation is becoming more complex due to the increase in generation of renewable energy. The recent series of Learning To Run a Power Network (L2RPN) competitions have encouraged the use of artificial agents to assist human dispatchers in operating power grids. However, the combinatorial nature of the action space poses a challenge to both conventional optimizers and learned controllers. Action space factorization, which breaks down decision-making into smaller sub-tasks, is one approach to tackle the curse of dimensionality. In this study, we propose a centrally coordinated multi-agent (CCMA) architecture for action space factorization. In this approach, regional agents propose actions and subsequently a coordinating agent selects the final action. We investigate several implementations of the CCMA architecture, and benchmark in different experimental settings against various L2RPN baseline approaches. The CCMA architecture exhibits higher sample efficiency and superior final performance than the baseline approaches. The results suggest high potential of the CCMA approach for further application in higher-dimensional L2RPN as well as real-world power grid settings.

Figures

Figures reproduced from arXiv: 2502.08681 by the authors.

Figure 1
Figure 1. Diagram of the Feedback Control Framework used in this research. The goal to be achieved (and maintained) in the system is an [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Left: Hierarchy proposed by Manczak et al [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. An overview of all possible multi-agent architectures. First the maximum line loading in the current observation [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The mean number of timesteps survived on all validation scenarios throughout training for the 5-bus network without opponent [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The mean performance of the Single RL Agent compared to the individual seeds of the Greedy-RL and RL-RL agent on the [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Performance of the Greedy, Greedy-RL and Single RL agents on the 36-bus network without opponent over time: (a) training, [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 37 canonical work pages

  1. [1]

    Melike Baykal-Gursoy. 2010. Semi-Markov Decision Pro- cesses. Wiley Encyclopedia of Operations Research and Management Science (2010). https://doi.org/10.1002/ 9780470400531.eorms0757

    work page 2010

  2. [2]

    Anandsingh Chauhan, Mayank Baranwal, and Ansuma Ba- sumatary. 2023. Powrl: A reinforcement learning framework for robust management of power networks. In Proceedings of the AAAI Conference on Artificial Intelligence , V ol. 37. 14757–14764

    work page 2023

  3. [3]

    Peter Dayan and Geoffrey E Hinton. 1992. Feudal reinforce- ment learning. Advances in neural information processing systems 5 (1992)

    work page 1992

  4. [4]

    Matthijs de Jong, Jan Viebahn, and Yuliya Shapovalova

    work page

  5. [5]

    Benjamin Donnot. 2019. Source code for grid2op.Opponent.randomLineOpponent. https: //grid2op.readthedocs.io/en/dev_multiagent/ _modules/grid2op/Opponent/randomLineOpponent. html

    work page 2019

  6. [6]

    Benjamin Donnot. 2020. Grid2op- A testbed platform to model sequential decision making in power systems. https: //GitHub.com/rte-france/grid2op. 11 de Mol et al

    work page 2020

  7. [7]

    Benjamin Donnot. 2020. L2RPN Baselines- Repository host- ing reference baselines for the L2RPN challenge. https: //github.com/Grid2op/l2rpn-baselines/

    work page 2020

  8. [8]

    Matthias Dorfer, Anton R Fuxjäger, Kristian Kozak, Patrick M Blies, and Marcel Wasserer. 2022. Power grid congestion management via topology optimization with AlphaZero. arXiv preprint arXiv:2211.05612 (2022)

    arXiv 2022

Show all 47 references
  1. [9]

    Shixiang Gu, Ethan Holly, Timothy Lillicrap, and Sergey Levine. 2017. Deep reinforcement learning for robotic manip- ulation with asynchronous off-policy updates. In 2017 IEEE international conference on robotics and automation (ICRA). IEEE, 3389–3396

    2017

  2. [10]

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. InInterna- tional conference on machine learning. PMLR, 1861–1870

    2018

  3. [11]

    Majid Heidarifar, Panagiotis Andrianesis, Pablo Ruiz, Michael C Caramanis, and Ioannis Ch Paschalidis. 2021. An optimal transmission line switching and bus splitting heuristic incorporating AC and N-1 contingency constraints. Interna- tional Journal of Electrical Power & Energ...

    2021

  4. [12]

    Alex Kendall, Jeffrey Hawke, David Janz, Przemyslaw Mazur, Daniele Reda, John-Mark Allen, Vinh-Dieu Lam, Alex Bew- ley, and Amar Shah. 2019. Learning to drive in a day. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 8248–8254

    2019

  5. [13]

    Hans-Jurgen Koglin and Holger Müller. 1982. Corrective switching: a new dimension in optimal load flow.International Journal of Electrical Power & Energy Systems 4, 2 (1982), 142–149

    1982

  6. [14]

    Tu Lan, Jiajun Duan, Bei Zhang, Di Shi, Zhiwei Wang, Ruisheng Diao, and Xiaohu Zhang. 2020. AI-based au- tonomous line flow control via topology adjustment for max- imizing time-series ATCs. In 2020 IEEE Power & Energy Society General Meeting (PESGM). IEEE, 1–5

    2020

  7. [15]

    Malte Lehna, Clara Holzhüter, Sven Tomforde, and Christoph Scholz. 2024. Hugo–highlighting unseen grid options: Com- bining deep reinforcement learning with a heuristic target topology approach. Sustainable Energy, Grids and Networks 39 (2024), 101510

    2024

  8. [16]

    Malte Lehna, Jan Viebahn, Antoine Marot, Sven Tomforde, and Christoph Scholz. 2023. Managing power grids through topology actions: A comparative study between advanced rule- based and reinforcement learning agents. Energy and AI 14 (2023), 100276

    2023

  9. [17]

    Yuanlong Li, Yonggang Wen, Dacheng Tao, and Kyle Guan

  10. [18]

    Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Joseph Gonzalez, Ken Goldberg, and Ion Stoica

  11. [19]

    Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E Gonzalez, and Ion Stoica. 2018. Tune: A Research Platform for Distributed Model Selection and Training. arXiv preprint arXiv:1807.05118 (2018)

    2018 arXiv

  12. [20]

    Emily Little, Sandrine Bortolotti, Jean-Yves Bourmaud, Efthymios Karangelos, and Yannick Perez. 2021. Optimal transmission topology for facilitating the growth of renewable power generation. In 2021 IEEE Madrid PowerTech. IEEE, 1–6

    2021

  13. [21]

    Michael L Littman. 1994. Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings 1994. Elsevier, 157–163

    1994

  14. [22]

    Shunyu Liu, Yanzhen Zhou, Mingli Song, Guangquan Bu, Jianbo Guo, and Chun Chen. 2024. Progressive decision- making framework for power system topology control. Expert Systems with Applications 235 (2024), 121070

    2024

  15. [23]

    Gianvito Losapio, Davide Beretta, Marco Mussi, Al- berto Maria Metelli, and Marcello Restelli. 2024. State and Action Factorization in Power Grids. arXiv:2409.04467 [eess.SY] https://arxiv.org/ abs/2409.04467

    2024 arXiv

  16. [24]

    Trieu T Mai, Paige Jadun, Jeffrey S Logan, Colin A McMil- lan, Matteo Muratori, Daniel C Steinberg, Laura J Vimmer- stedt, Benjamin Haley, Ryan Jones, and Brent Nelson. 2018. Electrification futures study: scenarios of electric technology adoption and power consumption for the...

    2018

  17. [25]

    Blazej Manczak, Jan Viebahn, and Herke van Hoof. 2023. Hi- erarchical Reinforcement Learning for Power Network Topol- ogy Control. https://arxiv.org/abs/2311.02129

    2023 arXiv

  18. [26]

    Antoine Marot, Benjamin Donnot, Karim Chaouache, Adrian Kelly, Qiuhua Huang, Ramij-Raja Hossain, and Jochen L Cremer. 2022. Learning to run a power network with trust. Electric Power Systems Research212 (2022), 108487

    2022

  19. [27]

    Antoine Marot, Benjamin Donnot, Gabriel Dulac-Arnold, Adrian Kelly, Aidan O’Sullivan, Jan Viebahn, Mariette Awad, Isabelle Guyon, Patrick Panciatici, and Camilo Romero. 2021. Learning to run a power network challenge: a retrospective analysis. In NeurIPS 2020 Competition and D...

    2021

  20. [28]

    Antoine Marot, Benjamin Donnot, Sami Tazi, and Patrick Panciatici. 2018. Expert system for topological remedial action discovery in smart grids. In Mediterranean Conference on Power Generation, Transmission, Distribution and Energy Conversion (MEDPOWER 2018). IET, 1–6

    2018

  21. [29]

    Ian Osband and Benjamin Van Roy. 2014. Near-optimal re- inforcement learning in factored mdps. Advances in Neural Information Processing Systems 27 (2014)

    2014

  22. [30]

    Patrick Panciatici, Gabriel Bareux, and Louis Wehenkel. 2012. Operating in the fog: Security management under uncertainty. IEEE Power and Energy Magazine10, 5 (2012), 40–49

    2012

  23. [31]

    polixer. 2020. Winner of L2RPN ICAPS 2021. https:// github.com/polixir/L2RPN_2021

    2020

  24. [32]

    Pablo Ariel Ruiz, Evgeniy Goldis, Aleksandr M Rudkevich, Michael C Caramanis, C Russ Philbrick, and Justin M Foster

  25. [33]

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. 2020. Mastering atari, go, chess and shogi by planning with a learned model. Nature 588, 7839 (2020), 604–609

    2020

  26. [34]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  27. [35]

    Gaëtan Serré, Eva Boguslawski, Benjamin Donnot, Adrien Pavão, Isabelle Guyon, and Antoine Marot. 2022. Reinforce- ment learning for Energies of the future and carbon neutrality: a Challenge Design. arXiv preprint arXiv:2207.10330 (2022)

    2022

  28. [36]

    Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press

    2018

  29. [37]

    Richard S Sutton, Doina Precup, and Satinder Singh. 1999. Between MDPs and semi-MDPs: A framework for tempo- ral abstraction in reinforcement learning. Artificial Intel- ligence 112, 1 (1999), 181–211. https://doi.org/10. 1016/S0004-3702(99)00052-1

    1999

  30. [38]

    Erica van der Sar, Alessandro Zocca, and Sandjai Bhulai

  31. [39]

    Jan Viebahn, Sjoerd Kop, Joost van Dijk, Hariadi Budaya, Marja Streefland, Davide Barbieri, Paul Champion, Mario Jothy, Vincent Renault, and Simon Tindemans. 2024. GridOp- tions Tool: Real-World Day-Ahead Congestion Management using Topological Remedial Actions. CIGRE Session ...

    2024

  32. [40]

    Jan Viebahn, Matija Naglic, Antoine Marot, Benjamin Donnot, and Simon H Tindemans. 2022. Potential and challenges of AI-powered decision support for short-term system operations. CIGRE Session 2022 (2022)

    2022

  33. [41]

    Deunsol Yoon, Sunghoon Hong, Byung-Jun Lee, and Kee- Eung Kim. 2020. Winning the l2rpn challenge: Power grid management via semi-markov afterstate actor-critic. In Inter- national Conference on Learning Representations

    2020

  34. [42]

    forgetting

    Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. 2022. The surprising effectiveness of ppo in cooperative multi-agent games. Ad- vances in Neural Information Processing Systems 35 (2022), 24611–24624. 13 de Mol et al. A Comparative Analys...

    2022

  35. [2016]

    Security-constrained transmission topology control 12 Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control MILP formulation using sensitivity factors.IEEE Transactions on Power Systems 32, 2 (2016), 1597–1605

    2016

  36. [2017]

    arXiv preprint arXiv:1712.09381 85 (2017)

    Ray rllib: A composable and scalable reinforcement learning library. arXiv preprint arXiv:1712.09381 85 (2017)

    2017 arXiv

  37. [2019]

    IEEE transactions on cybernetics 50, 5 (2019), 2002–2013

    Transforming cooling optimization for green data cen- ter via deep reinforcement learning. IEEE transactions on cybernetics 50, 5 (2019), 2002–2013

    2019

  38. [2023]

    arXiv:2310.02605 [cs.LG] https: //arxiv.org/abs/2310.02605

    Multi-Agent Reinforcement Learning for Power Grid Topology Optimization. arXiv:2310.02605 [cs.LG] https: //arxiv.org/abs/2310.02605

    arXiv

  39. [2024]

    arXiv preprint arXiv:2407.19865 (2024)

    Imitation Learning for Intra-Day Power Grid Operation through Topology Actions. arXiv preprint arXiv:2407.19865 (2024)

    2024 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.