Uncertainty-aware Power System Planning via Gradient Descent
Pith reviewed 2026-06-29 00:34 UTC · model grok-4.3
The pith
A projected stochastic gradient descent method lets power system planners account for day-ahead and real-time uncertainty and still obtain lower total costs while meeting renewable targets.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By solving the two-stage planning problem with projected stochastic gradient descent, a primal-dual framework, and exponential moving average smoothing, the method produces investment decisions that, when evaluated in a two-stage day-ahead and real-time simulation, yield lower total system costs than a perfect-knowledge baseline while still achieving the prescribed renewable integration targets.
What carries the argument
projected stochastic gradient descent algorithm combined with a primal-dual optimization framework and exponential moving average smoothing strategy to solve the two-stage uncertain expansion planning problem
If this is right
- Investment portfolios selected by the method meet renewable integration targets at lower total system cost than portfolios chosen under perfect-knowledge assumptions.
- Time-coupling constraints for generator ramping and energy storage remain enforceable while uncertainty is modeled across both operational stages.
- The same computational approach can be used to compare technology mixes under different renewable penetration targets without collapsing operations into a single stage.
- Planning models that retain explicit two-stage decisions avoid the underinvestment in transmission and flexibility previously shown to result from ignoring operational uncertainty.
Where Pith is reading between the lines
- Planners facing higher renewable shares may need to replace single-stage models with two-stage gradient-based solvers to prevent systematic under-procurement of flexible assets.
- The method's convergence behavior under real data sets could be tested by replacing the exponential moving average smoother with alternative stabilization techniques.
- Extending the framework to include transmission or distribution network constraints would show whether the cost advantage persists when network physics are modeled at higher fidelity.
Load-bearing premise
The two-stage day-ahead and real-time simulation used to score the investment plans accurately reflects the operational uncertainties, ramping limits, and market interactions that determine real costs and renewable performance.
What would settle it
Running the uncertainty-aware investment plans through an operational simulation that includes additional uncertainties or constraints omitted from the two-stage model and finding that total costs exceed those of the perfect-knowledge baseline would falsify the claimed advantage.
Figures
read the original abstract
Power system planning models provide important guidance on long-term investment strategies with significant socio-economic impact. To remain computationally manageable, however, such planning models compromise on the level of complexity with which power system operations and physics are captured. A common approach in most planning models is to collapse multi-stage power system operational processes into a single stage and, as a result, give up on the ability to account for uncertainty in each operational stage. In light of newly emerging load patterns and the continuing adoption of weather-dependent stochastic renewable generation, this uncertainty, however, becomes increasingly impactful on operations, and ignoring it has been shown to cause underinvestment in transmission capacity and flexible resources. In this work, we present a computational approach for power system expansion planning that explicitly considers two-stage day-ahead (DA) and real-time (RT) operational decisions under uncertainty while retaining time-coupling constraints to allow modeling generator ramping and energy storage. To solve the resulting optimization problem efficiently, we employ a projected stochastic gradient descent algorithm combined with a primal-dual optimization framework and an exponential moving average smoothing strategy to improve convergence stability. We evaluate the resulting investment decisions within a two-stage DA and RT simulation framework and compare them with a classic expansion planning model that assumes perfect knowledge of renewable generation. Our experiments show that the proposed framework achieves lower total system costs while ensuring that the implemented technology portfolio achieves set renewable integration targets.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a projected stochastic gradient descent algorithm augmented with a primal-dual framework and exponential moving average smoothing to solve a two-stage day-ahead/real-time power system expansion planning problem that retains ramping and storage constraints while explicitly modeling uncertainty. Investment decisions obtained from this solver are evaluated in a matching two-stage DA/RT simulation and reported to yield lower realized system costs than a perfect-foresight baseline while still satisfying renewable integration targets.
Significance. If the reported cost reductions prove robust, the work would demonstrate a scalable gradient-based route to uncertainty-aware planning that avoids the usual single-stage aggregation of operations, addressing documented underinvestment in flexibility. The combination of stochastic gradient methods with primal-dual updates for a multi-stage planning model with time-coupling constraints would be a technical contribution worth documenting, provided convergence behavior and implementation details are supplied.
major comments (2)
- [Abstract] Abstract: the headline empirical claim (lower total system costs while meeting renewable targets) is stated without any description of the test system, number of scenarios, uncertainty sampling procedure, or statistical significance of the cost differences; these omissions make it impossible to assess whether the advantage is attributable to the proposed method rather than to modeling choices.
- [Abstract] Evaluation framework (implicit in the comparison to the perfect-knowledge baseline): the two-stage DA/RT simulator used to score the investment decisions appears to employ the same operational constraints, uncertainty realizations, and market rules as the planning model itself; no out-of-sample validation against a higher-fidelity operational model or real market data is described, leaving open the possibility that reported savings arise from internal consistency rather than improved planning under realistic uncertainty.
Simulated Author's Rebuttal
We thank the referee for their constructive comments, which highlight opportunities to strengthen the presentation of our results. We address each major comment below.
read point-by-point responses
-
Referee: [Abstract] Abstract: the headline empirical claim (lower total system costs while meeting renewable targets) is stated without any description of the test system, number of scenarios, uncertainty sampling procedure, or statistical significance of the cost differences; these omissions make it impossible to assess whether the advantage is attributable to the proposed method rather than to modeling choices.
Authors: We agree that the abstract would benefit from additional context on the experimental setup. The body of the manuscript reports results on a modified IEEE 118-bus system using 100 scenarios drawn from an empirical distribution of renewable generation; cost differences were evaluated across repeated simulation runs. We will revise the abstract to include a concise statement of the test system, scenario count, sampling approach, and consistency of the observed cost reductions. revision: yes
-
Referee: [Abstract] Evaluation framework (implicit in the comparison to the perfect-knowledge baseline): the two-stage DA/RT simulator used to score the investment decisions appears to employ the same operational constraints, uncertainty realizations, and market rules as the planning model itself; no out-of-sample validation against a higher-fidelity operational model or real market data is described, leaving open the possibility that reported savings arise from internal consistency rather than improved planning under realistic uncertainty.
Authors: The evaluation deliberately employs the identical two-stage operational model and uncertainty realizations for both the proposed method and the perfect-foresight baseline. This controlled design isolates the effect of uncertainty-aware planning on investment decisions. We acknowledge that the absence of out-of-sample testing against higher-fidelity models or real market data is a limitation. We will revise the manuscript to clarify the rationale for the current evaluation framework and to add an explicit discussion of this limitation together with suggestions for future validation. revision: yes
Circularity Check
No circularity; algorithmic solver evaluated against external baseline.
full rationale
The paper formulates a two-stage planning optimization and solves it via projected stochastic gradient descent with primal-dual updates and EMA smoothing. Investment decisions are then evaluated in a separate two-stage DA/RT simulation and compared to a perfect-foresight planning baseline. No load-bearing step reduces by the paper's own equations to a fitted parameter, self-citation, or input quantity by construction. The derivation chain consists of standard optimization techniques applied to an explicitly stated model; the empirical claim rests on out-of-sample simulation comparison rather than internal redefinition.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Power system operations can be approximated by linear constraints and two-stage decisions under sampled uncertainty.
Reference graph
Works this paper leans on
-
[1]
Available at SSRN 5169721 , year=
Degleris, Anthony and El Gamal, Abbas and Rajagopal, Ram , title =. Available at SSRN 5169721 , year=
-
[2]
JuMP 1.0: Recent improvements to a modeling language for mathematical optimization , journal =
Lubin, Miles and Dowson, Oscar and Garcia, Joaquim Dias and Huchette, Joey and Legat, Beno. JuMP 1.0: Recent improvements to a modeling language for mathematical optimization , journal =. 2023 , publisher =
2023
-
[3]
Two-Stage ML-Guided Decision Rules for Sequential Decision Making under Uncertainty , journal =
Rosemberg, Andrew and Street, Alexandre and Vallad. Two-Stage ML-Guided Decision Rules for Sequential Decision Making under Uncertainty , journal =
-
[4]
Flexible Differentiable Optimization via Model Transformations , journal =
Besan. Flexible Differentiable Optimization via Model Transformations , journal =. 2023 , doi =
2023
-
[5]
European Journal of Operational Research , volume =
Pineda, Salvador and Morales, Juan M and Boomsma, Trine K , title =. European Journal of Operational Research , volume =. 2016 , publisher =
2016
-
[6]
International Conference on Machine Learning , pages =
Amos, Brandon and Kolter, J Zico , title =. International Conference on Machine Learning , pages =. 2017 , organization =
2017
-
[7]
2013 , publisher =
Morales, Juan M and Conejo, Antonio J and Madsen, Henrik and Pinson, Pierre and Zugno, Marco , title =. 2013 , publisher =
2013
-
[8]
Annals of Operations Research , volume =
Pozo, David and Sauma, Enzo and Contreras, Javier , title =. Annals of Operations Research , volume =. 2017 , publisher =
2017
-
[9]
Rutgers Weather Research and Forecasting Model , year =
-
[10]
Annual Technology Baseline 2024 , year =
2024
-
[11]
and Siddik, M
Shehabi, Arman and Newkirk, Aaron and Smith, Sam and Hubbard, Alex and Lei, N. and Siddik, M. and others , title =. 2024 , doi =
2024
-
[12]
Union, Europ. Directive 2009/28/EC of the European Parliament and of the Council of 23 April 2009 on the promotion of the use of energy from renewable sources and amending and subsequently repealing Directives 2001/77/EC and 2003/30/EC , journal =
2009
-
[13]
Statistical analysis of wind power forecast error , journal =
Bludszuweit, Hans and Dom. Statistical analysis of wind power forecast error , journal =. 2008 , publisher =
2008
-
[14]
2014 , publisher =
Ela, Erik and Milligan, Michael and Bloom, Aaron and Botterud, Audun and Townsend, Aaron and Levin, Todd , title =. 2014 , publisher =
2014
-
[15]
Farghal, S. A. and Aziz, M. R. Abdel , title =. IEEE Transactions on Power Systems , volume =. 1988 , publisher =
1988
-
[16]
IEEE Power Engineering Review , number =
Caramanis, Michael C and Tabors, Richard D and Nochur, Kumar S and Schweppe, Fred C , title =. IEEE Power Engineering Review , number =. 2010 , publisher =
2010
-
[17]
Applied Energy , volume =
De Jonghe, Cedric and Delarue, Erik and Belmans, Ronnie and D'haeseleer, William , title =. Applied Energy , volume =. 2011 , publisher =
2011
-
[18]
2011 IEEE Power and Energy Society General Meeting , pages =
Palmintier, Bryan and Webster, Mort , title =. 2011 IEEE Power and Energy Society General Meeting , pages =. 2011 , organization =
2011
-
[19]
IEEE Transactions on Power Systems , volume =
Jin, Shan and Botterud, Audun and Ryan, Sarah M , title =. IEEE Transactions on Power Systems , volume =. 2014 , publisher =
2014
-
[20]
IEEE Transactions on Power Systems , volume =
Arabali, Amirsaman and Ghofrani, Mahmoud and Etezadi-Amoli, Mehdi and Fadali, Mohammed Sami and Moeini-Aghtaie, Moein , title =. IEEE Transactions on Power Systems , volume =. 2014 , publisher =
2014
-
[21]
IEEE Transactions on Power Systems , volume =
Orfanos, George A and Georgilakis, Pavlos S and Hatziargyriou, Nikos D , title =. IEEE Transactions on Power Systems , volume =. 2012 , publisher =
2012
-
[22]
Journal of Regulatory Economics , volume =
Munoz, Francisco D and Sauma, Enzo E and Hobbs, Benjamin F , title =. Journal of Regulatory Economics , volume =. 2013 , publisher =
2013
-
[23]
IEEE Transactions on Power Systems , volume =
Nanduri, Vishnu and Das, Tapas K and Rocha, Patricio , title =. IEEE Transactions on Power Systems , volume =. 2009 , publisher =
2009
-
[24]
Generation capacity expansion in liberalized electricity markets: A stochastic MPEC approach , journal =
Wogrin, Sonja and Centeno, Efraim and Barqu. Generation capacity expansion in liberalized electricity markets: A stochastic MPEC approach , journal =. 2011 , publisher =
2011
-
[25]
IEEE Transactions on Power Systems , volume =
Morales, Juan M and Pinson, Pierre and Madsen, Henrik , title =. IEEE Transactions on Power Systems , volume =. 2012 , publisher =
2012
-
[26]
A representation and economic interpretation of a two-level programming problem , journal =
Fortuny-Amat, Jos. A representation and economic interpretation of a two-level programming problem , journal =. 1981 , publisher =
1981
-
[27]
Mathematical Programming , volume =
McCormick, Garth P , title =. Mathematical Programming , volume =. 1976 , publisher =
1976
-
[28]
Annals of Operations Research , volume =
Zare, M Hosein and Borrero, Juan S and Zeng, Bo and Prokopyev, Oleg A , title =. Annals of Operations Research , volume =. 2019 , publisher =
2019
-
[29]
Computers & Chemical Engineering , volume =
Castro, Pedro M , title =. Computers & Chemical Engineering , volume =. 2015 , publisher =
2015
-
[30]
Top , volume =
Constante-Flores, Gonzalo and Conejo, Antonio J and Constante-Flores, S , title =. Top , volume =. 2022 , publisher =
2022
-
[31]
International Conference on Artificial Intelligence and Statistics , pages =
Lorraine, Jonathan and Vicol, Paul and Duvenaud, David , title =. International Conference on Artificial Intelligence and Statistics , pages =. 2020 , organization =
2020
-
[32]
Efficient and modular implicit differentiation , journal =
Blondel, Mathieu and Berthet, Quentin and Cuturi, Marco and Frostig, Roy and Hoyer, Stephan and Llinares-L. Efficient and modular implicit differentiation , journal =
-
[33]
International Conference on ``European Finance, Business and Regulation'' (EUFIRE 2020), Tofan, M., Bilan, I., Cigu, E
Iarmenco, Mihaela and Donos, Evlampie and Plotnic, Olesea , title =. International Conference on ``European Finance, Business and Regulation'' (EUFIRE 2020), Tofan, M., Bilan, I., Cigu, E. (eds), Iasi, Romania , pages =
2020
-
[34]
Computers & Operations Research , volume=
Capacity expansion of stochastic power generation under two-stage electricity markets , author=. Computers & Operations Research , volume=. 2016 , publisher=
2016
-
[35]
IEEE Transactions on Power Systems , volume=
Security-constrained unit commitment with volatile wind power generation , author=. IEEE Transactions on Power Systems , volume=. 2008 , publisher=
2008
-
[36]
Advances in Energy Systems: The Large-scale Renewable Energy Integration Challenge , pages=
Handling renewable energy variability and uncertainty in power system operation , author=. Advances in Energy Systems: The Large-scale Renewable Energy Integration Challenge , pages=. 2019 , publisher=
2019
-
[37]
Nature Energy , volume=
Energy storage solutions to decarbonize electricity through enhanced capacity expansion modelling , author=. Nature Energy , volume=. 2023 , publisher=
2023
-
[38]
Energy Economics , volume=
On representation of energy storage in electricity planning models , author=. Energy Economics , volume=. 2024 , publisher=
2024
-
[39]
Energy Reports , volume=
Joint expansion planning of distribution network with uncertainty of demand load and renewable energy , author=. Energy Reports , volume=. 2022 , publisher=
2022
-
[40]
IET Renewable Power Generation , volume=
Stochastic multi-stage joint expansion planning of transmission system and energy hubs in the presence of correlated uncertainties , author=. IET Renewable Power Generation , volume=. 2023 , publisher=
2023
-
[41]
Electric Power Systems Research , volume=
Battery energy storage systems in transmission network expansion planning , author=. Electric Power Systems Research , volume=. 2017 , publisher=
2017
-
[42]
IEEE Access , year=
A review of evolving challenges in transmission expansion planning problems , author=. IEEE Access , year=
-
[43]
IEEE Transactions on Power Systems , volume=
Incorporating non-convex operating characteristics into bi-level optimization electricity market models , author=. IEEE Transactions on Power Systems , volume=. 2019 , publisher=
2019
-
[44]
Neural Computation , volume=
Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation , author=. Neural Computation , volume=. 2022 , publisher=
2022
-
[45]
Computational Management Science , volume=
Why there is no need to use a big-M in linear bilevel optimization: A computational study of two ready-to-use approaches , author=. Computational Management Science , volume=. 2023 , publisher=
2023
-
[46]
SIAM Journal on Optimization , volume=
KKT reformulation and necessary conditions for optimality in nonsmooth bilevel optimization , author=. SIAM Journal on Optimization , volume=. 2014 , publisher=
2014
-
[47]
International conference on machine learning , pages=
Bilevel programming for hyperparameter optimization and meta-learning , author=. International conference on machine learning , pages=. 2018 , organization=
2018
-
[48]
International conference on machine learning , pages=
Bilevel optimization: Convergence analysis and enhanced design , author=. International conference on machine learning , pages=. 2021 , organization=
2021
-
[49]
Energy , volume =
Wind data introduce error in time-series reduction for capacity expansion modelling , author =. Energy , volume =. 2022 , doi =
2022
-
[50]
IEEE Transactions on Sustainable Energy , volume=
Energy storage sizing taking into account forecast uncertainties and receding horizon operation , author=. IEEE Transactions on Sustainable Energy , volume=. 2016 , publisher=
2016
-
[51]
2017 , publisher=
First-order methods in optimization , author=. 2017 , publisher=
2017
-
[52]
arXiv e-prints , pages=
A Method for Learning to Solve Parametric Bilevel Optimization with Coupling Constraints , author=. arXiv e-prints , pages=
-
[53]
2013 , publisher=
Integrating renewables in electricity markets: operational problems , author=. 2013 , publisher=
2013
-
[54]
Operations research , volume=
A single-settlement, energy-only electric power market for unpredictable and intermittent participants , author=. Operations research , volume=. 2010 , publisher=
2010
-
[55]
Annals of Operations Research , volume=
The impact of short-term variability and uncertainty on long-term power planning , author=. Annals of Operations Research , volume=. 2020 , publisher=
2020
-
[56]
arXiv preprint arXiv:2603.00394 , year=
Uncertainty-Aware Grid Planning in the Real World: A Method Enabling Large-Scale, Two-Stage Adaptive Robust Optimization for Capacity Expansion Planning , author=. arXiv preprint arXiv:2603.00394 , year=
-
[57]
Journal of Scientific Computing , volume=
A two-phase stochastic momentum-based algorithm for nonconvex expectation-constrained optimization , author=. Journal of Scientific Computing , volume=. 2025 , publisher=
2025
-
[58]
Advances in neural information processing systems , volume=
Momentum-based variance reduction in non-convex sgd , author=. Advances in neural information processing systems , volume=
-
[59]
Stochastic Processes and their Applications , volume=
Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds , author=. Stochastic Processes and their Applications , volume=. 1987 , publisher=
1987
-
[60]
International Conference on Machine Learning , pages=
On gradient descent ascent for nonconvex-concave minimax problems , author=. International Conference on Machine Learning , pages=. 2020 , organization=
2020
-
[61]
Mathematical Programming , volume=
Stochastic first-order methods for convex and nonconvex functional constrained optimization , author=. Mathematical Programming , volume=. 2022 , publisher=
2022
-
[62]
arXiv preprint arXiv:2305.19225 , year=
Learning decision-focused uncertainty sets in robust optimization , author=. arXiv preprint arXiv:2305.19225 , year=
-
[63]
arXiv preprint arXiv:2510.25986 , year=
A General and Streamlined Differentiable Optimization Framework , author=. arXiv preprint arXiv:2510.25986 , year=
-
[64]
Applied Energy , volume=
The wind integration national dataset (wind) toolkit , author=. Applied Energy , volume=. 2015 , publisher=
2015
-
[65]
Neurocomputing , volume=
On hyperparameter optimization of machine learning algorithms: Theory and practice , author=. Neurocomputing , volume=. 2020 , publisher=
2020
-
[66]
2026 , month = apr, url =
2026
-
[67]
North American Electric Reliability Corporation (NERC), White Paper , year=
Characteristics and risks of emerging large loads , author=. North American Electric Reliability Corporation (NERC), White Paper , year=
-
[68]
Integrated Investment and Policy Planning for Power Systems via Differentiable Scenario Generation
Integrated Investment and Policy Planning for Power Systems via Differentiable Scenario Generation , author=. arXiv preprint arXiv:2604.14410 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[69]
IEEE Control Systems Letters , year=
Value-oriented forecast combinations for unit commitment , author=. IEEE Control Systems Letters , year=
-
[70]
IEEE Transactions on Power Systems , volume=
Fast security-constrained optimal power flow through low-impact and redundancy screening , author=. IEEE Transactions on Power Systems , volume=. 2020 , publisher=
2020
-
[71]
SIAM Journal on Optimization , volume=
Combining Progressive Hedging with a Frank--Wolfe Method to Compute Lagrangian Dual Bounds in Stochastic Mixed-Integer Programming , author=. SIAM Journal on Optimization , volume=. 2018 , publisher=
2018
-
[72]
Ho, Jonathan and Becker, Jonathon and Brown, Maxwell and Brown, Patrick and Chernyakhovskiy, Ilya and Cohen, Stuart and Cole, Wesley and Corcoran, Sean and Eurek, Kelly and Frazier, Will and others , title =. 2021 , month =. doi:10.2172/1788425 , url =
-
[73]
2009 , publisher=
Implicit functions and solution mappings , author=. 2009 , publisher=
2009
-
[74]
Advances in neural information processing systems , volume=
Task-based end-to-end model learning in stochastic optimization , author=. Advances in neural information processing systems , volume=
-
[75]
, author=
An Updated Version of the IEEE RTS 24-Bus System for Electricity Market and Power System Operation Studies. , author=. 2016 , publisher=
2016
-
[76]
IEEE Transactions on power apparatus and systems , number=
IEEE reliability test system , author=. IEEE Transactions on power apparatus and systems , number=. 1979 , publisher=
1979
-
[77]
On the Resolution of Stochastic MPECs over Networks: Distributed Implicit Zeroth-Order Gradient Tracking Methods , author=. arXiv preprint arXiv:2505.22916 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[78]
(2026) Data-Driven Uncertainty-aware Power System Planning via Gradient Descent Code Supplement
https://github.com/MehrnoushGhazanfariharandi/Uncertainty-aware-PSP-via-SGD. (2026) Data-Driven Uncertainty-aware Power System Planning via Gradient Descent Code Supplement
2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.