Recognition: unknown
Diagnostic Modelling: a framework of principles for responsible energy systems modelling
Pith reviewed 2026-05-10 00:45 UTC · model grok-4.3
The pith
Energy systems models yield policy insights that remain valid despite uncertainties by uncovering mechanistic explanations.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Diagnostic Modelling is a framework wherein modellers critically interrogate their models and explore uncertainties to uncover mechanistic explanations that offer policy-relevant insights. Mechanistic explanations provide fundamental understanding that remains valid despite model uncertainty and does not depend on detailed knowledge of a specific model. By adopting a more open and transparent approach to engaging with energy systems models, Diagnostic Modelling encourages the participation of a broader range of decision-makers, thereby building consensus in support of the energy transition.
What carries the argument
Diagnostic Modelling, the framework of principles for modellers to critically interrogate energy systems optimisation models, explore uncertainties, and extract mechanistic explanations that hold independent of any single model's details.
If this is right
- Mechanistic explanations extracted this way remain valid even when the original model has uncertainties or is revised.
- Policy insights become usable without requiring stakeholders to understand the full technical details of the model.
- Greater openness in the modelling process draws in a wider group of decision-makers beyond technical experts.
- Shared mechanistic understandings help build broader agreement on steps for the energy transition.
Where Pith is reading between the lines
- The same interrogation steps could be applied to models in related areas such as climate policy or infrastructure planning to find robust explanations.
- Re-running past energy studies with Diagnostic Modelling might show which recommendations are consistent across different model versions.
- Practical tests could track whether teams using the framework produce reports that lead to faster or more inclusive policy adoption.
Load-bearing premise
That adopting a more open and transparent approach through Diagnostic Modelling will encourage the participation of a broader range of decision-makers and thereby build consensus in support of the energy transition.
What would settle it
A comparison of stakeholder engagement and policy consensus levels in energy projects that use standard model reporting versus those that apply Diagnostic Modelling to surface mechanistic explanations.
Figures
read the original abstract
Energy systems optimisation models are a leading tool for informing decisions in the energy transition. However, these models often remain opaque, and results are frequently presented without a clear discussion of their epistemic limitations. We propose Diagnostic Modelling as a framework wherein modellers critically interrogate their models and explore uncertainties to uncover mechanistic explanations that offer policy-relevant insights. Mechanistic explanations provide fundamental understanding that remains valid despite model uncertainty and does not depend on detailed knowledge of a specific model. By adopting a more open and transparent approach to engaging with energy systems models, Diagnostic Modelling encourages the participation of a broader range of decision-makers, thereby building consensus in support of the energy transition.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes 'Diagnostic Modelling' as a framework for responsible energy systems modelling. Modellers should critically interrogate their models and explore uncertainties to uncover mechanistic explanations that offer policy-relevant insights. These explanations are claimed to provide fundamental understanding that remains valid despite model uncertainty and does not depend on detailed knowledge of a specific model. By promoting a more open and transparent approach, the framework is said to encourage broader decision-maker participation and build consensus in support of the energy transition.
Significance. If the framework can be operationalized, it has the potential to improve transparency and epistemic awareness in energy systems optimisation models, shifting emphasis from opaque numerical outputs to robust mechanistic insights that support policy. This normative contribution is timely for the energy transition and could foster more inclusive modelling practices, though its value hinges on providing concrete principles and evidence rather than remaining at the definitional level.
major comments (2)
- [Abstract] Abstract: The core assertion that 'Mechanistic explanations provide fundamental understanding that remains valid despite model uncertainty and does not depend on detailed knowledge of a specific model' is presented axiomatically without elaboration, derivation, or illustrative example. This property is load-bearing for the entire proposal, as the framework's claimed advantage over standard modelling rests on it.
- [Abstract] Abstract: The claim that Diagnostic Modelling 'encourages the participation of a broader range of decision-makers, thereby building consensus in support of the energy transition' is central to the paper's motivation and significance but is advanced without any supporting mechanism, case study, reference to stakeholder literature, or hypothetical application.
minor comments (2)
- The title promises 'a framework of principles' but the abstract provides no enumeration or description of these principles; a dedicated section or table listing them explicitly would improve usability.
- The manuscript would benefit from at least one worked example or vignette showing how Diagnostic Modelling would be applied to an existing energy model to demonstrate the uncovering of mechanistic explanations.
Simulated Author's Rebuttal
We thank the referee for their constructive comments, which highlight opportunities to strengthen the presentation of our proposed Diagnostic Modelling framework. We respond to each major comment below and commit to revisions that address the concerns while preserving the manuscript's focus as a normative framework proposal.
read point-by-point responses
-
Referee: [Abstract] Abstract: The core assertion that 'Mechanistic explanations provide fundamental understanding that remains valid despite model uncertainty and does not depend on detailed knowledge of a specific model' is presented axiomatically without elaboration, derivation, or illustrative example. This property is load-bearing for the entire proposal, as the framework's claimed advantage over standard modelling rests on it.
Authors: We agree that the abstract presents this property concisely and without an example. The body of the manuscript derives the property from established concepts in the philosophy of science on mechanistic explanations, which prioritise causal structures over model-specific parameters or outputs. To make this more accessible in the abstract, we will add a short illustrative example demonstrating how a mechanistic insight (such as the requirement for dispatchable capacity to manage renewable variability) can be identified independently of particular model formulations or uncertainty ranges. revision: yes
-
Referee: [Abstract] Abstract: The claim that Diagnostic Modelling 'encourages the participation of a broader range of decision-makers, thereby building consensus in support of the energy transition' is central to the paper's motivation and significance but is advanced without any supporting mechanism, case study, reference to stakeholder literature, or hypothetical application.
Authors: We accept that the abstract advances this claim without explicit support. The manuscript explains the mechanism via the framework's focus on transparent interrogation and mechanistic insights that reduce the need for specialised modelling expertise, thereby lowering participation barriers. Relevant literature on stakeholder engagement and open modelling practices is referenced in the body. As the paper proposes a conceptual framework rather than an empirical study, a full case study lies outside its scope. We will revise the abstract to briefly reference the mechanism and add a concise hypothetical application in the discussion section to illustrate potential effects on decision-maker involvement. revision: partial
Circularity Check
No significant circularity in the proposed framework
full rationale
The manuscript advances a purely normative and definitional framework of principles for energy systems modelling. It contains no mathematical derivations, equations, fitted parameters, predictions, or empirical claims that could reduce to their own inputs by construction. The central assertion—that Diagnostic Modelling yields mechanistic explanations valid despite model uncertainty—is presented as a prescriptive recommendation rather than a deductive result derived from prior quantitative work or self-citations. No load-bearing steps invoke uniqueness theorems, ansatzes smuggled via citation, or renaming of known results. The proposal is self-contained as a set of guiding principles independent of any fitted data or prior author-specific theorems.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Energy systems optimisation models are a leading tool for informing decisions in the energy transition.
- ad hoc to paper Mechanistic explanations provide fundamental understanding that remains valid despite model uncertainty and does not depend on detailed knowledge of a specific model.
invented entities (1)
-
Diagnostic Modelling framework
no independent evidence
Reference graph
Works this paper leans on
-
[1]
IEA https://www.iea.org/reports/world-energy- outlook-2024 (2024)
World Energy Outlook 2024 – Analysis. IEA https://www.iea.org/reports/world-energy- outlook-2024 (2024)
2024
-
[2]
& Lofstedt, R
Bouder, F. & Lofstedt, R. Risk and uncertainty: what it means for the Dutch energy transition and how a tolerability of risk approach could help. Journal of Risk Research 27, 625–636 (2024)
2024
-
[3]
& Moslener, U
Haas, C., Jahns, H., Kempa, K. & Moslener, U. Deep uncertainty and the transition to a low- carbon economy. Energy Research & Social Science 100, 103060 (2023)
2023
-
[4]
Climate Policy Uncertainty and Investment Risk – Analysis
OECD. Climate Policy Uncertainty and Investment Risk – Analysis. IEA https://www.iea.org/reports/climate-policy-uncertainty-and-investment-risk (2017)
2017
-
[5]
& Keirstead, J
Pfenninger, S., Hawkes, A. & Keirstead, J. Energy systems modeling for twenty-first century energy challenges. Renewable and Sustainable Energy Reviews 33, 74–86 (2014)
2014
-
[6]
Hall, L. M. H. & Buckley, A. R. A review of energy systems models in the UK: Prevalent usage and categorisation. Applied Energy 169, 607–628 (2016)
2016
-
[7]
DeCarolis, J. et al. Formalizing best practice for energy system optimization modelling. Applied Energy 194, 184–198 (2017)
2017
-
[8]
Chang, M. et al. Trends in tools and approaches for modelling the energy transition. Applied Energy 290, 116731 (2021)
2021
-
[9]
& Stolten, D
Lopion, P ., Markewitz, P ., Robinius, M. & Stolten, D. A review of current challenges and trends in energy systems modeling. Renewable and Sustainable Energy Reviews 96, 156–166 (2018)
2018
-
[10]
& van Vuuren, D
van Beek, L., Oomen, J., Hajer, M., Pelzer, P . & van Vuuren, D. Navigating the political: An analysis of political calibration of integrated assessment modelling in light of the 1.5 °C goal. Environmental Science & Policy 133, 193–202 (2022)
2022
-
[11]
Morgan, M. G. & Keith, D. W. Improving the way we think about projecting future energy use and emissions of carbon dioxide. Climatic Change 90, 189–215 (2008)
2008
-
[12]
Keep it complex
Stirling, A. Keep it complex. Nature 468, 1029–1031 (2010)
2010
-
[13]
Popper, K. R. The Logic of Scientific Discovery. (Psychology Press, 2002)
2002
-
[14]
Lempert, R. J. Robust Decision Making (RDM). in Decision Making under Deep Uncertainty: From Theory to Practice (eds Marchau, V. A. W. J., Walker, W. E., Bloemen, P . J. T. M. & Popper, S. W.) 23–51 (Springer International Publishing, Cham, 2019). doi:10.1007/978-3-030-05252-2_2
-
[15]
Exploratory Modeling for Policy Analysis
Bankes, S. Exploratory Modeling for Policy Analysis. Operations Research 41, 435–449 (1993)
1993
-
[16]
F., Hunter, K
DeCarolis, J. F., Hunter, K. & Sreepathi, S. The case for repeatable analysis with energy economy optimization models. Energy Economics 34, 1845–1853 (2012)
2012
-
[17]
DeCarolis, J. F. Using modeling to generate alternatives (MGA) to expand our thinking on energy futures. Energy Economics 33, 145–152 (2011)
2011
-
[18]
Opening Up
Stirling, A. “Opening Up” and “Closing Down”: Power, Participation, and Pluralism in the Social Appraisal of Technology. Science, Technology, & Human Values 33, 262–294 (2008). 14
2008
-
[19]
& Giampietro, M
Saltelli, A. & Giampietro, M. What is wrong with evidence based policy, and how can it be improved? Futures 91, 62–71 (2017)
2017
-
[20]
Models and the common good
Saltelli, A. Models and the common good. Environmental Modelling & Software 188, 106430 (2025)
2025
-
[21]
& Funtowicz, S
Saltelli, A. & Funtowicz, S. When All Models Are Wrong. Issues in Science and Technology 30, 79–85 (2014)
2014
-
[22]
Puy, A. et al. Models with higher effective dimensions tend to produce more uncertain estimates. Science Advances 8, eabn9450 (2022)
2022
-
[24]
A short comment on statistical versus mathematical modelling
Saltelli, A. A short comment on statistical versus mathematical modelling. Nat Commun 10, 3870 (2019)
2019
-
[25]
Experiments versus models: New phenomena, inference and surprise
Morgan, M. Experiments versus models: New phenomena, inference and surprise. Journal of Economic Methodology 12, 317–329 (2005)
2005
-
[26]
van Asselt, M. B. A. & Rotmans, J. Uncertainty in Integrated Assessment Modelling. Climatic Change 54, 75–105 (2002)
2002
-
[27]
& Praktiknjo, A
Priesmann, J., Nolting, L. & Praktiknjo, A. Are complex energy system models more accurate? An intra-model comparison of power system optimization models. Applied Energy 255, 113783 (2019)
2019
-
[28]
Yue, X. et al. A review of approaches to uncertainty assessment in energy system optimization models. Energy Strategy Reviews 21, 204–217 (2018)
2018
-
[29]
Rosen, R. A. & Guenther, E. The economics of mitigating climate change: What can we know? Technological Forecasting and Social Change 91, 93–106 (2015)
2015
-
[30]
Integrated assessment models of climate change: An incomplete overview
Dowlatabadi, H. Integrated assessment models of climate change: An incomplete overview. Energy Policy 23, 289–296 (1995)
1995
-
[31]
C., Mealy, P
Way, R., Ives, M. C., Mealy, P . & Farmer, J. D. Empirically grounded technology forecasts and the energy transition. Joule 6, 2057–2082 (2022)
2057
-
[32]
Korteling, J. E. (Hans), Paradies, G. L. & Sassen-van Meer, J. P . Cognitive bias and how to improve sustainable decision making. Front. Psychol. 14, (2023)
2023
-
[33]
Parker, P . et al. Progress in integrated assessment and modelling1. Environmental Modelling & Software 17, 209–217 (2002)
2002
-
[34]
& Anger-Kraavi, A
Doukas, H., Nikas, A., González-Eguino, M., Arto, I. & Anger-Kraavi, A. From Integrated to Integrative: Delivering on the Paris Agreement. Sustainability 10, 2299 (2018)
2018
-
[35]
Pindyck, R. S. Climate Change Policy: What Do the Models Tell Us? Journal of Economic Literature 51, 860–872 (2013)
2013
-
[36]
DeCarolis, J. F. Using modeling to generate alternatives (MGA) to expand our thinking on energy futures. Energy Economics 33, 145–152 (2011). 15
2011
-
[37]
& Keppo, I
Trutnevyte, E., McDowall, W., Tomei, J. & Keppo, I. Energy scenario choices: Insights from a retrospective review of UK energy futures. Renewable and Sustainable Energy Reviews 55, 326–337 (2016)
2016
-
[38]
& Strachan, N
Gambhir, A., Butnar, I., Li, P .-H., Smith, P . & Strachan, N. A Review of Criticisms of Integrated Assessment Models and Proposed Approaches to Address These, through the Lens of BECCS. Energies 12, 1747 (2019)
2019
-
[39]
Epstein, J. M. Why Model? Journal of Artificial Societies and Social Simulation 11, 1–12 (2008)
2008
-
[40]
Stark, P . B. Pay No Attention to the Model Behind the Curtain. Pure Appl. Geophys. 179, 4121–4145 (2022)
2022
-
[41]
& Strachan, N
Trutnevyte, E., Guivarch, C., Lempert, R. & Strachan, N. Reinvigorating the scenario technique to expand uncertainty consideration. Climatic Change 135, 373–379 (2016)
2016
-
[42]
& Brown, T
Neumann, F. & Brown, T. Broad ranges of investment configurations for renewable power systems, robust to cost uncertainty and near-optimality. iScience 26, 106702 (2023)
2023
-
[43]
Sterman, J. D. All models are wrong: reflections on becoming a systems scientist. System Dynamics Review 18, 501–531 (2002)
2002
-
[44]
Mersch, M., Markides, C. N. & Dowell, N. M. The impact of the energy crisis on the UK’s net- zero transition. iScience 26, (2023)
2023
-
[45]
& Doolan, M
Featherston, C. & Doolan, M. A Critical Review of the Criticisms of System Dynamics. (System Dynamics Society, 2012)
2012
-
[46]
Forrester, J. W. Lessons from system dynamics modeling. System Dynamics Review 3, 136– 149 (1987)
1987
-
[47]
Sarker, I. H. Data Science and Analytics: An Overview from Data-Driven Smart Computing, Decision-Making and Applications Perspective. SN COMPUT. SCI. 2, 377 (2021)
2021
-
[48]
Keppo, I. et al. Exploring the possibility space: taking stock of the diverse capabilities and gaps in integrated assessment models. Environ. Res. Lett. 16, 053006 (2021)
2021
-
[49]
& Weyant, J
Peace, J. & Weyant, J. Insights Not Numbers: The Appropriate Use of Economic Models. (2008)
2008
-
[50]
G., Weyant, J
Huntington, H. G., Weyant, J. P . & Sweeney, J. L. Modeling for insights, not numbers: the experiences of the energy modeling forum. Omega 10, 449–462 (1982)
1982
-
[51]
Jaramillo, P . et al. The Open Energy Outlook 2023. https://www.cmu.edu/energy/_files/documents/oeo-report-2023.pdf (2023)
2023
-
[52]
Net-Zero America
America, N.-Z. Net-Zero America. Net-Zero America https://netzeroamerica.princeton.edu/ (2021)
2021
-
[53]
Annual Energy Outlook 2025 - U.S
EIA. Annual Energy Outlook 2025 - U.S. Energy Information Administration (EIA). https://www.eia.gov/outlooks/aeo/index.php (2025)
2025
-
[54]
Saltelli, A. et al. Why so many published sensitivity analyses are false: A systematic review of sensitivity analysis practices. Environmental Modelling & Software 114, 29–39 (2019). 16
2019
-
[55]
Simon, H. A. Prediction and Prescription in Systems Modeling. Operations Research 38, 7–14 (1990)
1990
-
[56]
& Graça Carvalho, M
Lund, H., Duić, N., Krajac˘ić, G. & Graça Carvalho, M. da. Two energy system analysis models: A comparison of methodologies and results. Energy 32, 948–954 (2007)
2007
-
[57]
Luderer, G. et al. The economics of decarbonizing the energy system—results and insights from the RECIPE model intercomparison. Climatic Change 114, 9–37 (2012)
2012
-
[58]
Prina, M. G. et al. Comparison methods of energy system frameworks, models and scenario results. Renewable and Sustainable Energy Reviews 167, 112719 (2022)
2022
-
[59]
Gates, W. L. et al. An Overview of the Results of the Atmospheric Model Intercomparison Project (AMIP I). https://journals.ametsoc.org/view/journals/bams/80/1/1520- 0477_1999_080_0029_aootro_2_0_co_2.xml (1999)
1999
-
[60]
Covey, C. et al. An overview of results from the Coupled Model Intercomparison Project. Global and Planetary Change 37, 103–133 (2003)
2003
-
[61]
W., Berkhout, F
Geels, F. W., Berkhout, F. & van Vuuren, D. P . Bridging analytical approaches for low-carbon transitions. Nature Clim Change 6, 576–583 (2016)
2016
-
[62]
How the Tiger Bush Got Its Stripes: ‘How Possibly’ vs
Bokulich, A. How the Tiger Bush Got Its Stripes: ‘How Possibly’ vs. ‘How Actually’ Model Explanations. The Monist 97, 321–338 (2014)
2014
-
[63]
Mayo, D. G. Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars. (Cambridge University Press, Cambridge, 2018). doi:10.1017/9781107286184
-
[64]
Cano Renteria, E., Schwartz, J. A. & Jenkins, J. Evaluating advanced nuclear fission technologies for future decarbonized power grids. Applied Energy 398, 126395 (2025)
2025
-
[65]
Nature 276, 429–429 (1978)
Rothschild’s numerate arrogance. Nature 276, 429–429 (1978)
1978
-
[66]
& Blechinger, P
Krumm, A., Süsser, D. & Blechinger, P . Modelling social aspects of the energy transition: What is the current representation of social factors in energy models? Energy 239, 121706 (2022)
2022
-
[67]
Maeda, E. E. et al. Black Boxes and the Role of Modeling in Environmental Policy Making. Front. Environ. Sci. 9, (2021)
2021
-
[68]
How can quantitative policy analysis inform the energy transition? The case of electrification
Vaishnav, P . How can quantitative policy analysis inform the energy transition? The case of electrification. Front. Sustain. Energy Policy 2, (2023)
2023
-
[69]
Victor Valentine, S., Sovacool, B. K. & Brown, M. A. Frame envy in energy policy ideology: A social constructivist framework for wicked energy problems. Energy Policy 109, 623–630 (2017)
2017
-
[70]
Sovacool, B. K. & Brown, M. A. Deconstructing facts and frames in energy research: Maxims for evaluating contentious problems. Energy Policy 86, 36–42 (2015)
2015
-
[71]
Opening new institutional spaces for grappling with uncertainty: A constructivist perspective
Duncan, R. Opening new institutional spaces for grappling with uncertainty: A constructivist perspective. Environmental Impact Assessment Review 38, 151–154 (2013)
2013
-
[72]
Climate Change Politics Through a Constructivist Prism
Pfefferle, T. Climate Change Politics Through a Constructivist Prism. E-International Relations https://www.e-ir.info/2014/06/18/climate-change-politics-through-a-constructivist-prism/ (2014)
2014
-
[73]
When does power listen to truth? A constructivist approach to the policy process
Haas, P . When does power listen to truth? A constructivist approach to the policy process. Journal of European Public Policy 11, 569–592 (2004). 17
2004
-
[74]
& Strachan, N
Usher, W. & Strachan, N. Critical mid-term uncertainties in long-term decarbonisation pathways. Energy Policy 41, 433–444 (2012)
2012
-
[75]
Environment, U. N. Emissions Gap Report 2023 | UNEP - UN Environment Programme. https://www.unep.org/resources/emissions-gap-report-2023 (2023)
2023
-
[76]
Justice as a measure of energy transition success
Nock, D. Justice as a measure of energy transition success. Nat Energy 11, 5–6 (2026)
2026
-
[77]
Mara, T. A. & Tarantola, S. Variance-based sensitivity indices for models with dependent inputs. Reliability Engineering & System Safety 107, 115–121 (2012)
2012
-
[78]
F., Babaee, S., Li, B
DeCarolis, J. F., Babaee, S., Li, B. & Kanungo, S. Modelling to generate alternatives with an energy system optimization model. Environmental Modelling & Software 79, 300–310 (2016)
2016
-
[79]
Does cost optimization approximate the real-world energy transition? Energy 106, 182–193 (2016)
Trutnevyte, E. Does cost optimization approximate the real-world energy transition? Energy 106, 182–193 (2016)
2016
-
[80]
black box
Süsser, D. et al. Model-based policymaking or policy-based modelling? How energy models and energy policy interact. Energy Research & Social Science 75, 101984 (2021). S1. Decision-making methods and tools for the energy transition S1.1. A broad classification of methods and tools for decision-making Decision-making methods can be broadly classified as: q...
2021
-
[81]
In: Dancer, A., Garc´ıa-Prada, O., Kirwan, F
Large, uncertain input datasetsS18,S179,S181,S183 . Optimistic decision representation S179,S186– S190 technology capacity and its utilisationS18. Technological fidelityS4,S179. (optimisation, rationality, perfect information, efficient markets)S13,S179,S181. Cost as primary objective misrepresents real- world decision- makingS10,S185. Geographic informat...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.