Pith. sign in

REVIEW 5 major objections 6 minor 72 references

AI-Driven Climate Policy Scenario Generation for Sub-Saharan Africa

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that generative AI, grounded in historical COP documents, can generate climate policy scenarios for Sub-Saharan Africa that experts accept 88% of the time.

desk verdict A modest, honest RAG application for policy scenario generation; the evaluation design makes the headline claim conditional rather than proven. read the letter →

arxiv 2505.18694 v1 pith:7KHDHRBN submitted 2025-05-24 cs.AI

Rafiu Adekoya Badekale , Adewale Akinfaderin This is my paper · ORCID
classification cs.AI
keywords climatepolicyscenariosgenerativeAIretrieval-augmentedgenerationSub-SaharanAfricaenergytransitionautomatedevaluationlargelanguagemodelsscenarioplanning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a small generative language model, grounded in historical COP conference documents through retrieval-augmented generation, can produce climate policy scenarios for Sub-Saharan Africa that are coherent, relevant, plausible, and diverse. Of 34 generated responses, 30 passed the authors' validation, and automated quality scores from a human climate expert and two other language models averaged above 0.70 on faithfulness, answer relevancy, and context utilization. The claim matters because traditional scenario methods, such as integrated assessment models, need detailed country-level data that many Sub-Saharan African countries lack, so an AI pipeline that works from international climate documents could offer a faster route to policy options. A sympathetic reader would take this as a proof of concept for data-constrained energy-transition planning, not as a replacement for expert review.

What carries the argument

The mechanism has two parts. The generation side is a retrieval-augmented pipeline: 94 COP documents are chunked, embedded, and stored in a vector database; each zero-shot prompt retrieves the most relevant passages, and the language model composes a scenario from that context. The evaluation side is a set of three automated metrics—faithfulness, answer relevancy, and context utilization—applied by a human expert and two LLM judges, with Spearman correlation used to compare evaluator rankings. The faithfulness metric tests whether the answer stays grounded in the retrieved context rather than hallucinating; answer relevancy tests whether the response addresses the actual prompt; context utilization tests whether the retrieved context is focused.

What would settle it

A decisive check is to recruit a second independent climate expert with regional expertise to score the same 30 generated responses using the paper's 1-5 rubric; if inter-rater agreement with the first expert is weak (Spearman below approximately 0.5) or the new mean scores fall below 0.70, the expert-validation claim would not survive. An out-of-sample test would compare generated scenarios against expert-written policy scenarios on the same prompts, with automated scores recomputed on that labeled set.

Watch

Extended reading notes

Core claim

The central discovery is that the llama3.2-3B model, using retrieval-augmented generation over historical COP documents and zero-shot prompts, generates policy scenarios that expert validation accepts: 30 of 34 responses (88%) were judged to reflect the intended impacts in their prompts, and all three evaluators assigned mean scores above 0.70 across faithfulness, answer relevancy, and context utilization. The paper interprets the score pattern, including high answer relevancy (0.993 for the human evaluator) and strong Spearman correlations between human and model faithfulness rankings, as evidence that generative AI can produce usable, region-specific scenarios and that embedding-based automated evaluation is a workable substitute when human evaluators are scarce.

Load-bearing premise

The conclusion holds only if the automatic scoring metrics—computed without ground-truth labels and partly against the same COP documents used to generate the answers—validly measure scenario quality and the single human climate expert is an unbiased judge.

Editorial extensions

If this is right

  • A retrieval-augmented generator grounded in COP documents can produce region-specific energy-transition scenarios for Sub-Saharan Africa without fine-tuning.
  • Automated evaluation with LLM judges can serve as a first-pass quality filter when human expert time is limited, though it aligns with humans on faithfulness more than on answer relevancy.
  • Zero-shot prompting is sufficient for many scenario themes, but fails on prompts where the model treats negative outcomes, such as an 'energy poverty trap,' as a policy scenario rather than a consequence of policy failure.
  • The scenario outputs are structured enough to support workshop-style planning exercises, such as ministry-level energy-transition discussions, when combined with human oversight.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: an out-of-sample evaluation against expert-written scenarios would test whether the high automated scores reflect scenario quality or the circularity of scoring against the same documents used for generation.
  • Beyond the paper: a panel of multiple independent regional experts with reported inter-rater reliability would separate model capability from evaluator subjectivity.
  • Beyond the paper: a direct comparison with traditional IAM-based or expert-crafted scenarios would clarify whether this approach adds value beyond speed and accessibility.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This manuscript proposes a retrieval-augmented generation (RAG) approach to climate policy scenario generation for Sub-Saharan Africa. Using 94 UNFCCC COP documents as the knowledge base and llama3.2-3B at temperature zero, the authors generate 34 zero-shot scenarios on topics such as renewable energy adoption and technology transfer. Two authors manually validate 30 of the 34 responses (88%), and the three RAGAs metrics—faithfulness, answer relevancy, and context utilization—are computed for the validated responses by a human climate expert and two LLM evaluators (gemma2-2B and mistral-7B). The paper reports mean metric scores above 0.70 and a mixture of Spearman correlations among evaluators, and concludes that generative AI can produce coherent, relevant, plausible, and diverse policy scenarios suitable for data-constrained regions.

Significance. The application target is worthwhile: scenario generation for climate policy in Sub-Saharan Africa is a real need, and the authors provide a transparent, reproducible pipeline with public code and data. The paper also demonstrates a concrete use of small, locally runnable LLMs (llama3.2-3B) with RAG rather than large closed APIs, which is appropriate for low-resource settings. If the evaluation concerns were addressed, the framework could be a useful methodological starting point. As it stands, the strengths are the real deployment context, the public artifacts, and the honest discussion of human/automated evaluation tension; the weaknesses are the self-referential nature of the main evaluation and the absence of baselines and diversity metrics.

major comments (5)
  1. [§2.3 and Table 1] The automated and human scores are reported only for the 30 responses that the two authors pre-selected as valid; the 4 rejected responses are discarded before any metric is applied. As a result, the abstract's two headline numbers—88% expert validation and mean metric scores above 0.70—are not independent lines of evidence; they are computed on the same author-filtered sample. Moreover, faithfulness is assessed against the retrieved COP context, which is the same context that conditioned llama3.2-3B at temperature zero, so a high faithfulness score largely confirms that the output stayed close to its conditioning input rather than establishing factual accuracy or plausibility with respect to the world. The paper should either report metrics on all 34 responses, or explicitly treat the validation and metric stages as one filter-plus-evaluation pipeline and temper claims accordingly.
  2. [Abstract and Table 2] The statement that Spearman correlation confirms a high degree of alignment is not supported by the reported coefficients. Faithfulness correlations are strong (0.814–0.919), but answer relevancy correlations with the human evaluator are only 0.311 for both LLMs, and context utilization human–mistral is 0.401; human–gemma2 context utilization is 0.859. The paper should disaggregate the correlation claim by metric and discuss the weak answer-relevancy alignment, which directly bears on the claim that automated evaluation is reliable.
  3. [Abstract and Table 1] The claim that generated scenarios are diverse is not operationalized anywhere. Table 1 contains only faithfulness, answer relevancy, and context utilization; no diversity metric (e.g., pairwise embedding distance, topic coverage, or scenario count per theme) is reported. Without a defined and measured diversity quantity, the diversity component of the headline claim is unsupported.
  4. [§2.3 and Table 1] No baseline is included for interpreting the absolute score levels. Mean scores near or above 0.7 are presented as evidence of quality, but with no comparison condition—for example, non-RAG generation, expert-written scenarios, or a random-retrieval control—the reader cannot tell whether the framework adds value or merely reflects the ease of the evaluation task. Adding at least one baseline would make the central claim testable.
  5. [§2.3 and Appendix C] The human evaluation rests on a single unnamed human climate expert, and the initial validation is performed by the two authors; no statement of independence or inter-rater reliability is provided. Since the entire external-grounding argument depends on this human oracle, the paper should disclose whether the expert is an author, report the detailed rubric used (beyond the appendix), and, ideally, include a second independent rater or an agreement statistic.
minor comments (6)
  1. [Appendix B] The rejected response is shown for a prompt that is not among the 30 validated prompts in Appendix A; the narrative should clarify which prompt variant generated the rejected response, since the validator note attributes the failure to the prompting approach.
  2. [§2.1] Figure 1 is referenced in the text but the figure itself does not appear in the manuscript; please include the flow diagram or remove the reference.
  3. [References] The bibliography lists many entries that are never cited in the body (e.g., [8], [9], [15], [29], [33], [37]); please either cite them in the relevant sections or remove them.
  4. [Abstract] The phrase 'effectively generate scenarios' should be 'effectively generates scenarios' to agree with the singular subject 'generative AI.'
  5. [§3.1] The use of the ratio of SD to mean as evidence that a mean is representative is informal; reporting confidence intervals or a formal reliability statistic would be more appropriate.
  6. [Abstract and §4] The abstract states that the method ensures robustness even under limited data conditions, but no limited-data ablation or data-scarcity experiment appears in the paper; this claim should be removed or supported with evidence.

Circularity Check

2 steps flagged · score 5.0 of 10

RAGAs evaluation is partly self-referential: faithfulness/context scores are measured against the same COP contexts that conditioned generation, and all scores are computed only on the 30 author-validated responses.

  1. self definitional [Section 2.2 (Scenario generation) and Section 2.3 (Scenario validation and evaluation)]
    "We used llama3.2-3B to generate scenarios with 34 zero-shot prompts, via a RAG pipeline grounded in the UN COP documents. ... Faithfulness which refers to the idea that the answer should be grounded in the given context to avoid hallucinations."

    The generator conditions on retrieved COP contexts (temperature zero), and the faithfulness/context-utilization metrics reward answers that are grounded in that same retrieved context. The reported faithfulness (0.760 human, 0.966 gemma2-2B, 0.848 mistral-7B) and context utilization (0.713/0.939/0.980) therefore largely measure how closely the output stayed to its conditioning input, not whether the scenario is factually accurate or plausible in the real world. The abstract's equation of faithfulness with 'factually accurate' thus rests on a metric whose reference document is the generation input, making the support self-referential by construction.

  2. other [Section 2.3, Table 1]
    "Two authors independently validated the generated scenarios. Out of 34 generated responses, 30 (88%) were deemed valid (highly depicting designated impacts provided in the corresponding prompt). We evaluated the quality of the validated responses using a three-metric Retrieval-Augmented Generation Assessment (RAGAs) framework."

    The 88% pass rate and the mean metric scores are not independent evidence: Table 1 is computed only on the 30 responses the two authors had already judged 'valid,' and the 4 rejected responses are never scored. Thus the high means (e.g., answer relevancy 0.993 for the human evaluator) are conditional on the authors' own validity filter. The evaluation is applied to the already-validated subset and then reported as confirming the framework, so the validation and the scores form a single author-filtered loop rather than two independent confirmations.

full rationale

The paper contains no formal derivation that reduces to its inputs, and there is no load-bearing self-citation chain or imported uniqueness theorem. The circularity is in the evaluation loop. Prompt themes were identified via the same RAG setup (Section 2.2), and faithfulness/context utilization are computed against the same COP documents that conditioned llama3.2-3B during generation, so high scores partly reflect adherence to the conditioning context rather than independent factual accuracy or plausibility. Second, the RAGAs metrics are applied only to the 30 responses that the two authors had already selected as valid, so the 88% validation and the high mean scores are not independent lines of evidence. Some independent content remains: the UNFCCC corpus is a fixed external source, the human climate expert provides a non-automated judgment, and RAGAs is a standard published benchmark, so this is not a pure tautology. The diversity claim is asserted without a diversity metric in Table 1, which further weakens the central claim but is an evidential gap rather than a circular step. Overall, the central claim is partially supported by self-referential evaluation and author-filtered scoring, warranting a moderate circularity score rather than a high one.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No parameters are fitted to optimize evaluation scores; the listed hyperparameters are hand-chosen preprocessing and generation settings. The central assumptions are the sufficiency of the COP corpus, the validity of automated RAGAs metrics for speculative scenario quality, and the reliability of the human and LLM evaluators. No new entities are introduced.

free parameters (3)
  • chunk_size = 1000
    Hand-chosen chunk size in RecursiveCharacterTextSplitter; affects retrieval granularity but is not fitted to scenario quality.
  • chunk_overlap = 100
    Hand-chosen overlap between chunks in the preprocessing step; a configuration choice rather than a fitted parameter.
  • temperature = 0
    Sampling temperature set to zero for consistency in generation; not tuned against evaluation metrics.
assumptions (4)
  • domain assumption UNFCCC COP documents are a sufficient knowledge base for plausible Sub-Saharan Africa energy transition policy scenarios.
    The entire RAG pipeline uses only the 94 collected COP documents as grounding; no national climate plans or SSA-specific socioeconomic data are included.
  • domain assumption RAGAs faithfulness, answer relevancy, and context utilization scores are valid proxies for scenario quality for speculative future policies.
    Evaluation in Section 2.3 relies on these automated metrics without ground-truth scenario quality labels; their validity for open-ended policy narratives is assumed.
  • domain assumption The single human climate expert is a reliable and independent evaluator of scenario quality.
    Section 2.3 and Appendix C describe one human evaluator; the paper does not disclose whether this evaluator is one of the two authors or report inter-rater reliability.
  • domain assumption LLM evaluators (gemma2-2B, mistral-7B) produce meaningful comparisons with human judgment.
    Used as automated evaluators in Section 2.3; the paper itself reports weak correlations with the human on answer relevancy and one context utilization pair.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AI-Driven Climate Policy Scenario Generation for Sub-Saharan Africa." pith.science (2026). https://pith.science/paper/7KHDHRBN

@misc{pith2026250518694,
  author       = {Pith},
  title        = {Pith review of: AI-Driven Climate Policy Scenario Generation for Sub-Saharan Africa},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7KHDHRBN}},
  note         = {Machine review of arXiv:2505.18694}
}
read the original abstract

Climate policy scenario generation and evaluation have traditionally relied on integrated assessment models (IAMs) and expert-driven qualitative analysis. These methods enable stakeholders, such as policymakers and researchers, to anticipate impacts, plan governance strategies, and develop mitigation measures. However, traditional methods are often time-intensive, reliant on simple extrapolations of past trends, and limited in capturing the complex and interconnected nature of energy and climate issues. With the advent of artificial intelligence (AI), particularly generative AI models trained on vast datasets, these limitations can be addressed, ensuring robustness even under limited data conditions. In this work, we explore the novel method that employs generative AI, specifically large language models (LLMs), to simulate climate policy scenarios for Sub-Saharan Africa. These scenarios focus on energy transition themes derived from the historical United Nations Climate Change Conference (COP) documents. By leveraging generative models, the project aims to create plausible and diverse policy scenarios that align with regional climate goals and energy challenges. Given limited access to human evaluators, automated techniques were employed for scenario evaluation. We generated policy scenarios using the llama3.2-3B model. Of the 34 generated responses, 30 (88%) passed expert validation, accurately reflecting the intended impacts provided in the corresponding prompts. We compared these validated responses against assessments from a human climate expert and two additional LLMs (gemma2-2B and mistral-7B). Our structured, embedding-based evaluation framework shows that generative AI effectively generate scenarios that are coherent, relevant, plausible, and diverse. This approach offers a transformative tool for climate policy planning in data-constrained regions.

Figures

Figures reproduced from arXiv: 2505.18694 by the authors.

Figure 1
Figure 1. Flow diagram of study 2.2 Scenario generation We used llama3.2-3B to generate scenarios with 34 zero-shot prompts, via a RAG pipeline grounded in the UN COP documents. The prompts were crafted from themes such as renewable energy adoption and technology transfer, identified via the same RAG setup. We set temperature to zero for consistency. To ground the generated scenarios in actionable steps (e.g., KPIs, milestone… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 47 canonical work pages

  1. [1]

    Pörtner et al., Eds., Cambridge, U.K.: Cambridge Univ

    Intergovernmental Panel on Climate Change, “Africa,” in Climate Change 2022 – Impacts, Adaptation and Vulnerability: Working Group II Contribution to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, H.-O. Pörtner et al., Eds., Cambridge, U.K.: Cambridge Univ. Press, 2023, pp. 1285–1456, doi:10.1017/9781009325844

    doi:10.1017/9781009325844 2022

  2. [2]

    AI for global climate cooperation: Modeling global climate negotiations, agreements, and long-term cooperation in RICE-N,

    T. Zhang et al., “AI for global climate cooperation: Modeling global climate negotiations, agreements, and long-term cooperation in RICE-N,” arXiv preprint arXiv:2208.07004, 2022 (under review)

    arXiv 2022

  3. [3]

    A review of criticisms of integrated assessment models and proposed approaches to address these, through the lens of BECCS,

    A. Gambhir, I. Butnar, P.-H. Li, P. Smith, and N. Strachan, "A review of criticisms of integrated assessment models and proposed approaches to address these, through the lens of BECCS," Energies, vol. 12, no. 9, May 2019, Art. no. 1747, doi:10.3390/en12091747

    doi:10.3390/en12091747 2019

  4. [4]

    RAGAs: Automated evaluation of re- trieval augmented generation,

    S. Es, J. James, L. E. Anke, and S. Schockaert, “RAGAs: Automated evaluation of re- trieval augmented generation,” in Proc. 18th Eur. Chapter Assoc. Comput. Linguistics Conf.: Syst. Demonstrations , St. Julians, Malta, Mar. 2024, pp. 150–158. [Online]. Available: https://aclanthology.org/2024.eacl-demo.16/

    work page 2024

  5. [5]

    AI-based scenario generation for future planning: An exploratory study using GPT-3,

    D. Jungwirth and D. Haluza, "AI-based scenario generation for future planning: An exploratory study using GPT-3," J. Curr. Trends Comp. Sci. Res. , vol. 2, no. 2, pp. 57-67, Apr. 2023. [Online]. Available: https://www.opastpublishers.com/peer-review/aibased-scenario-generation- for-future-planningrnan-exploratory-study-using-gpt3-5371.html

    work page 2023

  6. [6]

    Chain-of-thought prompting elicits reasoning in large language mod- els,

    J. Wei et al. , "Chain-of-thought prompting elicits reasoning in large language mod- els," Adv. Neural Inf. Process. Syst. , vol. 35, pp. 24824-24837, 2022. [Online]. Avail- able: https://papers.nips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4- Abstract-Conference.html

    work page 2022

  7. [7]

    Training language models to follow instructions with human feedback,

    L. Ouyang et al., "Training language models to follow instructions with human feedback," Adv. Neural Inf. Process. Syst., vol. 35, pp. 27730–27744, 2022

    work page 2022

  8. [8]

    GPT-4 technical report,

    J. Achiam et al., "GPT-4 technical report," arXiv preprint arXiv:2303.08774, 2024

    arXiv 2024

Show all 72 references
  1. [9]

    A review of scenario planning,

    M. Amer, T. U. Daim, and A. Jetter, "A review of scenario planning," Futures, vol. 46, pp. 23-40, Feb. 2013, doi:10.1016/j.futures.2012.10.003

    2013 doi

  2. [10]

    Global futures of energy, climate, and policy: Qualitative and quantitative foresight towards 2055,

    D. Ansari, F. Holz, and H. B. Tosun, "Global futures of energy, climate, and policy: Qualitative and quantitative foresight towards 2055," DIW Berlin Discussion Paper, no. 1782, Jan. 2019, doi:10.2139/ssrn.3324127

    2019 doi

  3. [11]

    Saving the United Nations Framework Convention on Climate Change from itself: Complementary and alternative approaches,

    A. Antypas, "Saving the United Nations Framework Convention on Climate Change from itself: Complementary and alternative approaches," Ekistics and the New Habitat: The Problems and Science of Human Settlements, vol. 82, no. 3, pp. 3-8, Jun. 2024, doi:10.53910/26531313- E2022823668

    2024 doi

  4. [12]

    Reliable, adaptable, and attributable language models with retrieval,

    A. Asai et al., "Reliable, adaptable, and attributable language models with retrieval," arXiv preprint arXiv:2403.03187, 2024. 5

    2024 arXiv

  5. [13]

    A cookbook of self-supervised learning,

    R. Balestriero et al., "A cookbook of self-supervised learning,"arXiv preprint arXiv:2304.12210, 2023

    2023 arXiv

  6. [14]

    Language models are few-shot learners,

    T. Brown et al. , "Language models are few-shot learners," Adv. Neu- ral Inf. Process. Syst. , vol. 33, pp. 1877-1901, 2020. [Online]. Available: https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html

    1901

  7. [15]

    RAG does not work for enterprises,

    T. Bruckhaus, "RAG does not work for enterprises," arXiv preprint arXiv:2406.04369, 2024

    2024 arXiv

  8. [16]

    Sparks of artificial general intelligence: Early experiments with GPT-4,

    S. Bubeck et al., "Sparks of artificial general intelligence: Early experiments with GPT-4," arXiv preprint arXiv:2303.12712, 2023

    2023 arXiv

  9. [17]

    Benchmarking large language models in retrieval- augmented generation,

    J. Chen, H. Lin, X. Han, and L. Sun, "Benchmarking large language models in retrieval- augmented generation," in Proc. 38th AAAI Conf. Artif. Intell., Vancouver, Canada, Feb. 20-27, 2024, vol. 38, no. 16, pp. 17754-17762, doi:10.1609/aaai.v38i16.29728

    2024 doi

  10. [18]

    Realgen: Retrieval augmented generation for controllable traffic scenarios,

    W. Ding, Y . Cao, D. Zhao, C. Xiao, and M. Pavone, "Realgen: Retrieval augmented generation for controllable traffic scenarios," in Proc. 18th Eur. Conf. Comput. Vis. - ECCV 2024, Milan, Italy, vol. 15120, pp. 93-110, Sep. 29-Oct. 4, 2024, doi:10.1007/978-3-031-73033-7_6

    2024 doi

  11. [19]

    Retrieval-augmented generation for large language models: A survey,

    Y . Gaoet al., "Retrieval-augmented generation for large language models: A survey," arXiv preprint arXiv:2312.10997, 2024

    2024 arXiv

  12. [20]

    Enhancing RAG with active learning on conversation records: Reject incapables and answer capables,

    X. Geng, H. Wang, J. Wang, W. Liu, and R. Li, "Enhancing RAG with active learning on conversation records: Reject incapables and answer capables,"arXiv preprint arXiv:2502.09073, 2025

    2025 arXiv

  13. [21]

    TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension,

    M. Joshi, E. Choi, D. Weld, and L. Zettlemoyer, "TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension," in Proc. 55th Annu. Meet. Assoc. Comput. Linguist. (ACL), Vancouver, Canada, Jul. 2017, pp. 1601-1611, doi:10.18653/v1/P17-1147

    2017 doi

  14. [22]

    QASC: A dataset for question answering via sentence composition,

    T. Khot, P. Clark, M. Guerquin, P. Jansen, and A. Sabharwal, "QASC: A dataset for question answering via sentence composition," in Proc. 34th AAAI Conf. Artif. Intell., New York, USA, Feb. 7-12, 2020, vol. 34, no. 05, pp. 8082-8090, doi:10.1609/aaai.v34i05.6319

    2020 doi

  15. [23]

    Generation constraint scaling can mitigate hallucination,

    G. Kollias, P. Das, and S. Chaudhury, "Generation constraint scaling can mitigate hallucination," presented at the ICML 2024 Workshop Large Lang. Models Cogn., Vienna, Austria, Jul. 27,

    2024

  16. [24]

    Natural questions: A benchmark for question answering research,

    T. Kwiatkowski et al., "Natural questions: A benchmark for question answering research," Trans. Assoc. Comput. Linguist., vol. 7, pp. 453-466, Aug. 2019, doi:10.1162/tacl_a_00276

    2019 doi

  17. [25]

    Retrieval-augmented generation for knowledge-intensive NLP tasks,

    P. Lewiset al., "Retrieval-augmented generation for knowledge-intensive NLP tasks," inProc. 34th Int. Conf. Neural Inf. Process. Syst. (NeurIPS), Vancouver, BC, Canada, Dec. 2020, pp. 9459-9474, doi:10.5555/3495724.3496517

    2020

  18. [26]

    From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning,

    M. Li et al., "From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning," in Proc. 2024 Conf. North Amer. Chapter Assoc. Comput. Linguist.: Human Lang. Technol. (NAACL-HLT), Mexico City, Mexico, Jun. 2024, pp. 7602- 7635, doi:10...

    2024 doi

  19. [27]

    Evaluating verifiability in generative search engines,

    N. Liu, T. Zhang, and P. Liang, "Evaluating verifiability in generative search engines," in Proc. Findings Assoc. Comput. Linguist.: EMNLP 2023, Singapore, Dec. 2023, pp. 7001-7025, doi:10.18653/v1/2023.findings-emnlp.467

    2023 doi

  20. [28]

    CRUD-RAG: A comprehensive Chinese benchmark for retrieval-augmented generation of large language models,

    Y . Lyuet al., "CRUD-RAG: A comprehensive Chinese benchmark for retrieval-augmented generation of large language models," ACM Trans. Inf. Syst., vol. 43, no. 2, pp. 1-32, Jan. 2025, doi:10.1145/3701228

    2025 doi

  21. [29]

    A perspective on the next generation of Earth system model scenarios: towards representative emission pathways (REPs),

    M. Meinshausen et al., “A perspective on the next generation of Earth system model scenarios: towards representative emission pathways (REPs),” Geosci. Model Dev., vol. 17, no. 11, pp. 4533–4559, Jun. 2024, doi:10.5194/gmd-17-4533-2024. 6

    2024 doi

  22. [30]

    A critical appraisal of the international and national legal framework on climate change,

    H. S. Samaradiwakara, “A critical appraisal of the international and national legal framework on climate change,” in Proc. 27th Int. Forestry Environ. Symp., Sri Lanka, Jan. 27-28, 2023, vol. 27, pp. 98–106, doi:10.31357/fesympo.v27.7101

    2023 doi

  23. [31]

    Ocean Risks in SIDS and LDCs,

    K. Tokunaga, A. Blandon, R. Blasiak, J.-B. Jouffray, C. C. C. Wabnitz, and A. V . Norström, “Ocean Risks in SIDS and LDCs,” Ocean Risk and Resilience Action Alliance (ORRAA), Oct

  24. [32]

    Knowledgeable in-context tuning: Exploring and exploiting factual knowledge for in-context learning,

    J. Wang, C. Wang, C. Tan, J. Huang, and M. Gao, “Knowledgeable in-context tuning: Exploring and exploiting factual knowledge for in-context learning,” in Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, Jun. 2024, pp. 3261–3280, doi: ...

    2024 doi

  25. [33]

    Evaluating process-based integrated assessment models of climate change mitigation,

    C. Wilson et al., "Evaluating process-based integrated assessment models of climate change mitigation," Climatic Change, vol. 166, pp. 1-22, May 2021, doi:10.1007/s10584-021-03099-9

    2021 doi

  26. [34]

    An explanation of in-context learning as implicit Bayesian inference,

    S. M. Xie, A. Raghunathan, P. Liang, and T. Ma, "An explanation of in-context learning as implicit Bayesian inference," presented at the 10th Int. Conf. Learn. Represent. (ICLR), virtual, Apr. 25-29, 2022. [Online]. Available: https://openreview.net/forum?id=RdJVFCHjUMI

    2022

  27. [35]

    The UNFCCC and the Paris Agreement,

    Y . Yamineva and N. Löther, "The UNFCCC and the Paris Agreement," inElgar Encyclopedia of Climate Policy, D. J. Fiorino, T. A. Eisenstadt, and M. K. Ahluwalia, Eds., Cheltenham, U.K.: Edward Elgar Publishing, 2024, pp. 249-252, doi:10.4337/9781802209204.ch47

    2024 doi

  28. [36]

    A survey of large language models,

    W. X. Zhao et al., "A survey of large language models," arXiv preprint arXiv:2303.18223, 2025

    2025 arXiv

  29. [37]

    RAGEval: Scenario specific RAG evaluation dataset generation frame- work,

    K. Zhu et al. , "RAGEval: Scenario specific RAG evaluation dataset generation frame- work,"arXiv preprint arXiv:2408.01262, 2025. 7 Appendix This section provides additional details to support the main findings of the paper, enhancing trans- parency and reproducibility. The ap...

    2025 arXiv

  30. [40]

    Consider policy incentives, grid integration challenges, and mechanisms for ensuring equitable access to clean energy."

    Prompt 1: "Generate potential future climate policy scenarios for Sub-Saharan Africa that focus on accelerating renewable energy adoption. Consider policy incentives, grid integration challenges, and mechanisms for ensuring equitable access to clean energy."

  31. [41]

    Include regulatory frameworks, financial incentives, and projected long-term benefits."

    Prompt 2: "Develop climate policy scenarios in which Sub-Saharan Africa prioritizes energy efficiency as a pillar of sustainable development. Include regulatory frameworks, financial incentives, and projected long-term benefits."

  32. [42]

    Explore community-led initiatives, financing models, and regulatory support."

    Prompt 3: "Propose future policy pathways that drive equitable access to renewable energy so- lutions in off-grid communities across Sub-Saharan Africa. Explore community-led initiatives, financing models, and regulatory support."

  33. [43]

    Explore how governments might incentivize production, build supporting infrastructure, and navigate geopolitical opportunities and risks in global energy markets."

    Prompt 4: "Develop climate policy scenarios where Sub-Saharan Africa prioritizes green hydrogen as a key energy export. Explore how governments might incentivize production, build supporting infrastructure, and navigate geopolitical opportunities and risks in global energy markets."

  34. [44]

    Highlight key focus areas, potential breakthroughs, and the role of international collaboration."

    Prompt 5: "Imagine future climate policy scenarios where Sub-Saharan Africa invests heavily in research and development of new climate technologies. Highlight key focus areas, potential breakthroughs, and the role of international collaboration."

  35. [45]

    Consider applications in emissions monitoring, predictive modeling, and adaptive policymaking."

    Prompt 6: "Generate scenarios in which artificial intelligence (AI) and digital technologies are integrated into climate policy implementation across Sub-Saharan Africa. Consider applications in emissions monitoring, predictive modeling, and adaptive policymaking."

  36. [46]

    Consider pathways that emphasize economic diversification, workforce retraining initiatives, and regulatory reforms."

    Prompt 7: "Explore different climate policy scenarios in which Sub-Saharan Africa reduces fossil fuel dependency and transitions to a clean energy economy. Consider pathways that emphasize economic diversification, workforce retraining initiatives, and regulatory reforms."

  37. [47]

    Consider potential economic benefits, implementation hurdles, and regional cooperation models."

    Prompt 8: "Develop climate policy scenarios in which Sub-Saharan Africa adopts carbon pric- ing and emissions trading as primary tools for reducing greenhouse gas emissions. Consider potential economic benefits, implementation hurdles, and regional cooperation models."

  38. [48]

    Discuss waste-to-energy innovations, material reuse, and industrial symbiosis."

    Prompt 9: "Explore policy scenarios where Sub-Saharan Africa integrates circular economy principles into energy policy to promote sustainability. Discuss waste-to-energy innovations, material reuse, and industrial symbiosis."

  39. [49]

    Discuss key regional and international partnerships (e.g., IAEA, COP agreements), infrastructure development needs, and financial feasibility."

    Prompt 10: "Explore viable climate policy scenarios in which Sub-Saharan Africa adopts nu- clear energy as part of its energy transition. Discuss key regional and international partnerships (e.g., IAEA, COP agreements), infrastructure development needs, and financial feasibility."

  40. [50]

    Discuss policy incentives for electric vehicle adoption, challenges in urban transport planning, and the role of biofuels and hydrogen as alternative energy sources."

    Prompt 11: "Develop climate policy scenarios for reducing transportation-related emissions in Sub-Saharan Africa. Discuss policy incentives for electric vehicle adoption, challenges in urban transport planning, and the role of biofuels and hydrogen as alternative energy sources."

  41. [51]

    Highlight local innovations, policy enablers, and community-driven approaches." 8

    Prompt 12: "Imagine future scenarios where decentralized energy efficiency initiatives sig- nificantly reduce energy poverty in Sub-Saharan Africa. Highlight local innovations, policy enablers, and community-driven approaches." 8

  42. [52]

    Consider incentives for reforestation, carbon sequestration policies, and indigenous land management practices."

    Prompt 13: "Develop potential future climate policy scenarios for Sub-Saharan Africa that emphasize sustainable land use and afforestation. Consider incentives for reforestation, carbon sequestration policies, and indigenous land management practices."

  43. [53]

    Consider public-private partnerships, sovereign green bonds, and risk mitigation strategies."

    Prompt 14: "Generate possible future scenarios where Sub-Saharan Africa accelerates climate finance through international funding mechanisms. Consider public-private partnerships, sovereign green bonds, and risk mitigation strategies."

  44. [54]

    Address governance reforms, inter-agency coordination, and financial resource allocation."

    Prompt 15: "Propose detailed policy scenarios for Sub-Saharan Africa that strengthen institu- tional capacity for climate adaptation. Address governance reforms, inter-agency coordination, and financial resource allocation."

  45. [55]

    Discuss cross-sectoral collaboration, funding integration, and legislative frameworks."

    Prompt 16: "Explore future policy pathways where Sub-Saharan Africa mainstreams climate adaptation into national development plans. Discuss cross-sectoral collaboration, funding integration, and legislative frameworks."

  46. [56]

    Consider land rights, traditional resource management practices, and equitable governance structures."

    Prompt 17: "Explore policy pathways where indigenous knowledge and local community-led solutions shape climate adaptation strategies in Sub-Saharan Africa. Consider land rights, traditional resource management practices, and equitable governance structures."

  47. [57]

    Include capacity-building initiatives, knowledge-sharing platforms, and policy uptake metrics."

    Prompt 18: "Generate policy scenarios where empowering local communities through climate education and advocacy leads to stronger grassroots climate action. Include capacity-building initiatives, knowledge-sharing platforms, and policy uptake metrics."

  48. [58]

    Consider skills development, stakeholder collaboration, and policy implementation."

    Prompt 19: "Imagine future climate policy scenarios where capacity building for climate governance strengthens institutional responses to climate challenges in Sub-Saharan Africa. Consider skills development, stakeholder collaboration, and policy implementation."

  49. [59]

    Include plausible timeframes, critical actors, and key funding mechanisms."

    Prompt 20: "Please generate potential future policy scenarios for Sub-Saharan Africa that focus on building climate-resilient infrastructure—particularly around water management and disaster risk reduction. Include plausible timeframes, critical actors, and key funding mechanisms."

  50. [60]

    Include metrics and expected outcomes."

    Prompt 21: "Propose detailed policy scenarios for Sub-Saharan Africa focusing on the health sector’s adaptation to climate-induced challenges, such as heatwaves, vector-borne diseases, and flood-related health crises. Include metrics and expected outcomes."

  51. [61]

    Consider international donors, private capital, and novel funding mechanisms, and explain major opportunities and risks."

    Prompt 22: "Generate possible future scenarios in which Sub-Saharan Africa accelerates green finance for large-scale renewable energy projects. Consider international donors, private capital, and novel funding mechanisms, and explain major opportunities and risks."

  52. [62]

    Prompt 23: "What are potential future climate policy pathways in Sub-Saharan Africa if carbon markets and offset schemes become mainstream? Outline how governments, regional bodies, and local communities might participate or benefit."

  53. [63]

    How might local institutions be strengthened to adopt, maintain, and innovate on climate- related technologies (e.g., solar, wind, climate-smart agriculture)?"

    Prompt 24: "Imagine several policy scenarios in which external technology transfer accelerates. How might local institutions be strengthened to adopt, maintain, and innovate on climate- related technologies (e.g., solar, wind, climate-smart agriculture)?"

  54. [64]

    Include considerations of political will, stakeholder conflicts, and resource allocation."

    Prompt 25: "Generate a set of climate policy scenarios for Sub-Saharan Africa that show how collaboration between local, national, and regional bodies might evolve. Include considerations of political will, stakeholder conflicts, and resource allocation."

  55. [65]

    Consider equity issues, social inclusion, and mechanisms for ensuring marginalized groups have a voice."

    Prompt 26: "Develop future policy pathways in which local communities and grassroots movements play a pivotal role in shaping national climate strategies. Consider equity issues, social inclusion, and mechanisms for ensuring marginalized groups have a voice."

  56. [66]

    Explore how climate stressors could exacerbate conflict or incentivize deeper regional cooperation."

    Prompt 27: "Propose climate policy scenarios addressing transboundary resource management (e.g., shared water basins, pastoral lands) in Sub-Saharan Africa. Explore how climate stressors could exacerbate conflict or incentivize deeper regional cooperation."

  57. [67]

    Include measurable outcomes and ethical considerations." 9

    Prompt 28: "Please propose climate policy scenarios that prioritize mitigation efforts with the largest public-health co-benefits, such as reducing indoor air pollution from traditional biomass cooking. Include measurable outcomes and ethical considerations." 9

  58. [68]

    Consider political inertia, minimal international support, and accelerating climate impacts, and explore the long-term social and economic consequences."

    Prompt 29: "Describe a range of worst-case ‘business as usual’ climate policy scenarios for Sub-Saharan Africa. Consider political inertia, minimal international support, and accelerating climate impacts, and explore the long-term social and economic consequences."

  59. [69]

    How might governments, communities, and private actors innovate or pivot policy approaches in this high-risk future?" Follow-up prompts

    Prompt 30: "Generate scenarios in which Sub-Saharan Africa experiences more extreme climate impacts than currently predicted. How might governments, communities, and private actors innovate or pivot policy approaches in this high-risk future?" Follow-up prompts

  60. [70]

    Prompt 1: What specific metrics can be used to measure the success of each scenario? Identify Key Performance Indicators (KPIs) that track progress and milestones that indicate meaningful advancements

  61. [71]

    Consider factors such as economic conditions, policy frameworks, technology readiness, public perception, and environmental constraints

    Prompt 2: Identify the key drivers that will influence the success or failure of each scenario. Consider factors such as economic conditions, policy frameworks, technology readiness, public perception, and environmental constraints. Additionally, analyze how these key drivers ...

  62. [72]

    African Renaissance

    Prompt 3: Outline a clear implementation roadmap with well-defined milestones. Define short- term (0-2 years), medium-term (3-7 years), and long-term (8+ years) milestones. Highlight critical decision points, dependencies, and risks that could impact progress. Provide specific...

  63. [2021]

    Available: https://gmri-org-production.s3.amazonaws.com/documents/ORRAA- Ocean-Risks.pdf

    [Online]. Available: https://gmri-org-production.s3.amazonaws.com/documents/ORRAA- Ocean-Risks.pdf

  64. [2024]

    Available: https://openreview.net/forum?id=FCsxpYu80U

    [Online]. Available: https://openreview.net/forum?id=FCsxpYu80U

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.