Pith. sign in

REVIEW 4 major objections 4 minor 34 references

AfriEconQA shows that exact numerical and temporal grounding in long economic reports is an open problem for AI.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

An LLM-generated QA benchmark over World Bank African economic reports is evaluated on a small sample and found hard, but the abstract and body report conflicting dataset sizes, model names, and scores.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection A needed benchmark idea buried under an internally inconsistent manuscript; the abstract and full text report incompatible numbers, and the gold labels are unvalidated LLM outputs. the 4 major comments →

arxiv 2601.15297 v3 pith:KQQU2ICB submitted 2026-01-06 cs.CL

AfriEconQA: A Benchmark for Quantitative and Temporal Reasoning over World Bank Economic Reports

Edward Ajayi , Mustapha Alaba , David Stephen This is my paper
classification cs.CL
keywords question answeringretrieval-augmented generationeconomic dataWorld Bank reportsAfrican economiestemporal reasoningnumerical reasoningbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces AfriEconQA, a benchmark built from World Bank economic reports on African countries, containing thousands of question-answer pairs that require a system to retrieve the exact supporting passage and preserve precise numbers and dates. Its central claim is that the combination of exact grounding and temporal precision over long, repetitive institutional documents is not tested by existing QA benchmarks and is largely unsolved: language models without retrieval answer almost none of the questions, and even the best retrieval-augmented configuration reaches only about half the answer-quality score. If true, the benchmark documents a 'parametric vacuum' — African economic indicators are effectively missing from model pretraining — and provides a hard testbed for systems that must ground answers in authoritative sources.

Core claim

On its own terms, the paper establishes two findings. First, a dataset: AfriEconQA, with 8,937 QA instances (the abstract of one version reports 4,309) in five reasoning categories — factoid, list, multiple choice, synthesis, and comparison — each paired with an evidence snippet and source metadata, generated from 236 World Bank reports (or 220 in the abstract's count) so that questions require distinguishing historical actuals from future projections. Second, an empirical result: in evaluations on a stratified sample, zero-shot models score near zero, retrieval-augmented generation lifts performance substantially, yet the best configuration still achieves low exact match and F1, with the ha

What carries the argument

The central object is the benchmark itself: a corpus of World Bank reports chunked into about 65,000 retrieval units and a set of QA instances whose gold answers must satisfy a verbatim constraint — every percentage, fiscal year, and monetary value in an answer must appear character-for-character in the cited evidence snippet. The evaluation machinery pairs this corpus with a hybrid retrieval pipeline (BM25 plus dense embeddings fused by reciprocal rank fusion) that feeds a language-model generator. The verbatim constraint is what turns the benchmark into a test of exact numerical and temporal grounding rather than of paraphrase or general familiarity.

Load-bearing premise

The load-bearing premise is that the gold answers are correct: the methodology describes an automated auditing layer that enforces verbatim numerical matching but no human review of the final labels (the abstract's mention of human validation does not appear in the full-text methods), so any systematic labeling error would make every reported accuracy figure unreliable.

What would settle it

Take a random sample of 100–200 AfriEconQA instances and have two independent analysts with economics background verify each gold answer against the cited evidence snippet without seeing any model output; if a nontrivial fraction of labels are wrong, ambiguous, or unsupported, then the benchmark's difficulty is an artifact of the generation pipeline rather than a property of the models.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Retrieval augmentation is a prerequisite for reliable QA on African economic data: without it, models score near zero.
  • Hybrid retrieval (lexical plus dense, fused) produces better final answers than either retrieval strategy alone, even when it slightly lowers ranking precision.
  • Open-weight models can match or exceed proprietary models on exact-match extraction, suggesting they are a viable choice when strict numerical fidelity is required.
  • List and comparison questions remain largely unsolved, indicating that single-pass retrieval cannot supply the multiple, non-contiguous evidence chunks these questions require.
  • The benchmark offers a reusable, provenance-linked testbed for temporal disambiguation between historical outturns and forward-looking projections in long institutional reports.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the parametric vacuum is real, a natural check is whether models released after this corpus's cutoff improve on the same questions; such improvement would signal either data contamination or genuinely new pretraining data, both of which would erode the benchmark's difficulty.
  • Because the corpus draws on a single institution's reporting style, the difficulty may partly be a genre effect; rebuilding the same pipeline from central-bank or national statistics publications would show whether the benchmark measures a general reasoning gap or a house style.
  • The verbatim numerical constraint may over-penalize models that paraphrase the same number correctly; adding a semantically-equivalent-number evaluation would separate grounding failures from formatting differences.
  • The paper's abstract mentions a stratified human-validation subset for gold-label auditing, but the full methodology describes only automated filtering; resolving this discrepancy matters because label errors would cap the reported accuracies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces AfriEconQA, a question-answering benchmark built from World Bank reports on African economies, with QA pairs generated synthetically and evaluated under retrieval-augmented generation (RAG) configurations. The authors claim that current LLMs have a 'parametric knowledge vacuum' for African economic indicators and that even strong RAG pipelines struggle on the benchmark, which they offer as a hard testbed for numerical and temporal grounding. The full-text version reports 8,937 QA instances from 236 reports, a filtered subset of a 10,018 synthetic pool, and an 11-configuration evaluation on a stratified sample of 300 questions. However, the abstract and the full text disagree on nearly every headline number—corpus size, instance count, test-set size, model names, and reported F1—and the gold labels are produced by an LLM pipeline with no described human validation. No data or code is released, so the benchmark's central claims are currently unverifiable.

Significance. If the results held up, AfriEconQA would fill a real gap: a focused benchmark for document-grounded QA over African economic reporting, combining temporal disambiguation, exact numerical extraction, and a taxonomy of reasoning types. The paper also raises a worthwhile question about the parametric coverage of African economic data and whether open-weight models can be competitive on extractive tasks. Those strengths are, however, entirely contingent on the validity of the gold labels and the reproducibility of the numbers. At present the manuscript does not provide the data, code, or human-annotation evidence needed to establish either, so the significance is potential rather than demonstrated.

major comments (4)
  1. [§3.3-§3.4, §5.2, §7] The gold answers are generated by an LLM pipeline and filtered only by an automated 'auditing layer' enforcing verbatim string constraints. The full text nowhere describes human validation, annotator counts, inter-annotator agreement, or label-error rates; the abstract's claim of a 'stratified human-validation subset for gold-label auditing' is unsupported by any experimental detail in the body. Since Tables 2–4 evaluate models against these LLM-generated labels, and the LLM-Judge metric uses GPT-4o-mini as scorer, systematic label noise or string-matching artifacts could produce the observed low scores independently of model capability. The Limitations section (§7) also omits label noise or annotation disagreement as a risk. A human-label audit with error statistics and a human performance baseline on the 300-item sample is necessary to calibrate the benchmark's difficulty.
  2. [Abstract vs. §3.1, §5, §6] The abstract and full text report incompatible numbers: 220 vs. 236 source reports, 4,309 vs. 8,937 QA instances, an n=862 test split vs. an n=300 evaluation sample, and different model names (Qwen 3.6 35B/DeepSeek v4-pro/Gemma 4 12B IT vs. GPT-5 Mini/GPT-4o/Qwen 32B). The reasoning-category counts in the abstract (Factoid 1,093, List 879, Multiple Choice 944, Synthesis 710, Comparison 683) sum to 4,309 and do not match Table 1's counts (3,237, 1,705, 1,265, 1,205, 1,525) which sum to 8,937. The abstract's headline '0.545 F1' does not correspond to any row in Table 2. This is not a cosmetic discrepancy: a reader cannot determine which corpus, models, or splits the conclusions are based on, and every reported result inherits the ambiguity.
  3. [§6, §7, §9] The evaluation is conducted on a stratified sample of 300 questions rather than the full benchmark, yet no confidence intervals or significance tests are reported. The paper draws broad conclusions—e.g., 'current language models possess virtually no parametric knowledge of African economic indicators'—from a single zero-shot configuration on this small sample. More importantly, the dataset and code are only promised 'upon publication'; no link, dataset card, or release artifact is provided. For a benchmark paper, the data and code are the primary contribution, and without them the reader cannot verify the evidence-answer alignment, the temporal anchoring logic, or the exact split used. At minimum, release the 300-item evaluation sample and the full annotation/generation protocol.
  4. [§6.1, Table 4] The 'parametric vacuum' conclusion is presented as a robust finding but no contamination analysis is performed. The claim that African economic indicators are 'virtually absent from pre-training corpora' requires checking whether the exact reports or QA pairs appear in the training data of the evaluated models; without such a check, the zero-shot failure could partly reflect benchmark-specific formatting or sycophantic failure modes rather than a genuine knowledge gap. The claim as stated exceeds what the evidence supports. A simple overlap/contaimination test on the 236-document corpus against the evaluated models' training corpora is feasible and should be reported.
minor comments (4)
  1. [§4.2, Eq. (1)] The RRF constant η is never given a value. RRF results are sensitive to η; please report the value used and, ideally, a small sensitivity analysis.
  2. [§5.2, Table 4 caption] Table 4's caption mentions an 'LLM-Judge threshold≥0.7', but the text in §5.2 only describes a semantic score 0.0–1.0. Clarify how the threshold is applied: e.g., is it a binarization rule, a filtering criterion, or a condition for reporting a response as correct?
  3. [§1, §9] The abstract describes an 'agentic generation pipeline' but §3.3 and §9 describe a synthetic pipeline with an 'economic-expert persona' and no agentic loop. Please align the terminology or describe what agentic behavior was used.
  4. [§7] The Limitations section does not mention label noise, possible annotation errors, or the absence of human validation; adding these would make the limitations of the benchmark more transparent.

Circularity Check

0 steps flagged

No circular derivation: the benchmark's claims rest on an empirical evaluation of externally sourced documents, not on a self-referential fit.

full rationale

AfriEconQA is a new benchmark whose gold answers are generated by an LLM pipeline (§3.3), filtered by an automated auditing layer that enforces verbatim numerical grounding (§3.4), and scored with standard IR/QA metrics plus LLM-as-a-judge (§5.2). The central claims—that models display a parametric knowledge vacuum and that RAG systems achieve only limited precision—are empirical measurements on this constructed testbed, not quantities that reduce by construction to the inputs. The gold labels are anchored to external World Bank source documents via character-level constraints, so a low EM/F1 is not statistically forced by any fitted parameter. There is no self-citation chain, no imported uniqueness theorem, and no ansatz smuggled via citation. The abstract mentions a stratified human-validation subset that the full text does not document, and the paper contains internal inconsistencies in corpus size (220 vs. 236) and instance counts (4,309 vs. 8,937); however, these are data-quality and reproducibility concerns, not instances of the specific circularity patterns defined here. Per the hard rules, circularity requires exhibiting a specific reduction (e.g., Eq. X = Eq. Y by construction or a fitted parameter renamed as a prediction), and no such reduction is present.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The paper introduces no physical or named entity beyond the benchmark itself. Its load-bearing assumptions are the validity of LLM-generated labels, the fidelity of publication-date-based temporal anchoring, and the representativeness of the 300-question sample.

free parameters (2)
  • η (RRF constant)
    Used in Eq. (1) for Reciprocal Rank Fusion but never assigned a numeric value anywhere in the paper, so hybrid retrieval scores are not reproducible as written.
  • LLM-Judge accuracy threshold = 0.7
    Table 4 defines per-category accuracy using an LLM-Judge threshold ≥ 0.7; the threshold is chosen by the authors and directly affects all reported accuracy numbers.
axioms (3)
  • domain assumption World Bank report publication date can be used to convert relative temporal expressions to absolute fiscal years.
    §3.3 applies 'temporal anchoring' logic that assumes the document's publication metadata matches the fiscal period of the data, but reports often contain data from prior periods.
  • ad hoc to paper An automated LLM-based auditing layer yields ground-truth answers without human verification.
    §3.4 describes only automated verification and filtering; the abstract's claimed human-validation subset is absent from the full text. The correctness of all benchmark labels rests on this assumption.
  • domain assumption The stratified sample of 300 questions is representative of the full 8,937-question benchmark.
    §6 evaluates only on n=300; §7 acknowledges this limitation, yet §10 draws conclusions about the benchmark as a whole.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of AfriEconQA: A Benchmark for Quantitative and Temporal Reasoning over World Bank Economic Reports." pith.science (2026). https://pith.science/paper/KQQU2ICB

@misc{pith2026260115297,
  author       = {Pith},
  title        = {Pith review of: AfriEconQA: A Benchmark for Quantitative and Temporal Reasoning over World Bank Economic Reports},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KQQU2ICB}},
  note         = {Machine review of arXiv:2601.15297}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Reliable question answering over long institutional documents requires more than topical retrieval: a system must localize the exact passage that supports a claim and preserve precise numerical and temporal detail when the same indicator recurs across years, countries, and projection horizons in dense, repetitive prose. Existing question-answering benchmarks rarely test this combination of exact grounding and temporal precision at document scale. We introduce AfriEconQA, a benchmark for document-grounded question answering built from 220 World Bank economic reports on African economies, a corpus whose claims are tied to specific fiscal periods, countries, and projection states. AfriEconQA contains 4,309 evidence-linked QA instances across five reasoning categories: Factoid (1,093), List (879), Multiple Choice (944), Synthesis (710), and Comparison (683), targeting quantitative extraction, set recovery, discrimination, causal integration, and cross-period or cross-country reasoning. Each instance carries sup- porting evidence and source provenance and is constructed through an agentic generation pipeline with evidence-grounding checks and a stratified human-validation subset for gold-label auditing. We evaluate Qwen 3.6 35B, DeepSeek v4-pro, and Gemma 4 12B IT under zero-shot, oracle, and hybrid retrieval-augmented generation (RAG) conditions on the held-out test split (n = 862). Across all three models, retrieval yields substantial gains, yet the best RAG system reaches only 0.545 F1, with residual errors concentrated in list extraction, synthesis, and temporally scoped comparison. AfriEconQA therefore poses a hard open challenge for exact numerical and temporal grounding over long institutional economic reports.

Figures

Figures reproduced from arXiv: 2601.15297 by David Stephen, Edward Ajayi, Mustapha Alaba.

Figure 1
Figure 1. Figure 1: Example of a factoid question-answer pair from the dataset with supporting evidence. In this study, we operationalize AfriEconQA to characterize the current state of domain-specific retrieval and reasoning. We frame our evaluation around three primary research questions: • RQ1: To what extent do state-of-the-art models rely on para￾metric memory versus external retrieval for niche economic data? • RQ2: How… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 11 linked inside Pith

  1. [1]

    2025.BIOMED QA: KNOWLEDGE GRAPH REASONING WITH PRE-AND POST-RETRIEV AL GRAPH RAG FOR BIOMEDICAL QUESTION ANSWER- ING SYSTEM

    Husan Almusawi. 2025.BIOMED QA: KNOWLEDGE GRAPH REASONING WITH PRE-AND POST-RETRIEV AL GRAPH RAG FOR BIOMEDICAL QUESTION ANSWER- ING SYSTEM. Master’s thesis. Cleveland State University

    2025

  2. [2]

    Samuel Amouyal, Tomer Wolfson, Ohad Rubin, Ori Yoran, Jonathan Herzig, and Jonathan Berant. 2023. QAMPARI: A benchmark for open-domain questions with many answers. InProceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM). 97–110

    2023

  3. [3]

    Haniel G Cavalcante, Jéferson N Soares, and José EB Maia. 2021. Question expansion in a question-answering system in a closed-domain system.Int. J. Comput. Appl183, 23 (2021), 1–5

    2021

  4. [4]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation.arXiv preprint arXiv:2402.03216 (2024)

    Pith/arXiv arXiv 2024

  5. [5]

    Gordon V Cormack, Charles LA Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval. 758–759

    2009

  6. [6]

    Xavier Daull, Patrice Bellot, Emmanuel Bruno, Vincent Martin, and Elisabeth Murisasco. 2025. Complex QA and language models hybrid architectures, Survey. arXiv preprint arXiv:2302.09051(2025)

    arXiv 2025

  7. [7]

    Caner Derici, Yiğit Aydin, Çiğdem Yenialaca, Nihal Yağmur Aydin, Günizi Kartal, Arzucan Özgür, and Tunga Güngör. 2018. A closed-domain question answer- ing framework using reliable resources to assist students.Natural Language Engineering24, 5 (2018), 725–762

    2018

  8. [8]

    Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, and Ge Li

  9. [9]

    Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Yang Wang, Zhiheng Huang, Bing Xiang, and Dan Roth. 2023. RobustQA: Benchmarking the Robustness of Domain Adaptation for Open-Domain Question Answering. In Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics, Toronto, Canada, 4294–4311

    2023

  10. [10]

    Ivan Iaroshev, Ramalingam Pillai, Leandro Vaglietti, and Thomas Hanne. 2024. Evaluating Retrieval-Augmented Generation Models for Financial Report Ques- tion and Answering.Applied Sciences14, 20 (2024)

    2024

  11. [11]

    Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. 2023. Financebench: A new benchmark for financial question answering.arXiv preprint arXiv:2311.11944(2023)

    Pith/arXiv arXiv 2023

  12. [12]

    Seonok Kim. 2025. Medbiolm: Optimizing medical and biological qa with fine- tuned large language models and retrieval-augmented generation.arXiv preprint arXiv:2502.03004(2025)

    Pith/arXiv arXiv 2025

  13. [13]

    Gustavo Kuratomi, Paulo Pirozelli, Fabio G Cozman, and Sarajane M Peres. 2025. A RAG-Based Institutional Assistant.arXiv preprint arXiv:2501.13880(2025)

    Pith/arXiv arXiv 2025

  14. [14]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. InText summarization branches out. 74–81

    2004

  15. [15]

    Jiahua Liu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2019. XQA: A cross- lingual open-domain question answering dataset. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2358–2368

    2019

  16. [16]

    Shayne Longpre, Yi Lu, and Joachim Daiber. 2021. MKQA: A linguistically diverse benchmark for multilingual open domain question answering.Transactions of the Association for Computational Linguistics9 (2021), 1389–1406

    2021

  17. [17]

    Shiwen Ni, Guhong Chen, Shuaimin Li, Xuanang Chen, Siyi Li, Bingli Wang, Qiyao Wang, Xingjian Wang, Yifan Zhang, Liyang Fan, Chengming Li, Ruifeng Xu, Le Sun, and Min Yang. 2025. A Survey on Large Language Model Benchmarks. arXiv preprint arXiv:2508.15361(2025)

    Pith/arXiv arXiv 2025

  18. [18]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics. 311–318

    2002

  19. [19]

    Yinzhu Quan and Zefang Liu. 2024. EconLogicQA: A Question-Answering Bench- mark for Evaluating Large Language Models in Economic Sequential Reasoning. arXiv preprint arXiv:2405.07938(2024)

    Pith/arXiv arXiv 2024

  20. [20]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text.arXiv preprint arXiv:1606.05250(2016)

    Pith/arXiv arXiv 2016

  21. [21]

    Cheol Ryu, Seolhwa Lee, Subeen Pang, Chanyeol Choi, Hojun Choi, Myeonggee Min, and Jy-Yong Sohn. 2023. Retrieval-based evaluation for LLMs: a case study in Korean legal QA. InProceedings of the Natural Legal Language Processing Workshop 2023. 132–137

    2023

  22. [22]

    Matthew Saad and Zakariya Qawaqneh. 2024. Closed Domain Question- Answering Techniques in an Institutional Chatbot. In2024 4th International Conference on Electrical, Computer, Communications and Mechatronics Engineer- ing (ICECCME). 1–8

    2024

  23. [23]

    Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kalu- arachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (RAG) models for open domain question answering.Transactions of the Association for Computational Linguistics 11 (2023), 1–17

    2023

  24. [24]

    Wei Tang, Yixin Cao, Yang Deng, Jiahao Ying, Bo Wang, Yizhe Yang, Yuyue Zhao, Qi Zhang, Xuan-Jing Huang, Yu-Gang Jiang, et al . 2025. Evowiki: Evaluating llms on evolving knowledge. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 948–964

    2025

  25. [25]

    Zhen Wang. 2022. Modern question answering datasets and benchmarks: A survey.arXiv preprint arXiv:2206.15030(2022)

    Pith/arXiv arXiv 2022

  26. [26]

    Jiexin Wang, Adam Jatowt, and Masatoshi Yoshikawa. 2022. Archivalqa: A large- scale benchmark dataset for open-domain question answering over historical news collections. InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3025–3035

    2022

  27. [27]

    John S Whissell and Charles LA Clarke. 2011. Improving document clustering using Okapi BM25 feature weighting.Information retrieval14, 5 (2011), 466–487

    2011

  28. [28]

    2023.Financial & Economic Analysis: Differences and Similarities

    World Bank. 2023.Financial & Economic Analysis: Differences and Similarities. World Bank, EEX Global Knowledge Unit (IEEGK)

    2023

  29. [29]

    World Bank. 2025. Documents & Reports — All Documents | The World Bank. https://documents.worldbank.org/en/publication/documents-reports. Accessed: 2025-12-04

    2025

  30. [30]

    Wenhao Yu, Meng Jiang, Peter Clark, and Ashish Sabharwal. 2023. Ifqa: A dataset for open-domain question answering under counterfactual presuppositions.arXiv preprint arXiv:2305.14010(2023)

    Pith/arXiv arXiv 2023

  31. [31]

    Zixuan Zhang, Revanth Gangi Reddy, Kevin Small, Tong Zhang, and Heng Ji

  32. [32]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems36 (2023), 46595–46623

    2023

  33. [33]

    Towards Better Generalization in Open-Domain Question Answering by Mitigating Context Memorization.arXiv preprint arXiv:2404.01652(2024)

    Pith/arXiv arXiv 2024

  34. [2024]

    Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models.arXiv preprint arXiv:2402.15938(2024)

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.