REVIEW 4 major objections 4 minor 31 references
Unbiased Evaluation of Large Language Models from a Causal Perspective
T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Standard benchmarks overstate LLM ability because models have memorized a share of test answers; a rule-based intervention protocol called BOAT removes that share and yields lower, contamination-free scores.
desk verdict A useful contamination diagnostic that overreaches; the 'unbiased' label needs support it doesn't have. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is BOAT, a set of deterministic atomic interventions on input variables, framed as causal manipulations in a directed acyclic graph whose nodes are input variables, latent variables, and the correct answer. Each intervention changes one atomic component -- for example, adding a 'no correct answer' option, replacing an option with one from another question, shuffling order, replacing letters with numerals, converting the question into true/false statements, or jittering numbers -- instead of relying on an LLM to generate new questions. The supporting identity is the bias decomposition $E[\epsilon(\hat{\phi}_{D'})^2] = E[\epsilon(\hat{\phi}_D)^2] + 2\mathrm{Cov}(\epsilon(\hat{\phi}_D), \Delta) + 2E[\epsilon(\hat{\phi}_D)]E[\Delta] + E[\Delta^2]$, which separates original benchmark bias from bias introduced by the protocol and dictates that new interventions should be negatively correlated with existing benchmark biases.
What would settle it
Take a model evaluated on a benchmark whose questions were all created after the model's training cutoff, so no contamination is possible, and run BOAT on those same questions. If BOAT accuracy falls well below vanilla accuracy on this uncontaminated set, then the accuracy decline cannot be entirely attributed to removing memorized answers, and the paper's central interpretation in Section 6.2 would be falsified.
Extended reading notes
Core claim
The central discovery claimed is that a substantial portion of LLM correctness on standard multiple-choice and math benchmarks comes from exposure to test questions during training, and that this portion can be isolated by rule-based interventions. The Unbiased Evaluator builds Bags Of Atomic InTerventions (BOAT): the question stem, options, labels, and answer set are systematically altered through distractor hints, distractor questions, answer removal, option shuffling, label replacement, binary transformation, and numeric jittering, while the ground-truth reasoning is preserved. Evaluated this way, GPT-4o falls from 94.5 to 89.4 on ARC-C and from 83.5 to 68.8 on MMLU, Yi1.5-34B-Chat falls from 93.9 to 71.8 and 77.7 to 56.7, and a Llama2-13B model fine-tuned on the original MMLU test set drops from 96.6 to 37.1, which the paper reads as evidence that BOAT strips memorization advantages. The method also reports a perfect Kendall rank correlation with LiveBench scores, supporting the claim that it measures something closer to true ability.
Load-bearing premise
The load-bearing premise is that the atomic interventions in BOAT preserve the reasoning a question tests, so the only systematic difference between original and BOAT accuracy is the removal of memorized or contaminated answers; if the rewrites increase difficulty or change the required skill, the claimed drop in accuracy does not measure contamination.
Editorial extensions
If this is right
- If BOAT scores equal uncontaminated capability, then published leaderboard gaps on ARC-C, MMLU, and GSM8K overstate model ability; using the reported drops, open-source models such as Yi1.5-34B-Chat lose more than 20 points, so ranking by BOAT changes the picture.
- A model fine-tuned on the original test set can score near-perfect on the original benchmark yet fail BOAT, so BOAT can serve as a direct test for whether a reported score is contaminated.
- The pattern that Binary Transformation causes the largest drop indicates that models often answer multiple-choice questions without verifying each option, so intervention-level scores give a mechanism-level interpretation of failures.
- The bias decomposition gives a concrete criterion for future protocol design: measure the covariance between a new protocol's delta bias and the original benchmark's bias, and prefer protocols whose related term is negative.
- Because BOAT reuses existing benchmarks and only changes the input presentation, it offers a low-cost way to keep old benchmarks usable after contamination has occurred.
Reading between the lines
- If the contamination interpretation is correct, BOAT per-question outcomes ('original true, BOAT false') can be reused as a training-free contamination detector: the proportion of such flips per benchmark roughly estimates the memorized fraction, and benchmarks could publish these flip rates.
- The same intervention logic could be applied to open-ended generation by treating instruction variations and distractor context as interventions; the paper leaves this as future work, but the causal framing suggests a general recipe.
- One testable implication beyond the paper: fine-tuning on BOAT-augmented versions of a benchmark should improve BOAT scores more than vanilla scores, because it would train invariance to atomic changes; if it does not, the gap is largely added difficulty rather than contamination.
- The reported perfect Kendall correlation with LiveBench is computed on a small set of models; a stronger test would be to compare BOAT ranks against multiple contamination-free benchmarks across many more models.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a causal framework for LLM evaluation, decomposing evaluation bias into original, related, and independent terms (Proposition 3.1). It empirically probes two biases in Agents-as-an-Evaluator methods (data bias and model bias) using purpose-designed metrics, and then proposes the Unbiased Evaluator, a rule-based protocol called BOAT (Bags Of Atomic InTerventions) that applies atomic interventions to existing benchmarks. Experiments on ARC-C, MMLU, and GSM8K show accuracy drops relative to vanilla evaluation, which the authors attribute to removal of benchmark contamination; additional results include confusion-matrix analyses, intervention ablations, scaling trends, human verification of intervened labels, and correlation with LiveBench.
Significance. If the central claim were established, the Unbiased Evaluator would be a practically valuable tool: it would provide contamination-resistant, interpretable LLM evaluation using existing benchmarks with almost no additional cost. The paper deserves credit for a clear, rule-based intervention framework, a human-verification check on label correctness, and an explicit fine-tuning test for contamination. However, the load-bearing assumption—that the atomic interventions preserve the construct and difficulty of the original tasks—is not validated, and the theoretical result is an algebraic identity that carries no operational force. The fine-tuning experiment itself shows a residual advantage for a contaminated model, which contradicts the claim that BOAT removes contamination. As presented, the evidence does not support the paper's headline claims of unbiasedness.
major comments (4)
- [§4.2, §6.2, Table 2] The interpretation that the accuracy drops in Table 2 reflect 'the decrease of contamination' assumes that each BOAT intervention preserves task difficulty and the construct being measured. This is not established. Interventions such as Binary Transformation (turning a 4-option MCQ into four true/false judgments), Distractor Question (requiring the model to identify which of two questions matches the options), and Answer Removal (replacing the correct answer with 'None of the above') plausibly add reasoning demands and meta-skills unrelated to contamination. The human verification in §6.7 confirms only that the labels of the intervened questions are correct (99.3–99.9%), not that the intervention preserves difficulty or measures the same ability. Without such evidence, the observed drops cannot be uniquely attributed to contamination, and the label 'unbiased' is unsupported.
- [§3.1, Eq. (13) and Appendix D.1] Proposition 3.1 is a tautological expansion of the definition of delta bias. Equation (9) defines Δ = ϵ(φ̂_D′) − ϵ(φ̂_D), and then Eq. (13) is exactly E[(ϵ_D + Δ)²] = E[ϵ_D²] + 2Cov(ϵ_D,Δ) + 2E[ϵ_D]E[Δ] + E[Δ²], which follows from E[Δϵ_D] = Cov(ϵ_D,Δ) + E[ϵ_D]E[Δ]. This is an algebraic identity, not a substantive theorem about evaluation bias. It provides no operational criterion for constructing an unbiased protocol, nor does it establish that any particular protocol (including BOAT) achieves unbiasedness. The claim in the Discussion of §4.2 that the design of BOAT is 'guided by the findings from Proposition 3.1' is therefore not supported by the proposition itself.
- [§6.2, Table 3] The fine-tuning experiment is the paper's direct evidence that BOAT removes contamination, but the results show the opposite of a complete removal. Llama2-13B fine-tuned on the original MMLU test set achieves 96.6 on the vanilla test set but 37.1 under BOAT, versus 33.7 for the unfine-tuned model under BOAT. The residual 3.4-point advantage indicates that the contaminated model still benefits from its exposure to the original test set even under BOAT. The paper's interpretation that the model 'struggles to perform well' understates this gap; a truly contamination-free evaluation should show no advantage for the fine-tuned model, or at least the paper should explain why a 3.4-point residual is negligible. Without such an explanation, the claim that BOAT eliminates performance inflation caused by data contamination is not supported.
- [§6.3, Table 4] The correlation with LiveBench is computed over only five models after excluding GPT-4-Turbo and Yi1.5-34B-Chat. With n=5, a Kendall correlation of 1.000 is not a meaningful demonstration of reliability; the paper reports no confidence interval, permutation test, or p-value, and with five points a perfect rank correlation can arise from a single pairwise ordering. The claim that the Unbiased Evaluator 'aligns more closely with LiveBench' is therefore based on very weak evidence. Reporting the exact models and a significance test would be needed to support this conclusion.
minor comments (4)
- [Figure 6 caption] The caption lists 'DQ, DH, AE, OS, LR and BT' and then glosses them as 'Distractor Hint, Distractor Question, Answer Removal, Option Shuffling, Label Replacement and Binary Transformation', but the abbreviations used elsewhere (e.g., Table 7) map DQ to Distractor Question and DH to Distractor Hint. This inconsistency should be fixed.
- [§3.3, Eqs. (5)–(7)] The thresholds tu=5 and tl=6 are introduced without justification or sensitivity analysis. Since RCE, ROC, and RUC are computed with hard sign functions based on these thresholds, it would strengthen the bias analysis to show that the conclusions are stable to reasonable variations in tu and tl.
- [Related Works and References] The citation '(Chen et al.; Ye et al., 2024)' is incomplete because 'Chen et al.' has no year or full reference; also, in the Conclusions the phrase 'an new evaluation protocol' should be 'a new evaluation protocol', and in Appendix D.1 'rephased' should be 'rephrased'.
- [§5.1, Implementation Details] The description 'the answers extracted from different questions were not identical' would be clearer as 'the answers extracted from different questions were ensured to be non-identical' or, better, specify the exact deduplication criterion used during Answer Removal.
Circularity Check
The theoretical bias decomposition (Prop. 3.1) is an algebraic identity built into the definition of Δ; the core BOAT and contamination results are independent empirical claims, though the contamination attribution depends on an unproven difficulty-preservation assumption.
-
self definitional
[Section 3.1 (Proposition 3.1, Eq. 2) and Appendix D.1 (Eqs. 8-13)]
"ϵ( ˆϕD′) = ϵ( ˆϕD) + ∆ (9) ... E[ϵ( ˆϕD′)2] = E[ϵ( ˆϕD)2] + 2Cov(ϵ( ˆϕD), ∆) + 2E[ϵ( ˆϕD)]E[∆] + E[∆2] (13). Proposition 3.1 offers a pivotal perspective ... that any newly introduced biases should ideally mitigate and, more importantly, counteract existing biases in the original benchmark."
Eq. (9) defines Δ as ϵ(ˆϕ_D')−ϵ(ˆϕ_D), so Eq. (13) is only the algebraic expansion of E[(ϵ_D+Δ)^2]. The 'original/related/independent' decomposition is an identity, not a derived property of evaluation protocols; the 'pivotal perspective' that new biases should counteract old ones is a restatement of the sign of the covariance term. The theoretical formulation is thus equivalent to its own definitions by construction. The later BOAT evaluation and contamination experiments are not inferred from this identity, but the paper presents Prop. 3.1 as the theoretical foundation for the protocol, so the claimed theoretical insight reduces to its input.
full rationale
The paper's central empirical work—rule-based BOAT interventions, the accuracy drops in Table 2, the fine-tuning contamination probe in Table 3, and the LiveBench agreement in Table 4—is self-contained and externally benchmarked; no fitted parameter is renamed as a prediction, and no load-bearing uniqueness or ansatz is imported from the authors' prior work. The one genuine circularity is Proposition 3.1: after defining Δ as the difference in estimator bias, the claimed decomposition into 'original', 'related', and 'independent' terms is the standard expansion of a squared sum and carries no additional empirical content. This undermines the rhetorical claim that the protocol design is 'guided by' a substantive theoretical finding, but it does not by itself invalidate the empirical comparisons. The Section 6.2 statement that 'the decline of accuracy in Table 2 actually reflects the decrease of contamination' is flagged as under-supported—no evidence establishes that the atomic interventions preserve construct and difficulty—but this is a validity/causal-attribution gap rather than a circular reduction, because the paper never defines contamination as the BOAT accuracy drop. For these reasons the circularity score is moderate: the theoretical framing is definitionally circular, while the central evaluation results retain independent content.
Assumptions & free parameters
free parameters (4)
- tu (threshold for 'NO' in probing metrics) =
5
- tl (threshold for 'Yes') =
6
- Intervention probabilities in BOAT =
0.5 for DH, DQ, AR, OS, LR; 0.1 for BT
- Strength parameter p in probing task =
varied from 0 to 1
assumptions (5)
- domain assumption The original benchmark and the transformed benchmark estimate the same fixed true capability phi (Eqs. 8-9).
- domain assumption Collective opinion (consensus among other LLMs) is a reliable reference for correctness, as assumed in RCE/ROC/RUC (Section 3.3).
- domain assumption LiveBench provides a valid external ordering of model capability for validating evaluation reliability (Section 6.3).
- ad hoc to paper The causal DAG representation of evaluation (V_input, V_latent, R) and the interventions preserve the causal effect of interest (Section 4.1).
- ad hoc to paper Atomic interventions in BOAT preserve the construct being measured while removing memorization shortcuts.
Cite this review
Pith. "Pith review of Unbiased Evaluation of Large Language Models from a Causal Perspective." pith.science (2026). https://pith.science/paper/OZN6LXXS
@misc{pith2026250206655,
author = {Pith},
title = {Pith review of: Unbiased Evaluation of Large Language Models from a Causal Perspective},
year = {2026},
howpublished = {\url{https://pith.science/paper/OZN6LXXS}},
note = {Machine review of arXiv:2502.06655}
}
read the original abstract
Benchmark contamination has become a significant concern in the LLM evaluation community. Previous Agents-as-an-Evaluator address this issue by involving agents in the generation of questions. Despite their success, the biases in Agents-as-an-Evaluator methods remain largely unexplored. In this paper, we present a theoretical formulation of evaluation bias, providing valuable insights into designing unbiased evaluation protocols. Furthermore, we identify two type of bias in Agents-as-an-Evaluator through carefully designed probing tasks on a minimal Agents-as-an-Evaluator setup. To address these issues, we propose the Unbiased Evaluator, an evaluation protocol that delivers a more comprehensive, unbiased, and interpretable assessment of LLMs.Extensive experiments reveal significant room for improvement in current LLMs. Additionally, we demonstrate that the Unbiased Evaluator not only offers strong evidence of benchmark contamination but also provides interpretable evaluation results.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,
-
[5]
URL https://arxiv. org/abs/2402, 10669. Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457,
-
[7]
Dai, S., Xu, C., Xu, S., Pang, L., Dong, Z., and Xu, J. Unify- ing bias and unfairness in information retrieval: A survey of challenges and opportunities with large language mod- els. arXiv preprint arXiv:2404.11457,
-
[8]
Nphard- eval: Dynamic benchmark on reasoning ability of large language models via complexity classes
Fan, L., Hua, W., Li, L., Ling, H., and Zhang, Y . Nphard- eval: Dynamic benchmark on reasoning ability of large language models via complexity classes. arXiv preprint arXiv:2312.14890,
-
[9]
Should chatgpt be biased? challenges and risks of bias in large language models
Ferrara, E. Should chatgpt be biased? challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03738,
-
[10]
L., Shen, J., Hu, J., Han, X., Huang, Y ., Zhang, Y ., et al
He, C., Luo, R., Bai, Y ., Hu, S., Thai, Z. L., Shen, J., Hu, J., Han, X., Huang, Y ., Zhang, Y ., et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad- level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008, 2024a. He, Y ., Li, S., Liu, J., Tan, Y ., Wang, W., Huang, H., Bu, X., Guo, H., Hu, C., Zheng, B....
-
[11]
Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825,
-
[12]
10 Unbiased Evaluation of Large Language Models from a Causal Perspective Jiang, M., Liu, K., Zhong, M., Schaeffer, R., Ouyang, S., Han, J., and Koyejo, S. Does data contamination make a difference? insights from intentionally contaminating pre-training data for language models. In ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundati...
work page 2024
Show all 31 references
-
[13]
Deduplicating train- ing data makes language models better
Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., and Carlini, N. Deduplicating train- ing data makes language models better. arXiv preprint arXiv:2107.06499,
-
[14]
S3eval: A synthetic, scalable, systematic evalua- tion suite for large language models
Lei, F., Liu, Q., Huang, Y ., He, S., Zhao, J., and Liu, K. S3eval: A synthetic, scalable, systematic evalua- tion suite for large language models. arXiv preprint arXiv:2310.15147,
-
[15]
An open source data contamination report for llama series models
Li, Y . An open source data contamination report for llama series models. arXiv preprint arXiv:2310.17589,
-
[16]
Holistic evaluation of language models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y ., Narayanan, D., Wu, Y ., Kumar, A., et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110,
-
[18]
R., Susnjak, T., Arachchilage, N., Liu, T., Wat- ters, P., and Halgamuge, M
McIntosh, T. R., Susnjak, T., Arachchilage, N., Liu, T., Wat- ters, P., and Halgamuge, M. N. Inadequacies of large language model benchmarks in the era of generative ar- tificial intelligence. arXiv preprint arXiv:2402.09880 ,
-
[19]
Training on the benchmark is not all you need
Ni, S., Kong, X., Li, C., Hu, X., Xu, R., Zhu, J., and Yang, M. Training on the benchmark is not all you need. arXiv preprint arXiv:2409.01790,
-
[20]
Riddell, M., Ni, A., and Cohan, A
URL https://openai.com/ index/hello-gpt-4o/. Riddell, M., Ni, A., and Cohan, A. Quantifying contamina- tion in evaluating code generation capabilities of language models. arXiv preprint arXiv:2403.04811,
-
[21]
A., Garc ´ıa-Ferrero, I., Etxaniz, J., de Lacalle, O
Sainz, O., Campos, J. A., Garc ´ıa-Ferrero, I., Etxaniz, J., de Lacalle, O. L., and Agirre, E. Nlp evaluation in trouble: On the need to measure llm data contamination for each benchmark. arXiv preprint arXiv:2310.18018,
-
[22]
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615,
-
[23]
M., Hauth, A., Millican, K., et al
Team, G., Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Sori- cut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805,
-
[24]
Llama 2: Open foundation and fine- tuned chat models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y ., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09288 ,
-
[25]
Glue: A multi-task benchmark and analysis plat- form for natural language understanding
Wang, A. Glue: A multi-task benchmark and analysis plat- form for natural language understanding. arXiv preprint arXiv:1804.07461,
-
[27]
Livebench: A challenging, contamination-free llm benchmark
White, C., Dooley, S., Roberts, M., Pal, A., Feuer, B., Jain, S., Shwartz-Ziv, R., Jain, N., Saifullah, K., Naidu, S., et al. Livebench: A challenging, contamination-free llm benchmark. arXiv preprint arXiv:2406.19314,
-
[29]
Justice or prejudice? quantifying biases in llm-as-a-judge
Ye, J., Wang, Y ., Huang, Y ., Chen, D., Zhang, Q., Moniz, N., Gao, T., Geyer, W., Huang, C., Chen, P.-Y ., et al. Justice or prejudice? quantifying biases in llm-as-a-judge. arXiv preprint arXiv:2410.02736,
-
[30]
X., Chen, X., Lin, Y ., Wen, J.-R., and Han, J
Zhou, K., Zhu, Y ., Chen, Z., Chen, W., Zhao, W. X., Chen, X., Lin, Y ., Wen, J.-R., and Han, J. Don’t make your llm an evaluation benchmark cheater. arXiv preprint arXiv:2311.01964,
-
[31]
Z., Yang, D., and Xie, X
Zhu, K., Chen, J., Wang, J., Gong, N. Z., Yang, D., and Xie, X. Dyval: Graph-informed dynamic evaluation of large language models. arXiv preprint arXiv:2309.17167,
-
[2018]
Training verifiers to solve math word problems
Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168,
-
[2019]
W., Jiao, Y
Wei, J., Karina, N., Chung, H. W., Jiao, Y . J., Papay, S., Glaese, A., Schulman, J., and Fedus, W. Measuring short- form factuality in large language models. arXiv preprint arXiv:2411.04368,
-
[2021]
L., Barocas, S., Daum ´e III, H., and Wallach, H
Blodgett, S. L., Barocas, S., Daum ´e III, H., and Wallach, H. Language (technology) is power: A critical survey of” bias” in nlp. arXiv preprint arXiv:2005.14050,
2005 arXiv
-
[2022]
Let’s verify step by step
Lightman, H., Kosaraju, V ., Burda, Y ., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. arXiv preprint arXiv:2305.20050,
-
[2023]
M., Gebru, T., McMillan-Major, A., and Shmitchell, S
Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and trans- parency, pp. 610–623,
2021
-
[2024]
org/CorpusID:268232499
URL https://api.semanticscholar. org/CorpusID:268232499. Bai, J., Bai, S., Chu, Y ., Cui, Z., Dang, K., Deng, X., Fan, Y ., Ge, W., Han, Y ., Huang, F., et al. Qwen technical report. arXiv preprint arXiv:2309.16609,
-
[2025]
E., and Stoica, I
Yang, S., Chiang, W.-L., Zheng, L., Gonzalez, J. E., and Stoica, I. Rethinking benchmark and contamination for language models with rephrased samples. arXiv preprint arXiv:2311.04850,
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.