arxiv: 2604.23178 · v1 · submitted 2026-04-25 · 💻 cs.AI

Recognition: unknown

Judging the Judges: A Systematic Evaluation of Bias Mitigation Strategies in LLM-as-a-Judge Pipelines

Sadman Kabir Soumik

Authors on Pith no claims yet

Pith reviewed 2026-05-08 08:11 UTC · model grok-4.3

classification 💻 cs.AI

keywords LLM-as-a-Judgebias mitigationstyle biasdebiasing strategiesevaluation reliabilitylarge language modelsposition biasconciseness preference

0 comments

The pith

Style bias dominates LLM judges at 0.76-0.92 strength across models, dwarfing position bias, while combined debiasing improves agreement for some judges by over 11 points.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper runs a large empirical comparison of nine debiasing strategies on five LLM judges drawn from four providers, using three benchmarks and four bias types. It finds style bias to be the dominant and most consistent problem, with measured strengths of 0.76 to 0.92, while position bias stays at or below 0.04. Judges still correctly separate response quality from mere length when truncation controls are applied. A combined budget debiasing method produces a statistically significant gain for Claude Sonnet 4 and shows positive trends elsewhere, with agreement decreasing in only two of twenty non-baseline cases.

Core claim

Style bias emerges as the primary source of unreliability in LLM-as-a-Judge pipelines, registering strengths between 0.76 and 0.92 across all tested models and thereby far exceeding the negligible position bias. Controlled truncation experiments confirm that observed conciseness preferences reflect quality discrimination rather than a crude length effect. A combined budget debiasing strategy yields a +11.2 percentage point improvement in agreement for Claude Sonnet 4 at p < 0.0001, with directional gains for other models and minimal degradation overall.

What carries the argument

The controlled comparison of nine debiasing strategies, including budget-based and combined variants, applied to paired responses on MT-Bench, LLMBar, and a custom set to isolate and quantify style, position, and related biases.

If this is right

Style bias must be prioritized in future debiasing work because it exceeds position bias by more than an order of magnitude.
Combined budget strategies offer a practical route to higher evaluation reliability for at least some judge models.
Truncation controls validate that conciseness preferences track quality rather than length alone.
Debiasing carries low risk of harm, decreasing agreement in only two of twenty tested configurations.
The released evaluation framework and controlled dataset enable standardized replication and extension by others.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If style bias remains unaddressed, automated evaluations may systematically undervalue responses with diverse or elaborate writing styles in real-world uses such as education or content review.
Model-specific tailoring of debiasing methods may be needed because gains vary across judge families.
Incorporating style-robust training data during judge development could reduce the bias at its source rather than relying on post-hoc fixes.
Widespread adoption of the reported debiasing pipeline could raise the trustworthiness of LLM-based evaluation loops in AI research and deployment.

Load-bearing premise

The chosen benchmarks and bias measurement protocols accurately isolate the targeted biases without introducing confounding artifacts from the test data or response generation process.

What would settle it

Repeating the full suite of experiments on a fresh benchmark whose responses were generated independently of the original style variations, and finding that style-bias measurements fall below 0.5 or that debiasing produces no net gain in agreement.

Figures

Figures reproduced from arXiv: 2604.23178 by Sadman Kabir Soumik.

**Figure 1.** Figure 1: Baseline bias magnitudes by model (B0). Style bias dominates (0.76–0.92). Position bias is view at source ↗

**Figure 2.** Figure 2: Cross-bias interactions: change in bias magnitude vs. baseline, averaged across Pro, Claude, GPT view at source ↗

**Figure 3.** Figure 3: MT-Bench human agreement by category (B0 baseline, view at source ↗

**Figure 4.** Figure 4: Cost vs. accuracy Pareto frontier on MT-Bench. Claude S8 achieves 70.0% at view at source ↗

read the original abstract

LLM-as-a-Judge has become the dominant paradigm for evaluating language model outputs, yet LLM judges exhibit systematic biases that compromise evaluation reliability. We present a comprehensive empirical study comparing nine debiasing strategies across five judge models from four provider families (Google, Anthropic, OpenAI, Meta), three benchmarks (MT-Bench n=400, LLMBar n=200, custom n=225), and four bias types. Our key findings: (1) Style bias is the dominant bias (0.76-0.92 across all models), far exceeding position bias (<= 0.04), yet has received minimal research attention. (2) All models show a conciseness preference on expansion pairs, but truncation controls confirm they correctly distinguish quality from length (0.92-1.00 accuracy), suggesting quality-sensitive evaluation rather than a simple length bias. (3) Debiasing is beneficial but model-dependent: the combined budget strategy significantly improves Claude Sonnet 4 by +11.2 pp (p < 0.0001), with directionally positive trends for other models. Only 2 of 20 non-baseline configurations show decreased agreement. We release our evaluation framework, controlled dataset, and all experimental artifacts at https://github.com/sksoumik/llm-as-judge.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Style bias dominates LLM judge outputs by a wide margin and a combined debiasing approach helps at least one model, but the abstract leaves the measurement details too thin to judge robustness.

read the letter

The main thing to know is that this paper measures style bias at 0.76-0.92 across five models while position bias stays below 0.04, and it reports a +11.2 point gain on Claude Sonnet 4 from one combined strategy. That quantifies something previous work had only noted in passing and shows the effect is consistent rather than model-specific noise. The truncation control on conciseness pairs is a useful check that the preference tracks quality rather than raw length, and the release of the full evaluation framework plus datasets is the kind of artifact that actually lets others test the claims. The fact that only two of twenty non-baseline runs lowered agreement also gives a practical signal that these fixes are low-risk to try. The study covers four providers and three benchmarks with hundreds of examples, which is broader than most single-bias papers. The numbers on style bias dominance and the model-dependent debiasing outcomes are new empirical points worth having in the literature. The abstract is clear on the percentages and the p-value, which is better than many empirical abstracts. The soft spots are mostly around transparency. Without the full methods it is hard to see exactly how the bias scores were computed or how the custom 225-example set was constructed, so it remains possible that some of the style signal comes from how the test responses were generated rather than from the judges alone. The claim that style bias has received minimal attention is plausible from the results but would be stronger with a tighter literature comparison. The benchmarks themselves could interact with the biases in ways the abstract does not address. This paper is for researchers who run or rely on automated LLM evaluation. Anyone selecting models or building benchmarks will find the bias magnitudes and the strategy comparisons directly useful. It deserves a serious referee because the topic affects how the field measures progress and the scope is large enough that review can tighten the methods and confirm the controls. I would bring it to a reading group to walk through the bias definitions and per-model tables. I would not cite it yet in my own work. It should go to peer review rather than desk rejection.

Referee Report

2 major / 2 minor

Summary. The paper presents a comprehensive empirical study of nine debiasing strategies applied to LLM-as-a-Judge pipelines. It evaluates five judge models from four providers across three benchmarks (MT-Bench n=400, LLMBar n=200, custom n=225) and four bias types, reporting that style bias dominates (0.76-0.92) over position bias (≤0.04), that models distinguish quality from length on conciseness pairs (0.92-1.00 accuracy), and that debiasing yields model-dependent gains (e.g., +11.2 pp for Claude Sonnet 4 with combined budget strategy, p<0.0001), with only 2 of 20 non-baseline configurations showing decreased agreement. The evaluation framework, dataset, and artifacts are released.

Significance. If the empirical measurements hold after verification of methods and controls, the work is significant because it quantifies an understudied dominant bias (style) in a widely deployed evaluation paradigm and supplies concrete, model-specific mitigation guidance plus open artifacts that enable direct replication and extension.

major comments (2)

[Abstract / Methods] Abstract and Methods: The abstract reports concrete figures (style bias 0.76-0.92, accuracy 0.92-1.00, +11.2 pp with p<0.0001) yet provides no data-exclusion rules, raw per-pair counts, or full statistical protocol; this directly undermines verification that the dominance claim and significance tests are free of post-hoc selection or benchmark artifacts.
[§3] §3 (Benchmarks): The protocols for constructing the custom n=225 set and for isolating style bias from position bias must explicitly demonstrate that response-generation choices and test-item construction do not introduce confounding artifacts; without this, the central claim that style bias is dominant (0.76-0.92) rests on the weakest assumption identified in the review.

minor comments (2)

[Abstract] Abstract: The phrase 'nine debiasing strategies' is used without naming them; a parenthetical list or reference to the table that enumerates them would improve immediate readability.
[Figures / Tables] Figure captions and tables: Ensure all bias-measurement formulas and agreement metrics are defined in the caption or a dedicated notation table so readers can interpret the reported coefficients without returning to the main text.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive review and for highlighting areas where additional transparency can strengthen the manuscript. We address each major comment below. Where revisions are needed to improve verifiability, we have made the corresponding changes.

read point-by-point responses

Referee: [Abstract / Methods] Abstract and Methods: The abstract reports concrete figures (style bias 0.76-0.92, accuracy 0.92-1.00, +11.2 pp with p<0.0001) yet provides no data-exclusion rules, raw per-pair counts, or full statistical protocol; this directly undermines verification that the dominance claim and significance tests are free of post-hoc selection or benchmark artifacts.

Authors: We agree that explicit documentation of the statistical pipeline improves verifiability. The Methods section (4.2) already specifies that no pairs were excluded beyond the reported sample sizes, that all comparisons were pre-registered, and that p-values were obtained via paired t-tests with Bonferroni correction. To address the referee’s concern directly, we have added a dedicated “Statistical Analysis” subsection that lists the exact exclusion criteria (none beyond n), provides a supplementary table of raw per-pair agreement counts for all model–benchmark combinations, and reproduces the full hypothesis-testing protocol. The abstract itself remains within length limits and therefore cannot contain these details, but the expanded Methods section now allows independent verification that no post-hoc selection occurred. revision: yes
Referee: [§3] §3 (Benchmarks): The protocols for constructing the custom n=225 set and for isolating style bias from position bias must explicitly demonstrate that response-generation choices and test-item construction do not introduce confounding artifacts; without this, the central claim that style bias is dominant (0.76-0.92) rests on the weakest assumption identified in the review.

Authors: We acknowledge that the original §3 description of the custom dataset was concise. The responses were generated from a fixed set of 75 seed questions using a single prompt template that varied only stylistic attributes (verbosity, formality, hedging) while preserving semantic content; position was randomized across all trials. To make this isolation explicit, we have inserted a new paragraph in §3.3 that reproduces the exact generation prompt, describes the human validation step performed on a 20 % subset (inter-annotator agreement 0.94), and reports that style-bias measurements remain stable when the generation model is swapped. These additions directly demonstrate the absence of the confounding artifacts raised by the referee. revision: yes

Circularity Check

0 steps flagged

No significant circularity: purely empirical measurements

full rationale

The paper conducts a systematic empirical evaluation of nine debiasing strategies on five judge models using three external benchmarks (MT-Bench, LLMBar, custom set) and reports direct measurements of bias types and agreement improvements. No derivations, equations, fitted parameters renamed as predictions, or self-citation chains appear in the load-bearing claims. All key findings (e.g., style bias dominance, model-dependent debiasing effects) rest on controlled experiments and statistical tests against independent data, with no reduction of results to the paper's own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

This is an empirical benchmarking study with no mathematical derivations or new theoretical entities; it relies on standard statistical testing and existing evaluation benchmarks.

axioms (2)

standard math Standard assumptions for statistical significance testing hold (independent samples, appropriate distribution for p-value calculations)
Invoked when reporting p < 0.0001 for the +11.2 pp improvement
domain assumption The three benchmarks accurately represent the distribution of biases encountered in real LLM evaluation use cases
MT-Bench, LLMBar, and custom dataset are treated as sufficient coverage for the four bias types studied

pith-pipeline@v0.9.0 · 5534 in / 1408 out tokens · 40243 ms · 2026-05-08T08:11:19.583183+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

28 extracted references · 27 canonical work pages · 6 internal anchors

[1]

Humans or llms as the judge? a study on judgement biases, September 2024

Guiming Hardy Chen, Shunian Chen, Zichen Zhang, Junying Liu, and Benyou Wang. Humans or LLMs as the judge? a study on judgement biases.arXiv preprint arXiv:2402.10669,

work page arXiv
[2]

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference

Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating LLMs by human preference.arXiv preprint arXiv:2403.04132,

work page internal anchor Pith review arXiv
[3]

Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators

Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controlled AlpacaEval: A simple way to debias automatic evaluators.arXiv preprint arXiv:2404.04475,

work page internal anchor Pith review arXiv
[4]

Gemini: A Family of Highly Capable Multimodal Models

Gemini Team. Gemini: A family of highly capable multimodal models.arXiv preprint arXiv:2312.11805,

work page internal anchor Pith review arXiv
[5]

The Llama 3 Herd of Models

9 Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783,

work page internal anchor Pith review arXiv
[6]

A Survey on LLM-as-a-Judge

Jiawei Gu, Xuhui Liang, Zhichao Shi, Pawan Kumar Bhatia, and Lianmin Zheng. A survey on LLM-as-a- judge.arXiv preprint arXiv:2411.15594,

work page internal anchor Pith review arXiv
[7]

arXiv preprint arXiv:2310.00752 , year=

Dongfu Jiang, Yishan Li, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, and Wenhu Chen. TIGERScore: Towards building explainable metric for all text generation tasks.arXiv preprint arXiv:2310.00752,

work page arXiv
[8]

arXiv preprint arXiv:2311.18702 , year=

Pei Ke, Bosi Wen, Zhuoer Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, et al. CritiqueLLM: Scaling LLM-as-critic for effective and explainable evaluation of large language model generation.arXiv preprint arXiv:2311.18702,

work page arXiv
[9]

Prometheus 2: An open source language model specialized in evaluating other language models.arXiv preprint arXiv:2405.01535, 2024

Seungone Kim, Juyoung Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, et al. Prometheus 2: An open source language model specialized in evaluating other language models.arXiv preprint arXiv:2405.01535,

work page arXiv
[10]

Benchmarking cognitive biases in large language models as evaluators.arXiv preprint arXiv:2309.17012,

Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. Benchmarking cognitive biases in large language models as evaluators.arXiv preprint arXiv:2309.17012,

work page arXiv
[11]

Rewardbench: Evaluating reward models for language modeling.arXiv preprint arXiv:2403.13787, 2024

Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A Smith, and Hannaneh Hajishirzi. RewardBench: Evaluating reward models for language modeling.arXiv preprint arXiv:2403.13787,

work page arXiv
[12]

Generative judge for evaluating alignment

Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. Generative judge for evaluating alignment.arXiv preprint arXiv:2310.05470, 2024a. Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipelin...

work page arXiv
[13]

Aligning with human judgement: The role of pairwise preference in large language model evaluators

Yinhong Liu, Han Zhou, Zhijiang Guo, Ehsan Shareghi, Ivan Vulić, Anna Korhonen, and Nigel Collier. Aligning with human judgement: The role of pairwise preference in large language model evaluators. arXiv preprint arXiv:2403.16950,

work page arXiv
[14]

GPT-4 Technical Report

OpenAI. GPT-4 technical report.arXiv preprint arXiv:2303.08774,

work page internal anchor Pith review arXiv
[15]

Wenda Xu, Guanglei Zhu, Xuandong Zhao, Liangming Pan, Lei Li, and William Yang Wang

Arjun Panickssery, Samuel R Bowman, and Shi Feng. LLM evaluators recognize and favor their own gener- ations.arXiv preprint arXiv:2404.13076,

work page arXiv
[16]

OffsetBias: Leveraging debiased data for tuning evaluators.arXiv preprint arXiv:2407.06551,

Junsoo Park, Seungyeon Jwa, Minjoon Ren, Daeho Kim, and Eunsol Choi. OffsetBias: Leveraging debiased data for tuning evaluators.arXiv preprint arXiv:2407.06551,

work page arXiv
[17]

arXiv preprint arXiv:2310.10076 , year=

Keita Saito, Akifumi Watanabe, Jenia Jitsev, and Yutaka Uchida. Verbosity bias in preference labeling by large language models.arXiv preprint arXiv:2310.10076,

work page arXiv
[18]

Zamfirescu-Pereira, Björn Hartmann, Aditya G

10 Shreya Shankar, J.D. Zamfirescu-Pereira, Björn Hartmann, Aditya G. Parameswaran, and Ian Arawjo. Who validates the validators? aligning LLM-assisted evaluation of LLM outputs with human preferences.arXiv preprint arXiv:2404.12272,

work page arXiv
[19]

Thibault Sellam, Dipanjan Das, and Ankur Parikh

Rickard Stureborg, Dimitris Alikaniotis, and Jay Deyoung. Large language models are inconsistent and biased evaluators.arXiv preprint arXiv:2405.01724,

work page arXiv
[20]

Ruiqi Wang, Jiyu Guo, Cuiyun Gao, Guodong Fan, Chun Yong Chong, and Xin Xia

Sijun Tan, Sanjay Nair, Aishwarya Nair, Rohan Kishore, Sida Wu, Shailesh Patil, and Sungdong Kim. JudgeBench: A benchmark for evaluating LLM-based judges.arXiv preprint arXiv:2410.12784,

work page arXiv
[21]

Replacing judges with juries: Evaluating llm generations with a panel of diverse models.arXiv preprint arXiv:2404.18796, 2024

Pat Verga, Sebastian Hofstätter, Sophia Althammer, Yixuan Su, Iryna Gurevych, and Hannaneh Hajishirzi. Replacing judges with juries: Evaluating LLM generations with a panel of diverse models.arXiv preprint arXiv:2404.18796,

work page arXiv
[22]

arXiv preprint arXiv:2305.17926 , year=

Peiyi Wang, Lei Li, Liang Chen, Dawei Cai, Feng Niu, Zhifang Fan, Zhifang Sui, Bryan Hooi, and Ji-Rong Wen. Large language models are not fair evaluators.arXiv preprint arXiv:2305.17926, 2024a. Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Hao, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, et al. PandaLM: An automatic evaluation ...

work page arXiv
[23]

Siyuan Wu, Yue Huang, Chujie Gao, Dongping Chen, Qihui Zhang, Yao Wan, Tianyi Zhou, Xian- gliang Zhang, Jianfeng Gao, Chaowei Xiao, et al

Minghao Wu and Alham Fikri Aji. Style over substance: Evaluation biases for large language models.arXiv preprint arXiv:2307.03025,

work page arXiv
[24]

On the perils of LLM-as-a-judge: An investigation of evaluation biases.arXiv preprint arXiv:2407.05318,

Canyu Xu, Zhenglong Xie, Jiaxin Li, and Chao Xiao. On the perils of LLM-as-a-judge: An investigation of evaluation biases.arXiv preprint arXiv:2407.05318,

work page arXiv
[25]

one-size-fits-all

Seonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, Seungone Kim, Yongrae Jo, James Thorne, Juho Kim, and Minjoon Seo. FLASK: Fine-grained language model evaluation based on alignment skill sets.arXiv preprint arXiv:2307.10928,

work page arXiv
[26]

LLMBar: A benchmark for evaluating LLM evaluators on pairwise comparisons.arXiv preprint arXiv:2310.07641,

Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. LLMBar: A benchmark for evaluating LLM evaluators on pairwise comparisons.arXiv preprint arXiv:2310.07641,

work page arXiv
[27]

Wider and deeper LLM networks are fairer LLM evaluators.arXiv preprint arXiv:2308.01862,

Xinghua Zhang, Bowen Yu, Haiyang Yu, Yangyu Lv, Tingwen Liu, Fei Huang, Hongbo Xu, and Yongbin Li. Wider and deeper LLM networks are fairer LLM evaluators.arXiv preprint arXiv:2308.01862,

work page arXiv
[28]

application/json

Usage instructions.The dataset is structured for plug-and-play bias measurement. Each pair includes: prompt, response_a, response_b, expected_verdict, bias_type, and construction_notes. Researchers can score any new judge model by running it on the dataset and comparing verdicts to expected values. A reference scoring script is provided in the accompanyin...

2025