Pith. sign in

REVIEW 3 major objections 6 minor 38 references

Who Endorsed It? Measuring Authority Bias Across Expertise Levels in Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Language models defer more strongly to higher-status endorsers, even when the endorsement is wrong.

desk verdict A well-motivated graded authority-bias study whose main table reports impossible accuracy gains under the stated paired design; the steering half is promising but under-reported. read the letter →

arxiv 2601.13433 v4 pith:YFUEKYHQ submitted 2026-01-19 cs.CL cs.LG

Priyanka Mary Mammen , Emil Joswin , Shankar Venkitachalam This is my paper
classification cs.CLcs.LG
keywords authoritybiasmisleadingendorsementspersona-basedpromptingsteeringvectorsactivationengineeringLLMreasoningsycophancymultiple-choiceQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that language models do not just follow whoever endorses an answer, but follow endorsers in a strict order of perceived expertise. By keeping the endorsed content identical and varying only the persona—high-schooler to professor, medical student to board-certified physician, law student to senior counsel—the authors show across 4 datasets and 11 models that correct endorsements help more, and misleading endorsements hurt more, as source expertise rises. The misleading-expert effect is especially worrying: models become more confident in the wrong answer, and reasoning models with chain-of-thought remain susceptible. The paper further claims the bias is encoded as a direction in the model's internal activations, and that subtracting a 'high-expertise' steering vector restores accuracy under misleading endorsements.

What carries the argument

The paper's load-bearing object is the 'high-expertise steering vector': the mean difference in residual-stream activations between the highest- and lowest-expertise persona responses on a 100-question set controlled for ground truth, with the four responses per question differing only in expertise-typical wording. Subtracting this vector from the residual stream during inference is the intervention claimed to neutralize authority bias; adding it is claimed to amplify trust in low-credibility sources. The behavioral half of the study is carried by three metrics—delta accuracy, delta entropy, and robustness rate—computed against a no-endorsement baseline.

What would settle it

Take the 100-question steering-vector set (Appendix A.1) and compare it with the MCQ evaluation items on AQuA-RAT, LEXam, MedMCQA, and MedQA; if any question appears in both, recompute the post-steering accuracy on a guaranteed disjoint split—the improvement must survive. Separately, present the same endorsement sentence attributed to low and high expertise labels; if the accuracy ordering disappears, the hierarchy is confounded with phrasing differences.

Watch

Extended reading notes

Core claim

The central claim is a monotonic authority gradient: delta accuracy grows with the endorser's expertise for correct endorsements, and falls with expertise for incorrect endorsements, while delta entropy shows increased confidence in the erroneous answers. This holds across algebraic, legal, and medical multiple-choice benchmarks and across both reasoning and non-reasoning models. The authors also claim the bias is mechanistically real: a steering vector computed as the mean residual-stream difference between the highest and lowest expertise personas, when subtracted during inference, reduces the influence of expert endorsements and improves accuracy, while adding it amplifies the persuasiven

Load-bearing premise

The steering half of the paper assumes the 100-question set used to compute the 'high expertise' vector is disjoint from the items used to measure post-steering accuracy; the appendix does not state this, so if the sets overlap the reported improvement is an in-sample fit rather than a generalizable intervention.

Editorial extensions

If this is right

  • Model evaluation should treat endorsement source as a first-class variable: a model can appear more accurate when an expert agrees and dangerously wrong when an expert is misinformed.
  • Deploying LLMs in medical or legal decision support without addressing authority bias means an authoritative but incorrect input is likely to be adopted with high confidence.
  • Reasoning-augmented models are not a safeguard: chain-of-thought does not eliminate deference to status cues.
  • Activation steering offers a concrete inference-time mitigation direction: subtract the high-expertise direction to restore reliance on the model's own knowledge.
  • The gradient's consistency across domains and model families suggests a general property of current LLM training, not a quirk of one benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the steering vector was built from 100 questions in the same three domains used for evaluation, and the appendix does not state that the vector-building and evaluation sets are disjoint; if they overlap, the reported post-steering gains are partly in-sample and need confirmation on held-out items.
  • Editorial inference: the 'expertise' vector may be capturing linguistic style markers (formality, hedging, jargon) rather than social authority per se; a test that swaps only the status label while holding wording identical would separate the two.
  • Editorial inference: a natural extension is to test conflicting authorities or mixed-expertise panels, since real-world endorsements rarely come from a single source; the paper's single-endorser design cannot predict which cue wins.
  • Editorial inference: the monotonicity of the gradient is itself testable at finer granularity—e.g., whether a Nobel laureate or celebrity doctor produces more deference than a board-certified physician—which would show whether the effect saturates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies whether LLMs exhibit a systematic 'authority bias' when an endorsement is attributed to sources of varying expertise. Using four multiple-choice reasoning datasets (AQuA-RAT, LEXam, MedMCQA, MedQA), 11 open-weight models, and four domain-specific personas per dataset, the authors report changes in accuracy, output entropy, and answer robustness under correct and incorrect endorsements. They claim that models are increasingly susceptible to misleading endorsements as source expertise increases, and that this bias is mechanistically encoded in the residual stream, such that subtracting a 'high-expertise' steering vector restores accuracy under misleading endorsements. The paper reports delta accuracy, delta entropy, and robustness rates in four large tables, and describes steering-vector experiments in an appendix.

Significance. If the behavioral claim were established, the paper would make a useful contribution to the growing literature on sycophancy, authority bias, and social desirability effects in LLMs. The scope is broad: 11 models across 4 datasets and 4 expertise tiers, including both reasoning and non-reasoning models. The additional attempt to localize the bias mechanistically via residual-stream steering is also potentially valuable. However, the paper's primary evidence is logically inconsistent with its own definitions: under a paired evaluation, an incorrect endorsement can only leave accuracy unchanged or decrease it, yet Table 1 contains numerous positive delta-accuracy values for incorrect endorsements. The absence of any uncertainty quantification further weakens the empirical claims, and the steering-vector analysis lacks the details needed to rule out in-sample fitting. As written, the central claims are not adequately supported.

major comments (3)
  1. [§3.3, Eq. (1); Table 1 (a)–(d)] The main behavioral evidence is internally inconsistent. With a paired protocol (same question set, only the prompt changes), an incorrect endorsement can only cause the model to stay wrong or become wrong; accuracy cannot increase. Yet Table 1 reports many positive ΔAcc values under 'Incorrect/Misleading Endorsement,' e.g., Phi-4 on AQuA-RAT (+0.142, +0.102, +0.063, +0.035 across expertise levels), Qwen-2.5-14B on MedMCQA (+0.209, +0.185, +0.119), and LLaMA-3.1-8B on LEXam (+0.197). These values are only possible if the baseline and endorsed accuracies are computed on different question subsets, or if the 'incorrect' endorsement sometimes points to the correct answer. The paper never states that the same items are used across conditions, nor does it provide item-level counts. Without a paired evaluation, the reported gradients could reflect question difficulty differences rather than au
  2. [§3.3, §4.1; Table 1] No uncertainty quantification is reported anywhere. The paper gives no sample sizes, standard errors, confidence intervals, or significance tests for any ΔAcc, ΔEnt, or robustness value. With 11 models and 4 datasets, many of the reported gradients across expertise levels could be within random variation. For example, the difference between +0.063 and +0.035 for Phi-4 on AQuA-RAT is not interpretable without knowing the number of items and the variance. The paper should report item counts, paired bootstrap or confidence intervals, and appropriate multiple-comparison considerations before claiming a 'clear hierarchical pattern.'
  3. [Appendix A.1, §4.2; Figures 2–3] The steering-vector claim is not adequately supported. The 100-question contrast set used to compute the 'high-expertise' vector is never stated to be disjoint from the evaluation datasets (AQuA-RAT, LEXam, MedMCQA, MedQA). If the steering vector is derived from the same or overlapping items, the reported post-steering improvement is an in-sample fit. The appendix also omits key implementation details: which layers are steered, what coefficient is used, how the vector is normalized, and whether the evaluation prompts are identical to the contrast prompts. Figures 2 and 3 are referenced but contain no numeric results or axis/caption descriptions. Without these details and a clear disjointness statement, the mechanistic conclusion that subtracting the vector 'neutralizes the bias' is not established.
minor comments (6)
  1. [§2, §3.3] The term 'incorrect suggestion' is used interchangeably with 'incorrect/misleading endorsement' in the tables. Since the accuracy argument depends on what the endorsement points to, the paper should define precisely whether the persona always endorses a specific wrong option and whether the option is fixed across all conditions.
  2. [Table 1] The tables are extremely dense and difficult to read. The column headers wrap in a way that obscures which numbers correspond to ΔAcc, Rob, and ΔEnt. A long-form table or a figure with error bars would be much clearer.
  3. [Appendix A.1] The appendix says 'see Appendix for examples' of the four response variations, but no examples are actually provided. Without concrete prompt templates, the study is difficult to reproduce.
  4. [§5, References] Some references are incomplete or formatted inconsistently, e.g., 'Saito et al.' has no year, and several entries use 'arXiv preprint' without venue information. The reference for Mistral-7B is incorrectly given as a CLIP-to-DINO paper.
  5. [§8] The ethical considerations mention personas such as 'Chief Medical Officer' and 'senior judge,' but the experiments use 'Board-Certified Physician' and 'Senior Legal Counsel.' Align the language.
  6. [§4.2] The sentence 'all the models that we considered improve its performance' is vague: it should specify the magnitude of improvement, the number of models, and the exact evaluation protocol for the post-steering accuracy.

Circularity Check

0 steps flagged · score 0.0 of 10

No exhibited circularity; the central behavioral contrast is an independent controlled comparison, while the steering-vector and ΔAcc issues are unresolved validity risks rather than reductions-by-construction.

full rationale

The main behavioral claim is obtained from a direct controlled comparison: §2 says 'For each MCQ, we design three kinds of prompts: i) baseline ... ii) correct endorsement ... iii) incorrect suggestion,' and Eq. 1 defines ΔAcc = Acc_endorse − Acc_base. This is an independent, parameter-free contrast on standard benchmarks (AQuA-RAT, LEXam, MedMCQA, MedQA); no parameter is fitted to the data and then renamed as a prediction. The representation-engineering method is cited to external prior work (Turner et al., 2023; Zou et al., 2023), not to the authors' own prior results, so there is no load-bearing self-citation chain. Two concerns are flagged but do not meet the bar for an exhibited circular step. First, Table 1 reports positive ΔAcc values under 'Incorrect/Misleading Endorsement' (e.g., Qwen-2.5-14B on AQuA-RAT: +0.185, +0.157, +0.122, +0.114). Under the paper's paired design and its definition of an incorrect suggestion as a persona endorsing the incorrect answer, a per-item accuracy delta for such endorsements cannot be positive when there is exactly one correct option; the positive values therefore imply unpaired question subsets or a violated definition. This is an internal-validity problem, not a reduction of the conclusion to its inputs. Second, Appendix A.1 says 'We computed the steering vector as the mean difference in residual stream activations between the highest and lowest expertise personas' on a 100-question set, and §4.2 reports that 'subtracting the vector from the model’s residual stream ... all the models that we consider improve its performance to varying degrees while answering MCQs with misleading suggestions.' The appendix never states that the evaluation MCQs are disjoint from those 100 questions; if they overlap, the improvement would be an in-sample fit. Because the paper does not state the split, this is omitted support rather than a quoted circular reduction. Honest non-finding: no circularity is demonstrated, so the score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central behavioral claim rests on unvalidated persona labels and an assumed monotonic expertise ordering. The mechanistic claim rests on unreported steering parameters and on a possible overlap between the steering contrast set and the evaluation set. No new theoretical entities are introduced.

free parameters (3)
  • Steering vector subtraction strength/coefficient = not reported
    The intervention requires choosing how much of the 'high expertise' vector to subtract from the residual stream; no value or sweep is reported in Appendix A.1.
  • Steering layers = not reported ('middle layers')
    The paper states only that the biggest effect happens in middle layers, without specifying which layers were modified. Layer choice is a performance-affecting parameter.
  • Steering contrast set size = 100 questions
    Appendix A.1 selects 100 questions across three fields; no justification or coverage analysis is given.
assumptions (4)
  • domain assumption Persona labels such as 'Board-Certified Physician' and 'Senior Legal Counsel' induce a monotonic perceived-expertise gradient in the model, and the labels are the only systematic difference between prompts.
    Section 3.1 defines the four-tier hierarchy; no control prompt with an unnamed or contradictory-status source is included, so prompt wording differences are confounded with expertise.
  • ad hoc to paper The 100-question steering set from Appendix A.1 is representative of the evaluation data and disjoint from it; the paper does not state this.
    Appendix A.1 describes constructing the vector but never states an overlap check; if overlapping, the steering results are in-sample.
  • domain assumption Output-token entropy is a valid proxy for model confidence and is comparable across prompts and models.
    Equation (2) defines delta entropy; no calibration or validation of entropy as confidence is provided.
  • domain assumption Simple MCQ endorsements of the form 'persona endorses option X' represent real-world misleading expert advice.
    The authors themselves acknowledge in the Limitations that real-world misinformation is more sophisticated; the simplified prompt format may not generalize.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Who Endorsed It? Measuring Authority Bias Across Expertise Levels in Language Models." pith.science (2026). https://pith.science/paper/YFUEKYHQ

@misc{pith2026260113433,
  author       = {Pith},
  title        = {Pith review of: Who Endorsed It? Measuring Authority Bias Across Expertise Levels in Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YFUEKYHQ}},
  note         = {Machine review of arXiv:2601.13433}
}
read the original abstract

Prior research demonstrates that performance of language models on reasoning tasks can be influenced by suggestions, hints and endorsements. However, the influence of endorsement source credibility remains underexplored. We investigate whether language models exhibit systematic bias based on the perceived expertise of the provider of the endorsement. Across 4 datasets spanning mathematical, legal, and medical reasoning, we evaluate 11 models using personas representing four expertise levels per domain. Our results reveal that models are increasingly susceptible to incorrect/misleading endorsements as source expertise increases, with higher-authority sources inducing not only accuracy degradation but also increased confidence in wrong answers. We also show that this authority bias is mechanistically encoded within the model and a model can be steered away from the bias, thereby improving its performance even when an expert gives a misleading endorsement.

Figures

Figures reproduced from arXiv: 2601.13433 by the authors.

Figure 1
Figure 1. We design our experiment across various domains (math reasoning, medical, and legal MCQs) where [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Model accuracy for incorrect endorsement [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 2
Figure 2. Model accuracy for incorrect endorsement [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 22 linked inside Pith

  1. [1]

    Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, and 1 others. 2025. Phi-4-reasoning technical report. arXiv preprint arXiv:2504.21318

    arXiv 2025

  2. [2]

    Marah Abdin, Jyoti Aneja, Harkirat Behl, S \'e bastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, and 1 others. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905

    arXiv 2024

  3. [3]

    Haozhe An, Christabel Acquaye, Colin Wang, Zongxia Li, and Rachel Rudinger. 2024. Do large language models discriminate in hiring decisions on the basis of race, ethnicity, and gender? arXiv preprint arXiv:2406.10486

    arXiv 2024

  4. [4]

    Noel F Ayoub, Karthik Balakrishnan, Marc S Ayoub, Thomas F Barrett, Abel P David, and Stacey T Gray. 2024. Inherent bias in large language models: a random sampling analysis. Mayo Clinic Proceedings: Digital Health, 2(2):186--191

    2024

  5. [5]

    Guiming Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024. Humans or llms as the judge? a study on judgement bias. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8301--8327

    2024

  6. [6]

    Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, and 1 others. 2025. Reasoning models don't always say what they think. arXiv preprint arXiv:2505.05410

    arXiv 2025

  7. [7]

    Yann Dubois, Bal \'a zs Galambosi, Percy Liang, and Tatsunori B Hashimoto. 2024. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475

    arXiv 2024

  8. [8]

    Yu Fan, Jingwei Ni, Jakob Merane, Yang Tian, Yoan Hermstr \"u wer, Yinya Huang, Mubashara Akhtar, Etienne Salimbeni, Florian Geering, Oliver Dreyer, and 1 others. 2025. Lexam: Benchmarking legal reasoning on 340 law exams. arXiv preprint arXiv:2505.12864

    arXiv 2025

Show all 38 references
  1. [9]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  2. [10]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  3. [11]

    Dongsheng Jiang, Yuchen Liu, Songlin Liu, Jin'e Zhao, Hao Zhang, Zhen Gao, Xiaopeng Zhang, Jin Li, and Hongkai Xiong. 2023. From clip to dino: Visual encoders shout in multi-modal large language models. arXiv preprint arXiv:2310.08825

    2023 arXiv

  4. [12]

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421

    2021

  5. [13]

    Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. 2024. Benchmarking cognitive biases in large language models as evaluators. In Findings of the Association for Computational Linguistics: ACL 2024, pages 517--545

    2024

  6. [14]

    Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. arXiv preprint arXiv:1705.04146

    2017 arXiv

  7. [15]

    Zehao Liu and Xi Lin. 2025. Breaking minds, breaking systems: Jailbreaking large language models via human-like psychological manipulation. arXiv preprint arXiv:2512.18244

    2025

  8. [16]

    Team Olmo, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, and 1 others. 2025. Olmo 3. arXiv preprint arXiv:2512.13961

    2025 arXiv

  9. [17]

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pages 248--260. PMLR

    2022

  10. [18]

    Junsoo Park, Seungyeon Jwa, Ren Meiying, Daeyoung Kim, and Sanghyuk Choi. 2024. Offsetbias: Leveraging debiased data for tuning evaluators. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1043--1067

    2024

  11. [19]

    Pouya Pezeshkpour and Estevam Hruschka. 2023. Large language models sensitivity to the order of options in multiple-choice questions, 2023. URL https://arxiv. org/abs/2308.11483

    2023 arXiv

  12. [20]

    Verbosity bias in preference labeling by large language models

    Keita Saito, Akifumi Wachi, Koki Wataoka, and Youhei Akimoto. Verbosity bias in preference labeling by large language models. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following

    2023

  13. [21]

    Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, and 1 others. 2023. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548

    2023 arXiv

  14. [22]

    Lin Shi, Chiyu Ma, Wenhua Liang, Xingjian Diao, Weicheng Ma, and Soroush Vosoughi. 2024. Judging the judges: A systematic study of position bias in llm-as-a-judge. arXiv preprint arXiv:2406.07791

    2024

  15. [23]

    Andreas Stephan, Dawei Zhu, Matthias A enmacher, Xiaoyu Shen, and Benjamin Roth. 2025. From calculation to adjudication: Examining llm judges on mathematical reasoning tasks. In Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM ^2 ), pages 759--773

    2025

  16. [24]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram \'e , Morgane Rivi \`e re, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786

    2025 arXiv

  17. [25]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , and 1 others. 2024 a . Gemma 2: Improving open language models at a practical size. arXiv preprint arX...

    2024 arXiv

  18. [26]

    Qwen Team and 1 others. 2024 b . Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2(3)

    2024 arXiv

  19. [27]

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. 2023. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248

    2023 arXiv

  20. [28]

    Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and 1 others. 2024. Large language models are not fair evaluators. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu...

    2024

  21. [29]

    Qian Wang, Zhanzhi Lou, Zhenheng Tang, Nuo Chen, Xuandong Zhao, Wenxuan Zhang, Dawn Song, and Bingsheng He. 2025. Assessing judging bias in large reasoning models: An empirical study. arXiv preprint arXiv:2504.09946

    2025 arXiv

  22. [30]

    Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. 2023. Simple synthetic data reduces sycophancy in large language models. arXiv preprint arXiv:2308.03958

    2023 arXiv

  23. [31]

    Minghao Wu and Alham Fikri Aji. 2023. Style over substance: Evaluation biases for large language models.” arxiv. arXiv preprint arXiv:2307.03025

    2023 arXiv

  24. [32]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

    2025 arXiv

  25. [33]

    Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, and 1 others. 2024. Justice or prejudice? quantifying biases in llm-as-a-judge. arXiv preprint arXiv:2410.02736

    2024 arXiv

  26. [34]

    Zheng Zhang, Peilin Zhao, Deheng Ye, and Hao Wang. 2025. Enhancing jailbreak attacks on llms via persona prompts. arXiv preprint arXiv:2507.22171

    2025

  27. [35]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595--46623

    2023

  28. [36]

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, and 1 others. 2023. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405

    2023 arXiv

  29. [37]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  30. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.