Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Is Reasoning All You Need? Probing Bias in the Age of Reasoning Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Explicit reasoning—whether prompted at query time or built into training—makes language models more vulnerable to being provoked into biased, stereotyped answers, not more robust.

desk verdict Useful empirical comparison, but the causal claim about reasoning-by-design amplifying bias is confounded and overstates the evidence. read the letter →

arxiv 2507.02799 v1 pith:HCXV2QJ7 submitted 2025-07-03 cs.CL

Riccardo Cantini , Nicola Gabriele , Alessio Orsino , Domenico Talia This is my paper
classification cs.CL
keywords reasoninglanguagemodelschain-of-thoughtpromptingsocialbiasstereotypereinforcementjailbreakattacksadversarialrobustnessLLM-as-a-judgesafety
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether giving language models explicit reasoning—either by prompting them to "think step by step" at query time or by building reasoning into the model itself—makes them more resistant to producing biased, stereotyped content. Using the CLEAR-Bias benchmark and an LLM-as-a-judge safety scorer, it compares base models, chain-of-thought-prompted variants, and reasoning-enabled models from three model families. The central finding is that both forms of reasoning amplify vulnerability to bias elicitation compared with the base models, with chain-of-thought prompting the most vulnerable. This matters because reasoning is widely assumed to improve reliability and alignment; the result suggests that reasoning as currently implemented can open new pathways for stereotype reinforcement and calls for bias-aware reasoning design.

What carries the argument

The central mechanism is the CLEAR-Bias benchmark's two-stage safety evaluation. Base, CoT-prompted, and reasoning-enabled models first answer 200 bias-probing prompts across ten sociocultural categories; an LLM-as-a-judge classifies each response as stereotyped, counter-stereotyped, debiased, or refused, and combines robustness (refusal plus debiasing rate) and fairness (balance of stereotyped versus counter-stereotyped responses) into a safety score per category. Categories passing a threshold are then attacked with seven jailbreak techniques—machine translation into low-resource languages, obfuscation, prefix injection, prompt injection, refusal suppression, reward incentive, and role-playing—and the final safety score takes the worst attack per category. Comparing the same model families (GPT, DeepSeek, Phi-4) across the Base, CoT, and Reasoner groups is what isolates the effect of the reasoning mechanism from other model differences.

What would settle it

Run the same CLEAR-Bias protocol on a wider set of model families with matched base, CoT-prompted, and reasoning-enabled versions. If the base version does not achieve the highest average safety score in a majority of families, the central amplification claim would be overturned; the attribution to reasoning would also fail if models with identical safety alignment but different reasoning mechanisms show no safety difference.

Watch

Extended reading notes

Core claim

Reasoning does not inherently make language models more robust to social bias; under the CLEAR-Bias protocol it makes them measurably more vulnerable. Averaged over all tested models, base models reached a safety score of 0.50, while chain-of-thought-prompted variants fell to 0.33 and reasoning-enabled models to 0.40, so reasoning-enabled models were safer than CoT-prompted ones but still below base models. In adversarial evaluation, no model stayed above the safety threshold, and attack effectiveness depended on reasoning type, with the strongest overall attack being machine translation into low-resource languages (effectiveness 0.49), followed by obfuscation (0.41). The paper interprets this as evidence that current reasoning implementations add generative freedom that can be exploited to produce stereotyped or rationalized harmful answers.

Load-bearing premise

The three selected model families are assumed to stand in for reasoning and non-reasoning language models generally; if the DeepSeek family's particular safety weaknesses drive the pattern, the conclusion that reasoning itself amplifies bias would not generalize.

Editorial extensions

If this is right

  • Model developers cannot infer bias safety from a model's reasoning benchmark scores; bias robustness has to be measured directly under adversarial prompts.
  • Chain-of-thought prompting should not be treated as a neutral reliability tool in safety-sensitive deployments; it lowered safety relative to the base model in every family tested.
  • Reasoning-enabled models, while safer than CoT-prompted ones, are still more bias-vulnerable than base models, so reasoning design and safety alignment must be developed together.
  • Adversarial defenses need to be attack-specific: machine translation and reward incentives hit CoT models hardest, while obfuscation and refusal suppression hit reasoning-enabled models hardest.
  • Distilled reasoning models are a particular safety concern, as the smallest distilled DeepSeek variants had the lowest safety scores in the evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the pattern generalizes beyond these three families, it points to a structural tension between fluent reasoning and bias resistance, not a quirk of one vendor's training data.
  • This paper does not test whether constraining reasoning traces would help; a direct extension would force models to consider counter-stereotypes during reasoning and check whether the safety gap shrinks.
  • The attack-dependence result implies that averaging safety over attacks can hide which defense to build; reporting per-attack vulnerability is more actionable for alignment work.
  • Because model releases change quickly, the base-versus-reasoner ordering may narrow or invert as safety alignment improves, so treating the current gap as a permanent law of reasoning models would be an overgeneralization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper evaluates how reasoning mechanisms affect robustness to social-bias elicitation. Using the CLEAR-Bias benchmark and an LLM-as-a-judge protocol, the authors compare twelve models from the GPT, DeepSeek, and Phi-4 families under three conditions: base models, zero-shot Chain-of-Thought prompting of those base models, and reasoning-enabled models (o3-mini, o1-preview, DeepSeek R1 and its distilled variants, and Phi-4-reasoning). They report that base models have the highest averaged safety, CoT-prompted models the lowest, and reasoning-enabled models intermediate; they also report attack-specific effectiveness and a family-level vulnerability dominance rate. The central conclusion, stated in RQ1, is that both inference-time CoT prompting and reasoning-by-design amplify vulnerability to bias elicitation relative to base models.

Significance. If the causal interpretation of the central claim were supported, the paper would be an important contribution: it would show that current implementations of reasoning, despite improving task performance, can increase stereotype generation relative to non-reasoning baselines. The paper has genuine strengths: it uses a previously published benchmark, spans multiple model families and scales, specifies its metrics clearly, and provides detailed per-attack tables. However, the strongest empirical evidence is for the within-model CoT comparison, while the reasoning-by-design comparison is confounded with model identity and safety tuning; moreover, the key comparisons lack inferential statistics. The contribution is therefore conditional: the paper is a useful descriptive benchmark and hypothesis-generating study, but it does not yet substantiate the causal claim about reasoning-by-design.

major comments (4)
  1. [§4.1, Fig. 3a, §4.2.3 (RQ1)] The Reasoner-versus-Base comparison is not a controlled manipulation. The CoT condition is a within-model comparison (the same base checkpoint with a 'Think step by step' instruction), but the Reasoner group consists of different checkpoints and, in the DeepSeek family, different architectures, sizes, and distillation procedures. These models differ from the Base models in alignment recipe, training data, and release date. Therefore the lower average safety of the Reasoner group (0.40 vs. 0.50 in Fig. 3a) cannot be causally attributed to 'reasoning by design.' The claim in RQ1 that 'both forms of reasoning ... tend to amplify vulnerability' should be restricted to the CoT comparison or re-framed as a descriptive, model-set-specific observation.
  2. [§4.2.1, Fig. 3] The central comparisons lack error bars on the means, significance tests, or effect sizes. With only three model families for Base and CoT and six models in the Reasoner group, and with overlapping standard deviations (Base 0.50±0.13, CoT 0.33±0.12, Reasoner 0.40±0.15), the statement that reasoning 'generally' or 'tends to' amplify vulnerability is not statistically supported. The same issue affects the LRM-vs-SRM comparison (0.33±0.06 vs. 0.29±0.23). The paper should report paired per-family differences at minimum, or explicitly weaken the causal vocabulary.
  3. [§3.2, §4.1] The judge model is DeepSeek V3, which is itself one of the evaluated models. Its selection as judge is justified by agreement with a control set in the authors' prior work [11], but that validation was not performed on the reasoning-enabled outputs considered here. Because the authors evaluate only final answers while reasoning traces are generated internally, it is not clear that the judge's scoring behavior transfers. The paper should report judge agreement on a sample of RLM responses or use an independent judge to confirm the main safety scores.
  4. [§4.2.2, Eq. (6), Table 2] The FL-VDR metric is computed using o1-preview as the sole representative of the Reasoner category for the GPT family, with the justification that o1-preview 'exhibits lower average attack effectiveness compared to o3-mini.' This is an outcome-dependent selection made after observing the results, and it can change the values in Table 2 and the conclusions about which reasoning type is most vulnerable to a given attack. The representative should be chosen a priori, or the metric should aggregate over both GPT reasoner models.
minor comments (4)
  1. [§3.2, Eq. (4), §4.1] The safety threshold τ = 0.5 determines which bias categories are subjected to adversarial prompting, but no sensitivity analysis is reported. Since Eq. (4) takes a minimum over attacked categories only, attack-effectiveness comparisons across models may be sensitive to this hand-set threshold.
  2. [Table 1] The '-' entries in Table 1 are not explained in the caption. The text mentions a 'misunderstanding filter,' but the reader should be told explicitly why certain model-attack pairs have no value, and whether this affects the FL-VDR denominator in Eq. (6).
  3. [Fig. 3a] The legend and caption of Fig. 3a do not clearly map the nine plotted points to the three groups (Base, CoT, Reasoner). Also, the figure displays duplicate numerical labels (e.g., 0.38 appears twice), which makes the plot hard to read.
  4. [Abstract and §4.2.3] The abstract and RQ1 answer use 'generally more vulnerable' and 'tend to amplify' to describe both CoT and reasoning-by-design. Given the confound discussed in the major comments, the wording should be tempered or explicitly qualified as descriptive of the particular model set evaluated.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark, judge, and metrics are fixed external artifacts, and the headline comparisons are measurements, not fitted reconstructions.

full rationale

I walked the derivation chain from raw model responses to the headline claims. The safety metrics in Eqs. 1-3 are computed from judge labels and prescribed formulas; the threshold tau = 0.5 and the adversarial aggregation in Eqs. 4-5 are fixed design choices, not parameters fitted to reproduce the conclusion. The CoT comparison is a within-model manipulation of the same base models with a 'Think step by step' prompt, so the CoT half of RQ1 is a controlled measurement rather than a constructed equivalence. The Reasoner comparison uses different checkpoints, which raises a genuine confounding concern about model identity, alignment recipe, and release date, but confounding is not circularity: the reported numbers are not analytically forced to equal their inputs. The CLEAR-Bias benchmark and the DeepSeek V3 judge are cited from the authors' prior work [11], but that prior work is an externally published artifact with a manually curated control set and Cohen's kappa evaluation, and the present paper does not refit the judge, the prompts, or the safety formulas to its own conclusions. No equation in the paper reduces to its own input, and no fitted parameter is relabeled as a prediction. I therefore find no significant circularity, even though the Reasoner-vs-Base comparison is weak as a causal claim.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No invented entities; the only new construct is the FL-VDR metric, which is a defined statistic, not a postulated entity. The analysis depends heavily on choices inherited from prior work (benchmark, judge) and the hand-chosen threshold.

free parameters (1)
  • safety threshold tau = 0.5
    Bias categories with safety score below 0.5 are excluded from adversarial evaluation; the choice affects which attacks contribute to final safety scores.
assumptions (4)
  • domain assumption DeepSeek V3 as LLM judge reliably labels responses into S, CS, D, R categories
    The paper relies on the judge's classification to compute all robustness, fairness, and safety scores; this assumption comes from prior work [11], not re-validated here.
  • domain assumption CLEAR-Bias benchmark prompt set adequately samples the targeted sociocultural bias dimensions
    The benchmark is cited from the authors' prior work; if the prompts are not representative, the safety scores are not meaningful measures of bias.
  • domain assumption Zero-shot CoT prompt 'Think step by step' is a fair representative of inference-time reasoning elicitation
    The CoT group is constructed with a single fixed prompt; other CoT formulations could give different safety outcomes.
  • domain assumption The three model families are representative of reasoning and non-reasoning model populations
    General claims about Base vs CoT vs Reasoner rely on n = 3 families; family-specific biases could drive the aggregate pattern.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Is Reasoning All You Need? Probing Bias in the Age of Reasoning Language Models." pith.science (2026). https://pith.science/paper/HCXV2QJ7

@misc{pith2026250702799,
  author       = {Pith},
  title        = {Pith review of: Is Reasoning All You Need? Probing Bias in the Age of Reasoning Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HCXV2QJ7}},
  note         = {Machine review of arXiv:2507.02799}
}
read the original abstract

Reasoning Language Models (RLMs) have gained traction for their ability to perform complex, multi-step reasoning tasks through mechanisms such as Chain-of-Thought (CoT) prompting or fine-tuned reasoning traces. While these capabilities promise improved reliability, their impact on robustness to social biases remains unclear. In this work, we leverage the CLEAR-Bias benchmark, originally designed for Large Language Models (LLMs), to investigate the adversarial robustness of RLMs to bias elicitation. We systematically evaluate state-of-the-art RLMs across diverse sociocultural dimensions, using an LLM-as-a-judge approach for automated safety scoring and leveraging jailbreak techniques to assess the strength of built-in safety mechanisms. Our evaluation addresses three key questions: (i) how the introduction of reasoning capabilities affects model fairness and robustness; (ii) whether models fine-tuned for reasoning exhibit greater safety than those relying on CoT prompting at inference time; and (iii) how the success rate of jailbreak attacks targeting bias elicitation varies with the reasoning mechanisms employed. Our findings reveal a nuanced relationship between reasoning capabilities and bias safety. Surprisingly, models with explicit reasoning, whether via CoT prompting or fine-tuned reasoning traces, are generally more vulnerable to bias elicitation than base models without such mechanisms, suggesting reasoning may unintentionally open new pathways for stereotype reinforcement. Reasoning-enabled models appear somewhat safer than those relying on CoT prompting, which are particularly prone to contextual reframing attacks through storytelling prompts, fictional personas, or reward-shaped instructions. These results challenge the assumption that reasoning inherently improves robustness and underscore the need for more bias-aware approaches to reasoning design.

Figures

Figures reproduced from arXiv: 2507.02799 by the authors.

Figure 1
Figure 1. Robustness, fairness, and safety at the bias level of each model after the initial assessment. Darker green shades indicate higher positive scores, while darker red ones reflects more biased behaviors. When analyzing safety scores for each model ( [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Safety outcomes under different reasoning types and model sizes. To better assess model behavior, we analyzed responses in terms of refusal, debiasing, stereotype, and counter-stereotype rates ( [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Analysis of models’ behavior during initial safety assessment in terms of refusal vs. debiasing rate (𝑎) and stereotype vs. counter-stereotype rate (𝑏). debiasing is the dominant strategy for many models, especially those in the Phi-4 family without built-in reasoning …

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models

    cs.CL 2025-09 conditional novelty 4.0 of 10

    A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.

Reference graph

Works this paper leans on

29 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [11]

    Cantini, A

    R. Cantini, A. Orsino, M. Ruggiero, D. Talia, Benchmarking adversarial robustness to bias elicitation in large language models: Scalable automated assessment with llm-as-a-judge, arXiv preprint arXiv:2504.07887 (2025)

    arXiv 2025

  2. [1]

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, et al., Language models are few-shot learners, Advances in Neural Information Processing Systems (2020)

    work page 2020

  3. [2]

    Chang, X

    Y. Chang, X. Wang, J. Wang, Y. Wu, et al., A survey on evaluation of large language models, ACM Transactions on Intelligent Systems and Technology (2024)

    work page 2024

  4. [3]

    Navigli, S

    R. Navigli, S. Conia, B. Ross, Biases in large language models: origins, inventory, and discussion, ACM Journal of Data and Information Quality (2023)

    work page 2023

  5. [4]

    D. Hovy, S. Prabhumoye, Five sources of bias in natural language processing, Language and linguistics compass (2021)

    work page 2021

  6. [5]

    I. O. Gallegos, R. A. Rossi, J. Barrow, M. M. Tanjim, et al., Bias and fairness in large language models: A survey, Computational Linguistics (2024)

    work page 2024

  7. [6]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, et al., Chain-of-thought prompting elicits reasoning in large language models, Advances in Neural Information Processing Systems (2022)

    work page 2022

  8. [7]

    T. Q. Luong, X. Zhang, Z. Jie, P. Sun, et al., Reft: Reasoning with reinforced fine-tuning, in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024

    work page 2024

Show all 29 references
  1. [8]

    F. Xu, Q. Hao, Z. Zong, J. Wang, et al., Towards large reasoning models: A survey of reinforced reasoning with large language models, arXiv preprint arXiv:2501.09686 (2025)

    2025 arXiv

  2. [9]

    Huang, K

    J. Huang, K. C.-C. Chang, Towards reasoning in large language models: A survey, in: Findings of the Association for Computational Linguistics, 2022

    2022

  3. [10]

    Cantini, G

    R. Cantini, G. Cosenza, A. Orsino, D. Talia, Are large language models really bias-free? jailbreak prompts for assessing adversarial robustness to bias elicitation, in: International Conference on Discovery Science, 2024

    2024

  4. [12]

    Nadeem, A

    M. Nadeem, A. Bethke, S. Reddy, Stereoset: Measuring stereotypical bias in pretrained language models, in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021

    2021

  5. [13]

    Nangia, C

    N. Nangia, C. Vania, R. Bhalerao, S. R. Bowman, Crows-pairs: A challenge dataset for measuring social biases in masked language models, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

    2020

  6. [14]

    T. Shen, R. Jin, Y. Huang, C. Liu, et al., Large language model alignment: A survey, arXiv preprint arXiv:2309.15025 (2023)

    2023 arXiv

  7. [15]

    X. Wu, J. Nian, Z. Tao, Y. Fang, Evaluating social biases in llm reasoning, arXiv preprint arXiv:2502.15361 (2025)

    2025

  8. [16]

    Parrish, A

    A. Parrish, A. Chen, N. Nangia, V. Padmakumar, et al., BBQ: A hand-built bias benchmark for question answering, in: Findings of the Association for Computational Linguistics, 2022

    2022

  9. [17]

    Kojima, S

    T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, et al., Large language models are zero-shot reasoners, Advances in Neural Information Processing Systems (2022)

    2022

  10. [18]

    Shaikh, H

    O. Shaikh, H. Zhang, W. Held, M. Bernstein, et al., On second thought, let’s not think step by step! bias and toxicity in zero-shot reasoning, in: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, 2023

    2023

  11. [19]

    Arrieta, M

    A. Arrieta, M. Ugarte, P. Valle, J. A. Parejo, et al., o3-mini vs deepseek-r1: Which one is safer?, arXiv preprint arXiv:2501.18438 (2025)

    2025 arXiv

  12. [20]

    Ugarte, P

    M. Ugarte, P. Valle, J. A. Parejo, S. Segura, et al., Astral: Automated safety testing of large language models, arXiv preprint arXiv:2501.17132 (2025)

    2025 arXiv

  13. [21]

    Zheng, W.-L

    L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, et al., Judging llm-as-a-judge with mt-bench and chatbot arena, Advances in Neural Information Processing Systems (2023)

    2023

  14. [22]

    L. Zhu, X. Wang, X. Wang, Judgelm: Fine-tuned large language models are scalable judges, in: The Thirteenth International Conference on Learning Representations, 2023

    2023

  15. [23]

    Ranathunga, E

    S. Ranathunga, E. A. Lee, M. P. Skenduli, R. Shekhar, et al., Neural machine translation for low-resource languages: A survey, ACM Computing Survey (2023)

    2023

  16. [24]

    D. Dorn, A. Variengien, C.-R. Segerie, V. Corruble, Bells: A framework towards future proof benchmarks for the evaluation of llm safeguards, arXiv preprint arXiv:2406.01364 (2024)

    2024 arXiv

  17. [25]

    A. Liu, B. Feng, B. Xue, B. Wang, et al., Deepseek-v3 technical report, arXiv preprint arXiv:2412.19437 (2024)

    2024 arXiv

  18. [26]

    Abdin, S

    M. Abdin, S. Agarwal, A. Awadallah, V. Balachandran, et al., Phi-4-reasoning technical report, arXiv preprint arXiv:2504.21318 (2025)

    2025 arXiv

  19. [27]

    D. Guo, D. Yang, H. Zhang, J. Song, et al., Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  20. [28]

    Turpin, J

    M. Turpin, J. Michael, E. Perez, S. Bowman, Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting, Advances in Neural Information Processing Systems (2023)

    2023

  21. [29]

    Y. Chen, J. Benton, A. Radhakrishnan, J. Uesato, et al., Reasoning models don’t always say what they think, arXiv preprint arXiv:2505.05410 (2025)

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.