Beauty and the Bias: Exploring the Impact of Attractiveness on Multimodal Large Language Models

Aditya Gulati; Bruno Lepri; Moreno D'Inc\`a; Nicu Sebe; Nuria Oliver

REVIEW 3 major objections 4 minor 5 references

Beauty and the Bias: Exploring the Impact of Attractiveness on Multimodal Large Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper reports that seven open-source multimodal large language models give systematically different forced-choice answers about the same person when their face is beautified, associating beautified faces with positive traits and…

desk verdict A credible, large-scale measurement that beauty filters shift MLLM judgments in a stereotype-consistent direction, but the 'only difference is attractiveness' claim is overstated and the abstract numbers need a sanity check. read the letter →

arxiv 2504.16104 v3 pith:ENSDN5IM submitted 2025-04-16 cs.CY

Aditya Gulati , Moreno D'Incà , Nicu Sebe , Bruno Lepri , Nuria Oliver This is my paper · ORCID

classification cs.CY

keywords attractivenessbiashaloeffectmultimodallargelanguagemodelsbeautyfiltersintersectionalalgorithmicfairnessfaceperceptioncognitiveinAI

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a well-documented human judgment bias — the "attractiveness halo effect," in which attractive people are assumed to have positive traits — is also present in multimodal large language models (MLLMs), systems that answer text prompts about images. To test this, the authors show 462 faces to seven open-source MLLMs both in original form and after a commercial beauty filter, and ask 91 forced-choice questions about jobs, personality traits, and social conditions. The paper reports that attractiveness changed model answers in 86.2% of scenarios on average, and that beautified faces were significantly more often assigned positive traits, such as confidence and trustworthiness, in most of the sentiment scenarios, which is direct evidence of the halo effect in machines. The result matters because MLLMs are beginning to inform real decisions in hiring, education, and professional evaluation, where physical attractiveness should be irrelevant.

What carries the argument

The load-bearing device is a paired-image counterfactual: every one of the 462 identities appears twice, once as the original photograph and once passed through a beauty filter, so the input pair is intended to differ only in perceived attractiveness. For each of the 91 scenarios, the MLLM must choose between two options; responses are converted to a Stereotype Consistency Score (SCS), the fraction of times an image is assigned to the stereotyped choice, averaged over the four possible orderings of the two options and three random seeds to remove position and sampling effects. An attractiveness bias is declared when a Kruskal-Wallis test at $p<0.01$ finds that the score distributions for originals and beautified versions differ; the halo effect is the directional case in which beautified faces are more often matched to the positive trait. Wilcoxon paired rank tests across the 91 scenarios, with Bonferroni correction, then quantify how gender, age, and race change the strength of the attractiveness bias.

What would settle it

Re-run the 91 scenarios with an attractiveness manipulation that does not touch low-level image properties — for example, morph each face by a fixed amount toward a consensus-attractive prototype while holding age, skin texture, lighting, expression, and identity constant. If the original and manipulated versions no longer produce significantly different answer distributions at $p<0.01$, the reported bias is an artifact of the beauty filter rather than an attractiveness halo effect in the models.

Watch

Extended reading notes

Core claim

The central claim is stated directly: physical attractiveness biases the decisions made by MLLMs. Using the same individual's face in original and beautified versions, the paper observes statistically significant differences in forced-choice answers across a large majority of 91 scenarios, with the effect present in every one of the seven models tested. The attractiveness halo effect appears in the sentiment trait scenarios, where beautified faces are more likely than original faces to be labelled confident, trustworthy, happy, kind, or competent; the paper reports this in 94.8% of the relevant scenarios. The bias is not uniform: it is stronger for female faces than male faces, beauty filters amplify gender bias in most models, and the effect of race and age is attenuated by beautification in several models. The authors conclude that attractiveness is an "invisible" decision cue for MLLMs, operating implicitly and intersecting with demographic stereotypes.

Load-bearing premise

The entire measurement rests on the assumption that the beauty filter changes only perceived attractiveness and leaves every other decision-relevant property of the face unchanged; if the filter also shifts apparent age, skin texture, lighting, or expression, the answer differences could be caused by those changes rather than by attractiveness.

Editorial extensions

If this is right

If the result holds, any vision-based screening or assessment system built on an MLLM can silently favor more attractive-looking people even in settings where looks carry no information.
The human attractiveness halo effect transfers to machines: the same person is judged more trustworthy, confident, or competent when their face is beautified, so appearance editing can change model-based evaluations.
Attractiveness bias is intersectional: it is strongest for female faces, and beautification amplifies gender bias in most of the seven models, while reducing age and race biases in some.
Fairness audits that check only gender, race, or age will miss a bias that operates on appearance and modulates those demographic biases; attractiveness should be an explicit evaluation dimension.
The paired original/beautified image protocol provides a reusable probe for appearance-based bias in open MLLMs, including for models not covered in this study.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A beauty filter also changes skin texture, apparent age, and lighting; until attractiveness is manipulated independently of those cues, part of the measured bias could come from filter artifacts rather than from attractiveness as humans perceive it.
The forced-choice format may understate the bias in free-form conversation, where a model can hedge or explain; asking the same models open-ended questions about the same paired faces is a direct next test.
If the halo effect is encoded in pretraining associations, then text-only language models may show a similar effect when faces are described verbally, which would make the bias about language statistics rather than vision.
A practical extension would be to use this same original-versus-beautified contrast as a routine fairness probe during MLLM release testing, complementing demographic parity checks.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this is the first controlled paired-image study I've seen that directly tests attractiveness bias in MLLMs using natural faces. The protocol is clean—same individual, original vs. beautified, 462 faces, 91 scenarios, seven open models, order-invariant scoring, three seeds. That is a real step beyond the synthetic-image benchmarks in the prior work, and the empirical pattern is consistent: beautified faces are significantly more likely to be assigned positive traits, and the effect shows up across all seven models. The large Kruskal-Wallis statistics in the appendix support the qualitative conclusion, and the gender intersection result (stronger attractiveness bias for female faces) aligns with the human literature. I find the core finding credible and worth taking seriously.

Where the paper is softer: first, the abstract numbers don't match the results tables. The abstract says 86.2% of scenarios and 94.8% for the halo effect; Table 2 and the text say 83.8% and 92.6%. That's a minor but embarrassing inconsistency that must be fixed. Second, the load-bearing assumption that beauty filters change 'only attractiveness' is asserted, not demonstrated. The paper's own discussion admits filters make people look younger, and they also smooth skin, alter lighting, and leave a filter signature. So the measured shifts could partly reflect filter artifacts or an "edited/idealized" cue rather than attractiveness per se. The human-rating validation from the prior study shows the manipulation works on attractiveness perception, but it doesn't show other decision-relevant features are invariant. This is the main interpretive caveat, and it means the specific percentages are probably upper bounds on the pure attractiveness effect. Third, the per-scenario KW tests are run at p<0.01 with no multiple-comparison correction across 91 scenarios, so some false positives are likely. The effect sizes are large enough that the main conclusion survives, but the reported proportions are optimistic. Fourth, code and data are "available upon request"—for a paper whose contribution is a dataset and protocol, that is weak; they should ship them.

Who is this for? Anyone working on fairness evaluation of vision-language models, and people studying how cognitive biases transfer to AI. It deserves a serious referee: the design is thoughtful, the finding is important if it holds, and the weaknesses are addressable rather than fatal. My recommendation: send it to review, and ask the authors to reconcile the numbers, release the artifacts, and directly test the filter-confound concern—e.g., by measuring which low-level features change or by including a control condition with a non-attractiveness filter.

Referee Report

3 major / 4 minor

Summary. This paper reports an empirical study of whether multimodal large language models (MLLMs) exhibit an attractiveness bias. The authors use a dataset of 462 individuals, each with an original face image and a beautified version, and 91 forced-choice social judgment scenarios spanning stereotyped jobs, traits, and conditions. Seven open-source MLLMs are queried with each image under four option orderings and three seeds, yielding over seven million prompts. The authors define an attractiveness bias as a statistically significant Kruskal-Wallis difference between the distributions of stereotype-consistency scores for original versus beautified faces, and an attractiveness halo effect as the same bias in sentiment scenarios where the beautified face is more likely to be assigned the positive trait. The paper reports that attractiveness affects decisions in the large majority of scenarios (83.8% per Table 2; 86.2% in the abstract and Section 5), that the halo effect appears in 92.6% or 94.8% of relevant scenarios depending on the section, and that gender, age, and race biases interact with attractiveness, with female faces showing a stronger attractiveness bias and beautified images amplifying gender bias in most models. The paper concludes that MLLMs replicate the human attractiveness halo effect and calls for intersectional bias mitigation.

Significance. If the reported effects are real, this is a meaningful and timely contribution: it provides the first large-scale evidence that open-source MLLMs alter forced-choice judgments about the same person when facial attractiveness is manipulated, and it connects this behavior to a well-documented human cognitive bias. The study has noteworthy strengths: it covers seven diverse open-source models, uses paired original/beautified images of the same identities, averages over option orderings and random seeds, reports per-scenario statistics in the appendices, and makes the experimental design transparent enough to be replicated. The main risk is interpretive: the claim that the observed differences are specifically due to attractiveness, rather than to other visual changes introduced by beauty filters, rests on an invariance assumption that is asserted but not directly tested. The inconsistency among the headline percentages (83.8% vs 86.2%; 92.6% vs 94.8%) also needs to be resolved before the central quantitative claims can be taken at face value.

major comments (3)

[Section 3.2; Section 5; Figure 1] The central inference, that differences between original and beautified faces isolate attractiveness, depends on the claim in Section 3.2 that "the only difference between them is attractiveness, with minimal confounds." The cited human-rating validation from Gulati et al. (2024) shows that beautified faces are perceived as more attractive, but it does not establish that other decision-relevant visual properties are invariant. Beauty filters typically alter skin texture, apparent age, lighting, local contrast, and introduce a common filter signature, and MLLMs may exploit any of these cues. Because the paper's RQ1 and RQ2 conclusions are precisely about attractiveness, the authors should provide direct evidence on this invariance, for example by measuring low-level image similarity between paired images, testing a filter-detection classifier, or evaluating models on control tasks where attractiveness should be irrelevant (e.g., identity, expression, or age judgment). Without such a control, the reported shifts in phi_i could be driven partly by filter artifacts rather than by attractiveness as humans perceive it.
[Abstract; Section 4; Section 5; Table 2] The headline numbers are inconsistent across the manuscript. The abstract states that attractiveness impacts decisions in 86.2% of scenarios and that the halo effect appears in 94.8% of relevant scenarios, while Table 2 reports an average of 83.8% for attractiveness bias and Section 4 reports 92.6% for the halo effect in the 33 sentiment scenarios. Section 5 repeats 86.2% and 92.6%. The paper does not explain whether these are different aggregation methods, different scenario subsets, or errors. Since these percentages are the paper's central quantitative claims, the authors must reconcile the numbers and specify the exact computation underlying each reported figure.
[Section 3.5; Section 4] The attractiveness-bias test compares the distributions of phi_i over original faces and their paired beautified faces using a Kruskal-Wallis test, which treats the two samples as independent groups. Because each beautified image is derived from a specific original image of the same individual, the observations are paired. A paired test (e.g., Wilcoxon signed-rank test) or a mixed-effects model with identity as a random effect would use the pairing structure and is more directly aligned with the paper's claim of comparing "the same individuals" before and after beautification. The authors should either justify the unpaired test or report paired analyses; this is not necessarily fatal given the large effects, but it is a methodological mismatch that affects the statistical framing.

minor comments (4)

[Appendix G, Tables 24-30] The captions say "Out of 19 scenarios" for the race-stereotyped jobs, but there are 12 such scenarios and each table has 12 rows. This appears to be a copy-paste error from the gender-stereotyped jobs appendix and should be corrected.
[Section 4, RQ2] The sentence "In all scenarios and for all models (except for 3 out of 31 scenarios for DeepSeek and 1 out of 30 for Qwen2)" is confusing because Tables 12 and 14 report different totals (28 out of 33 and 28 out of 33 significant scenarios, with 3 and 1 opposite-direction cases respectively). The text should state the denominator and direction conventions clearly.
[Table 2 and Tables 31-33] The table formatting uses colored shading with a legend that may not survive black-and-white printing; the authors should ensure all information is also encoded textually or with patterns.
[Section 3.5] The definition of the Bias section uses H with subscripts interchangeably for the Kruskal-Wallis statistic and the hypothesis label; for clarity, the paper should consistently distinguish the test statistic from the hypothesis being tested.

Circularity Check

0 steps flagged · score 0.0 of 10

Measurement study; no derivation-level circularity; possible filter confound is a validity concern, not a circular step.

full rationale

The paper is an empirical measurement study: it defines an attractiveness bias as a Kruskal-Wallis significant difference in MLLM forced-choice responses between original and beautified versions of the same face, and then reports the proportion of scenarios where this difference is observed. There is no fitted parameter, no derived quantity that reduces to an input, and no uniqueness theorem. The main reliance on prior work is the beautified-face dataset and its human-rating validation, both from [Gulati et al. 2024], which shares the first author. That self-citation is used to establish that beautified faces are perceived as more attractive; however, the human ratings are external empirical data collected outside this paper's experiment, and the MLLM responses are new measurements against a fixed stimulus set, so the citation is independent support rather than a circular premise. The most serious threat to the central claim is the asserted invariance of beauty filters: the paper states "the only difference between them is attractiveness, with minimal confounds" without directly testing whether low-level visual changes (skin texture, age cues, filter signature) drive the differences. That is a validity/confounding concern, not a circularity, because the reported statistics are not constructed to equal the assumption; they could in principle have come out null or in the opposite direction. Hence no circular step is identifiable, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on treating a selfie-filter manipulation as a clean change in perceived attractiveness, on treating output differences as bias, and on labeling stereotype directions from US labor data and prior stereotype taxonomies. The human-rating validation of the manipulation comes from a prior paper by the same first author and co-authors, which is independent of the MLLM measurements but not independent of the research group. There are no fitted numerical parameters; the numbers in the paper come from significance testing over measured responses. The p<0.01 threshold and the restriction to 33 sentiment scenarios for the halo effect are analytic choices, not fitted constants.

assumptions (5)

domain assumption The beautified version of each face differs from the original only in perceived attractiveness, with all other identity-relevant properties held constant.
Stated in Section 3.2 as 'the only difference between them is attractiveness, with minimal confounds.' The validation is a self-cited human-rating study; beauty filters also change skin texture, age cues, and other low-level features that MLLMs may exploit.
domain assumption A statistically significant difference between model responses to original and beautified images is a valid measure of attractiveness bias in the model.
This equates observed output differences with an internal bias; alternative explanations such as filter artifacts or prompt sensitivity are not fully controlled.
domain assumption The 91 scenarios' 'stereotyped choices' reflect genuine societal stereotypes relevant to Western and US contexts.
Job pairs are sourced from BLS 2022 labor force data and prior stereotype benchmarks; this grounds stereotype direction but imposes a US and Western frame on race and occupation.
domain assumption Forced-choice binary prompts with no neutral option elicit a valid measure of model judgment rather than prompting artifacts.
Authors state in Section 3.2 that neutral responses were disallowed to minimize noise; this presupposes the binary choice is meaningful despite potential position and format bias.
standard math Nonparametric hypothesis tests (Kruskal-Wallis, Wilcoxon) with p<0.01 appropriately measure response distribution differences.
Used throughout Section 3.5; however, the paired structure of original and beautified images is analyzed with an unpaired Kruskal-Wallis test, weakening sensitivity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Beauty and the Bias: Exploring the Impact of Attractiveness on Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/ENSDN5IM

@misc{pith2026250416104,
  author       = {Pith},
  title        = {Pith review of: Beauty and the Bias: Exploring the Impact of Attractiveness on Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ENSDN5IM}},
  note         = {Machine review of arXiv:2504.16104}
}

read the original abstract

Physical attractiveness matters. It has been shown to influence human perception and decision-making, often leading to biased judgments that favor those deemed attractive in what is referred to as the "attractiveness halo effect". While extensively studied in human judgments in a broad set of domains, including hiring, judicial sentencing or credit granting, the role that attractiveness plays in the assessments and decisions made by multimodal large language models (MLLMs) is unknown. To address this gap, we conduct an empirical study with 7 diverse open-source MLLMs evaluated on 91 socially relevant scenarios and a diverse dataset of 924 face images - corresponding to 462 individuals both with and without beauty filters applied to them. Our analysis reveals that attractiveness impacts the decisions made by MLLMs in 86.2% of the scenarios on average, demonstrating substantial bias in model behavior in what we refer to as an attractiveness bias. Similarly to humans, we find empirical evidence of the existence of the attractiveness halo effect in 94.8% of the relevant scenarios: attractive individuals are more likely to be attributed positive traits, such as intelligence or confidence, by MLLMs than unattractive individuals. Furthermore, we uncover gender, age and race biases in a significant portion of the scenarios which are also impacted by attractiveness, particularly in the case of gender, highlighting the intersectional nature of the algorithmic attractiveness bias. Our findings suggest that societal stereotypes and cultural norms intersect with perceptions of attractiveness in MLLMs in a complex manner. Our work emphasizes the need to account for intersectionality in algorithmic bias detection and mitigation efforts and underscores the challenges of addressing biases in modern MLLMs.

Figures

Figures reproduced from arXiv: 2504.16104 by the authors.

**Figure 2.** Visual depiction of the 91 scenarios used in the experiments: 39 scenarios were about jobs, further divided in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Bonferroni-corrected Wilcoxon Paired Rank Test across scenarios to evaluate the strength of the attractiveness [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Bonferroni-corrected Wilcoxon Paired Rank Test across scenarios to evaluate the strength of the attractiveness [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 2 canonical work pages

[2]

[Dion et al.(1972)]Karen Dion, Ellen Berscheid, and Elaine Walster

Leveraging Large Language Models to Measure Gender Bias in Gendered Languages.arXiv preprint arXiv:2406.13677 (2024). [Dion et al.(1972)]Karen Dion, Ellen Berscheid, and Elaine Walster. 1972. What is beautiful is good.Journal of Personality and Social Psychology24, 3 (1972), 285–290.https://doi.org/10.1037/h0033731 [Dong et al.(2024)]Xiaoyi Dong, Pan Zhan...

arXiv 2024
[3]

IOS Press, 145–161

Investigating Labeler Bias in Face Annotation for Machine Learning. IOS Press, 145–161. https: //doi.org/10.3233/faia240191 [Hall et al.(2022)]MelissaHall,LaurensvanderMaaten,LauraGustafson,MaxwellJones,andAaronAdcock.2022. A Systematic Study of Bias Amplification. arXiv:2201.11706 [cs.LG]https://arxiv.org/abs/2201.11706 [Hall et al.(2023)]Siobhan Mackenz...

arXiv 2022
[4]

InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol

Identifying implicit social biases in vision-language models. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 7. 547–561. [Hosoda et al.(2003)]Megumi Hosoda, Eugene F Stone-Romero, and Gwen Coats. 2003. The effects of physical attractiveness on job-related outcomes: A meta-analysis of experimental studies.Personnel Psychology56, ...

work page arXiv 2003
[5]

Job 1” corresponds to male stereotyped jobs (“Stereotyped Choice

Benchmarking Cognitive Biases in Large Language Models as Evaluators. InFindings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Associa- tion for Computational Linguistics, Bangkok, Thailand, 517–545.https://doi.org/10.18653/v1/2024. findings-acl.29 [Kotek et al.(2023)]HadasKotek,RikkerDoc...

arXiv 2023
[2024]

Cash and Robert N

Beyond the Surface: Comprehensive Analysis of Implicit Bias in Vision-Language Models.https: //hdl.handle.net/11380/1350126 [Cash and Kilcullen(1985)]Thomas F. Cash and Robert N. Kilcullen. 1985. The Aye of the Beholder: Susceptibility to Sexism and Beautyism in the Evaluation of Managerial Applicants1.Journal of Applied Social Psychology15, 4 (June 1985)...

work page arXiv 1985

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [2]

[Dion et al.(1972)]Karen Dion, Ellen Berscheid, and Elaine Walster

Leveraging Large Language Models to Measure Gender Bias in Gendered Languages.arXiv preprint arXiv:2406.13677 (2024). [Dion et al.(1972)]Karen Dion, Ellen Berscheid, and Elaine Walster. 1972. What is beautiful is good.Journal of Personality and Social Psychology24, 3 (1972), 285–290.https://doi.org/10.1037/h0033731 [Dong et al.(2024)]Xiaoyi Dong, Pan Zhan...

arXiv 2024

[2] [3]

IOS Press, 145–161

Investigating Labeler Bias in Face Annotation for Machine Learning. IOS Press, 145–161. https: //doi.org/10.3233/faia240191 [Hall et al.(2022)]MelissaHall,LaurensvanderMaaten,LauraGustafson,MaxwellJones,andAaronAdcock.2022. A Systematic Study of Bias Amplification. arXiv:2201.11706 [cs.LG]https://arxiv.org/abs/2201.11706 [Hall et al.(2023)]Siobhan Mackenz...

arXiv 2022

[3] [4]

InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol

Identifying implicit social biases in vision-language models. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 7. 547–561. [Hosoda et al.(2003)]Megumi Hosoda, Eugene F Stone-Romero, and Gwen Coats. 2003. The effects of physical attractiveness on job-related outcomes: A meta-analysis of experimental studies.Personnel Psychology56, ...

work page arXiv 2003

[4] [5]

Job 1” corresponds to male stereotyped jobs (“Stereotyped Choice

Benchmarking Cognitive Biases in Large Language Models as Evaluators. InFindings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Associa- tion for Computational Linguistics, Bangkok, Thailand, 517–545.https://doi.org/10.18653/v1/2024. findings-acl.29 [Kotek et al.(2023)]HadasKotek,RikkerDoc...

arXiv 2023

[5] [2024]

Cash and Robert N

Beyond the Surface: Comprehensive Analysis of Implicit Bias in Vision-Language Models.https: //hdl.handle.net/11380/1350126 [Cash and Kilcullen(1985)]Thomas F. Cash and Robert N. Kilcullen. 1985. The Aye of the Beholder: Susceptibility to Sexism and Beautyism in the Evaluation of Managerial Applicants1.Journal of Applied Social Psychology15, 4 (June 1985)...

work page arXiv 1985