REVIEW 5 cited by
See It from My Perspective: How Language Affects Cultural Bias in Image Understanding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Vision-language models (VLMs) can respond to queries about images in many languages. However, beyond language, culture affects how we see things. For example, individuals from Western cultures focus more on the central figure in an image while individuals from East Asian cultures attend more to scene context. In this work, we characterize the Western bias of VLMs in image understanding and investigate the role that language plays in this disparity. We evaluate VLMs across subjective and objective visual tasks with culturally diverse images and annotations. We find that VLMs perform better on the Western split than on the East Asian split of each task. Through controlled experimentation, we trace one source of this bias in image understanding to the lack of diversity in language model construction. While inference in a language nearer to a culture can lead to reductions in bias, we show it is much more effective when that language was well-represented during text-only pre-training. Interestingly, this yields bias reductions even when prompting in English. Our work highlights the importance of richer representation of all languages in building equitable VLMs.
Forward citations
Cited by 5 Pith papers
-
CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity
Current VLMs score well on Chinese-art recognition QA but collapse on style-to-period inference, expert-style long-form appreciation, and authenticity discrimination under visual confounds.
-
AI vs Human Expert Reasoning: Assessing Agreements in Building Typology Predictions based on Street View Imagery
Commercial vision-language models classify building construction, use, and storeys from Jakarta street-view images with about 70% agreement with human experts, with storeys easiest and current use hardest.
-
Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists?
On the ACON benchmark, any-to-any models do not consistently beat specialist model pairs on cyclic consistency, but show weak latent-space consistency in equivariance tests.
-
Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)
Placing demographic audience information in system prompts rather than user prompts shifts sentiment and ranking outputs across six commercial LLMs, but the design confounds position with instruction content.
-
Scaling Pre-training to One Hundred Billion Data for Vision Language Models
Scaling VLM pretraining from 10B to 100B image-text pairs yields saturation on standard benchmarks but large gains on cultural diversity, low-resource language retrieval, and subgroup disparity.
Discussion (0). Continue with ORCID to comment.