Pith. sign in

REVIEW 4 cited by

WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.12705 v5 pith:ENJEXFD4 submitted 2024-10-16 cs.CL cs.AIcs.CV

Genta Indra Winata , Frederikus Hudi , Patrick Amadeus Irawan , David Anugraha , Rifki Afina Putri , Yutong Wang , Adam Nohejl , Ubaidillah Ariq Prathama
show 43 more authors
Nedjma Ousidhoum Afifa Amriani Anar Rzayev Anirban Das Ashmari Pramodya Aulia Adila Bryan Wilie Candy Olivia Mawalim Ching Lam Cheng Daud Abolade Emmanuele Chersoni Enrico Santus Fariz Ikhwantri Garry Kuwanto Hanyang Zhao Haryo Akbarianto Wibowo Holy Lovenia Jan Christian Blaise Cruz Jan Wira Gotama Putra Junho Myung Lucky Susanto Maria Angelica Riera Machin Marina Zhukova Michael Anugraha Muhammad Farid Adilazuarda Natasha Santosa Peerat Limkonchotiwat Raj Dabre Rio Alexander Audino Samuel Cahyawijaya Shi-Xiong Zhang Stephanie Yulia Salim Yi Zhou Yinxuan Gui David Ifeoluwa Adelani En-Shiun Annie Lee Shogo Okada Ayu Purwarianti Alham Fikri Aji Taro Watanabe Derry Tanti Wijaya Alice Oh Chong-Wah Ngo
This is my paper · ORCID
classification cs.CLcs.AIcs.CV
keywords benchmarkknowledgelanguagelanguagesmulticulturalansweringcontextscuisines
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages

    cs.CL 2025-10 conditional novelty 6.0 of 10

    Across nine Asian languages, multilingual LLMs favor Western cultural entities in 30-40% of culturally grounded contexts, with model-specific sentiment biases and extraction accuracy gaps.

  2. CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries

    cs.AI 2025-01 conditional novelty 6.0 of 10

    CultureVerse is a 188-country, 19k-concept visual QA benchmark, and fine-tuning open VLMs on it improves cultural accuracy, but the main evaluation shares concepts between training and test sets.

  3. Benchmarking Multimodal Models for Ukrainian Language Understanding Across Academic and Cultural Domains

    cs.CL 2024-11 conditional novelty 6.0 of 10

    Introduces ZNO-Vision, a 4,306-item Ukrainian multimodal exam benchmark, plus a translated VQA set and a 20-dish cuisine test, and finds only Gemini, Claude, and Qwen2-VL-72B beat the chance baseline.

  4. The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes

    cs.CY 2025-02 conditional novelty 4.0 of 10

    A design retrospective of World Wide Dishes identifies three dimensions of community ambassador labor, trust building, accessibility, and cultural contextualization, as essential to participatory dataset creation.

Pith tools