pith. machine review for the scientific record. sign in

arxiv: 2605.00119 · v1 · submitted 2026-04-30 · 💻 cs.CL · cs.AI

Recognition: unknown

Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues

Muhammad Dehan Al Kautsar , Saeed Almheiri , Momina Ahsan , Bilal Elbouardi , Younes Samih , Sarfraz Ahmad , Amr Keleg , Omar El Herraoui
show 8 more authors
Kareem Elzeky Abed Alhakim Freihat Mohamed Anwar Zhuohan Xie Junhong Liang Mohammad Rustom Al Nasar Preslav Nakov Fajri Koto
Authors on Pith no claims yet

Pith reviewed 2026-05-09 20:35 UTC · model grok-4.3

classification 💻 cs.CL cs.AI
keywords Arabic dialectscultural reasoningLLM evaluationdialogue datasetsMSA translationdialect generation
0
0 comments X

The pith

LLMs perform worse on Arabic dialects than on Modern Standard Arabic across cultural dialogue tasks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces ArabCulture-Dialogue, a dataset of conversations from 13 Arabic-speaking countries presented in both Modern Standard Arabic and local dialects across 12 daily topics. It defines three tasks on this data: multiple-choice cultural reasoning, translation between standard and dialect forms, and generating responses in a steered dialect. Experiments on several models show a consistent performance drop when the input shifts from standard Arabic to dialects. A sympathetic reader would care because most existing Arabic AI tests use only short standard-Arabic snippets and therefore miss how cultural understanding actually works in spoken daily life.

Core claim

We introduce ArabCulture-Dialogue, a culturally grounded conversational dataset covering 13 Arabic-speaking countries in both MSA and each country's respective dialect across 12 daily-life topics. We form three benchmarking tasks from the dataset: multiple-choice cultural reasoning, machine translation between MSA and dialects, and dialect-steering generation. Experiments indicate that the performance gap between MSA and Arabic dialects still exists, with models performing worse on all three tasks in the dialectal setup compared to the MSA one.

What carries the argument

The ArabCulture-Dialogue dataset of paired MSA and dialect dialogues from 13 countries, used to create the three tasks of multiple-choice reasoning, translation, and steered generation.

If this is right

  • Future Arabic LLM evaluations must include dialectal dialogues to avoid overestimating model capability.
  • Models that close the MSA-dialect gap on these tasks would handle everyday cultural interactions more reliably.
  • Translation and generation quality between standard and dialect forms remains a clear bottleneck.
  • The 54 fine-grained subtopics provide a structured way to diagnose where cultural reasoning fails.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Training data that includes more spoken dialect from multiple countries could narrow the observed gap.
  • Real-world Arabic chat systems in the Middle East and North Africa may currently deliver weaker cultural alignment than MSA-only tests suggest.
  • The same paired-dialogue approach could be applied to other languages with strong standard-versus-spoken divides.
  • Re-testing the dataset on newer models would show whether the gap is shrinking over time.

Load-bearing premise

The new dataset accurately captures culturally rich nuances in dialogues from the 13 countries and the three tasks validly measure cultural reasoning capabilities in LLMs.

What would settle it

Run the same three tasks on ArabCulture-Dialogue with current models and observe no measurable performance difference between the MSA and dialect versions, or show that the dialogues do not reflect real cultural patterns in those countries.

Figures

Figures reproduced from arXiv: 2605.00119 by Abed Alhakim Freihat, Amr Keleg, Bilal Elbouardi, Fajri Koto, Junhong Liang, Kareem Elzeky, Mohamed Anwar, Mohammad Rustom Al Nasar, Momina Ahsan, Muhammad Dehan Al Kautsar, Omar El Herraoui, Preslav Nakov, Saeed Almheiri, Sarfraz Ahmad, Younes Samih, Zhuohan Xie.

Figure 2
Figure 2. Figure 2: Dataset construction pipeline of ArabCulture-Dialogue. After the initial dialogue generation by GPT-5, all subsequent stages, including revision, dialect localization, style post-editing, and quality control, are performed through human annotation, resulting in a fully human-curated dataset. During revision, the annotators verify the lin￾guistic correctness, naturalness, and cultural appro￾priateness of th… view at source ↗
Figure 3
Figure 3. Figure 3: The impact of SFT on the generated re￾sponses of Gemma-2 (a multilingual LLM), for the Di￾alect Steering task. 6 Conclusion and Future Work We introduce ArabCulture-Dialogue, the first cul￾turally grounded conversational dataset covering 13 Arabic-speaking countries, spanning both MSA and corresponding dialects across 12 everyday do￾mains and 54 fine-grained subtopics, with a total of 343,804 words. We use… view at source ↗
read the original abstract

There is a significant gap in evaluating cultural reasoning in LLMs using conversational datasets that capture culturally rich and dialectal contexts. Most Arabic benchmarks focus on short text snippets in Modern Standard Arabic (MSA), overlooking the cultural nuances that naturally arise in dialogues. To address this gap, we introduce ArabCulture-Dialogue, a culturally grounded conversational dataset covering 13 Arabic-speaking countries, in both MSA and each country's respective dialect, spanning 12 daily-life topics and 54 fine-grained subtopics. We utilize the dataset to form three benchmarking tasks: (i) multiple-choice cultural reasoning, (ii) machine translation between MSA and dialects, and (iii) dialect-steering generation. Our experiments indicate that the performance gap between MSA and Arabic dialects still exists, whereby the models perform worse on all three tasks in the dialectal setup, compared to the MSA one.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The paper introduces ArabCulture-Dialogue, a parallel conversational dataset of culturally grounded dialogues in Modern Standard Arabic (MSA) and the respective dialects of 13 Arabic-speaking countries, spanning 12 daily-life topics and 54 subtopics. The dataset is used to define three benchmarking tasks: (i) multiple-choice cultural reasoning, (ii) machine translation between MSA and dialects, and (iii) dialect-steering generation. Experiments across multiple LLMs show consistent performance degradation on all three tasks in the dialectal setting relative to the MSA setting.

Significance. If the central empirical observation holds, the work is significant because it supplies a native-speaker-curated, parallel MSA-dialect resource that directly exposes gaps in current LLMs' handling of dialectal and culturally nuanced Arabic dialogue. The explicit release of the dataset and the parallel construction enable reproducible follow-up work and falsifiable tests of cultural-reasoning claims in multilingual NLP.

minor comments (3)
  1. [Results] The abstract states that models perform worse on dialectal versions but supplies no quantitative deltas, confidence intervals, or statistical tests; the main text should include these in the results section to allow readers to assess the magnitude and robustness of the reported gap.
  2. [Task Definitions] The description of task (iii) dialect-steering generation would benefit from an explicit example of the steering prompt and the exact metric used to score cultural appropriateness, as this task is the most open-ended of the three.
  3. [Dataset Construction] A dedicated limitations paragraph should explicitly note the coverage constraints (13 countries, 12 topics) and any potential annotator biases in the native-speaker curation process.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of our work and the recommendation for minor revision. The referee's summary accurately reflects the construction of ArabCulture-Dialogue, its coverage of 13 countries and 12 topics, and the three benchmarking tasks that demonstrate consistent performance degradation on dialectal Arabic relative to MSA.

Circularity Check

0 steps flagged

No significant circularity; purely empirical benchmarking

full rationale

The manuscript introduces the ArabCulture-Dialogue dataset via native-speaker curation across 13 countries and 12 topics, then applies it to three explicitly defined tasks (multiple-choice reasoning, MSA-dialect translation, dialect-steering generation). The central claim is a direct empirical comparison of LLM performance on MSA versus dialectal versions of the held-out data. No equations, fitted parameters, predictions, or self-citations are used to derive the reported gap; the results rest on standard evaluation protocols that remain externally falsifiable by replication on the released dataset.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 1 invented entities

The central claim rests on the representativeness and cultural grounding of the newly introduced ArabCulture-Dialogue dataset; no free parameters, mathematical axioms, or invented physical entities are involved.

invented entities (1)
  • ArabCulture-Dialogue dataset no independent evidence
    purpose: Provide culturally grounded conversational examples in MSA and dialects for LLM benchmarking
    Newly constructed for this paper; no independent prior evidence cited in abstract.

pith-pipeline@v0.9.0 · 5519 in / 1137 out tokens · 55734 ms · 2026-05-09T20:35:55.862251+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 28 canonical work pages · 3 internal anchors

  1. [1]

    A Lexical Distance Study of

    Kathrein Abu Kwaik and Motaz Saad and Stergios Chatzikyriakidis and Simon Dobnik , keywords =. A Lexical Distance Study of. Procedia Computer Science , volume =. 2018 , note =. doi:https://doi.org/10.1016/j.procs.2018.10.456 , url =

    work page doi:10.1016/j.procs.2018.10.456 2018

  2. [2]

    Proceedings of the Sixth Arabic Natural Language Processing Workshop , month = apr, year =

    Abdelali, Ahmed and Mubarak, Hamdy and Samih, Younes and Hassan, Sabit and Darwish, Kareem , editor =. Proceedings of the Sixth Arabic Natural Language Processing Workshop , month = apr, year =

  3. [3]

    Proceedings of the Sixth Arabic Natural Language Processing Workshop , month = apr, year =

    Abdul-Mageed, Muhammad and Zhang, Chiyu and Elmadany, AbdelRahim and Bouamor, Houda and Habash, Nizar , editor =. Proceedings of the Sixth Arabic Natural Language Processing Workshop , month = apr, year =

  4. [4]

    Alnumay, Yazeed and Barbet, Alexandre and Bialas, Anna and Darling, William and Desai, Shaan and Devassy, Joan and Duffy, Kyle and Howe, Stephanie and Lasche, Olivia and Lee, Justin and Shrinivason, Anirudh and Tracey, Jennifer , editor =. Command. Proceedings of the Sixth Workshop on African Natural Language Processing (AfricaNLP 2025) , month = jul, yea...

    work page doi:10.18653/v1/2025.africanlp-1.17 2025

  5. [5]

    and Nacar, Omer and Nagoudi, El Moatez Billah and Abdel-Salam, Reem and Atwany, Hanin and Nafea, Youssef and Yahya, Abdulfattah Mohammed and Alhamouri, Rahaf and Alsayadi, Hamzah A

    Alwajih, Fakhraddin and El Mekki, Abdellah and Magdy, Samar Mohamed and Elmadany, AbdelRahim A. and Nacar, Omer and Nagoudi, El Moatez Billah and Abdel-Salam, Reem and Atwany, Hanin and Nafea, Youssef and Yahya, Abdulfattah Mohammed and Alhamouri, Rahaf and Alsayadi, Hamzah A. and Zayed, Hiba and Shatnawi, Sara and Sibaee, Serry and Ech-chammakhy, Yasir a...

    work page doi:10.18653/v1/2025.acl-long.1579 2025

  6. [6]

    Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks , month = nov, year =

    Alwajih, Fakhraddin and El Mekki, Abdellah and Mubarak, Hamdy and Hawasly, Majd and Mohamed, Abubakr and Abdul-Mageed, Muhammad , editor =. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks , month = nov, year =. doi:10.18653/v1/2025.arabicnlp-sharedtasks.107 , pages =

    work page doi:10.18653/v1/2025.arabicnlp-sharedtasks.107 2025

  7. [7]

    Peacock: A Family of

    Alwajih, Fakhraddin and Nagoudi, El Moatez Billah and Bhatia, Gagan and Mohamed, Abdelrahman and Abdul-Mageed, Muhammad , editor =. Peacock: A Family of. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , month = aug, year =. doi:10.18653/v1/2024.acl-long.689 , pages =

    work page doi:10.18653/v1/2024.acl-long.689 2024

  8. [8]

    2025 , publisher =

    Ayash, Lama and Alhuzali, Hassan and Alasmari, Ashwag and Aloufi, Sultan , journal =. 2025 , publisher =

    2025

  9. [9]

    Bari, M Saiful and Alnumay, Yazeed and Alzahrani, Norah and Alotaibi, Nouf and Alyahya, Hisham and AlRashed, AlRashed and Mirza, Faisal and Alsubaie, Shaykhah and Alahmed, Hassan and Alabduljabbar, Ghadah and Alkhathran, Raghad and Almushayqih, Yousef and Alnajim, Raneem and Alsubaihi, Salman I and Al Mansour, Maryam and Hassan, Saad and Alrubaian, Majed ...

  10. [10]

    Hunzalah Hassan Bhatti and Firoj Alam , year =. Beyond. 2510.24328 , archiveprefix =

    work page internal anchor Pith review Pith/arXiv arXiv

  11. [11]

    Demographic Dialectal Variation in Social Media: A Case Study of A frican- A merican E nglish

    Blodgett, Su Lin and Green, Lisa and O. Demographic Dialectal Variation in Social Media: A Case Study of. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , month = nov, year =. doi:10.18653/v1/D16-1120 , pages =

    work page doi:10.18653/v1/d16-1120 2016

  12. [12]

    A Multidialectal Parallel Corpus of

    Bouamor, Houda and Habash, Nizar and Oflazer, Kemal , editor =. A Multidialectal Parallel Corpus of. Proceedings of the Ninth International Conference on Language Resources and Evaluation (. 2014 , address =

    2014

  13. [13]

    Bouamor, Houda and Habash, Nizar and Salameh, Mohammad and Zaghouani, Wajdi and Rambow, Owen and Abdulrahim, Dana and Obeid, Ossama and Khalifa, Salam and Eryani, Fadhl and Erdmann, Alexander and Oflazer, Kemal , editor =. The. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (. 2018 , address =

    2018

  14. [14]

    Assessing Cross-Cultural Alignment between C hat GPT and Human Societies: An Empirical Study

    Cao, Yong and Zhou, Li and Lee, Seolhwa and Cabello, Laura and Chen, Min and Hershcovich, Daniel , editor =. Assessing Cross-Cultural Alignment between. Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) , month = may, year =. doi:10.18653/v1/2023.c3nlp-1.7 , pages =

    work page doi:10.18653/v1/2023.c3nlp-1.7 2023

  15. [15]

    Findings of the Association for Computational Linguistics: EACL 2024 , month = mar, year =

    Bridging Cultural Nuances in Dialogue Agents through Cultural Value Surveys , author =. Findings of the Association for Computational Linguistics: EACL 2024 , month = mar, year =. doi:10.18653/v1/2024.findings-eacl.63 , pages =

    work page doi:10.18653/v1/2024.findings-eacl.63 2024

  16. [16]

    BERT: Pre- training of Deep Bidirectional Transformers for Language Understanding

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , editor =. Proceedings of the 2019 Conference of the North. 2019 , address =. doi:10.18653/v1/N19-1423 , pages =

    work page doi:10.18653/v1/n19-1423 2019

  17. [17]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , month = nov, year =

    El Mekki, Abdellah and Atou, Houdaifa and Nacar, Omer and Shehata, Shady and Abdul-Mageed, Muhammad , editor =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , month = nov, year =. doi:10.18653/v1/2025.emnlp-main.556 , pages =

    work page doi:10.18653/v1/2025.emnlp-main.556 2025

  18. [18]

    2025 , eprint =

    Abbas, Ummar and Ahmad, Mohammad Shahmeer and Alam, Firoj and Altinisik, Enes and Asgari, Ehsannedin and Boshmaf, Yazan and Boughorbel, Sabri and Chawla, Sanjay and Chowdhury, Shammur and Dalvi, Fahim and Darwish, Kareem and Durrani, Nadir and Elfeky, Mohamed and Elmagarmid, Ahmed and Eltabakh, Mohamed and Fatehkia, Masoomali and Fragkopoulos, Anastasios ...

    2025

  19. [19]

    2024 , url =

    ArXiv preprint , volume =. 2024 , url =

    2024

  20. [20]

    Aaron Grattafiori and Abhimanyu Dubey and Abhinav Jauhri and Abhinav Pandey and Abhishek Kadian and Ahmad Al-Dahle and Aiesha Letman and Akhil Mathur and Alan Schelten and Alex Vaughan and Amy Yang and Angela Fan and Anirudh Goyal and Anthony Hartshorn and Aobo Yang and Archi Mitra and Archie Sravankumar and Artem Korenev and Arthur Hinsvark and Arun Rao ...

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    , year =

    Habash, Nizar Y. , year =. Introduction to

  22. [22]

    Hala Technical Report Building

    Hammoud, Hasan Abed Al Kader and Zbib, Mohamad Bilal and Ghanem, Bernard , editor =. Hala Technical Report Building. Proceedings of the 2nd Workshop on. 2026 , address =. doi:10.18653/v1/2026.abjadnlp-1.32 , pages =

    work page doi:10.18653/v1/2026.abjadnlp-1.32 2026

  23. [23]

    Challenges and Strategies in Cross-Cultural

    Hershcovich, Daniel and Frank, Stella and Lent, Heather and de Lhoneux, Miryam and Abdou, Mostafa and Brandl, Stephanie and Bugliarello, Emanuele and Cabello Piqueras, Laura and Chalkidis, Ilias and Cui, Ruixiang and Fierro, Constanza and Margatina, Katerina and Rust, Phillip and S. Challenges and Strategies in Cross-Cultural. Proceedings of the 60th Annu...

    work page doi:10.18653/v1/2022.acl-long.482 2022

  24. [24]

    R., Jurafsky, D., and King, S

    Valentin Hofmann and Pratyusha Ria Kalluri and Dan Jurafsky and Sharese King , year =. Dialect prejudice predicts. 2403.00742 , archiveprefix =

    work page arXiv

  25. [25]

    Clive Holes , booktitle =. The. 2006 , address =

    2006

  26. [26]

    Demographic Factors Improve Classification Performance , author =. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , month = jul, year =. doi:10.3115/v1/P15-1073 , pages =

    work page doi:10.3115/v1/p15-1073

  27. [27]

    2024 , url =

    Jais Family Model Card , author =. 2024 , url =

    2024

  28. [28]

    Incorporating Dialectal Variability for Socially Equitable Language Identification

    Incorporating Dialectal Variability for Socially Equitable Language Identification , author =. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , month = jul, year =. doi:10.18653/v1/P17-2009 , pages =

    work page doi:10.18653/v1/p17-2009 2009

  29. [29]

    Findings of the

    Kadaoui, Karima and Atwany, Hanin and Al-Ali, Hamdan and Mohamed, Abdelrahman and Mekky, Ali and Tilga, Sergei and Fedorova, Natalia and Artemova, Ekaterina and Aldarmaki, Hanan and Kementchedjhieva, Yova , editor =. Findings of the. 2026 , address =. doi:10.18653/v1/2026.findings-eacl.18 , pages =

    work page doi:10.18653/v1/2026.findings-eacl.18 2026

  30. [30]

    Mohammed Kharma, Soohyeon Choi, Mohammed AlKhanafseh, and David Mohaisen

    Kargaran, Amir Hossein and Imani, Ayyoob and Yvon, Fran. Findings of the Association for Computational Linguistics: EMNLP 2023 , month = dec, year =. doi:10.18653/v1/2023.findings-emnlp.410 , pages =

    work page doi:10.18653/v1/2023.findings-emnlp.410 2023

  31. [31]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , month = dec, year =

    Keleg, Amr and Goldwater, Sharon and Magdy, Walid , editor =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , month = dec, year =. doi:10.18653/v1/2023.emnlp-main.655 , pages =

    work page doi:10.18653/v1/2023.emnlp-main.655 2023

  32. [32]

    Proceedings of ArabicNLP 2023 , month = dec, year =

    Keleg, Amr and Magdy, Walid , editor =. Proceedings of ArabicNLP 2023 , month = dec, year =. doi:10.18653/v1/2023.arabicnlp-1.31 , pages =

    work page doi:10.18653/v1/2023.arabicnlp-1.31 2023

  33. [33]

    2024 , address =

    Koto, Fajri and Mahendra, Rahmad and Aisyah, Nurul and Baldwin, Timothy , journal =. 2024 , address =. doi:10.1162/tacl_a_00726 , pages =

    work page doi:10.1162/tacl_a_00726 2024

  34. [34]

    Magdy, Samar Mohamed and Kwon, Sang Yun and Alwajih, Fakhraddin and Abdelfadil, Safaa Taher and Shehata, Shady and Abdul-Mageed, Muhammad , editor =. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , month = apr, year =. doi:10.1...

    work page doi:10.18653/v1/2025.naacl-long.613 2025

  35. [35]

    Arid and Hasanain, Maram and Kabbani, Tameem and Dalvi, Fahim and Chowdhury, Shammur Absar and Alam, Firoj , editor =

    Mousi, Basel and Durrani, Nadir and Ahmad, Fatema and Hasan, Md. Arid and Hasanain, Maram and Kabbani, Tameem and Dalvi, Fahim and Chowdhury, Shammur Absar and Alam, Firoj , editor =. Proceedings of the 31st International Conference on Computational Linguistics , month = jan, year =

  36. [36]

    B leu: a method for automatic evaluation of machine translation

    Papineni, Kishore and Roukos, Salim and Ward, Todd and Zhu, Wei-Jing , editor =. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , month = jul, year =. doi:10.3115/1073083.1073135 , pages =

    work page doi:10.3115/1073083.1073135

  37. [37]

    Qwen3 Technical Report

    Qwen3 technical report , author =. arXiv preprint arXiv:2505.09388 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  38. [38]

    Commonsense Reasoning in

    Sadallah, Abdelrahman and Tonga, Junior Cedric and Almubarak, Khalid and Almheiri, Saeed and Atif, Farah and Qwaider, Chatrine and Kadaoui, Karima and Shatnawi, Sara and Alesh, Yaser and Koto, Fajri , editor =. Commonsense Reasoning in. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , month ...

    work page doi:10.18653/v1/2025.acl-long.380 2025

  39. [39]

    arXiv preprint arXiv:2308.16149

    Sengupta, Neha and Sahu, Sunil Kumar and Jia, Bokang and Katipomu, Satheesh and Li, Haonan and Koto, Fajri and Marshall, William and Gosal, Gurpreet and Liu, Cynthia and Chen, Zhiming and Afzal, Osama Mohammed and Kamboj, Samta and Pandit, Onkar and Pal, Rahul and Pradhan, Lalit and Mujahid, Zain Muhammad and Baali, Massa and Han, Xudong and Bsharat, Sond...

    work page arXiv

  40. [40]

    2025 , howpublished =

    World. 2025 , howpublished =

    2025

  41. [41]

    and Callison-Burch, Chris , journal =

    Zaidan, Omar F. and Callison-Burch, Chris , journal =. 2014 , address =. doi:10.1162/COLI_a_00169 , pages =

    work page doi:10.1162/coli_a_00169 2014