\textit{Versteasch du mi?} Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language

John McCrae; Sina Ahmadi; Verena Platzgummer

REVIEW 5 major objections 6 minor 5 references

\textit{Versteasch du mi?} Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language

T0 review · 5 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLMs systematically tax and exclude non-standard dialects, and closing the gap requires policy change, not just better models.

desk verdict A readable, useful synthesis on dialect/LLM inequity, but the empirical premise is asserted rather than demonstrated — treat it as a position paper, not as a result. read the letter →

arxiv 2603.28213 v2 pith:I7D5VID3 submitted 2026-03-30 cs.CL

Verena Platzgummer , John McCrae , Sina Ahmadi This is my paper · ORCID

classification cs.CL

keywords tokenizationtaxnon-standardlanguageSouthTyroleandialectKurdishvarietieslinguisticstandardizationdigitaldivideLLMfairnesspolicy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that large language models do not merely under-serve non-standard language varieties—they actively penalize them. Through the cases of South Tyrolean dialects and Kurdish varieties, it argues that tokenizer fragmentation, missing ISO codes, and benchmark gaps combine with market forces and historical language policy to push dialect speakers into paying more and getting worse results. The authors contend that closing the 'dialect gap' therefore requires policy change, community data sovereignty, and new evaluation frameworks, not just technical fixes. A sympathetic reader would care because it reframes an apparently technical problem as a question of linguistic justice and digital citizenship.

What carries the argument

The load-bearing mechanism is the tokenization tax: because BPE tokenizers split text based on frequency in a training corpus, dialect words often break into five or six low-information byte fragments, tripling or quintupling per-token cost, shrinking the usable context window, and degrading performance. Alongside it, the ISO-639 code acts as a gatekeeper that determines whether a variety is even catalogued in NLP resources, and translation-pivot benchmarks define what counts as competence in a language.

What would settle it

A controlled evaluation on South Tyrolean and Southern Kurdish, comparing task accuracy and per-token cost against Standard German across several models; if dialect performance matched the standard when token counts and cost are equalized, the claim that pipelines impose a systematic penalty would weaken.

Watch

Extended reading notes

Core claim

The paper's central claim is that the digital language divide is maintained by a complex interplay of market forces, historical state policies, and the inherent biases of standardized data pipelines. For South Tyrolean, the lack of an ISO-639 code makes the variety invisible to NLP resource catalogs and benchmarks, while its non-standard orthography and the pull of Standard German leave it under-sampled and fragmented by BPE tokenizers. For Kurdish, varieties beyond Central and Northern Kurdish—Southern Kurdish, Laki, Zazaki, Hawrami—are almost entirely absent from corpora, models, benchmarks, and translation services, a marginalization that mirrors and reinforces political hierarchies. The

Load-bearing premise

The paper assumes, without a controlled measurement, that LLMs actually fail on South Tyrolean and Kurdish varieties; its evidence is an anecdotal 'yes' from a few models, a few tokenizer examples, and resource tables that cite no per-cell sources.

Editorial extensions

If this is right

If the analysis is right, dialect speakers pay more per prompt and get shorter conversations and poorer reasoning—a direct economic and quality penalty.
Adding dialect data or fine-tuning alone will not close the gap while tokenizers remain frequency-biased and per-token pricing persists.
Requiring 'dialect gap' reporting, as the paper urges, could turn the currently invisible performance cliff into a measurable, auditable inequality.
Benchmarks built by translating English questions actively mislead: a model can score well while failing culturally grounded tasks, so local, non-translated benchmarks are needed.
The EU AI Act's non-discrimination requirement could become a legal hook: an AI that fails a South Tyrolean citizen might count as discrimination.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension would be to measure dialect competence while controlling for tokenization—for instance, comparing task success against the number of tokens consumed—since the paper predicts that even equal-accuracy cases still suffer economically under per-token pricing.
The same pipeline biases likely generalize to other dialect continua beyond South Tyrolean and Kurdish, such as Arabic, Chinese, or African varieties, because the ISO-code and benchmark arguments are largely portable.
The tokenization tax could be quantified directly: tokenizing a fixed sentence in several varieties and comparing token counts and per-token prices across vendors would turn the paper's anecdotal evidence into a metric.
If the paper's policy recommendations were adopted, an observable consequence would be a shift from web-scraped, proprietary data to consent-based, open corpora with community audit—changing who controls language data.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this is a position/review essay, not an empirical paper. The core claims about tokenization unfairness, benchmark absence, and standardization bias come from the literature the paper cites, and the new evidence is a few handpicked examples plus two resource-inventory tables. That does not mean the argument is wrong; it is a coherent and mostly fair synthesis. But the manuscript should not be read as a demonstration that LLMs actually fail on South Tyrolean or Kurdish varieties.

What is genuinely useful: bringing South Tyrolean and Kurdish into one frame works well. The missing ISO-639 code point for South Tyrolean, and its absorption into 'Bavarian' in NLP resources, is concrete and underappreciated. The Kurdish resource tables give a quick sense of which varieties are supported and which are invisible. The survey of recent dialect benchmarks (DialectBench, Global MMLU, IRLBench, VoxLex) is up to date and largely accurate as far as I can tell. For a reader new to this area, this is a good map.

The soft spots are real and load-bearing. The 'V ersteasch du mi?' check in §5 is anecdotal; several models answering yes is weak support and could even cut the other way. The tokenizer examples use Irish, not the two case varieties, so they do not directly demonstrate the claimed tax for South Tyrolean or Kurdish. Tables 1, 3, and 4 lack per-cell sources, collection dates, and methodology; 'speaker counts based on the best information in Wikipedia' is not an adequate basis. The §6.2 literature survey attributed to 'AUTHOR' is a placeholder and unverifiable as presented. These are not cosmetic issues.

The central argument still holds up as a synthesis. The structural claim — that market forces, historical state policies, and data-pipeline biases combine to maintain the digital language divide — is consistent with the cited literature. But because the paper's own evidence is thin, the conclusion should be framed as an agenda or hypothesis, not as an established finding. If controlled evaluations later show strong dialect comprehension despite tokenization costs, the marginalization claim would need serious qualification.

Who gets value: people wanting a compact introduction to the dialect-gap debate, and policy readers looking for a map of actors and levers. It deserves a serious referee only if the venue accepts position papers and the authors are asked to fix the placeholder, source the tables, and either add minimal reproducible evaluations or explicitly narrow the empirical claims. As original empirical research, it is not ready.

Referee Report

5 major / 6 minor

Summary. The paper argues that the digital language divide is sustained by market forces, historical state policies, and biases in standardized data pipelines, and that non-standard varieties such as South Tyrolean German and Kurdish are therefore marginalized in LLMs and GenAI. It combines a sociolinguistic review of standardization and language policy with a computational-linguistics discussion of tokenization, benchmark design, and resource scarcity, then applies this framework to two case studies. For South Tyrolean it notes the lack of an ISO code, sparse resources, and an anecdotal comprehension check; for Kurdish it describes the unequal institutional support and computational resources across varieties. The conclusion proposes policy measures (dialect gap reporting, CSR credits, data sovereignty, and interdisciplinary synthesis) as necessary complements to technical fixes.

Significance. If its empirical premises hold, the paper provides a useful interdisciplinary bridge between critical sociolinguistics and NLP, and its policy proposals (e.g., dialect gap reporting, community data sovereignty) are concrete and actionable. It draws on a broad and appropriate literature, clearly structures the technical and sociopolitical dimensions, and identifies gaps that are real in the research landscape. Its main weaknesses are empirical: the load-bearing claim that LLMs actually fail on the two case-study varieties rests on anecdote and unverified inventories rather than measurement, and several resource tables lack sources or methodology. There are no original computational evaluations or machine-checked proofs; the contribution is conceptual and policy-oriented. The paper is therefore better framed as a position paper than an empirical demonstration, and the empirical claims should be either substantiated or explicitly scoped down.

major comments (5)

[§5, 'Versteasch du mi?' check] The only direct behavioral evidence about South Tyrolean is the statement that 'several Generative AI models responded affirmative when we asked "V ersteasch du mi?"'. This is anecdotal: no model names, versions, dates, prompt variants, number of trials, or evaluation criteria are given. More importantly, an affirmative answer to 'Do you understand me?' indicates at least some basic comprehension, which sits uneasily with the paper's premise that LLMs marginalize the dialect. The conclusion in §7 depends on that premise. I recommend a small controlled probe (multiple dialect sentences, several models, exact model/version metadata) or an explicit rephrasing of the paper's claim from 'LLMs fail on these varieties' to 'LLMs are not evaluated or optimized for these varieties.'
[§3, tokenization examples] The tokenization-tax argument is central to the charge of algorithmic discrimination, but the examples are not reproducible and are not from the case studies. The Irish word 'ionchomharthú' is attributed to 'bert-based-uncased' without a model version, input normalization, or access date, and no token counts are reported for South Tyrolean or Kurdish. The claim that 'a speaker of Kurdish or South Tyrolean literally pays more' is supported by citations to Petrov et al. (2023) and Ahia et al. (2023) for the general phenomenon, but no case-specific measurement is shown. Please provide reproducible tokenizer comparisons (model, version, date, token sequences, and token counts for representative South Tyrolean and Kurdish phrases) or soften the cost claim accordingly.
[§6.2 and Tables 3–4] Tables 3 and 4 are empirical pillars of the Kurdish case. Table 3 reports token counts, audio hours, and model/MT support with blank cells and no per-cell sources, definitions, or access dates; Table 4 does not visibly show inclusion/exclusion marks. The 'survey of the NLP literature carried out by AUTHOR' over 'over 100 papers' is unverifiable as presented: there is no search protocol, inclusion/exclusion criteria, or inter-coder reliability. The strong statement that Southern Kurdish, Laki, Zazaki, and Hawrami are 'entirely unsupported' (Table 3) should be backed by a documented, repeatable search procedure or hedged to 'not covered by the resources we surveyed.'
[Table 1] The speaker counts are based on 'the best information in Wikipedia,' which is not adequate provenance for demographic claims, and the OPUS/VLO columns lack definitions and access dates. Because Table 1 is used to argue that South Tyrolean's lack of an ISO code hampers its visibility in NLP resources, the absence of a systematic check for South Tyrolean-specific entries in OPUS and VLO weakens the point. Please state how the numbers were collected and whether any South Tyrolean-specific data are found under the Bavarian code.
[§7, causal conclusion] The conclusion asserts that the digital language divide 'is maintained by a complex interplay of market forces, historical state policies, and the inherent biases of standardized data pipelines.' The historical and political parts are well supported by the sociolinguistic literature, but the causal force of 'is maintained by' is stronger than the evidence presented, which does not adjudicate among alternative explanations (e.g., pure economic incentives or technical path dependence). I suggest framing this as a hypothesis or analytical framework rather than an empirically established statement, unless the authors add evidence that directly tests the relative contributions of these factors.

minor comments (6)

[Abstract and author list] The author name is typeset as 'V erena Platzgummer' with an unwanted space; the title's dialect phrase is also inconsistently formatted across the text.
[§3] 'does not bare any resemblance' should be 'bear any resemblance.' Also, the abbreviation 'URLs' for Under-Resourced Languages may be confusing to readers who identify URL as Uniform Resource Locator; consider spelling out the term at first use.
[Table 1 caption] The caption contains a typo: 'Virtual Language Obvservatory' should be 'Virtual Language Observatory.' Please also state the access date for OPUS and VLO data.
[§3, VoxLect] The benchmark name is written as 'V oxLect' in the text and 'Voxlect' elsewhere; please standardize the spelling (e.g., 'VoxLect').
[§6.2, Table 3] The model 'KuBERT' appears in Table 3 but is not cited or described in the text. Also, the table legend should explain whether blank cells mean 'no data' or 'no support.'
[References] Reference entries with '1 others' (e.g., Glaznieks et al. 2018) need full author lists or at least standard 'et al.' formatting.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the paper's central claim is an interpretive synthesis with independent external support; author self-citations are background, and the weak empirical premise is a correctness risk, not a circular step.

full rationale

The paper has no fitted parameters, no equations, and no quantitative prediction that could reduce to an input by construction. The central claim—that the digital language divide is maintained by market forces, historical state policies, and data-pipeline biases (§7)—is an argumentative synthesis built on external literature (e.g., Petrov et al. 2023; Bella et al. 2023; Faisal et al. 2024; Alber et al. 2024), not on a self-referential definition. The three author self-citations (Platzgummer 2021; Ahmadi and Anastasopoulos 2023; Ahmadi et al. 2025) are background references for sociolinguistic and resource claims that are also independently documented; none is invoked as a load-bearing uniqueness theorem or ansatz. Genuine weaknesses are present but are not circularity: §5's only direct model evidence is anecdotal ('several Generative AI models responded affirmative when we asked "V ersteasch du mi?"'), §6.2's 'survey of the NLP literature carried out by AUTHOR' is unverifiable as presented, and Table 1's speaker counts are 'based on the best information in Wikipedia.' These undermine the empirical premise but do not make the derivation circular. Per the rubric, the non-load-bearing self-citations warrant only a low score, with no circular step exhibited.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented physical entities appear. The paper's load-bearing assumptions are interpretive domain assumptions imported from cited sociolinguistic theory, plus unstated proxies (benchmark absence, ISO-code absence) for real-world harm.

assumptions (3)

domain assumption Tokenization patterns observed in BERT/OpenAI tokenizers generalize to current LLMs and directly produce user-facing cost/performance harm.
Section 3 uses two handpicked tokenizer examples to motivate a general 'tokenization tax', citing Ahia et al. and Petrov et al. for the pricing link, but does not measure the tax for the two case varieties.
domain assumption Institutional standardization and colonial language policy causally shape LLM training data and benchmark coverage.
Adopted from Schneider (2022, 2024) and Migge and Schneider (2025) in Sections 2 and 4.1; the paper treats this causal link as established rather than testing it.
domain assumption Absence from ISO-639 codes and from major benchmarks is a valid proxy for digital marginalization in LLMs.
Used throughout Sections 5 and 6; the paper argues ISO-code assignment is a 'gatekeeper' and benchmark absence indicates exclusion, but does not quantify how much code absence matters when models can still process text.

how reviews work

0 comments

Cite this review

Pith. "Pith review of \textit{Versteasch du mi?} Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language." pith.science (2026). https://pith.science/paper/I7D5VID3

@misc{pith2026260328213,
  author       = {Pith},
  title        = {Pith review of: \textitVersteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I7D5VID3}},
  note         = {Machine review of arXiv:2603.28213}
}

read the original abstract

The design of Large Language Models (LLMs) and generative artificial intelligence (GenAI) has been shown to be "unfair" to less-spoken languages (Petrov et al., 2023) and to deepen the digital language divide (Bella et al., 2023). Critical sociolinguistic work has also argued that these technologies are not only made possible by prior sociohistorical processes of linguistic standardisation, often grounded in European nationalist and colonial projects (Migge and Schneider, 2025), but also exacerbate epistemologies of language as "monolithic, monolingual, syntactically standardized systems of meaning" (Schneider, 2024, p. 5). In our paper, we draw on earlier work on the intersections of technology and language policy (Kelly-Holmes, 2019) and bring our respective expertise in critical sociolinguistics and computational linguistics to bear on an interrogation of these arguments. We take two different complexes of non-standard linguistic varieties in our respective repertoires-South Tyrolean dialects, which are widely used in informal communication in South Tyrol, Italy (Alber et al., 2024), as well as varieties of Kurdish-as starting points to an interdisciplinary exploration of the intersections between GenAI and linguistic variation and standardisation. We discuss both how LLMs can be made to deal with non-standard language from a technical perspective, and whether, when or how this can contribute to "democratic and decolonial digital and machine learning strategies" (Migge and Schneider, 2025, p. 12), which has direct policy implications.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 4 linked inside Pith

[242]

Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy

Routledge. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2021. The Pile: An 800GB dataset of diverse text for lan- guage modeling.CoRR, abs/2101.00027. Cecilia Gialdini. 2023. One minority, one lan- guage? evaluating linguistic justi...

arXiv 2021
[2020]

Emily Bender

Longformer: The long-document trans- former.arXiv preprint arXiv:2004.05150. Emily Bender. 2019. The #benderrule: On naming the languages we study and why it matters.The Gradient. Ruha Benjamin. 2019.Race after technology: Abo- litionist tools for the New Jim Code. Social Forces, Medford, MA. Steven Bird. 2020. Decolonising speech and lan- guage technolog...

arXiv 2004
[2021]

InProceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 445–451, Reyk- javik, Iceland (Online)

NorDial: A preliminary corpus of writ- ten Norwegian dialect use. InProceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 445–451, Reyk- javik, Iceland (Online). Linköping University Electronic Press, Sweden. Gábor Bella, Paula Helm, Gertraud Koch, and Fausto Giunchiglia. 2023. Towards bridging the digital language divid...

arXiv 2023
[2023]

alexa learned arabic

MADLAD-400: A multilingual and document-level large audited dataset. InAd- vances in Neural Information Processing Sys- tems 36: Annual Conference on Neural Infor- mation Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Didem Leblebici and May Rostom. 2025. “alexa learned arabic”: A translanguaging and multi- modal pers...

arXiv 2023
[2025]

Language in Society, pages 1–21

Language in the age of AI technology: From human to non-human authenticity, from public governance to privatised assemblages. Language in Society, pages 1–21. Dogu Ergil. 2000. The Kurdish question in Turkey. Journal of Democracy, 11(3):122–135. Fahim Faisal, Orevaoghene Ahia, Aarohi Sri- vastava, Kabir Ahuja, David Chiang, Yulia Tsvetkov, and Antonios An...

arXiv 2000

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [242]

Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy

Routledge. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2021. The Pile: An 800GB dataset of diverse text for lan- guage modeling.CoRR, abs/2101.00027. Cecilia Gialdini. 2023. One minority, one lan- guage? evaluating linguistic justi...

arXiv 2021

[2] [2020]

Emily Bender

Longformer: The long-document trans- former.arXiv preprint arXiv:2004.05150. Emily Bender. 2019. The #benderrule: On naming the languages we study and why it matters.The Gradient. Ruha Benjamin. 2019.Race after technology: Abo- litionist tools for the New Jim Code. Social Forces, Medford, MA. Steven Bird. 2020. Decolonising speech and lan- guage technolog...

arXiv 2004

[3] [2021]

InProceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 445–451, Reyk- javik, Iceland (Online)

NorDial: A preliminary corpus of writ- ten Norwegian dialect use. InProceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 445–451, Reyk- javik, Iceland (Online). Linköping University Electronic Press, Sweden. Gábor Bella, Paula Helm, Gertraud Koch, and Fausto Giunchiglia. 2023. Towards bridging the digital language divid...

arXiv 2023

[4] [2023]

alexa learned arabic

MADLAD-400: A multilingual and document-level large audited dataset. InAd- vances in Neural Information Processing Sys- tems 36: Annual Conference on Neural Infor- mation Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Didem Leblebici and May Rostom. 2025. “alexa learned arabic”: A translanguaging and multi- modal pers...

arXiv 2023

[5] [2025]

Language in Society, pages 1–21

Language in the age of AI technology: From human to non-human authenticity, from public governance to privatised assemblages. Language in Society, pages 1–21. Dogu Ergil. 2000. The Kurdish question in Turkey. Journal of Democracy, 11(3):122–135. Fahim Faisal, Orevaoghene Ahia, Aarohi Sri- vastava, Kabir Ahuja, David Chiang, Yulia Tsvetkov, and Antonios An...

arXiv 2000