Pith. sign in

REVIEW 2 major objections 1 minor 1 cited by

Morphological Irregularity Correlates with Frequency

T0 review · 2 major / 1 minor · reviewed 2026-05-25 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Analyses of 28 languages show higher-frequency words are more likely to be morphologically irregular.

desk verdict Broad multi-language correlation between frequency and neural-measured irregularity, but the proxy may inherit frequency biases from training. read the letter →

arxiv 1906.11483 v1 pith:4UKLVDVC submitted 2019-06-27 cs.CL

Shijie Wu , Ryan Cotterell , Timothy J. O'Donnell This is my paper · ORCID
classification cs.CL
keywords morphologicalirregularitywordfrequencyneuraltransductionmodelinformation-theoreticmeasurelinguisticparadigmscross-linguisticanalysismorphology
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper defines morphological irregularity through an information-theoretic measure of how predictable a word form is given others in its paradigm. Using a neural transduction model, it estimates this quantity across forms in 28 languages and conducts analyses that reveal a correlation with frequency. Higher-frequency items tend to be irregular, and irregular items tend to be high-frequency. The pattern strengthens when measured at the level of entire paradigms rather than isolated forms. A sympathetic reader would care because the result supplies broad empirical backing for longstanding linguistic ideas about how usage shapes structure and how abstract stems organize inflected words.

What carries the argument

An information-theoretic measure of irregularity based on the predictability of forms, estimated by a neural transduction model.

What would settle it

A new cross-linguistic dataset in which linguist-assigned irregularity ratings show no correlation with the model's scores, or in which frequency and irregularity are uncorrelated.

Watch

Extended reading notes

Core claim

The central claim is that morphological irregularity correlates with frequency: higher frequency items are more likely to be irregular and irregular items are more likely to be highly frequent. This holds across the 28 languages examined, is more robust when forms are grouped into whole paradigms, and supplies the first confirmation of this breadth for proposals from the linguistics literature.

Load-bearing premise

The neural model's estimate of irregularity matches the linguistic notion of irregularity instead of reflecting model-specific artifacts.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper defines an information-theoretic measure of morphological irregularity based on predictability from a neural transduction model, estimates it for forms across 28 languages, performs validatory and exploratory analyses, and reports a correlation with frequency: higher-frequency items tend to be more irregular and irregular items tend to be more frequent. The correlation strengthens when aggregated over paradigms, which the authors interpret as support for abstract stem/lexeme representations. Code is released.

Significance. If the central correlation survives controls for frequency-dependent artifacts in the neural model, the result would supply the broadest cross-linguistic empirical support yet for classic linguistic claims linking irregularity and frequency. The paradigm-level finding and the public code release are clear strengths that aid reproducibility and allow direct testing of the measure.

major comments (2)
  1. [Methods (neural model training and irregularity estimation)] Methods (neural transduction model and irregularity estimation): because the model is trained on the same frequency distribution later used for the correlation, any frequency-dependent optimization (better memorization or lower cross-entropy on high-count items) can directly affect the estimated irregularity score. Without frequency-balanced training, frequency-stratified held-out evaluation, or external validation against hand-labeled regular/irregular classes, the reported correlation risks being partly mechanical rather than linguistic.
  2. [Results (correlation analyses)] Results (paradigm-level aggregation): the claim that the correlation is 'more robust when aggregated at the level of whole paradigms' is central to the linguistic interpretation, yet the manuscript provides no explicit definition of how paradigms are constructed, no statistical comparison of form-level vs. paradigm-level effect sizes, and no error analysis showing that the improvement is not driven by a few high-frequency irregular paradigms.
minor comments (1)
  1. [Abstract] Abstract: 'irregular items are more likely be highly frequent' contains a missing 'to'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major point below, providing the strongest honest defense of the manuscript while acknowledging where revisions are warranted.

read point-by-point responses

  1. Referee: Methods (neural transduction model and irregularity estimation): because the model is trained on the same frequency distribution later used for the correlation, any frequency-dependent optimization (better memorization or lower cross-entropy on high-count items) can directly affect the estimated irregularity score. Without frequency-balanced training, frequency-stratified held-out evaluation, or external validation against hand-labeled regular/irregular classes, the reported correlation risks being partly mechanical rather than linguistic.

    Authors: We acknowledge the potential for frequency-dependent effects in model training. However, any such bias would improve predictability (reduce estimated irregularity) for high-frequency items, which works directly against the observed positive correlation between frequency and irregularity. The reported result is therefore conservative with respect to this artifact. The manuscript already includes validatory analyses comparing the measure against known morphological patterns in several languages; we will add an explicit discussion of this point in the revision. revision: partial

  2. Referee: Results (paradigm-level aggregation): the claim that the correlation is 'more robust when aggregated at the level of whole paradigms' is central to the linguistic interpretation, yet the manuscript provides no explicit definition of how paradigms are constructed, no statistical comparison of form-level vs. paradigm-level effect sizes, and no error analysis showing that the improvement is not driven by a few high-frequency irregular paradigms.

    Authors: We agree these details should be clarified. In the revised manuscript we will (i) explicitly define paradigm construction (forms grouped by shared lemma), (ii) report a direct statistical comparison of effect sizes between form-level and paradigm-level analyses, and (iii) include a robustness check or error analysis confirming the stronger paradigm-level correlation is not driven by a small number of high-frequency paradigms. These additions will be straightforward to implement from the existing data and code. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical correlation with externally estimated irregularity measure

full rationale

The paper defines irregularity via an information-theoretic predictability measure estimated by a neural transduction model, then correlates the resulting scores with frequency counts across 28 languages. This chain does not reduce the reported correlation to a self-definition, a fitted parameter renamed as a prediction, or a load-bearing self-citation; the model supplies an independent proxy for predictability that is not constructed from the frequency variable itself. No uniqueness theorems, ansatzes smuggled via citation, or renamings of known results appear in the derivation. The study is therefore self-contained against external benchmarks.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

Based solely on abstract; the central claim rests on the assumption that neural-model predictability is a valid proxy for linguistic irregularity and that the 28-language sample is representative. No free parameters or invented entities are named in the abstract.

assumptions (1)
  • domain assumption Neural transduction model output probabilities provide a faithful estimate of morphological predictability.
    Invoked when defining the information-theoretic irregularity measure from model predictions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Morphological Irregularity Correlates with Frequency." pith.science (2026). https://pith.science/paper/4UKLVDVC

@misc{pith2026190611483,
  author       = {Pith},
  title        = {Pith review of: Morphological Irregularity Correlates with Frequency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4UKLVDVC}},
  note         = {Machine review of arXiv:1906.11483}
}
read the original abstract

We present a study of morphological irregularity. Following recent work, we define an information-theoretic measure of irregularity based on the predictability of forms in a language. Using a neural transduction model, we estimate this quantity for the forms in 28 languages. We first present several validatory and exploratory analyses of irregularity. We then show that our analyses provide evidence for a correlation between irregularity and frequency: higher frequency items are more likely to be irregular and irregular items are more likely be highly frequent. To our knowledge, this result is the first of its breadth and confirms longstanding proposals from the linguistics literature. The correlation is more robust when aggregated at the level of whole paradigms--providing support for models of linguistic structure in which inflected forms are unified by abstract underlying stems or lexemes. Code is available at https://github.com/shijie-wu/neural-transducer.

Figures

Figures reproduced from arXiv: 1906.11483 by the authors.

Figure 1
Figure 1. Lemma paradigm tree 4 Modeling Morphological Inflection Our goal is to estimate P(w | `, σ,L−`) from data. We do this by using a structured probabilistic model of string transduction which we call pθ. In the following sections, we describe this model, how we handle syncretism in the model, our training (holdout and test) scheme, and our estimates of the degree of irregularity ι. 4.1 A Lemma-Based Model In linguistic… view at source ↗
Figure 2
Figure 2. Average degree of irregularity ι across lan￾guages. semi-regular patterns of inflection. Our approach however makes this impossible by strictly assign￾ing all forms from each lexeme to either train or test. It is important to ask, therefore, how well does our model predict the forms of heldout lexemes given this stricture? The results are displayed in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Correlations between irregularity and fre [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Correlations between irregularity and fre [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Agent-based models for the evolution of morphological alternation patterns

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    Multi-agent simulations with naturalistic lexicons and phonological rules show scale-free networks and Bernoulli adoption produce more plausible morphologies, evaluated by an LLM historical linguist debate system and ...

Reference graph

Works this paper leans on

41 extracted references · 41 canonical work pages · cited by 1 Pith paper

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

    work page

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

    work page

  3. [3]

    Farrell Ackerman and Robert Malouf. 2013. Morphological organization: T he low conditional entropy conjecture. Language, 89(3):429--464

    work page 2013

  4. [4]

    Adam Albright and Bruce Hayes. 2003. Rules vs. analogy in E nglish past tenses: A computational/experimental study. Cognition, 90(2):119--161

    work page 2003

  5. [5]

    Harald Baayen

    R. Harald Baayen. 2001. Word Frequency Distributions. Springer, Berlin, Germany

    work page 2001

  6. [6]

    Matthew Baerman, Dunstan Brown, and Greville G. Corbett. 2015. Understanding and measuring morphological complexity: An introduction. Oxford University Press

    work page 2015

  7. [7]

    Corbett, and D

    Matthew Baerman, Greville G. Corbett, and D. P. Brown. 2010. Defective Paradigms: Missing forms and what they tell us. Oxford University Press, Oxford, England

    work page 2010

  8. [8]

    Jean Berko. 1958. The child's learning of E nglish morphology. Word, 14:150--177

    work page 1958

Show all 41 references
  1. [9]

    Joan L. Bybee. 1985. Morphology: A Study of the Relation between Meaning and Form . John Benjamins, Amsterdam

    1985

  2. [10]

    Joan L. Bybee. 1991. Natural morphology: T he organization of paradigms and language acquisition. In Thom Huebner and Charles A. Ferguson, editors, Cross Currents in Second Language Acquisition and Linguistic Theory. John Benjamins Publishing Company

    1991

  3. [11]

    Chitashvili and R

    Revas J. Chitashvili and R. Harald Baayen. 1993. Word frequency distributions. Quantitative Text Analysis, pages 54--135

    1993

  4. [12]

    Ryan Cotterell, Christo Kirov, Mans Hulden, and Jason Eisner. 2018 a . On the complexity and typology of inflectional morphological systems. Transaction of the Association for Computational Linguistics ( TACL )

    2018

  5. [13]

    Mielke, and Jason Eisner

    Ryan Cotterell, Christo Kirov, Sebastian J. Mielke, and Jason Eisner. 2018 b . https://doi.org/10.18653/v1/N18-2087 Unsupervised disambiguation of syncretism in inflected lexicons . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Comp...

    2018 doi

  6. [14]

    e raldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra K \"u bler, David Yarowsky, Jason Eisner, and Mans Hulden

    Ryan Cotterell, Christo Kirov, John Sylak-Glassman, G \. e raldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra K \"u bler, David Yarowsky, Jason Eisner, and Mans Hulden. 2017 a . https://doi.org/10.18653/v1/K17-2001 CoNLL-SIGMORPHON 2017 shared task: U niv...

    2017 doi

  7. [15]

    Ryan Cotterell, John Sylak-Glassman, and Christo Kirov. 2017 b . Neural graphical models over strings for principal parts morphological paradigm completion. In Proceedings of the 15th Conference of the E uropean Chapter of the Association for Computational Linguistics ( EACL2017 )

    2017

  8. [16]

    Viviana Fratini, Joana Acha, and Itziar Laka. 2014. Frequency and morphological irregularity are independent variables. E vidence from a corpus study of S panish verbs. Corpus Linguistics and Linguistic Theory, 10(2):289 --314

    2014

  9. [17]

    Andrew Gelman and Jennifer Hill. 2007. Data Analysis using Regression and Multilevel/Hierarchical Models. Cambridge University Press, Cambridge

    2007

  10. [18]

    Martin Haspelmath and Andrea D. Sims. 2010. Understanding Morphology. Hodder Education

    2010

  11. [19]

    Jennifer Hay. 2003. Causes and Consequences of Word Structure. Routledge, New York, NY

    2003

  12. [20]

    Borja Herce. 2016. Why frequency and morphological irregularity are not independent variables in S panish: A response to F ratini et al. (2014). Corpus Linguistics and Linguistic Theory, 12(2)

    2016

  13. [21]

    Charles F. Hockett. 1954. Two models of grammatical description. Word, 10:210--231

    1954

  14. [22]

    Ferenc Kiefer. 2000. Regularity. In Morphologie: Ein internationales Handbuch zur Flexion und Wortbildung/Morphology: An international Handbook on Inflection and Word-Formation. Walter d e Gruyter, Berlin

    2000

  15. [23]

    Christo Kirov, Ryan Cotterell, John Sylak-Glassman, G \'e raldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sebastian Mielke, Arya D McCarthy, Sandra K \"u bler, et al. 2018. Unimorph 2.0: U niversal morphology. arXiv preprint arXiv:1810.11101

    2018

  16. [24]

    Kyle Mahowald, Isabelle Dautriche, Edward Gibson, and Steven Thomas Piantadosi. 2018. Word forms are structured for efficient use. Cognitive Science, 42(8):3116--3134

    2018

  17. [25]

    Marcus, Steven Pinker, Michael T

    Gary F. Marcus, Steven Pinker, Michael T. Ullman, Michelle Hollander, T. John Rosen, and Fei Xu. 1992. Overregularization in Language Acquisition. Monographs of the society for research in child development. University of Chicago Press, Chicago, IL

    1992

  18. [26]

    McClelland and Karalyn Patterson

    James L. McClelland and Karalyn Patterson. 2002 a . Rules or connections in past-tense inflections: W hat does the evidence rule out? Trends in Cognitive Sciences, 6(11):465--472

    2002

  19. [27]

    McClelland and Karalyn Patterson

    James L. McClelland and Karalyn Patterson. 2002 b . ` W ords or R ules' cannot exploit the regularity in exceptions. Trends in Cognitive Sciences, 6(11):464--465

    2002

  20. [28]

    O'Donnell

    Timothy J. O'Donnell. 2015. Productivity and Reuse in Language: A Theory of Linguistic Computation and Storage . The MIT Press, Cambridge, Massachusetts

    2015

  21. [29]

    Steven Pinker. 1999. Words and Rules. HarperCollins, New York, NY

    1999

  22. [30]

    Steven Pinker and Alan Prince. 1988. On language and connectionism: A nalysis of a parallel distributed processing model of language acquisition. Cognition, 28:73--193

    1988

  23. [31]

    Steven Pinker and Michael T. Ullman. 2002 a . Combination and structure, not gradedness, is the issue. Trends in Cognitive Sciences, 6(11):472--474

    2002

  24. [32]

    Steven Pinker and Michael T. Ullman. 2002 b . The past and future of the past tense debate. Trends in Cognitive Sciences, 6(11):456--463

    2002

  25. [33]

    Sandeep Prasada and Steven Pinker. 1993. Generalisation of regular and irregular morphological patterns. Language and Cognitive Processes, 8(1):1--56

    1993

  26. [34]

    Lawrence R. Rabiner. 1989. A tutorial on hidden M arkov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257--286

    1989

  27. [35]

    Rumelhart and James L

    David E. Rumelhart and James L. McClelland. 1986. On learning the past tenses of E nglish verbs. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition., volume 2, pages 216--271. Bradford Books/MIT Press, Cambridge, MA

    1986

  28. [36]

    Dorothy Siegel. 1974. Topics in English Morphology. Ph.D. thesis, Massachusetts Institute of Technology

    1974

  29. [37]

    Thomas Stolz, Hitomi Otsuka, Aina Urdze, and Johan v an d er Auwera. 2012. Introduction: I rregularity --- glimpses of a ubiquitous phenomenon. In Thomas Stolz, Hitomi Otsuka, Aina Urdze, and Johan v an d er Auwera, editors, Irregularity in Morphology (and Beyond), pages 7--38...

    2012

  30. [38]

    Gregory T. Stump. 2001. Inflection. In Handbook of Morphology. Blackwell, Oxford, England

    2001

  31. [39]

    Shijie Wu and Ryan Cotterell. 2019. Exact hard monotonic attention for character-level transduction. arXiv preprint arXiv:1905.06319

    2019

  32. [40]

    Charles D. Yang. 2002. Knowledge and Learning in Natural Language. Oxford linguistics. Oxford University Press, New York

    2002

  33. [41]

    Charles D. Yang. 2016. The Price of Productivity: H ow Children Learn to Break the Rules of Language . The MIT Press, Cambridge, Massachusetts

    2016

Pith tools

Reviewed May 25, 2026 · model on record in the stance chip above.