Pith. sign in

REVIEW 5 major objections 5 minor 43 references

Towards the Development of Balanced Synthetic Data for Correcting Grammatical Errors in Arabic: An Approach Based on Error Tagging Model and Synthetic Data Generating Model

T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-stage pipeline—an error tagger that predicts 26 Arabic error types from clean text and an AraT5 generator that turns clean sentences into corrupted ones—creates 30 million synthetic training pairs and, when used to train a GEC…

desk verdict A genuinely novel Arabic error-tagging and tag-conditioned synthetic data pipeline, but the headline GEC result is mislabeled and the evaluation protocol doesn't support the SOTA claim as written. read the letter →

arxiv 2502.05312 v1 pith:LCMXITNS submitted 2025-02-07 cs.CL cs.AI

Ahlam Alrehili , Areej Alhothali This is my paper · ORCID
classification cs.CLcs.AI
keywords Arabicgrammaticalerrorcorrectionsyntheticdatagenerationtaggingmulti-labelclassificationDeBERTa-v3AraT5back-translationQALB-14
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that Arabic grammatical error correction, a low-resource task, can be supplied with large-scale synthetic training data by splitting the problem into two learned steps: first predict which of 26 error types a clean sentence could plausibly contain, then have a text-to-text model rewrite the clean sentence into a version containing exactly those errors. Using a DeBERTa-v3 multi-label tagger and an AraT5 generator, the authors create 30,219,310 synthetic parallel sentence pairs from monolingual news corpora. They report that the tagger identifies error tags in clean sentences at 94.42% F1 on the combined QALB-14/15 test sets, and that a GEC model trained with their synthetic data beats the previous synthetic-data baseline and reaches 79.36 on QALB-14. If these numbers hold, the pipeline would give Arabic GEC a cheaper source of diverse training data that does not depend on manually annotated error corpora.

What carries the argument

The machinery is tag-conditioned reverse GEC. An error tagging model (DeBERTa-v3 optimized with distribution-balanced loss) converts a clean sentence into a 26-bit vector of ARETA error categories; that vector is serialized as a prefix ('grammar_error' followed by a string of 'a'/'b' positions), concatenated with the clean sentence, and fed to AraT5, which was first fine-tuned as a standard GEC model and then re-fine-tuned with inputs and outputs swapped. The tag string is what makes the corruption controllable and diverse: the generator is told which error types to inject, so rare error types can be oversampled and common ones undersampled. The resulting synthetic parallel pairs are used to train the final AraT5 GEC model, with comparisons run against the same correct sentences corrupted by the prior rule-based augmentation.

What would settle it

Annotate a random sample of the 30 million generated pairs and measure the fraction of source sentences that already contain a grammatical error before the generator corrupts them; if that fraction is substantial, the 'incorrect' half of many pairs is not a corruption of a correct sentence, and retraining on a filtered subset where sources are verified clean should change the reported QALB-14 score.

Watch

Extended reading notes

Core claim

The central claim is that error tags predicted from clean sentences alone are a sufficient steering signal for generating realistic Arabic errors. The pipeline first fine-tunes DeBERTa-v3 as a multi-label classifier over 26 ARETA error tags, using QALB-14, QALB-15, and ZAEBUC training text. It then fine-tunes AraT5 as a reverse GEC model: given a clean sentence prefixed with a 26-character tag string (each position indicating whether that error type is present), it emits a corrupted sentence. The same corpus is used to generate 30 million parallel pairs. The paper reports state-of-the-art tagger performance (94.42% F1 on combined QALB-14/15 test sets) and a QALB-14 GEC result of 79.36 (listed as F0.5 in the table and as F1 in the abstract), improving over the previous synthetic-data method when both are used to train AraT5.

Load-bearing premise

The whole pipeline assumes the monolingual source sentences are grammatically correct, but the paper states that after manual spelling and punctuation fixes, grammatical errors in the 1.5B-word and OSIAN corpora were left in place because correcting them would require human annotation.

Editorial extensions

If this is right

  • If the reported QALB-14 gain replicates, Arabic GEC can be improved without additional manually annotated parallel corpora, using monolingual news text alone.
  • The error tagger provides a reusable diagnostic tool: given any clean Arabic sentence, it predicts the types of mistakes a learner or native speaker would likely make, which could be used in writing feedback systems.
  • Because the generator is conditioned on explicit tags, the synthetic corpus can be rebalanced toward rare error types (ON, OS, MT) that currently get near-zero F1 in the tagger, potentially improving correction of those errors.
  • The released 30-million-pair dataset gives the Arabic NLP community a resource for training and evaluating GEC models beyond the small QALB-14/15 corpora.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the paper's logic is to filter the monolingual source sentences with the error tagger itself, keeping only sentences predicted to be clean, before generating corrupted pairs; this would directly address the paper's admitted problem that its news corpora already contain grammatical errors.
  • The same tag-conditioned corruption idea could transfer to other low-resource languages that have a clean-text monolingual corpus and an error-type taxonomy but no parallel GEC data.
  • The F1-versus-F0.5 inconsistency in the abstract suggests that the headline GEC number should be read as the F0.5 score until the authors clarify; the underlying comparison to the baseline is unaffected.
  • One could test whether tag diversity, not raw pair count, drives the gain by ablating the generator's tag string: corrupting the same sentences with random tags versus predicted tags.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a two-stage pipeline for generating synthetic Arabic grammatical error correction (GEC) data. First, a DeBERTa-v3-based multi-label classifier (the 'error tagging model') is trained on QALB-14, QALB-15, and ZAEBUC annotations produced by ARETA to predict which of 26 error types may appear in a sentence, using distribution-balanced loss and a tuned decision threshold. Second, an AraT5 model is fine-tuned as a tag-conditioned back-translation corrupter (the 'synthetic data generation model') to produce incorrect sentences from correct ones, conditioned on the predicted error tags. The authors use this pipeline to generate 30,219,310 synthetic sentence pairs from the 1.5-billion-word Arabic corpus and OSIAN, then fine-tune an AraT5 GEC model on the synthetic data and compare it against a GEC model trained on the synthetic data of Solyman et al. [10]. They report an error-tagging F1 of 94.42% and a QALB-14 GEC 'F1-Score' of 79.36%, which they describe as a new state of the art.

Significance. If the reported results were valid, the paper would make a useful contribution to low-resource Arabic GEC by introducing a tag-conditioned synthetic-data generation approach and releasing a large public dataset. The idea of using an error-tagging model to guide corruption is reasonable and goes beyond simple rule-based noising, and the authors provide details of several Arabic pre-trained models and their relative performance. However, the central quantitative claims are not supported by the manuscript as written: the headline metric is misstated, the error-tagging threshold is selected on the test set, the GEC comparison is not a clean synthetic-only evaluation and lacks a no-synthetic-data baseline, and the monolingual source corpus is acknowledged to contain grammatical errors that were not corrected, which threatens the validity of the 30-million-pair dataset. The paper's strengths—public data release and the general novelty of the pipeline—do not compensate for these load-bearing issues.

major comments (5)
  1. [Abstract; Table 8] The abstract states that 'we achieved a new state-of-the-art result of F1-Score: 79.36% in the QALB-14 Test set,' but Table 8 reports F1 = 75.87% and F0.5 = 79.36% for the proposed method. The headline number is an F0.5 score, not an F1 score, and F0.5 is not the standard metric used in QALB GEC evaluations. This misstatement directly affects the paper's central claim and must be corrected before the result can be assessed.
  2. [Section 4.1; Tables 5 and 6] The decision threshold for the multi-label error-tagging model is selected on both the development and test sets: the text says 'we tuned the threshold during training, validation, and testing,' and Tables 5 and 6 list 'Best F1 threshold' for the combined dev set and the combined test set. Selecting the threshold using the test set leaks test labels into model selection and inflates the reported test F1 (e.g., 0.94417 in Table 6). The threshold should be chosen on the development set only, and the test-set results should be recomputed accordingly.
  3. [Section 4.4; Table 8] The GEC evaluation does not support the claimed state-of-the-art result. The two AraT5 GEC models are first trained on synthetic data and then, according to Section 4.4, 'after 23 train epochs, we fine-tuned by using the QALB-14-L1 and QALB-15-L1 training data for two epochs,' while Table 8's caption says the model was 'trained using only synthetic data.' If the gold-data fine-tuning was applied to both arms, the comparison is controlled but the results are not synthetic-only and are not comparable to synthetic-only numbers in the literature; if it was applied only to the proposed model, the comparison is unfair. In addition, there is no no-synthetic-data AraT5 baseline and no comparison with published QALB-14 systems. The 'new state-of-the-art' claim is therefore unsupported as written.
  4. [Section 4.3] The synthetic data generation assumes that the monolingual source sentences used as the 'correct' side of each parallel pair are grammatically correct, but the paper states that both the 1.5B-word corpus and OSIAN 'contained spelling and grammatical errors' and that 'Grammatical errors were not corrected because they needed to be annotated by humans.' Any grammatical error in a source sentence therefore produces a (wrong, wrong) training pair, corrupting the 30,219,310-pair dataset. The authors need to either filter or correct the source sentences, or empirically analyze how often the corruption model outputs a different error than the source error and how this affects GEC training.
  5. [Section 6.1; Section 2.2] The paper claims that DeBERTa-v3 achieves 'state-of-the-art' results for the error-tagging task, but Section 2.2 states that no prior Arabic multi-label error-tagging model exists. A state-of-the-art claim requires comparison with existing systems for the same task; without any baseline, the claim is unsupported. If the authors mean that this is the first such model, they should say so explicitly rather than using the term 'state of the art.'
minor comments (5)
  1. [Section 5.1, Eq. (4)] Equation (4) writes the F0.5 denominator as 0.25 × (Precision + Recall), but the definition of F0.5 requires 0.25 × Precision + Recall. The correct formula is used in Eq. (8), and the reported F0.5 values in Table 8 are consistent with the correct formula, so this appears to be a typographical error.
  2. [Table 4, QALB-15-L2, CAMeLBERT-MSA row] The reported precision of 0.74039 and recall of 0.77324 do not produce an F1 score of 0.8002; the harmonic mean is approximately 0.7565. This suggests an arithmetic error in the table and should be corrected.
  3. [Table 8; Section 6.3] The metric is repeatedly written as 'BLUE-4'; the correct name is BLEU-4.
  4. [Section 4.3; Introduction] Several section cross-references are incorrect: Section 4.3 refers to 'the 5.1 subsections' and 'the 5.2 subsections,' and the Introduction's outline lists Section 4 twice. These references should be updated to the actual section numbering.
  5. [Section 6.1] The text reports AraBERTv02 precision as '8.909' and DeBERTa-v3-base precision as '8.91673,' which appear to be missing a leading decimal point; Table 5 shows values of 0.9109 and 0.91673.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation chain is grounded in external gold annotations and held-out test data, with no fitted parameter renamed as a prediction.

full rationale

The paper's main components are (1) an error-tagging multi-label classifier trained on ARETA-derived labels from QALB-14, QALB-15, and ZAEBUC, and evaluated on held-out QALB test sets, and (2) a reverse-GEC synthetic-data generator trained on QALB parallel pairs and used to create corrupted sentences from monolingual text, followed by (3) a GEC model trained on synthetic data and evaluated on QALB test sets against a Solyman et al. baseline. None of these steps reduces to its own inputs by construction: the tagger is supervised by external ARETA annotations, not by the GEC output; the generator is trained on real error-correction pairs; and the GEC comparison is controlled by using the same AraT5 model and the same fine-tuning procedure for both arms. The paper does contain reporting and attribution problems that are not circularity: the abstract's 'F1-Score: 79.36%' corresponds to F0.5 in Table 8, the caption says 'trained using only synthetic data' while Section 4.4 states both models were fine-tuned on QALB gold for two epochs, and Section 4.3 admits grammatical errors in the monolingual source were not corrected. These are correctness, metric, and data-quality concerns, not circular reductions. There is no load-bearing self-citation, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The central claims are therefore not circular, though their quantitative support should be scrutinized on reporting grounds.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no physical or formal entities. Its load-bearing assumptions are empirical: the completeness of the ARETA tag taxonomy, the grammatical correctness of the monolingual source sentences, the fidelity of AraT5 tag-conditioned corruption, and the accuracy of the gold corpora. The main free parameter is the classification threshold, which was tuned on the test set.

free parameters (4)
  • multi-label decision threshold for DeBERTa-v3 = 0.64 on combined test, 0.52 on dev
    Threshold selected by maximizing F1 on dev and test; test-set threshold tuning leaks test labels (Section 4.1, Tables 5-6).
  • learning rate for error tagging fine-tuning = 2e-5
    Reported in Section 5.2; chosen hyperparameter, no sweep or sensitivity analysis.
  • learning rate for AraT5 GEC fine-tuning = 5e-5
    Reported in Section 4.4; chosen hyperparameter.
  • training epochs for synthetic-data corruption model = 300
    Reported in Section 4.2; fixed with no validation-based stopping described.
assumptions (4)
  • domain assumption ARETA's 26 error tags, plus the authors' manual resolution of X/UNK labels, cover the full space of human-like Arabic grammar errors relevant to GEC.
    Section 4.1 uses these tags as the target space for the error tagging model and as the conditioning signal for corruption; if the taxonomy is incomplete, the generated data will miss error types.
  • domain assumption The monolingual news source sentences are acceptable 'correct' targets after the authors' cleaning.
    Section 4.3 leaves grammar errors uncorrected; if source sentences contain grammar errors, synthetic pairs are mislabeled.
  • domain assumption AraT5 conditioned on the predicted 26-character tag prefix generates corrupted sentences whose error types and difficulty match human errors.
    Sections 4.2 and 4.3; no human evaluation, no automatic error-type agreement measurement against real learner errors.
  • domain assumption QALB-14, QALB-15, and ZAEBUC gold corrections are accurate and representative of Arabic errors.
    Section 3.1, used as training/validation/test for both the tagger and GEC final fine-tuning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards the Development of Balanced Synthetic Data for Correcting Grammatical Errors in Arabic: An Approach Based on Error Tagging Model and Synthetic Data Generating Model." pith.science (2026). https://pith.science/paper/LCMXITNS

@misc{pith2026250205312,
  author       = {Pith},
  title        = {Pith review of: Towards the Development of Balanced Synthetic Data for Correcting Grammatical Errors in Arabic: An Approach Based on Error Tagging Model and Synthetic Data Generating Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LCMXITNS}},
  note         = {Machine review of arXiv:2502.05312}
}
read the original abstract

Synthetic data generation is widely recognized as a way to enhance the quality of neural grammatical error correction (GEC) systems. However, current approaches often lack diversity or are too simplistic to generate the wide range of grammatical errors made by humans, especially for low-resource languages such as Arabic. In this paper, we will develop the error tagging model and the synthetic data generation model to create a large synthetic dataset in Arabic for grammatical error correction. In the error tagging model, the correct sentence is categorized into multiple error types by using the DeBERTav3 model. Arabic Error Type Annotation tool (ARETA) is used to guide multi-label classification tasks in an error tagging model in which each sentence is classified into 26 error tags. The synthetic data generation model is a back-translation-based model that generates incorrect sentences by appending error tags before the correct sentence that was generated from the error tagging model using the ARAT5 model. In the QALB-14 and QALB-15 Test sets, the error tagging model achieved 94.42% F1, which is state-of-the-art in identifying error tags in clean sentences. As a result of our syntactic data training in grammatical error correction, we achieved a new state-of-the-art result of F1-Score: 79.36% in the QALB-14 Test set. We generate 30,219,310 synthetic sentence pairs by using a synthetic data generation model.

Figures

Figures reproduced from arXiv: 2502.05312 by the authors.

Figure 1
Figure 1. The error tagging model and the synthetic data generation model. [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Label imbalances in (QALB-14, QALB-15, and ZAEBUC). Over half of the data contain some labels, while [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Examples of data generated by our AraT5 synthetic data generation model. We use our error tagging model to [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 30 canonical work pages

  1. [10]

    Optimizing the impact of data augmentation for low-resource grammatical error correction

    Aiman Solyman, Marco Zappatore, Wang Zhenyu, Zeinab Mahmoud, Ali Alfatemi, Ashraf Osman Ibrahim, and Lubna Abdelkareim Gabralla. Optimizing the impact of data augmentation for low-resource grammatical error correction. Journal of King Saud University-Computer and Information Sciences, 35(6):101572, 2023

    work page 2023

  2. [1]

    A multilayer convolutional encoder-decoder neural network for grammati- cal error correction

    Shamil Chollampatt and Hwee Tou Ng. A multilayer convolutional encoder-decoder neural network for grammati- cal error correction. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    work page 2018

  3. [2]

    Approaching neural grammatical error correction as a low-resource machine translation task

    Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, and Kenneth Heafield. Approaching neural grammatical error correction as a low-resource machine translation task. arXiv preprint arXiv:1804.05940, 2018

    arXiv 2018

  4. [3]

    Challenges in arabic natural language processing

    Khaled Shaalan, Sanjeera Siddiqui, Manar Alkhatib, and Azza Abdel Monem. Challenges in arabic natural language processing. In Computational linguistics, speech and image processing for arabic language , pages 59–83. World Scientific, 2019

    work page 2019

  5. [4]

    The first qalb shared task on automatic text correction for arabic

    Behrang Mohit, Alla Rozovskaya, Nizar Habash, Wajdi Zaghouani, and Ossama Obeid. The first qalb shared task on automatic text correction for arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 39–47, 2014. 19 A PREPRINT - F EBRUARY 11, 2025

    work page 2014

  6. [5]

    The second qalb shared task on automatic text correction for arabic

    Alla Rozovskaya, Houda Bouamor, Nizar Habash, Wajdi Zaghouani, Ossama Obeid, and Behrang Mohit. The second qalb shared task on automatic text correction for arabic. In Proceedings of the Second workshop on Arabic natural language processing, pages 26–35, 2015

    work page 2015

  7. [6]

    Cmuq@ qalb-2014: An smt-based system for automatic arabic error correction

    Serena Jeblee, Houda Bouamor, Wajdi Zaghouani, and Kemal Oflazer. Cmuq@ qalb-2014: An smt-based system for automatic arabic error correction. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 137–142, 2014

    work page 2014

  8. [7]

    A web-based annotation framework for large-scale text correction

    Ossama Obeid, Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Kemal Oflazer, and Nadi Tomeh. A web-based annotation framework for large-scale text correction. In The Companion Volume of the Proceedings of IJCNLP 2013: System Demonstrations, pages 1–4, 2013

    work page 2013

Show all 43 references
  1. [8]

    Synthetic data with neural machine translation for automatic correction in arabic grammar

    Aiman Solyman, Wang Zhenyu, Tao Qian, Arafat Abdulgader Mohammed Elhag, Muhammad Toseef, and Zeinab Aleibeid. Synthetic data with neural machine translation for automatic correction in arabic grammar. Egyptian Informatics Journal, 22(3):303–315, 2021

    2021

  2. [9]

    Automatic arabic grammatical error correction based on expectation-maximization routing and target-bidirectional agreement

    Aiman Solyman, Zhenyu Wang, Qian Tao, Arafat Abdulgader Mohammed Elhag, Rui Zhang, and Zeinab Mahmoud. Automatic arabic grammatical error correction based on expectation-maximization routing and target-bidirectional agreement. Knowledge-Based Systems, 241:108180, 2022

    2022

  3. [11]

    Chatgpt for arabic grammatical error correction

    Sang Yun Kwon, Gagan Bhatia, El Moatez Billah Nagoud, and Muhammad Abdul-Mageed. Chatgpt for arabic grammatical error correction. arXiv preprint arXiv:2308.04492, 2023

    2023 arXiv

  4. [12]

    Automatic error type annotation for arabic

    Riadh Belkebir and Nizar Habash. Automatic error type annotation for arabic. arXiv preprint arXiv:2109.08068, 2021

    2021 arXiv

  5. [13]

    Arabic grammatical error detection using transformers-based pretrained language models

    Sarah AlOyaynaa and Yasser Kotb. Arabic grammatical error detection using transformers-based pretrained language models. In ITM Web of Conferences, volume 56, page 04009. EDP Sciences, 2023

    2023

  6. [14]

    A7’ ta: Data on a monolingual arabic parallel corpus for grammar checking

    Nora Madi and Hend S Al-Khalifa. A7’ ta: Data on a monolingual arabic parallel corpus for grammar checking. Data in brief, 22:237, 2019

    2019

  7. [15]

    Arabert: Transformer-based model for arabic language under- standing

    Wissam Antoun, Fady Baly, and Hazem Hajj. Arabert: Transformer-based model for arabic language under- standing. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9, 2020

    2020

  8. [16]

    Leveraging offensive language for sarcasm and sentiment detection in arabic

    Fatemah Husain and Ozlem Uzuner. Leveraging offensive language for sarcasm and sentiment detection in arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 364–369, 2021

    2021

  9. [17]

    Arbert & marbert: Deep bidirectional transformers for arabic

    Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. Arbert & marbert: Deep bidirectional transformers for arabic. arXiv preprint arXiv:2101.01785, 2020

    2020 arXiv

  10. [18]

    The interplay of variant, size, and task type in arabic pre-trained language models

    Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. The interplay of variant, size, and task type in arabic pre-trained language models. arXiv preprint arXiv:2103.06678, 2021

    2021 arXiv

  11. [19]

    Deberta: Decoding-enhanced bert with disentan- gled attention

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentan- gled attention. arXiv preprint arXiv:2006.03654, 2020

    2006 arXiv

  12. [20]

    Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing

    Pengcheng He, Jianfeng Gao, and Weizhu Chen. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543, 2021

    2021 arXiv

  13. [21]

    1.5 billion words arabic corpus

    Ibrahim Abu El-Khair. 1.5 billion words arabic corpus. arXiv preprint arXiv:1611.04033, 2016

    2016 arXiv

  14. [22]

    Osian: Open source international arabic news corpus-preparation and integration into the clarin-infrastructure

    Imad Zeroual, Dirk Goldhahn, Thomas Eckart, and Abdelhak Lakhouaja. Osian: Open source international arabic news corpus-preparation and integration into the clarin-infrastructure. In Proceedings of the fourth arabic natural language processing workshop, pages 175–182, 2019

    2019

  15. [23]

    A monolingual approach to contextualized word embeddings for mid-resource languages

    Pedro Javier Ortiz Suárez, Laurent Romary, and Benoît Sagot. A monolingual approach to contextualized word embeddings for mid-resource languages. arXiv preprint arXiv:2006.06202, 2020

    2006 arXiv

  16. [24]

    Asynchronous pipeline for processing huge corpora on medium to low resource infrastructures

    Pedro Javier Ortiz Suárez, Benoît Sagot, and Laurent Romary. Asynchronous pipeline for processing huge corpora on medium to low resource infrastructures. In 7th Workshop on the Challenges in the Management of Large Corpora (CMLC-7). Leibniz-Institut für Deutsche Sprache, 2019

    2019

  17. [25]

    Aligning books and movies: Towards story-like visual explanations by watching movies and reading books

    Yukun Zhu. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. arXiv preprint arXiv:1506.06724, 2015

    2015 arXiv

  18. [26]

    Openwebtext corpus, 2019

    Aaron Gokaslan, Vanya Cohen, E Pavlick, and S Tellex. Openwebtext corpus, 2019. URL http://Skylion007. github. io/OpenWebTextCorpus, page 9, 2019. 20 A PREPRINT - F EBRUARY 11, 2025

    2019

  19. [27]

    A simple method for commonsense reasoning

    Trieu H Trinh and Quoc V Le. A simple method for commonsense reasoning. arXiv preprint arXiv:1806.02847, 2018

    2018 arXiv

  20. [28]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020

    2020

  21. [29]

    mt5: A massively multilingual pre-trained text-to-text transformer

    L Xue. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934, 2020

    2010 arXiv

  22. [30]

    Byt5: Towards a token-free future with pre-trained byte-to-byte models

    Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. Byt5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306, 2022

    2022

  23. [31]

    Arat5: Text-to-text trans- formers for arabic language generation

    El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. Arat5: Text-to-text trans- formers for arabic language generation. arXiv preprint arXiv:2109.12068, 2021

    2021 arXiv

  24. [32]

    Arabic learner corpora (alc): a taxonomy of coding errors

    Abdullah Alfaifi and Eric Atwell. Arabic learner corpora (alc): a taxonomy of coding errors. In The 8th International Computing Conference in Arabic, 2012

    2012

  25. [33]

    Arabic learners written corpus: A resource for research and learning

    S Farwaneh and M Tamimi. Arabic learners written corpus: A resource for research and learning. retrieved 2 september, 2012, from the the university of arizona. the Center for Educational Resources in Culture, Language and Literacy web site: http://l2arabiccorpus. cercll. arizo...

    2012

  26. [34]

    Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop

    Nizar Habash and Owen Rambow. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of the 43rd annual meeting of the association for computational linguistics (ACL’05), pages 573–580, 2005

    2005

  27. [35]

    Mada+ tokan: A toolkit for arabic tokenization, diacritization, morphological disambiguation, pos tagging, stemming and lemmatization

    Nizar Habash, Owen Rambow, and Ryan Roth. Mada+ tokan: A toolkit for arabic tokenization, diacritization, morphological disambiguation, pos tagging, stemming and lemmatization. In Proceedings of the 2nd international conference on Arabic language resources and tools (MEDAR), C...

    2009

  28. [36]

    Zaebuc: An annotated arabic-english bilingual writer corpus

    Nizar Habash and David Palfreyman. Zaebuc: An annotated arabic-english bilingual writer corpus. InProceedings of the Thirteenth Language Resources and Evaluation Conference, pages 79–88, 2022

    2022

  29. [37]

    Balancing methods for multi-label text classification with long-tailed class distribution

    Yi Huang, Buse Giledereli, Abdullatif Köksal, Arzucan Özgür, and Elif Ozkirimli. Balancing methods for multi-label text classification with long-tailed class distribution. arXiv preprint arXiv:2109.04712, 2021

    2021 arXiv

  30. [38]

    A unified view of multi-label performance measures

    Xi-Zhu Wu and Zhi-Hua Zhou. A unified view of multi-label performance measures. In international conference on machine learning, pages 3780–3788. PMLR, 2017

    2017

  31. [39]

    An evaluation of statistical approaches to text categorization

    Yiming Yang. An evaluation of statistical approaches to text categorization. Information retrieval, 1:69–90, 1999

    1999

  32. [40]

    Multilabel classification

    Francisco Herrera, Francisco Charte, Antonio J Rivera, María J Del Jesus, Francisco Herrera, Francisco Charte, Antonio J Rivera, and María J del Jesus. Multilabel classification. Springer, 2016

    2016

  33. [41]

    Effective multi-label active learning for text classification

    Bishan Yang, Jian-Tao Sun, Tengjiao Wang, and Zheng Chen. Effective multi-label active learning for text classification. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 917–926, 2009

    2009

  34. [42]

    Better evaluation for grammatical error correction

    Daniel Dahlmeier and Hwee Tou Ng. Better evaluation for grammatical error correction. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 568–572, 2012

    2012

  35. [43]

    Transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in na...

    2020

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.