Pith. sign in

REVIEW 4 major objections 5 minor 46 references

Automated Essay Scoring Incorporating Annotations from Automated Feedback Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Automated essay scoring improves when the model sees argument-structure and error tags alongside the text.

desk verdict Component-annotation scoring on PERSUADE is a real but small effect; the paper overstates it, and Table 6 has an impossible summary row. read the letter →

arxiv 2505.22771 v2 pith:XJYZUW7H submitted 2025-05-28 cs.CL cs.AI

Christopher Ormerod This is my paper · ORCID
classification cs.CLcs.AI
keywords automatedessayscoringargumentativecomponentsgrammaticalerrorcorrectionPERSUADEcorpusModernBERTERRANTannotationbias
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tests whether feeding automated essay scoring (AES) models the same annotations that automated writing feedback systems produce can make scoring more accurate. Using the PERSUADE corpus of persuasive essays, the author augments each essay's text with XML tags marking argumentative components (lead, position, claim, evidence, and so on) and with tags marking spelling, punctuation, and grammar errors. The central claim is that these feedback-oriented annotations help an encoder-based fine-tuned language model agree better with human scores than the same model trained on plain text alone. The best configuration, component-annotated text, reaches a quadratic weighted kappa of 0.868 versus 0.860 for full text, both above the reported human inter-rater agreement of 0.745. A careful reader would care because this offers a path to scoring models that are both more accurate and more aligned with the rubric-based feedback students actually receive.

What carries the argument

The central mechanism is the XML-augmented input: the essay text is wrapped in <Lead>, <Position>, <Claim>, <Counterclaim>, <Rebuttal>, <Evidence>, <Concluding Statement> tags for argumentative components, and <Spelling>, <PunctOrth>, and <Grammar> tags for conventions errors. This markup lets a single long-context encoder model, ModernBERT, consume both the original text and the feedback-oriented structure at once. ModernBERT's rotary positional embeddings allow the scorer to process full essays without truncation, and the annotation model is the same architecture with a token-classification head. The argumentative-component tags and error tags are the load-bearing information added to the scorer's input, and the paper's premise is that these tags make rubric-relevant organization and language conventions explicit to the model.

What would settle it

Re-run the scoring experiments replacing the predicted argumentative-component tags with the corpus's gold human annotations; if the gold-annotated model does not outperform the full-text baseline (QWK 0.860), then the reported gains come from annotation artifacts rather than the argument structure itself. Separately, a human-annotated error set for a sample of PERSUADE essays would test whether the T5-ERRANT annotations used here are accurate enough to support the error-only and combined pipelines.

Watch

Extended reading notes

Core claim

The paper demonstrates, on 25,996 PERSUADE essays graded on a 1-6 holistic SAT-style rubric, that adding automated annotations to the scoring input improves agreement with human scores. A ModernBERT token classifier predicts seven argumentative component types, and a T5 grammatical-error-correction model followed by ERRANT produces spelling, punctuation, and grammar error tags; these are inserted into the essay text as XML. Averaged over ten training runs, the component-annotated model scores QWK 0.868 (range 0.867-0.870) and exact agreement 68.9%, compared with QWK 0.860 (range 0.859-0.862) and exact agreement 67.2% for the full-text baseline. Error annotations alone do not improve over full text (QWK 0.858), while combining argument and error annotations gives QWK 0.866. The author also reports standardized mean differences across demographic subgroups, finding that the combined-annotation model shows smaller negative bias for several groups, including English Language Learners, compared with the full-text model.

Load-bearing premise

The load-bearing premise is that the automated argumentative-component annotations are accurate enough on PERSUADE essays that the scoring gains reflect genuine rubric-relevant structure, and the error annotations, whose accuracy is never directly evaluated, are reliable enough not to corrupt the combined pipeline.

Editorial extensions

If this is right

  • If this holds, scoring engines can be made more accurate simply by feeding them annotations that automated writing feedback systems already generate, reusing existing infrastructure.
  • The component-annotated model beats the full-text model across all ten training runs, suggesting the gain is stable rather than a lucky seed.
  • Error annotations alone did not help, so the improvement appears to come from argumentative structure rather than from error flags.
  • XML-tagged output could let one model both score an essay and produce interpretable feedback for students, linking assessment with instruction.
  • The combined-annotation pipeline shows smaller negative bias for several demographic subgroups, hinting that annotation-informed scoring may reduce rather than exacerbate disparities.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If argument-structure tags are the source of the gain, then other argumentative writing corpora with discourse annotations should show similar improvements when scored with annotated input.
  • The error-only result may reflect annotation noise (the paper admits it has no direct accuracy measure for error tags) or it may mean conventions information is redundant with what the encoder already infers from text; a direct human-annotated error set would separate these possibilities.
  • Because the annotator model itself is trained on the same essays used for scoring, the pipeline could be evaluated as a fully automated loop where annotation quality is part of the scoring system, not a fixed external input.
  • If the bias reductions replicate, feedback-driven annotations could become a practical fairness intervention for AES, particularly for English Language Learners, though the small subgroup sizes make these SMD estimates uncertain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an automated essay scoring (AES) pipeline in which two feedback-oriented annotation streams are inserted into the input text: argumentative component tags (Lead, Position, Claim, etc.) produced by a ModernBERT token classifier, and spelling/grammar error tags produced by a T5 grammatical-error-correction model combined with ERRANT error classification. Using the PERSUADE corpus, the authors fine-tune ModernBERT scorers on full text, component-annotated text, error-annotated text, and combined-annotated text, reporting QWK, exact agreement, and SMD over ten training runs. The headline result is that component-annotated scoring achieves average QWK 0.868 (range 0.867–0.870) versus 0.860 (0.859–0.862) for full text, with all conditions above the 0.745 human baseline. The paper also reports subgroup SMDs and argues that the combined annotations mitigate some bias.

Significance. If the component-annotation result holds, it is a useful empirical contribution: it shows that a high-accuracy argumentative-component annotator can provide a small but reproducible scoring gain over a strong full-text fine-tuned transformer baseline, while also producing interpretable markup. The ten-run min/max reporting is a strength, and the non-overlapping QWK ranges for component-annotated versus full-text models make the main comparison credible. However, the broad abstract claim about 'feedback-oriented annotations' is only supported for the argumentative-component stream, not for the error-annotation stream, and the combined-condition row in Table 6 is internally inconsistent as printed. The error-annotation pipeline also lacks any direct accuracy evaluation, a limitation the paper itself acknowledges in Section 2.3.2. The contribution is therefore narrower than claimed, though the component-level finding is real and fixable.

major comments (4)
  1. [§3.2, Table 6] The Combined Annotations row is arithmetically impossible as printed: it reports average QWK 0.866 while also reporting a minimum QWK of 0.867. An average cannot be below its minimum. Because the paper's own stability argument relies on the min/max ranges, this row cannot be interpreted without the ten per-run values for QWK, exact agreement, and SMD. The combined condition is referenced in the abstract and in the bias discussion, so this inconsistency is load-bearing; it must be corrected with raw per-run data before any claim involving the combined pipeline is accepted.
  2. [Abstract; §3.2, Table 6; §2.3.2] The headline claim that 'incorporating feedback-oriented annotations into the scoring pipeline can enhance the accuracy of automated essay scoring' is not supported for the spelling/grammar error annotations. In Table 6, the Error Annotated condition (QWK 0.858, range 0.856–0.859) does not beat the Full Text baseline (0.860, range 0.859–0.862), and the ranges overlap. Section 2.3.2 explicitly concedes that 'we have no direct way of evaluating the accuracy of any annotations' for the T5-GEC/ERRANT stream, and Section 3.1 reports only 2795 spelling, 1401 grammatical, and 201 punctuation errors, noting that the pipeline 'does not seem to be uncovering as many errors as expected.' The central claim should be narrowed to argumentative-component annotations, or the error-annotation stream should be evaluated directly on PERSUADE.
  3. [§3.3, Table 7] The claim that combined annotations are 'mitigating some of the biases' rests on the Combined column of Table 7, but that condition's Table 6 row is internally inconsistent, so its SMDs are uninterpretable as printed. Moreover, even taken at face value, the Combined SMDs are larger than the Original SMDs for several subgroups (e.g., WC 0.15 vs 0.09, AP 0.52 vs 0.44). The bias discussion should be reframed as exploratory, and the primary bias comparison should be between the Full Text and Component Annotated conditions, which are both valid.
  4. [§2.3.2] The paper justifies the T5-GEC error annotations by citing benchmarks on JFLEG and BEA60k and stating that the model is comparable to ChatGPT and GPT-4. This is an indirect transfer argument, and the paper itself notes the lack of direct evaluation on PERSUADE. Because the error-only condition fails to help, the manuscript should either provide a direct evaluation of error-tag accuracy on this domain or explicitly state that the error-annotation results are inconclusive rather than supportive of the abstract's claim.
minor comments (5)
  1. [Figure 2] The label 'ModerBERT' is a typo; it should read 'ModernBERT'.
  2. [Throughout] There are several typographical slips, including 'pretaining' in Section 2.2.1, 'inter-rated reliability' in Section 2.1.2, and 'Ind..' in the Table 1 caption. These should be corrected.
  3. [§2.1.2] The parenthetical reference '(see (6))' after the reported human inter-rater reliability κ = 0.745 is misleading, because Equation (6) defines the QWK formula and the 0.745 value comes from the PERSUADE corpus paper. Please cite the source directly.
  4. [§3.2, Table 6] The paper reports no significance tests or confidence intervals. The QWK comparison between Component Annotated and Full Text is supported by non-overlapping min/max ranges, but the exact-agreement and SMD differences would benefit from a paired test or bootstrap intervals, and the error-condition comparison should not be described as a difference without such support.
  5. [§3.1, Table 5] It would be helpful to report the macro-averaged F1 over the seven component classes explicitly, since the text says it is the statistic of interest, rather than leaving the reader to compute it from the TP/FP/FN columns.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the annotation-to-scoring comparison is an empirical held-out evaluation; self-citations are background only.

full rationale

Walking the derivation chain, the claimed result is that incorporating feedback-oriented XML annotations into the essay text improves automated essay scoring on the PERSUADE corpus. The chain is empirical at every stage: (1) a ModernBERT token classifier is trained on the PERSUADE training-split argument-component annotations; (2) T5-GEC plus ERRANT is used to generate spelling/grammar error tags, whose accuracy the paper explicitly says it cannot directly evaluate (Section 2.3.2); (3) ModernBERT scoring models are fine-tuned on full text and on three annotated input variants; and (4) all are evaluated on the held-out PERSUADE test split, reporting QWK, exact agreement, and SMD in Table 6. The central improvement (Component Annotated QWK 0.868 vs Full Text QWK 0.860) is a measured test-set difference, not a quantity that reduces by construction to a fitted parameter or to the definition of the input. The annotation model is itself evaluated against the corpus ground truth in Table 5, so the self-citation to Ormerod et al. (2023) for annotation ability is supporting rather than load-bearing. Other self-citations concern bias motivation, prior AES work, or design choices and are not used to force the main result. The admitted inability to validate error annotations directly (Section 2.3.2) and the arithmetically inconsistent Combined row in Table 6 (average 0.866 with minimum 0.867) are substantive reporting and measurement concerns, but they are not circularity: the comparison does not rest on an identity between inputs and outputs. No circular step can be exhibited from the paper's equations or construction, so the score is low and reflects only the presence of minor non-load-bearing self-citations.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The scoring claim depends on the reliability of the PERSUADE scores, the accuracy of the annotation models, and the assumption that markup carries semantic signal rather than incidental formatting effects. The error annotations are an explicit unverified link.

free parameters (2)
  • ERRANT label grouping = 25 categories collapsed into Spelling, PunctOrth, Grammar
    Hand-chosen mapping in Section 2.1.3; changes the input representation for error-annotated scoring models.
  • annotation matching overlap threshold = 0.5
    The 50% overlap rule is taken from the Kaggle Feedback Prize competition (Section 2.3.1), not derived from this corpus; the reported F1 depends on it.
assumptions (4)
  • domain assumption PERSUADE human holistic scores are treated as ground truth for scoring accuracy.
    The evaluation compares predicted scores to the corpus scores, whose reported inter-rater agreement is kappa = 0.745 (Section 2.1.2), so the target itself is noisy.
  • domain assumption T5-GEC plus ERRANT produces accurate error annotations for PERSUADE essays.
    Explicitly unverified (Section 2.3.2); accuracy is inferred from JFLEG and BEA60k benchmarks, not from PERSUADE.
  • domain assumption The 50% overlap matching rule is a valid way to score argumentative annotation quality.
    Adopted from the Feedback Prize competition (Section 2.3.1); no analysis of sensitivity to this threshold.
  • domain assumption XML markup itself does not shift scoring outcomes independently of annotation semantics.
    No control condition with inert tags is run; the model sees the tags as ordinary tokens, so markup artifacts could contribute to the observed changes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automated Essay Scoring Incorporating Annotations from Automated Feedback Systems." pith.science (2026). https://pith.science/paper/XJYZUW7H

@misc{pith2026250522771,
  author       = {Pith},
  title        = {Pith review of: Automated Essay Scoring Incorporating Annotations from Automated Feedback Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XJYZUW7H}},
  note         = {Machine review of arXiv:2505.22771}
}
read the original abstract

This study illustrates how incorporating feedback-oriented annotations into the scoring pipeline can enhance the accuracy of automated essay scoring (AES). This approach is demonstrated with the Persuasive Essays for Rating, Selecting, and Understanding Argumentative and Discourse Elements (PERSUADE) corpus. We integrate two types of feedback-driven annotations: those that identify spelling and grammatical errors, and those that highlight argumentative components. To illustrate how this method could be applied in real-world scenarios, we employ two LLMs to generate annotations -- a generative language model used for spell correction and an encoder-based token-classifier trained to identify and mark argumentative elements. By incorporating annotations into the scoring process, we demonstrate improvements in performance using encoder-based large language models fine-tuned as classifiers.

Figures

Figures reproduced from arXiv: 2505.22771 by the authors.

Figure 1
Figure 1. An example of the model input using an ex [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. A diagram representing the scoring pipeline. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 19 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    AI@Meta . 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 Model Card

    work page 2024

  4. [4]

    Yigal Attali and Jill Burstein. 2006. https://ejournals.bc.edu/index.php/jtla/article/view/1650 Automated Essay Scoring With e-rater® V .2 . The Journal of Technology, Learning and Assessment, 4(3). Number: 3

    work page 2006

  5. [5]

    Peters, and Arman Cohan

    Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. https://doi.org/10.48550/arXiv.2004.05150 Longformer: The Long - Document Transformer . arXiv preprint. ArXiv:2004.05150 [cs]

    arXiv doi:10.48550/arxiv.2004.05150 2020

  6. [6]

    David Boulanger and Vivekanandan Kumar. 2020. https://doi.org/10.1007/978-3-030-49663-0_10 SHAPed Automated Essay Scoring : Explaining Writing Features ’ Contributions to English Writing Organization . In Intelligent Tutoring Systems , pages 68--78, Cham. Springer International Publishing

    work page doi:10.1007/978-3-030-49663-0_10 2020

  7. [7]

    Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. https://doi.org/10.18653/v1/P17-1074 Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics ( Volume 1: Long Papers ) , pages 793--805, Vancouver, Canada. Association for Compu...

    doi:10.18653/v1/p17-1074 2017

  8. [8]

    Jacob Cohen. 1960. https://doi.org/10.1177/001316446002000104 A Coefficient of Agreement for Nominal Scales . Educational and Psychological Measurement, 20(1):37--46. Publisher: SAGE Publications Inc

    doi:10.1177/001316446002000104 1960

Show all 46 references
  1. [9]

    Crossley, Perpetual Baffour, Yu Tian, Aigner Picou, Meg Benner, and Ulrich Boser

    Scott A. Crossley, Perpetual Baffour, Yu Tian, Aigner Picou, Meg Benner, and Ulrich Boser. 2022. https://doi.org/10.1016/j.asw.2022.100667 The persuasive essays for rating, selecting, and understanding argumentative and discourse elements ( PERSUADE ) corpus 1.0 . Assessing Wr...

    2022

  2. [10]

    Le, and Ruslan Salakhutdinov

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. 2019. https://doi.org/10.48550/arXiv.1901.02860 Transformer- XL : Attentive Language Models Beyond a Fixed - Length Context . arXiv preprint. ArXiv:1901.02860 [cs, stat]

    2019 doi arXiv

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. https://doi.org/10.48550/arXiv.1810.04805 BERT : Pre -training of Deep Bidirectional Transformers for Language Understanding . Technical Report arXiv:1810.04805, arXiv. ArXiv:1810.04805 [cs] type: article

    2018 doi arXiv

  4. [12]

    Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng. 2024. https://doi.org/10.48550/arXiv.2402.10171 Data Engineering for Scaling Language Models to 128K Context . arXiv preprint. ArXiv:2402.10171 [cs]

    2024 doi arXiv

  5. [13]

    Shai Gretz, Roni Friedman, Edo Cohen-Karlik, Assaf Toledo, Dan Lahav, Ranit Aharonov, and Noam Slonim. 2019. http://arxiv.org/abs/1911.11408 A Large -scale Dataset for Argument Quality Ranking : Construction and Analysis . arXiv preprint. ArXiv:1911.11408 [cs]

    2019 arXiv

  6. [14]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. https://doi.org/10.48550/arXiv.2006.03654 DeBERTa : Decoding -enhanced BERT with Disentangled Attention . arXiv preprint. Number: arXiv:2006.03654 arXiv:2006.03654 [cs]

    2021 doi arXiv

  7. [15]

    Shi Huawei and Vahid Aryadoust. 2023. https://doi.org/10.1007/s10639-022-11200-7 A systematic review of automated writing evaluation systems . Education and Information Technologies, 28(1):771--795

    2023 doi

  8. [16]

    Sai Muralidhar Jayanthi, Danish Pruthi, and Graham Neubig. 2020. https://doi.org/10.18653/v1/2020.emnlp-demos.21 NeuSpell : A Neural Spelling Correction Toolkit . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing : System Demonstrations ...

    2020 doi

  9. [17]

    Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya. 2020. https://doi.org/10.48550/arXiv.2001.04451 Reformer: The Efficient Transformer . Technical Report arXiv:2001.04451, arXiv. ArXiv:2001.04451 [cs, stat] type: article

    2020 doi arXiv

  10. [18]

    Katerina Korre and John Pavlopoulos. 2020. https://aclanthology.org/2020.latechclfl-1.10/ ERRANT : Assessing and Improving Grammatical Error Type Classification . In Proceedings of the 4th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage , Social Scienc...

    2020

  11. [19]

    Gyeong-Geon Lee, Ehsan Latif, Xuansheng Wu, Ninghao Liu, and Xiaoming Zhai. 2024. https://doi.org/10.1016/j.caeai.2024.100213 Applying large language models and chain-of-thought for automatic scoring . Computers and Education: Artificial Intelligence, 6:100213

    2024

  12. [20]

    Susan Lottridge, Amy Burkhardt, Christopher Ormerod, Sherri Woolf, Mackenzie Young, Milan Patel, Harry Wang, Julius Frost, Kevin McBeth, and Julie Benson. 2024. https://files.portal.cambiumast.com/corporate-site/documents/CAI-CSSC-NCME-Write-on-with-Cambi-White-Paper.pdf Write...

    2024

  13. [21]

    Nikita Martynov, Mark Baushenko, Anastasia Kozlova, Katerina Kolomeytseva, Aleksandr Abramov, and Alena Fenogenova. 2023. https://doi.org/10.48550/arXiv.2308.09435 A Methodology for Generative Spelling Correction via Natural Spelling Errors Emulation across Multiple Domains an...

    2023 doi arXiv

  14. [22]

    Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017. https://aclanthology.org/E17-2037 JFLEG : A Fluency Corpus and Benchmark for Grammatical Error Correction . In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguis...

    2017

  15. [23]

    Christopher Ormerod. 2022 a . https://doi.org/10.48550/arXiv.2202.11558 Short-answer scoring with ensembles of pretrained language models . arXiv preprint. ArXiv:2202.11558 [cs]

    2022 doi arXiv

  16. [24]

    Christopher Ormerod, Amy Burkhardt, Mackenzie Young, and Sue Lottridge. 2023. https://doi.org/10.48550/arXiv.2311.06239 Argumentation Element Annotation Modeling using XLNet . arXiv preprint. ArXiv:2311.06239 [cs]

    2023 doi arXiv

  17. [25]

    Harris, Milan Patel, Paul van Wamelen, Balaji Kodeswaran, Sharon Woolf, and Mackenzie Young

    Christopher Ormerod, Susan Lottridge, Amy E. Harris, Milan Patel, Paul van Wamelen, Balaji Kodeswaran, Sharon Woolf, and Mackenzie Young. 2022. https://doi.org/10.1007/s40593-022-00294-2 Automated Short Answer Scoring Using an Ensemble of Neural Networks and Latent Semantic An...

    2022 doi

  18. [26]

    Christopher Michael Ormerod. 2022 b . Mapping Between Hidden States and Features to Validate Automated Essay Scoring Using DeBERTa Models . Psychological Test and Assessment Modeling, 64(4):495--526

    2022

  19. [27]

    Christopher Michael Ormerod and Alexander Kwako. 2024. https://doi.org/10.48550/arXiv.2407.01873 Automated Text Scoring in the Age of Generative AI for the GPU -poor . arXiv preprint. ArXiv:2407.01873 [cs]

    2024 doi arXiv

  20. [28]

    Ellis Batten Page. 2003. Project Essay Grade : PEG . In Automated essay scoring: A cross-disciplinary perspective , pages 43--54. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US

    2003

  21. [29]

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. https://www.cs.ubc.ca/ amuham01/LING530/papers/radford2018improving.pdf Improving Language Understanding by Generative Pre -training

    2018

  22. [30]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. https://doi.org/10.48550/arXiv.1910.10683 Exploring the Limits of Transfer Learning with a Unified Text -to- Text Transformer . Technical Report ...

    2020 doi arXiv

  23. [31]

    Pedro Uria Rodriguez, Amir Jafari, and Christopher M. Ormerod. 2019. https://doi.org/10.48550/arXiv.1909.09482 Language models and Automated Essay Scoring . arXiv preprint. Number: arXiv:1909.09482 arXiv:1909.09482 [cs, stat]

    2019 doi arXiv

  24. [32]

    Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn. 2021. https://doi.org/10.48550/arXiv.2106.03830 A Simple Recipe for Multilingual Grammatical Error Correction . arXiv preprint. Number: arXiv:2106.03830 arXiv:2106.03830 [cs]

    2021 doi arXiv

  25. [33]

    Noam Shazeer. 2020. https://doi.org/10.48550/arXiv.2002.05202 GLU Variants Improve Transformer . arXiv preprint. ArXiv:2002.05202 [cs]

    2020 doi arXiv

  26. [34]

    Shermis and Ben Hamner

    Mark D. Shermis and Ben Hamner. 2013. https://doi.org/10.4324/9780203122761.CH19 Contrasting State -of-the- Art Automated Scoring of Essays . pages 335--368. Publisher: Routledge Handbooks Online

    2013 doi

  27. [35]

    Christian Stab and Iryna Gurevych. 2014 a . https://aclanthology.org/C14-1142 Annotating Argument Components and Relations in Persuasive Essays . In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics : Technical Papers , pages 1501--1510...

    2014

  28. [36]

    Christian Stab and Iryna Gurevych. 2014 b . https://doi.org/10.3115/v1/D14-1006 Identifying Argumentative Discourse Structures in Persuasive Essays . In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ) , pages 46--56, Doha, Qatar...

    2014 doi

  29. [37]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. https://doi.org/10.1016/j.neucom.2023.127063 RoFormer : Enhanced transformer with Rotary Position Embedding . Neurocomputing, 568:127063

    2024

  30. [38]

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. https://proceedings.neurips.cc/paper/2014/hash/a14ac55a4f27472c5d894ec1c3c743d2-Abstract.html Sequence to Sequence Learning with Neural Networks . In Advances in Neural Information Processing Systems , volume 27. Curran Assoc...

    2014

  31. [39]

    Assaf Toledo, Shai Gretz, Edo Cohen-Karlik, Roni Friedman, Elad Venezian, Dan Lahav, Michal Jacovi, Ranit Aharonov, and Noam Slonim. 2019. https://doi.org/10.18653/v1/D19-1564 Automatic Argument Quality Assessment - New Datasets and Methods . In Proceedings of the 2019 Confere...

    2019 doi

  32. [40]

    Masaki Uto and Yuto Uchida. 2020. https://doi.org/10.1007/978-3-030-52240-7_61 Automated Short - Answer Grading Using Deep Neural Networks and Item Response Theory . AIED

    2020 doi

  33. [41]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Attention is All you Need . In Advances in Neural In...

    2017

  34. [42]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. https://doi.org/10.48550/arXiv.1804.07461 GLUE : A Multi - Task Benchmark and Analysis Platform for Natural Language Understanding . Technical Report arXiv:1804.07461, arXiv. ArXiv:1...

    2019 doi arXiv

  35. [43]

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024. https://doi.org/10.48550/arXiv.2412.13663 Smarter, ...

    2024 doi arXiv

  36. [44]

    Williamson, Xiaoming Xi, and F

    David M. Williamson, Xiaoming Xi, and F. Jay Breyer. 2012. https://doi.org/10.1111/j.1745-3992.2011.00223.x A Framework for Evaluation and Use of Automated Scoring . Educational Measurement: Issues and Practice, 31(1):2--13. \_eprint: https://onlinelibrary.wiley.com/doi/pdf/10...

    2012

  37. [45]

    Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. 2020. https://proceedings.mlr.press/v119/xiong20b.html On Layer Normalization in the Transformer Architecture . In Proceedings of the 37th Internatio...

    2020

  38. [46]

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. https://proceedings.neurips.cc/paper/2019/hash/dc6a7e655d7e5840e66733e9ee67cc69-Abstract.html XLNet : Generalized Autoregressive Pretraining for Language Understanding . In Advanc...

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.