Pith. sign in

REVIEW 4 major objections 6 minor 37 references

Improve LLM-based Automatic Essay Scoring with Linguistic Features

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that appending ten hand-calculated linguistic features to a zero-shot LLM prompt raises agreement with human essay scores on both in-domain and out-of-distribution essays.

desk verdict Potentially useful prompt-level finding, but the evidence is not yet there: single-fold QWK without error bars, no control prompt, and an abstract that overstates the GPT-4 result. read the letter →

arxiv 2502.09497 v1 pith:2YEIFC3B submitted 2025-02-13 cs.CL cs.AI

Zhaoyi Joey Hou , Alejandro Ciuba , Xiang Lorraine Li This is my paper · ORCID
classification cs.CLcs.AI
keywords automaticessayscoringlinguisticfeatureszero-shotpromptingLLMasevaluatorcross-promptAESquadraticweightedkappapromptengineering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a zero-shot large language model scores student essays more accurately when the prompt is supplemented with a short list of hand-calculated linguistic features of the essay. On the ASAP dataset, Mistral-7B's quadratic weighted kappa with human graders rises from 0.454 with a plain prompt to 0.492 when ten features are included, without any fine-tuning. The improvement also transfers to the held-out ELLIPSE dataset, where the best feature set raises the score from 0.454 to 0.481. The result matters because it suggests a cheap, training-free route to better open-source essay grading, and because the gain is uneven across models, hinting at how LLM scores depend on prompt content.

What carries the argument

The mechanism is the prompt construction: the model is given a persona, the essay prompt, an analysis instruction, the essay text, and then an 'Additional Information' section listing precomputed linguistic features with the framing that higher feature values typically mean higher end scores. The features themselves come from a published set of essay-grading statistics, limited to ten to avoid excessively long prompts. A separate few-shot LLM parser converts the model's text output into structured scores. The argument rests on comparing quadratic weighted kappa across four feature settings—none, the single most correlated feature, the top three, and all ten—on two datasets.

What would settle it

Repeat the zero-shot evaluation on the same folds but replace the feature list with an equal-length list of random numbers, or with the same numbers presented without the correlation instruction; if QWK stays at the same level, the features are not the driver. A second, statistical test would compute bootstrap confidence intervals for the QWK differences; the claim is unsupported if the intervals for the with- and without-feature conditions overlap.

Watch

Extended reading notes

Core claim

The central claim is that appending an 'Additional Information' block listing ten precomputed essay statistics to an existing zero-shot prompt improves agreement with human holistic scores. The features—unique word count, word count, sentence count, lemma count, noun count, stopword count, Dale-Chall complex-word count, and two character counts—are presented to the model with the statement that higher values typically correspond to higher grades. Using the prompt template of a prior zero-shot baseline, the authors report that Mistral-7B improves from 0.454 to 0.492 average QWK on ASAP when all ten features are added, and from 0.454 to 0.481 on ELLIPSE with the top three features. They conclude that linguistic features can benefit LLM-based zero-shot AES, and that the benefit persists on out-of-distribution essays.

Load-bearing premise

The reported QWK improvements are caused by the linguistic features themselves, rather than by the longer prompt, the instruction telling the model that higher features mean higher grades, or chance variation across a single evaluation fold.

Editorial extensions

If this is right

  • If the central claim holds, educators can improve open-source LLM grading by computing a handful of text statistics and stating their positive correlation with scores, at no training cost.
  • The effect transfers across datasets, so the method supports the cross-prompt AES goal of scoring essays from prompts not seen during development.
  • The benefit is concentrated in the open-source model; the closed model shows mixed results, meaning the method's usefulness depends on the base LLM.
  • Even with the boost, a fine-tuned supervised model scores higher on ASAP, so prompt-only LLM grading still trails supervised methods.
  • The improved alignment between LLM and human scores suggests that additional linguistic context can partially substitute for the statistical prior that fine-tuning provides.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A plausible alternative reading, which the paper does not rule out, is that Mistral is simply following the prompt's stated rule that higher feature values mean higher scores; a control condition with random numbers, or with features presented without the correlation claim, would separate feature-driven improvement from instruction-following.
  • The single evaluation fold and absence of significance tests mean the reported QWK gaps could lie within noise; a bootstrap over the test essays would test whether 0.454 to 0.492 is a real difference.
  • The feature list is built from counts and readability statistics only; the same prompt scaffold could be tested with content-based features (e.g., rubric-aligned topic coverage) to see whether the mechanism generalizes beyond surface statistics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a zero-shot prompting method for automatic essay scoring (AES) that augments the prompt with handcrafted linguistic features of the essay, such as unique word count, lemma count, and Dale-Chall word count. The method is evaluated on two datasets, ASAP and ELLIPSE, with Mistral-7B-Instruct and GPT-4, using Quadratic Weighted Kappa (QWK) as the metric. The central claim is that incorporating linguistic features into existing zero-shot prompts notably improves score prediction, both in-domain (ASAP) and out-of-domain (ELLIPSE), without fine-tuning. The paper also compares against a supervised BERT baseline on ASAP and discusses the headroom for open-source LLMs. The main numerical support is Table 1, where Mistral-7B with Top-10 features reaches 0.492 on ASAP versus 0.454 without features, and with Top-3 features reaches 0.481 on ELLIPSE versus 0.454.

Significance. If the reported effect is real, the contribution is practically valuable: a simple, training-free prompt modification that improves LLM-based AES agreement with human scores, with some evidence of cross-prompt transfer. The paper is clearly written and releases code, uses two established datasets, and compares open and closed LLMs, which are strengths. However, the empirical evidence is currently too thin to support the central claim. The results are single-point QWK estimates without confidence intervals or significance tests, the feature condition is confounded with extra prompt text and an explicit correlation instruction, and feature/prompt variants were selected on the development or test data. These issues are fixable within the manuscript's scope, so the contribution is plausible but not yet demonstrated.

major comments (4)
  1. [Table 1, §4.1, §5.1] The central claim rests on single-point QWK estimates: Mistral Top-10 vs None on ASAP (.492 vs .454) and Top-3 vs None on ELLIPSE (.481 vs .454). No confidence intervals, bootstrap, or significance test is provided, and ASAP is evaluated on a single fold. These differences could be within sampling noise; the contribution statement that linguistic features 'notably improve' score prediction is not supported by the reported statistics without an uncertainty estimate.
  2. [§3.1, Appendix C.0.2] The 'additional information' condition adds both several lines of feature text and the explicit instruction that 'higher features typically mean higher end score.' There is no control prompt of matched length with non-informative statistics, so the observed gain could be due to extra token length, instruction-following, or the specific feature values rather than the features themselves. This confound is load-bearing because the paper's contribution is specifically the addition of linguistic features, not generic prompt extension.
  3. [§4.3, Appendix C] The feature variants and prompt phrasing were tuned on the ASAP dev set, and the ELLIPSE 'best' Top-3 result is identified after seeing test outputs. Since the paper reports three Mistral feature conditions and highlights the best-performing one on ELLIPSE to support the out-of-domain claim, the effect is inflated by multiple comparisons. The paper should report uncertainty for all conditions and, at a minimum, acknowledge this selection effect; ideally, the feature choice would be pre-registered or corrected.
  4. [Abstract, §5.2, Table 1] The abstract claims the hybrid method 'outperforms baseline models for both in-domain and out-of-domain writing prompts,' but Table 1 shows GPT-4 with Top-10 features scores lower on ASAP than GPT-4 with no features (.488 vs .499), and §5.2 concedes 'with the exception of GPT-4.' The abstract and the contributions section should be qualified to match the actual results and avoid overstating the improvement.
minor comments (6)
  1. [Appendix C.0.2] The Top-10 prompt lists 'total number of characters: 279' and later 'total number of characters: 1229' with no distinguishing label; one is the essay character length and the other the total character count. This duplicate label could confuse the model and should be fixed with separate names.
  2. [§5.1] The sentence beginning 'Another research Chen et al. (2024) that conducted a zero-shot prompting experiment...' is grammatically incomplete and the citation should be formatted properly.
  3. [§7] The Limitations section says 'there is only one open-source and one close-source one'; this should read 'only one open-source and one closed-source model.'
  4. [§4.3] The statement 'We use default sampling parameters in vllm framework during decoding' should specify the exact decoding parameters (e.g., temperature, top_p, max_tokens) for reproducibility, since temperature 0 is already mentioned but other defaults are not.
  5. [References] The bibliography entry for Devlin et al. is missing a publication year; the year should be added.
  6. [Appendix E] Appendix E reports 'Overall parsing error is less than 7%' without explaining how the error was measured or on which essays; a brief description would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the improvement claim is an empirical prompt-comparison, not a derivation from the features.

full rationale

The paper's central claim is that adding handcrafted linguistic features to zero-shot LLM prompts improves scoring agreement with human raters. This claim is evaluated by running LLMs with and without the feature text on held-out essays, and the reported QWK values are computed against human scores. No model parameter is fitted to the target scores within the paper, and no equation defines the predicted score in terms of the feature values; the features are computed from each essay and inserted as additional context. The prior correlation of these features with scores is cited from external work (Ridley et al.), not from the authors' own theorems or fitted values. The prompt does assert that higher feature values typically mean higher scores, which is an instruction that could partly explain the effect, but that is a confounding/interpretation concern, not a circular reduction: the outcome is measured independently of the feature values. The paper also reports all feature conditions (Unique, Top-3, and Top-10) including a case where GPT-4 degrades with features, so the headline result is not forced by construction. Concerns about single-fold evaluation, missing confidence intervals, and the absence of a length-matched control prompt are validity and robustness issues, not evidence of circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. The main cost is the feature-selection and prompt-tuning choices, plus the unverified correlation assumption.

free parameters (2)
  • number of linguistic features in prompt = 1, 3, or 10; best on dev was Top-10 for ASAP and Top-3 for ELLIPSE
    The set of features and its size were chosen based on the ASAP validation fold, an ad hoc selection that could overfit the dev set.
  • prompt phrasing variants = not fully specified
    Appendix C states limited prompt tuning on the ASAP dev set after failing to reproduce Stahl et al.; the exact variants tried are not listed.
assumptions (4)
  • domain assumption The selected linguistic features are positively correlated with the essay scores in both ASAP and ELLIPSE.
    The prompt asserts this correlation based on Ridley et al. for the original feature set, but the authors do not verify the correlation on ELLIPSE or on their re-implemented features. Entered in Section 3.1 prompt template and Section 3.2.
  • domain assumption Adding the Additional Information section to the prompt affects scoring only through the informative content of the features, not through prompt length or generic instruction effects.
    No control condition with non-informative extra text is run, so this assumption is untested. See Section 5.2.
  • standard math QWK between the LLM prediction and the human score is a valid and sufficient measure of AES performance.
    QWK is the standard AES metric from the ASAP Kaggle competition, used per Hamner et al. Section 4.2.
  • domain assumption The parsing module's less than 7% overall error does not materially bias the QWK scores.
    Appendix E reports parsing error but no analysis of its effect on the reported QWK values.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improve LLM-based Automatic Essay Scoring with Linguistic Features." pith.science (2026). https://pith.science/paper/2YEIFC3B

@misc{pith2026250209497,
  author       = {Pith},
  title        = {Pith review of: Improve LLM-based Automatic Essay Scoring with Linguistic Features},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2YEIFC3B}},
  note         = {Machine review of arXiv:2502.09497}
}
read the original abstract

Automatic Essay Scoring (AES) assigns scores to student essays, reducing the grading workload for instructors. Developing a scoring system capable of handling essays across diverse prompts is challenging due to the flexibility and diverse nature of the writing task. Existing methods typically fall into two categories: supervised feature-based approaches and large language model (LLM)-based methods. Supervised feature-based approaches often achieve higher performance but require resource-intensive training. In contrast, LLM-based methods are computationally efficient during inference but tend to suffer from lower performance. This paper combines these approaches by incorporating linguistic features into LLM-based scoring. Experimental results show that this hybrid method outperforms baseline models for both in-domain and out-of-domain writing prompts.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 20 canonical work pages

  1. [1]

    Personalized text generation with fine-grained linguistic control

    Bashar Alhafni, Vivek Kulkarni, Dhruv Kumar, and Vipul Raheja. Personalized text generation with fine-grained linguistic control. In Ameet Deshpande, EunJeong Hwang, Vishvak Murahari, Joon Sung Park, Diyi Yang, Ashish Sabharwal, Karthik Narasimhan, and Ashwin Kalyan, editors, Proceedings of the 1st Workshop on Personalization of Generative AI Systems (PER...

    work page 2024

  2. [2]

    Automated Scoring Using A Hybrid Feature Identification Technique

    Jill Burstein, Karen Kukich, Susanne Wolff, Chi Lu, Martin Chodorow, Lisa Braden-Harder , and Mary Dee Harris. Automated Scoring Using A Hybrid Feature Identification Technique . In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics , Volume 1 , pages 206--210, Montreal, Queb...

    work page arXiv 1998

  3. [3]

    Automated essay scoring by maximizing human-machine agreement

    Hongbo Chen and Ben He. Automated essay scoring by maximizing human-machine agreement. In David Yarowsky, Timothy Baldwin, Anna Korhonen, Karen Livescu, and Steven Bethard, editors, Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1741--1752, Seattle, Washington, USA, October 2013. Association for Computational...

    work page 2013

  4. [4]

    A multi-task automated assessment system for essay scoring

    Shigeng Chen, Yunshi Lan, and Zheng Yuan. A multi-task automated assessment system for essay scoring. In Andrew M. Olney, Irene-Angelica Chounta, Zitao Liu, Olga C. Santos, and Ig Ibert Bittencourt, editors, Artificial Intelligence in Education, pages 276--283, Cham, 2024. Springer Nature Switzerland. ISBN 978-3-031-64299-9

    work page 2024

  5. [5]

    A closer look into using large language models for automatic evaluation

    Cheng-Han Chiang and Hung-yi Lee. A closer look into using large language models for automatic evaluation. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8928--8942, Singapore, December 2023. Association for Computational Linguistics. doi:10.18653/v1/2023.findings-emnlp.5...

    doi:10.18653/v1/2023.findings-emnlp.599 2023

  6. [6]

    The English Language Learner Insight , Proficiency and Skills Evaluation ( ELLIPSE ) Corpus

    Scott Crossley. The English Language Learner Insight , Proficiency and Skills Evaluation ( ELLIPSE ) Corpus . International Journal of Learner Corpus Research, 9 0 (2), May 2024. doi:10.1075/ijlcr.22026.cro

    doi:10.1075/ijlcr.22026.cro 2024

  7. [7]

    The english language learner insight, proficiency and skills evaluation (ellipse) corpus

    Scott Crossley, Yu Tian, Perpetual Baffour, Alex Franklin, Youngmeen Kim, Wesley Morris, Meg Benner, Aigner Picou, and Ulrich Boser. The english language learner insight, proficiency and skills evaluation (ellipse) corpus. International Journal of Learner Corpus Research, 9 0 (2): 0 248--269, 2023

    work page 2023

  8. [8]

    Edgar Dale and Jeanne S. Chall. A Formula for Predicting Readability : Instructions . Educational Research Bulletin, 27 0 (2): 0 37--54, 1948. ISSN 1555-4023

    work page 1948

Show all 37 references
  1. [9]

    BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

  2. [10]

    Attention-based recurrent convolutional neural network for automatic essay scoring

    Fei Dong, Yue Zhang, and Jie Yang. Attention-based recurrent convolutional neural network for automatic essay scoring. In Roger Levy and Lucia Specia, editors, Proceedings of the 21st Conference on Computational Natural Language Learning ( C o NLL 2017) , pages 153--162, Vanco...

    2017 doi

  3. [11]

    Foltz, Darrell Laham, and Thomas K Landauer

    Peter W. Foltz, Darrell Laham, and Thomas K Landauer. Automated essay scoring: Applications to educational technology. In Betty Collis and Ron Oliver, editors, Proceedings of EdMedia + Innovate Learning 1999, pages 939--944, Seattle, WA USA, 1999. Association for the Advanceme...

    1999

  4. [12]

    GPTS core: Evaluate as you desire

    Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. GPTS core: Evaluate as you desire. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua...

    2024 doi

  5. [13]

    Llm-based nlg evaluation: Current status and challenges, 2024

    Mingqi Gao, Xinyu Hu, Jie Ruan, Xiao Pu, and Xiaojun Wan. Llm-based nlg evaluation: Current status and challenges, 2024. URL https://arxiv.org/abs/2402.01383

    2024 arXiv

  6. [14]

    The hewlett foundation: Automated essay scoring

    Ben Hamner, Jaison Morgan, lynnvandev, Mark Shermis, and Tom Vander Ark. The hewlett foundation: Automated essay scoring. https://kaggle.com/competitions/asap-aes, 2012. Kaggle

    2012

  7. [15]

    LLM -rubric: A multidimensional, calibrated approach to automated evaluation of natural language texts

    Helia Hashemi, Jason Eisner, Corby Rosset, Benjamin Van Durme, and Chris Kedzie. LLM -rubric: A multidimensional, calibrated approach to automated evaluation of natural language texts. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Me...

    2024 doi

  8. [16]

    spaCy : Industrial-strength Natural Language Processing in Python

    Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy : Industrial-strength Natural Language Processing in Python . 2020. doi:10.5281/zenodo.1212303

    2020 doi

  9. [17]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  10. [18]

    TDNN : A two-stage deep neural network for prompt-independent automated essay scoring

    Cancan Jin, Ben He, Kai Hui, and Le Sun. TDNN : A two-stage deep neural network for prompt-independent automated essay scoring. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    2018 doi

  11. [19]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems...

    2023

  12. [20]

    Conundrums in cross-prompt automated essay scoring: Making sense of the state of the art

    Shengjie Li and Vincent Ng. Conundrums in cross-prompt automated essay scoring: Making sense of the state of the art. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: L...

    2024

  13. [21]

    G -eval: NLG evaluation using gpt-4 with better human alignment

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G -eval: NLG evaluation using gpt-4 with better human alignment. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro...

    2023 doi

  14. [22]

    NLTK : The Natural Language Toolkit , May 2002

    Edward Loper and Steven Bird. NLTK : The Natural Language Toolkit , May 2002

    2002

  15. [23]

    Watheq Ahmad Mansour, Salam Albatarni, Sohaila Eltanbouly, and Tamer Elsayed. Can large language models automatically score proficiency of written essays? In Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, editors, Proceedi...

    2024

  16. [24]

    ASAP ++: Enriching the ASAP automated essay grading dataset with essay attribute scores

    Sandeep Mathias and Pushpak Bhattacharyya. ASAP ++: Enriching the ASAP automated essay grading dataset with essay attribute scores. In Nicoletta Calzolari, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Koiti Hasida, Hitoshi Isahara, Bente Maegaard, Joseph Ma...

    2018

  17. [25]

    Ellis B. Page. The Use of the Computer in Analyzing Student Essays . International Review of Education / Internationale Zeitschrift f \"u r Erziehungswissenschaft / Revue Internationale de l'Education , 14 0 (2): 0 210--225, 1968. ISSN 0020-8566

    1968

  18. [26]

    PyTorch : An Imperative Style , High-Performance Deep Learning Library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  19. [27]

    Chai, and Hwee Tou Ng

    Peter Phandi, Kian Ming A. Chai, and Hwee Tou Ng. Flexible domain adaptation for automated essay scoring using correlated linear regression. In Llu \' s M \`a rquez, Chris Callison-Burch, and Jian Su, editors, Proceedings of the 2015 Conference on Empirical Methods in Natural ...

    2015 doi

  20. [28]

    Prompt agnostic essay scorer: A domain generalization approach to cross-prompt automated essay scoring, 2020 a

    Robert Ridley, Liang He, Xinyu Dai, Shujian Huang, and Jiajun Chen. Prompt agnostic essay scorer: A domain generalization approach to cross-prompt automated essay scoring, 2020 a . URL https://arxiv.org/abs/2008.01441

    2020 arXiv

  21. [29]

    Prompt Agnostic Essay Scorer : A Domain Generalization Approach to Cross-prompt Automated Essay Scoring , August 2020 b

    Robert Ridley, Liang He, Xinyu Dai, Shujian Huang, and Jiajun Chen. Prompt Agnostic Essay Scorer : A Domain Generalization Approach to Cross-prompt Automated Essay Scoring , August 2020 b

    2020

  22. [31]

    Automated Cross-prompt Scoring of Essay Traits

    Robert Ridley, Liang He, Xin-yu Dai, Shujian Huang, and Jiajun Chen. Automated Cross-prompt Scoring of Essay Traits . Proceedings of the AAAI Conference on Artificial Intelligence, 35 0 (15): 0 13745--13753, May 2021 b . ISSN 2374-3468. doi:10.1609/aaai.v35i15.17620

    2021 doi

  23. [32]

    Exploring LLM prompting strategies for joint essay scoring and feedback generation

    Maja Stahl, Leon Biermann, Andreas Nehring, and Henning Wachsmuth. Exploring LLM prompting strategies for joint essay scoring and feedback generation. In Ekaterina Kochmar, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante , Ana \" s Tack, Victoria Yaneva, and ...

    2024

  24. [33]

    Neural automated essay scoring incorporating handcrafted features

    Masaki Uto, Yikuan Xie, and Maomi Ueno. Neural automated essay scoring incorporating handcrafted features. In Donia Scott, Nuria Bel, and Chengqing Zong, editors, Proceedings of the 28th International Conference on Computational Linguistics, pages 6077--6088, Barcelona, Spain ...

    2020 doi

  25. [34]

    On the use of bert for automated essay scoring: Joint learning of multi-scale essay representation

    Yongjie Wang, Chuang Wang, Ruobing Li, and Hui Lin. On the use of bert for automated essay scoring: Joint learning of multi-scale essay representation. In Marine Carpuat, Marie-Catherine de Marneffe , and Ivan Vladimir Meza Ruiz, editors, Proceedings of the 2022 Conference of ...

    2022 doi

  26. [35]

    Huggingface's transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R \'e mi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019

    1910 arXiv

  27. [36]

    A hierarchical bert-based transfer learning approach for multi-dimensional essay scoring

    Jin Xue, Xiaoyi Tang, and Liyan Zheng. A hierarchical bert-based transfer learning approach for multi-dimensional essay scoring. IEEE Access, 9: 0 125403--125415, 2021. doi:10.1109/ACCESS.2021.3110683

    2021

  28. [37]

    Is LLM a reliable reviewer? a comprehensive evaluation of LLM on automatic paper reviewing tasks

    Ruiyang Zhou, Lu Chen, and Kai Yu. Is LLM a reliable reviewer? a comprehensive evaluation of LLM on automatic paper reviewing tasks. In Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, editors, Proceedings of the 2024 Joint ...

    2024

  29. [38]

    A LLM -based ranking method for the evaluation of automatic counter-narrative generation

    Irune Zubiaga, Aitor Soroa, and Rodrigo Agerri. A LLM -based ranking method for the evaluation of automatic counter-narrative generation. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.