TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring

Salam Albatarni; Sohaila Eltanbouly; Tamer Elsayed

REVIEW 3 major objections 5 minor 28 references

TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a hybrid pipeline—an LLM that turns grading rubrics into trait-specific assessment questions, plus a simple regression model—sets a new state of the art for cross-prompt essay trait scoring on the ASAP dataset.

desk verdict The core hybrid idea is clever and worth engaging, but the "SOTA across all traits" claim only holds if you let the authors pick a different LLM for each trait. read the letter →

arxiv 2505.14577 v2 pith:XYC3HUO2 submitted 2025-05-20 cs.CL

Sohaila Eltanbouly , Salam Albatarni , Tamer Elsayed This is my paper · ORCID

classification cs.CL

keywords automatedessayscoringtraitcross-promptrubric-basedfeatureslargelanguagemodelsLLM-basedfeatureextractionASAPdatasetELLIPSE

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that trait-based Automated Essay Scoring can be made both more accurate and more interpretable by reframing the LLM's role: instead of asking the LLM to score an essay directly, TRATES asks it to convert each trait's grading rubric into a set of assessment questions, then to answer each question for a given essay as high, medium, or low. These LLM-generated answers are combined with generic writing-quality features and prompt-specific metadata, and a simple regression model is trained to predict trait scores for essays from prompts the model has never seen. The authors report new state-of-the-art performance on all eight ASAP traits, and a first cross-prompt baseline on the ELLIPSE dataset.

What carries the argument

The two-stage LLM feature chain: (1) rubric-to-question generation—an LLM is prompted to reformulate a trait rubric into a small set of short assessment questions, each targeting one scoring criterion; (2) question-to-answer extraction—the same LLM answers each question for an essay as high, medium, or low. These 3-valued answers are concatenated with a manually designed set of generic writing-quality features (length, readability, text variation, complexity, sentiment) and prompt-specific features (essay type, expected length, source length, grade level), and fed to a shallow neural-network regression model. A grade-level score scaling step maps different prompts' raw scores into a common [0–6] range before training, and predictions are mapped back for evaluation.

What would settle it

Re-run the ASAP cross-prompt experiments with an alternative, explicit score-alignment rule (e.g., plain per-prompt min-max scaling or a mapping derived directly from the rubrics' descriptors) while keeping all TRATES components identical; if the average QWK then fails to exceed the ProTACT baseline on all traits, the new-SOTA claim is not robust to the chosen grade scaling.

Watch

Extended reading notes

Core claim

On the ASAP dataset, TRATES improves average Quadratic Weighted Kappa over the strongest published baselines for cross-prompt trait scoring (ProTACT and Li & Ng), reaching new best scores on all eight traits when paired with the Starling or Gemma LLM. The ablation study shows that removing the trait-specific LLM features causes the largest drop in performance on most traits, indicating that these rubric-grounded features are the most informative of the three feature categories. The framework's success positions LLM-generated trait sub-questions as a practical feature source for cross-prompt scoring, rather than relying on LLMs as direct graders.

Load-bearing premise

The paper assumes that its hand-chosen grade-level score scaling—shaving one point off the maximum score for each grade below the top—makes scores from different ASAP prompts directly comparable; if that mapping is wrong, the cross-prompt training targets become misaligned and the reported gains could rest on an arbitrary transformation.

Editorial extensions

If this is right

If correct, direct LLM prompting for essay scores is not the decisive factor; what matters is the feature representation derived from rubrics.
A simple regression model on rich features can match or beat complex neural trait-scoring models, shifting the field's emphasis toward feature design and rubric grounding.
The released trait-specific questions enable automated, interpretable sub-trait feedback for students and teachers.
The approach transfers to new datasets (ELLIPSE) and to new traits given a rubric, making it a generic pipeline for trait scoring.
Because the LLM features are the most significant category, further improving question generation—with larger LLMs or better prompting—should directly improve scoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The grade-level score-scaling is hand-set; replacing it with a learned or data-driven mapping might change the cross-prompt results, and testing this would clarify how much of the SOTA gain depends on that choice.
The generated assessment questions themselves could be used as teaching feedback; the paper did not evaluate their pedagogical value, but if they are rubric-faithful, they offer a direct explanation of a trait score.
Because the framework only needs a rubric and an LLM, it could be adapted to holistic scoring or to non-ASAP rubrics, though holistic rubrics are more prompt-specific.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea here is genuinely nice: instead of asking an LLM to score an essay directly, TRATES uses the rubric to generate a set of trait-specific assessment questions, has the LLM answer them as high/medium/low, and then feeds those answers, alongside generic and prompt-specific features, into a simple regression model. That reconceptualization gives you interpretable sub-traits and a natural route to cross-prompt generalization. The paper does that clearly, ships the code and features, runs a sensible ablation, and validates on a second dataset. For those reasons alone, this is not a desk-reject paper.

What the paper does less well is support its central claim. The "new state-of-the-art across all traits" in the abstract and Section 6.2 is a union of per-trait best LLMs: in Table 1, Gemma wins ORG, WC, SF, CNV, and CNT, while Starling wins PA, NAR, and LNG. No single row beats both ProTACT and Li & Ng on every trait. So the SOTA claim describes a test-set oracle, not a deployable model. The authors never explain how a practitioner would choose the LLM for an unseen prompt. That is a substantive weakness, not a quibble.

The other soft spots are real but more contained. The score-scaling—reducing the max score by one point per grade level below the top—is a hand-picked transformation applied before training. The authors admit in the Limitations that it is debatable, and they are right to be cautious; it changes the regression targets and therefore every number in the paper. The baseline numbers for ASAP are quoted from previous papers, not re-run under TRATES's normalization and scaling protocol, so the comparison is not fully controlled. There are also no error bars or significance tests; QWK differences of one or two points could easily vanish with different random seeds.

None of this invalidates the central mechanism. The ablation study shows the LLM-generated features are the most important category on most traits, and the ELLIPSE results, while lower, still beat the available baselines. The paper would be strengthened by fixing a single LLM per trait using a validation-based rule, re-running the baselines under identical conditions, and reporting variance. As it stands, the contribution is a promising framework plus a set of reproducible features, not a clean SOTA result.

For a reading group, this is a good paper to argue about. I would cite it for the feature-generation trick. And I would send it to reviewers, because the idea is solid and all the major problems are fixable without changing the approach.

Referee Report

3 major / 5 minor

Summary. The paper proposes TRATES, a cross-prompt trait-scoring framework that uses an LLM to convert trait rubrics into sub-trait questions, answers those questions as High/Medium/Low, concatenates these trait-specific features with prompt-specific and generic writing-quality features, and trains a shallow neural network regressor per trait. Experiments on ASAP/ASAP++ report QWK across eight traits for three 7–9B LLMs, comparing against the ProTACT and Li & Ng baselines, with additional experiments on ELLIPSE and an ablation study of feature categories. The paper claims that TRATES establishes a new state of the art across all traits on ASAP and releases the generated trait-specific features.

Significance. If the claims are supported, the paper makes a useful contribution: it repurposes LLMs from direct graders to interpretable rubric-derived feature extractors, keeps the scoring model simple, and provides sub-trait questions that could serve as feedback. The public release of the generated features, the two-dataset evaluation, and the ablation study are explicit strengths that aid reproducibility. However, the headline claim of new state-of-the-art performance across all traits is not currently substantiated by a single instantiation of the framework, and the comparison with prior work mixes evaluation protocols. These issues are load-bearing for the central claim but appear fixable with a re-framed claim or additional experiments.

major comments (3)

[Abstract and Section 6.2, Table 1] The claim that TRATES 'establishes a new state-of-the-art performance across all traits' is supported only by a union of the best per-trait result across the three LLM rows (i, j, k). No single TRATES row outperforms both ProTACT and Li & Ng on all eight traits: Starling loses on WC (0.593 vs. 0.599), Gemma loses on PA (0.599 vs. 0.619), NAR (0.600 vs. 0.639), and LNG (0.521 vs. 0.596), and Llama loses on six traits. Because the paper does not specify a validation-based rule for choosing the LLM per trait on an unseen prompt, the reported per-trait best is not a single deployable model. The abstract and Section 6.2 should be revised to describe the result as a per-trait best over LLM choices, or the authors should provide a principled model-selection procedure and report the resulting single-system performance.
[Section 4.6, Section 8, and Appendix I] The grade-level score scaling is load-bearing for training a single cross-prompt model, but the assigned ranges [0,4] for grade 7, [0,5] for grade 8, and [0,6] for grade 10 are acknowledged in Section 8 to be 'intuitively determined after a thorough examination of the various rubrics.' This hand-chosen transformation changes the regression targets for different prompts, so the reported cross-prompt gains may be sensitive to the chosen ranges. The authors should provide a sensitivity analysis, for example comparing the grade-adjusted scaling to plain min-max scaling or other monotonic mappings, and show that the main conclusions are stable across reasonable choices.
[Section 5, Table 1] The comparison with ProTACT and Li & Ng is not apples-to-apples: those baseline scores are quoted from their original papers, while TRATES introduces a different feature-normalization protocol (train-set minimum and maximum values instead of per-prompt normalization) and the grade-level score-scaling described above. Since both protocol changes can affect QWK independently of the framework, the 'new SOTA' claim should be supported by re-running the baselines under the same normalization and scaling protocol, or by quantifying the effect of those protocol differences on the baseline scores.

minor comments (5)

[Table 2] The caption says bold and underlined values indicate the most and second-most important categories, but no underlining is visible in the table as typeset.
[Table 8] The hyperparameter numbering is inconsistent: 'Hidden layers' and 'Neurons per layer' are both numbered 3, so the list should be renumbered.
[Section 5, LLMs Selection] The 'Arena Elo benchmark' is cited only by a Hugging Face URL; a formal citation with an access date would be more appropriate.
[Section 6.1] The phrase 'by an average of 9 points' could be misread as 9 QWK units; consider writing '0.09 in QWK' or '9 points in the QWK scale used here.'
[Appendix I] The description of score-scaling should state explicitly how continuous predictions are mapped back to the original integer scales before QWK is computed, since rounding or clipping choices can affect the reported kappa values.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TRATES' predictions reduce to regression on human trait scores, not to its own inputs.

full rationale

The derivation chain is self-contained. TRATES generates trait-specific features by prompting an LLM with the trait rubric and the essay (Eqs. 1-2), concatenates them with hand-crafted generic and prompt-specific features, and trains a shallow regression model on human trait scores under leave-one-prompt-out cross-validation (Sections 4.1-4.5). The predicted target is the human trait score; the LLM outputs are only inputs to the regressor, so no prediction reduces by construction to a fitted value or to the LLM's own answer. The score-scaling step (Section 4.6 and Appendix I) is an acknowledged modeling choice made after inspecting the rubrics, not a parameter fitted to the target scores in a way that forces the reported QWK. The self-citation to Mansour et al. (2024) is only supporting evidence for the known failure of direct LLM scoring and is not load-bearing. The claim of 'new SOTA across all traits' may be an overclaim because the best per-trait cells come from different LLM variants in Table 1, but that is an evaluation and claims issue, not circular reasoning. No circular step is exhibited.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The framework does not introduce new physical entities or forces. Its load-bearing assumptions are the comparability of rubrics and scores across prompts, the reliability of LLM-generated sub-trait ratings, and the validity of the hand-chosen grade-level scaling. The free parameters that materially affect the reported results are the score-scaling ranges and the per-trait selection of the best LLM for the SOTA claim.

free parameters (3)

grade_level_score_scaling_ranges = [0,4] grade 7, [0,5] grade 8, [0,6] grade 10
Hand-chosen post hoc ranges used to unify ASAP score scales across grade levels; directly changes the regression targets and therefore all reported QWK values. The authors state the ranges were 'intuitively determined after a thorough examination of the various rubrics' (Section 8, Appendix I).
best_llm_per_trait_selection = Starling, Gemma, or Llama selected per trait for the SOTA claim
The 'new SOTA on all traits' claim is obtained by choosing the best performing LLM for each trait across three models, not by a single fixed configuration. This selection is made after seeing test results and inflates the apparent overall improvement.
high_medium_low_numeric_mapping = 3, 2, 1
LLM answers are mapped to 3/2/1 as feature values; this ordinal mapping is arbitrary but simple, and results could shift with different encodings.

assumptions (5)

domain assumption Trait rubrics are common across different prompts for the same trait
Section 3 states 'those rubrics are usually common across different prompts for specific traits', which is required for generated features to transfer across prompts. If rubrics are actually prompt-specific, the framework's feature set may be misaligned.
domain assumption LLM-generated assessment questions and high/medium/low answers are meaningful and reliable proxies for human trait judgments
The framework's core depends on the LLM accurately rating sub-traits; the paper provides no validation of the LLM answers against human ratings or rubric criteria, only indirect QWK improvements.
ad hoc to paper Grade-level adjusted score scaling makes scores from different prompts and rubrics comparable
The linear ranges [0,4], [0,5], [0,6] are chosen by the authors without independent justification. The entire cross-prompt training objective assumes this mapping is meaningful; if it is wrong, the model is trained on distorted targets.
domain assumption Using rubrics from source prompts only is sufficient to assess essays from an unseen target prompt
Section 4.6 assumes the diversity of source-prompt rubrics covers the target prompt's criteria. There is no verification that the generated questions are applicable to the held-out prompt's rubric and essay type.
domain assumption Generic writing-quality features from prior AES work transfer to trait scoring in a cross-prompt setting
The five generic feature categories are taken from Ridley et al. and reused without re-validation for trait-specific scoring, especially on the ELLIPSE English-learner dataset where the authors note their features may not generalize.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring." pith.science (2026). https://pith.science/paper/XYC3HUO2

@misc{pith2026250514577,
  author       = {Pith},
  title        = {Pith review of: TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYC3HUO2}},
  note         = {Machine review of arXiv:2505.14577}
}

read the original abstract

Research on holistic Automated Essay Scoring (AES) is long-dated; yet, there is a notable lack of attention for assessing essays according to individual traits. In this work, we propose TRATES, a novel trait-specific and rubric-based cross-prompt AES framework that is generic yet specific to the underlying trait. The framework leverages a Large Language Model (LLM) that utilizes the trait grading rubrics to generate trait-specific features (represented by assessment questions), then assesses those features given an essay. The trait-specific features are eventually combined with generic writing-quality and prompt-specific features to train a simple classical regression model that predicts trait scores of essays from an unseen prompt. Experiments show that TRATES achieves a new state-of-the-art performance across all traits on a widely-used dataset, with the generated LLM-based features being the most significant.

Figures

Figures reproduced from arXiv: 2505.14577 by the authors.

**Figure 1.** Overview of our TRATES framework. 3 Cross-prompt Trait-based AES In trait-based AES, a prompt p is defined as a tuple (ap, Tp, Ep), where ap is the prompt task description (a paragraph or so describing the writing task), Tp is a set {(t, rt)} of traits; each trait t (an aspect of student writing, such as organization or sentence fluency) is associated with a rubric rt (a set of criteria used to evaluate the specif… view at source ↗

**Figure 2.** LLM prompt for questions (i.e., features) [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

**Figure 3.** LLM prompt for feature extraction. Parameter Value #Essays 6,482 Grade levels 8–12 Number of prompts 44 Average prompt size 147 essays Average essay length 427 words Score range 1–5 (0.5 increments) [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: LLM prompt for Zero-shot LLM direct scor [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 12 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Yuan Chen and Xia Li. 2023. https://doi.org/10.18653/v1/2023.acl-long.83 PMAES : Prompt-mapping contrastive learning for cross-prompt automated essay scoring . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1489--1503, Toronto, Canada. Association for Computational Linguistics

doi:10.18653/v1/2023.acl-long.83 2023
[4]

Yuan Chen and Xia Li. 2024. https://aclanthology.org/2024.lrec-main.1118 PLAES : Prompt-generalized and level-aware learning framework for cross-prompt automated essay scoring . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 12775--12786, Torino, Italia. E...

work page 2024
[5]

Jacob Cohen. 1968. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin, 70(4):213

work page 1968
[6]

Scott Crossley, Yu Tian, Perpetual Baffour, Alex Franklin, Youngmeen Kim, Wesley Morris, Meg Benner, Aigner Picou, and Ulrich Boser. 2023. The english language learner insight, proficiency and skills evaluation (ellipse) corpus. International Journal of Learner Corpus Research, 9(2):248--269

work page 2023
[7]

Heejin Do, Yunsu Kim, and Gary Lee. 2024. https://aclanthology.org/2024.findings-eacl.115 Autoregressive score generation for multi-trait essay scoring . In Findings of the Association for Computational Linguistics: EACL 2024, pages 1659--1666, St. Julian ' s, Malta. Association for Computational Linguistics

work page 2024
[8]

Heejin Do, Yunsu Kim, and Gary Geunbae Lee. 2023. https://doi.org/10.18653/v1/2023.findings-acl.98 Prompt- and trait relation-aware cross-prompt essay trait scoring . In Findings of the Association for Computational Linguistics: ACL 2023, pages 1538--1551, Toronto, Canada. Association for Computational Linguistics

doi:10.18653/v1/2023.findings-acl.98 2023

Show all 28 references

[9]

Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Hyunseung Lim, Yoonsu Kim, Tak Yeon Lee, Hwajung Hong, Juho Kim, So-Yeon Ahn, and Alice Oh. 2024. https://doi.org/10.18653/v1/2024.customnlp4u-1.21 LLM -as-a-tutor in EFL writing education: Focusing on evaluation of student- LLM ...

2024 doi
[10]

Cancan Jin, Ben He, Kai Hui, and Le Sun. 2018. https://doi.org/10.18653/v1/P18-1100 TDNN : A two-stage deep neural network for prompt-independent automated essay scoring . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon...

2018 doi
[11]

Zixuan Ke and Vincent Ng. 2019. Automated Essay Scoring: A Survey of the State of the Art. In IJCAI, volume 19, pages 6300--6308

2019
[12]

Sanwoo Lee, Yida Cai, Desong Meng, Ziyang Wang, and Yunfang Wu. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.10 Unleashing large language models ' proficiency in zero-shot essay scoring . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages ...

2024 doi
[13]

Shengjie Li and Vincent Ng. 2024 a . https://aclanthology.org/2024.emnlp-main.991 Automated essay scoring: A reflection on the state of the art . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17876--17888, Miami, Florida, USA....

2024
[14]

Shengjie Li and Vincent Ng. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.414 Conundrums in cross-prompt automated essay scoring: Making sense of the state of the art . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: L...

2024 doi
[15]

Xia Li, Minping Chen, and Jian-Yun Nie. 2020. https://doi.org/https://doi.org/10.1016/j.knosys.2020.106491 Sednn: Shared and enhanced deep neural network model for cross-prompt automated essay scoring . Knowledge-Based Systems, 210:106491

2020
[16]

Watheq Ahmad Mansour, Salam Albatarni, Sohaila Eltanbouly, and Tamer Elsayed. 2024. https://aclanthology.org/2024.lrec-main.247 Can large language models automatically score proficiency of written essays? In Proceedings of the 2024 Joint International Conference on Computation...

2024
[17]

Sandeep Mathias and Pushpak Bhattacharyya. 2018. Asap++: Enriching the asap automated essay grading dataset with essay attribute scores. In Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018)

2018
[18]

Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37:124198--124235

2024
[19]

Ben Naismith, Phoebe Mulcaire, and Jill Burstein. 2023. https://doi.org/10.18653/v1/2023.bea-1.32 Automated evaluation of written discourse coherence using GPT -4 . In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), p...

2023 doi
[20]

Ellis B Page. 1966. The imminence of... grading essays by computer. The Phi Delta Kappan, 47(5):238--243

1966
[21]

Peter Phandi, Kian Ming A Chai, and Hwee Tou Ng. 2015. Flexible domain adaptation for automated essay scoring using correlated linear regression. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 431--439

2015
[22]

Robert Ridley, Liang He, Xin-yu Dai, Shujian Huang, and Jiajun Chen. 2021. Automated cross-prompt scoring of essay traits. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 13745--13753

2021
[23]

Robert Ridley, Liang He, Xinyu Dai, Shujian Huang, and Jiajun Chen. 2020. Prompt agnostic essay scorer: a domain generalization approach to cross-prompt automated essay scoring. arXiv preprint arXiv:2008.01441

2020 arXiv
[24]

Maja Stahl, Leon Biermann, Andreas Nehring, and Henning Wachsmuth. 2024. https://aclanthology.org/2024.bea-1.23 Exploring LLM prompting strategies for joint essay scoring and feedback generation . In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educat...

2024
[25]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

2023 arXiv
[26]

Jiayi Xie, Kaiwei Cai, Li Kong, Junsheng Zhou, and Weiguang Qu. 2022. Automated essay scoring via pairwise contrastive regression. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2724--2733

2022
[27]

Yancey, Geoffrey Laflair, Anthony Verardi, and Jill Burstein

Kevin P. Yancey, Geoffrey Laflair, Anthony Verardi, and Jill Burstein. 2023. https://doi.org/10.18653/v1/2023.bea-1.49 Rating short L 2 essays on the CEFR scale with GPT -4 . In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BE...

2023 doi
[28]

Banghua Zhu, Evan Frick, Tianhao Wu, Hanlin Zhu, and Jiantao Jiao. 2023. Starling-7b: Improving llm helpfulness & harmlessness with rlaif

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Yuan Chen and Xia Li. 2023. https://doi.org/10.18653/v1/2023.acl-long.83 PMAES : Prompt-mapping contrastive learning for cross-prompt automated essay scoring . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1489--1503, Toronto, Canada. Association for Computational Linguistics

doi:10.18653/v1/2023.acl-long.83 2023

[4] [4]

Yuan Chen and Xia Li. 2024. https://aclanthology.org/2024.lrec-main.1118 PLAES : Prompt-generalized and level-aware learning framework for cross-prompt automated essay scoring . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 12775--12786, Torino, Italia. E...

work page 2024

[5] [5]

Jacob Cohen. 1968. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin, 70(4):213

work page 1968

[6] [6]

Scott Crossley, Yu Tian, Perpetual Baffour, Alex Franklin, Youngmeen Kim, Wesley Morris, Meg Benner, Aigner Picou, and Ulrich Boser. 2023. The english language learner insight, proficiency and skills evaluation (ellipse) corpus. International Journal of Learner Corpus Research, 9(2):248--269

work page 2023

[7] [7]

Heejin Do, Yunsu Kim, and Gary Lee. 2024. https://aclanthology.org/2024.findings-eacl.115 Autoregressive score generation for multi-trait essay scoring . In Findings of the Association for Computational Linguistics: EACL 2024, pages 1659--1666, St. Julian ' s, Malta. Association for Computational Linguistics

work page 2024

[8] [8]

Heejin Do, Yunsu Kim, and Gary Geunbae Lee. 2023. https://doi.org/10.18653/v1/2023.findings-acl.98 Prompt- and trait relation-aware cross-prompt essay trait scoring . In Findings of the Association for Computational Linguistics: ACL 2023, pages 1538--1551, Toronto, Canada. Association for Computational Linguistics

doi:10.18653/v1/2023.findings-acl.98 2023