Pith. sign in

REVIEW 4 major objections 5 minor 40 references

A Zero-Shot LLM Framework for Automatic Assignment Grading in Higher Education

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a zero-shot LLM, guided only by a purpose-built prompt, can grade open-ended assignments in close agreement with human teaching assistants while delivering personalized feedback that students strongly prefer.

desk verdict A useful zero-shot grading pipeline, but the student survey compares AI feedback to a static solution, not to TA grading, so the paper's central claim is overstated. read the letter →

arxiv 2501.14305 v1 pith:X2OCXY6I submitted 2025-01-24 cs.CY cs.AI

Calvin Yeung , Jeff Yu , King Chau Cheung , Tat Wing Wong , Chun Man Chan , Kin Chi Wong , Keisuke Fujii This is my paper · ORCID
classification cs.CYcs.AI
keywords automatedgradingzero-shotlearninglargelanguagemodelspromptengineeringpersonalizedfeedbackstudentperceptionseducationtechnologyhigher
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a large language model can grade open-ended student assignments entirely through prompt engineering, with no fine-tuning or task-specific training. The proposed Automated Assignment Grading (AAG) system evaluates both computational and explanatory answers, and produces individualized feedback that names mistakes, explains them, and suggests improvements. In a real course, AAG scores correlated strongly with teaching-assistant scores on two open-ended questions (Pearson r = 0.75 and 0.82), and a survey of 104 students showed significant positive ratings and a strong preference for the AI feedback. The authors conclude that this combination of scalability and personalized feedback can improve student motivation, understanding, and preparedness relative to traditional grading.

What carries the argument

The load-bearing object is the evaluation prompt: a structured text that gives the LLM the assignment background, the specific question (or subquestion), a marking scheme it may refine or generate itself, the student's answer, and the correct answer, followed by instructions on scoring scale and the type of feedback to produce. Its work is to let a general-purpose model act as a grader and feedback generator without any parameter updates, turning the system's flexibility into a claimed advantage over few-shot approaches that require labeled datasets for each course or assignment.

What would settle it

A direct controlled study in the same course—where the same assignments are graded by TAs, by the AAG system, and then students are asked to choose between TA feedback and AAG feedback, or where independent expert graders rescore the 150 submissions—would settle whether the system genuinely outperforms traditional grading in agreement with ground truth and in student preference.

Watch

Extended reading notes

Core claim

The central claim is that a carefully engineered zero-shot prompt—containing the assignment question, a refined marking scheme, the student's response, the correct answer, and scoring instructions—is enough to make GPT-4 grade open-ended work and write tailored feedback without any additional training. The paper reports that on 150 real student submissions to two open-ended statistics questions, AAG scores had Pearson correlations of 0.75 and 0.82 with teaching-assistant scores, and a qualitative review found that some discrepancies came from TA errors rather than system errors. Survey responses from 104 students were significantly above the neutral midpoint on all Likert items, and 93 students said they preferred the solution plus AI comment over the solution alone. The authors interpret this as evidence that the system can deliver consistent, high-quality feedback that supports learning, especially for weaker students, who gave significantly higher ratings on four of the ten questions.

Load-bearing premise

The claim that the system beats traditional grading methods rests on a survey that asked students to compare AI feedback against a written solution alone, not against teaching-assistant grading, so the positive result could simply mean students like receiving extra comments rather than that they prefer AI to human graders.

Editorial extensions

If this is right

  • Assignments in fast-changing or specialized courses can be graded without building labeled training sets, since the system works with the question and a prompt.
  • Students receive feedback they can act on immediately, including why an answer is wrong and how to improve, at a scale that makes it feasible for large classes.
  • Teachers get an automatically generated performance summary that highlights common mistakes, which can guide review sessions.
  • Weaker students may benefit most: they rated the feedback significantly higher on identifying mistakes, actionable improvement steps, future problem-solving, and motivation.
  • Grading consistency could improve, since some observed discrepancies between AAG and human scores traced to human grading errors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the Q10 survey compared 'solution alone' with 'solution plus AI comment', the paper's conclusion about superiority over traditional grading is not yet supported; a head-to-head student preference test between TA feedback and AAG feedback would be a direct test.
  • Correlation with human scores does not by itself validate accuracy; if the human scores contain systematic errors, an LLM could correlate with those errors. Rescoring against an expert-created gold standard would show whether AAG is actually closer to correct answers than TAs are.
  • If zero-shot grading proves reliable, the natural next step is iterative resubmission—students revise and the system regrades—which the authors list as future work.
  • The same prompt-engineering pattern could transfer to written essays or lab reports, where marking schemes are more rubric-like, but the paper only demonstrates statistics assignments with short explanations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a zero-shot LLM framework (AAG) that grades student assignment submissions and generates personalized feedback without fine-tuning. The authors evaluate the system on two open-ended questions from a statistics course, reporting Pearson correlations of 0.75 and 0.82 between AAG and human teaching assistant (TA) scores for 150 submissions, and they administer a 10-question survey to 104 students about the feedback experience. The paper concludes that the AAG system is closely aligned with human grading and outperforms traditional grading methods in improving student motivation, understanding, and preparedness.

Significance. If the evidence supported the claims, the framework would be a valuable contribution to the automated grading literature: it avoids fine-tuning, provides tailored and actionable feedback, and evaluates student experience in a real higher-education setting. The authors also make code and data publicly available, which supports reproducibility. However, the current evidence does not establish the central comparative claim against traditional grading, and the grading-alignment evidence relies on a weak metric without agreement or uncertainty quantification. The framework's potential is real, but the paper overstates what its data show.

major comments (4)
  1. [Section 4.2, Section C, Conclusion] Section 4.2 states that Q10 asked students to choose between 'the traditional TA grading method and the AAG system feedback,' but the survey reproduced in Section C shows Q10 actually asks: 'Overall, which did you find more helpful for understanding: the solution alone or the solution with the AI feedback?' with options 'Solution alone' and 'Solution + AI Comment.' Questions Q1-Q9 are likewise all phrased as comparisons against 'the solution alone' or 'a general solution.' Therefore the 93/104 preference is for receiving AI comments in addition to a solution, not for AAG over TA grading. The conclusion's claim that the system is 'outperforming traditional grading methods' is unsupported by the instrument. This is the paper's headline student-facing claim and must be corrected—either by revising the claims to match what the survey actually measured or by running a comparison against TA feedback.
  2. [Section 4.1] Grading alignment is reported only as Pearson correlations of 0.75 and 0.82 for two open-ended questions, with no confidence intervals, no agreement metrics (e.g., quadratic weighted kappa, exact/adjacent agreement rates), and no ground-truth data, which the section explicitly acknowledges. Pearson correlation measures linear association, not agreement; two graders can be perfectly correlated yet systematically differ (e.g., one consistently assigns higher scores or uses a compressed scale). Without the distribution of score differences or at least per-grader means and standard deviations, the claim that AAG scores are 'closely aligned' with human evaluations is not established. The paper also asserts in Section 5 that the AAG system provides 'more consistent' grading than humans, but no inter-rater reliability comparison between TAs and AAG (or between TAs) is provided.
  3. [Section 3.2] The selection of GPT-4 among Llama-3, Qwen-2, Claude-3, and Gemini-1.5 is attributed to 'a comparative evaluation using identical prompts' with performance assessed by university lecturers, but no details of this evaluation are given: no data, criteria, number of lecturers, or results. This is a non-reproducible assertion that underlies the system's core model choice. Please provide the evaluation protocol and results, or explicitly state that the model choice was heuristic rather than empirically established.
  4. [Section 4.2 and Appendix B] The weak-versus-strong student comparison performs ten Mann-Whitney U tests and reports four significant differences at p<0.05 without any multiple-comparison correction. With ten independent tests, the expected number of false positives at alpha=0.05 is 0.5, and the family-wise error rate is about 40%. The conclusion that weaker students are more satisfied in four key areas is therefore fragile; please report adjusted p-values (e.g., Holm-Bonferroni) or a pre-specified analysis plan, and clarify whether the grouping variable is based on AAG scores, TA scores, or something else.
minor comments (5)
  1. [Section 2] The definition of few-shot learning as 'supervised learning with limited training epochs' is nonstandard; few-shot learning conventionally refers to learning from a limited number of labeled examples, not a limited number of epochs. Please correct or clarify.
  2. [Figure 5] The grading distributions in Figure 5 should include clear axis labels, legends for human and AAG scores, and perhaps overlaid densities; the current figure is difficult to interpret without these elements.
  3. [Appendix A, Table 3] For Q10, the binomial test statistic is reported as 0.8942, but a binomial test typically reports a proportion or a z-statistic; please clarify what this value represents and how the p-value was computed.
  4. [Section C] The survey header collects 'Your SID' despite reassuring students that responses would not impact assignment scores; if anonymity was intended, collecting student IDs is problematic. Please justify or remove the SID field.
  5. [Throughout] There are several grammatical and typographical issues, including 'less then or equal to' in Appendix A, 'these analysis highlights' in Section 4.1, and the label 'non-complementary input' in Figure 1, which is unclear and appears to mean 'optional input.' A careful copyedit is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper's evaluation is externally benchmarked against human TA scores, and its survey claims, while containing a wording mismatch, are not self-referential derivations.

full rationale

The AAG system's grading accuracy is assessed by Pearson correlation with human TA scores (r = 0.75 and 0.82), which is an external comparison rather than a quantity defined by the system's own inputs. No parameter is fitted from the target outcome, and no prediction is constructed from the data it claims to predict. The survey measures student self-reported perceptions of AI feedback compared with a solution-only condition; although this is self-report, it is not a derivation that reduces to its own inputs. There are no load-bearing self-citations: the cited works are standard external references (GPT-4, Llama, etc.), not prior papers by the same authors invoked to justify the central claim. The manuscript's own limitation statement acknowledges that direct validation against ground-truth grading was not possible, which is an honest caveat rather than a circular step. The notable issue is that Section 4.2 and the Conclusion describe Q10 as comparing AAG with traditional TA grading, whereas the actual survey instrument in Section C asks students to choose between 'the solution alone' and 'the solution with the AI feedback.' That is a validity or reporting mismatch, not circularity: the 93/104 preference is an empirical finding that may not support the conclusion drawn, but it is not used as an input to derive the outcome. Likewise, students rating the system's own feedback positively is a self-report limitation, not a constructional equivalence. Because no claimed result is equivalent by definition or by fitted construction to its inputs, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The system has no learnable parameters introduced by the authors; the central evidence rests on three assumptions: GPT-4 as a reliable zero-shot grader, self-reported survey responses as learning outcomes, and Pearson correlation as a measure of grading agreement.

assumptions (4)
  • domain assumption GPT-4, given a well-structured prompt and marking scheme, produces grading scores that align with human judgments.
    The entire system depends on this; invoked in Section 3.2 and tested indirectly in Section 4.1 via correlation.
  • domain assumption Student self-reported Likert ratings about feedback helpfulness are a valid proxy for actual learning improvement.
    Section 4.2 interprets survey responses as evidence that the system 'significantly improved' understanding and motivation.
  • domain assumption The two open-ended STAT1011 questions and the human TA scores used for comparison are representative of the broader grading tasks and standards in the course.
    Section 4.1 uses only two questions from one course; the conclusion generalizes to higher education.
  • domain assumption Pearson correlation between two sets of scores is an appropriate measure of grading agreement.
    Section 4.1 reports Pearson r as the main evidence of alignment, without agreement metrics or bias analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Zero-Shot LLM Framework for Automatic Assignment Grading in Higher Education." pith.science (2026). https://pith.science/paper/X2OCXY6I

@misc{pith2026250114305,
  author       = {Pith},
  title        = {Pith review of: A Zero-Shot LLM Framework for Automatic Assignment Grading in Higher Education},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X2OCXY6I}},
  note         = {Machine review of arXiv:2501.14305}
}
read the original abstract

Automated grading has become an essential tool in education technology due to its ability to efficiently assess large volumes of student work, provide consistent and unbiased evaluations, and deliver immediate feedback to enhance learning. However, current systems face significant limitations, including the need for large datasets in few-shot learning methods, a lack of personalized and actionable feedback, and an overemphasis on benchmark performance rather than student experience. To address these challenges, we propose a Zero-Shot Large Language Model (LLM)-Based Automated Assignment Grading (AAG) system. This framework leverages prompt engineering to evaluate both computational and explanatory student responses without requiring additional training or fine-tuning. The AAG system delivers tailored feedback that highlights individual strengths and areas for improvement, thereby enhancing student learning outcomes. Our study demonstrates the system's effectiveness through comprehensive evaluations, including survey responses from higher education students that indicate significant improvements in motivation, understanding, and preparedness compared to traditional grading methods. The results validate the AAG system's potential to transform educational assessment by prioritizing learning experiences and providing scalable, high-quality feedback.

Figures

Figures reproduced from arXiv: 2501.14305 by the authors.

Figure 1
Figure 1. Overview of the Zero-Shot LLM-Based AAG System. The bolded box highlights the core innovation of this study: an AAG system integrating prompt engineering, zero￾shot LLM capabilities, and tailored feedback for students. The dashed line box indicates non-complementary input. 2 Related Work Automated Grading Systems. Automated grading tools were first introduced in the 1960s for programming [19] and essay evaluation [2… view at source ↗
Figure 2
Figure 2. Evaluation prompt for the AAG system. The angle brackets < > indicate placeholders for specific content. The question and answer placeholders can represent multiple questions and answers, respectively, as a single question may include several subquestions and rely on previous answers. This prompt is designed to incorporate all prior context while specifically evaluating the last included question and its corre￾spond… view at source ↗
Figure 3
Figure 3. AAG system student feedback example [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: AAG system teachers feedback example. (A) and (B) provide summaries based on the student’s submission and the AAG score, while (C) presents a summary of the AAG feedback generated using LLM. For teachers, the system generates a “Performance Summary”, providing an overa…
Figure 5
Figure 5. Figure 5: Human and AAG system grading distribution of question 1 (left) and question 2 (right) in STAT1011. the refined marking scheme function, which ensures more consistent and precise grading. A further qualitative analysis of scoring discrepancies across all assignment ques…
Figure 6
Figure 6. Figure 6: Distribution of survey responses. For questions 1 through 9, responses follow a Likert scale where 5 indicates the highest rating and 1 indicates the lowest. The left plot displays the distribution of responses from all students, while the right plot shows the mean rat…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 29 canonical work pages

  1. [22]

    Kortemeyer, G.: Can an ai-tool grade assignments in an introductory physics course? arXiv preprint arXiv:2304.11221 (2023)

    work page Pith review arXiv 2023

  2. [1]

    arXiv preprint arXiv:2303.08774 (2023)

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  3. [2]

    In: Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I 21

    Azad, S., Chen, B., Fowler, M., West, M., Zilles, C.: Strategies for deploying unre- liable ai graders in high-transparency high-stakes exams. In: Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I 21. pp. 16–28. Springer (2020)

    work page 2020

  4. [3]

    Advances in neural information processing systems33, 1877–1901 (2020)

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

    2020

  5. [4]

    Proceed- ings of the Second Melbourne Computing Education Conventicle pp

    Burrows, S., D’Souza, D.: Management of teaching in a complex setting. Proceed- ings of the Second Melbourne Computing Education Conventicle pp. 1–8 (2005)

    work page 2005

  6. [5]

    Leveraging Large Language Model and Story-Based Gamification in Intelligent Tutoring System to Scaffold Introductory Programming Courses: A Design-Based Research Study

    Cao, C.: Leveraging large language model and story-based gamification in intelli- gent tutoring system to scaffold introductory programming courses: A design-based research study. arXiv preprint arXiv:2302.12834 (2023)

    work page Pith review arXiv 2023

  7. [6]

    Chen, C.F.E., Cheng, W.Y.E.C.: Beyond the design of automated writing evalua- tion: Pedagogical practices and perceived learning effectiveness in efl writing classes (2008)

    work page 2008

  8. [7]

    In: International Conference on Artificial Intelligence in Education

    Chen, S., Lan, Y., Yuan, Z.: A multi-task automated assessment system for essay scoring. In: International Conference on Artificial Intelligence in Education. pp. 276–283. Springer (2024)

    work page 2024

Show all 40 references
  1. [8]

    arXiv preprint arXiv:2407.05216 (2024)

    Chiang, C.H., Chen, W.C., Kuan, C.Y., Yang, C., Lee, H.y.: Large language model as an assignment evaluator: Insights, feedback, and challenges in a 1000+ student course. arXiv preprint arXiv:2407.05216 (2024)

    2024 arXiv

  2. [9]

    In: International Conference on Artificial Intelligence in Education

    Condor, A., Pardos, Z.: Explainable automatic grading with neural additive mod- els. In: International Conference on Artificial Intelligence in Education. pp. 18–31. Springer (2024)

    2024

  3. [10]

    Practical assessment, research & evaluation15(11), 1–12 (2010)

    De Winter, J.C., Dodou, D.: Five-point likert items: t test versus mann-whitney- wilcoxon. Practical assessment, research & evaluation15(11), 1–12 (2010)

    2010

  4. [11]

    arXiv preprint arXiv:1810.04805 (2018)

    Devlin, J.: Bert: Pre-training of deep bidirectional transformers for language un- derstanding. arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  5. [12]

    Yeung et al

    Dikli, S., Bleyle, S.: Automated essay scoring feedback for second language writers: How does it compare to instructor feedback? Assessing writing22, 1–17 (2014) 14 C. Yeung et al

    2014

  6. [13]

    arXiv preprint arXiv:2407.21783 (2024)

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al.: The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  7. [14]

    The Journal of Technology, Learning and Assessment 8(6) (2010)

    Grimes, D., Warschauer, M.: Utility in a fallible tool: A multi-site case study of au- tomated writing evaluation. The Journal of Technology, Learning and Assessment 8(6) (2010)

    2010

  8. [15]

    Haley, D., Thomas, P., De Roeck, A., Petre, M.: Measuring improvement in latent semantic analysis-based marking systems: using a computer to mark questions about html (2007)

    2007

  9. [16]

    arXiv preprint arXiv:2501.00274 (2024)

    Hashemi, H., Eisner, J., Rosset, C., Van Durme, B., Kedzie, C.: Llm-rubric: A multidimensional,calibratedapproachtoautomatedevaluationofnaturallanguage texts. arXiv preprint arXiv:2501.00274 (2024)

    2024 arXiv

  10. [17]

    In: Second Joint Conference on Lexical and Computational Semantics (* SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013)

    Heilman, M., Madnani, N.: Ets: Domain adaptation and stacking for short answer scoring. In: Second Joint Conference on Lexical and Computational Semantics (* SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013). pp. 275–279 (2013)

    2013

  11. [18]

    Hirschman,L.:Automatedgradingofshort-answertests.IEEEIntelligentSystems, Trends and Controversies section15(5), 22–37 (2000)

    2000

  12. [19]

    Communications of the ACM 3(10), 528–529 (1960)

    Hollingsworth, J.: Automatic graders for programming classes. Communications of the ACM 3(10), 528–529 (1960)

    1960

  13. [20]

    Hsu, S., Li, T.W., Zhang, Z., Fowler, M., Zilles, C., Karahalios, K.: Attitudes surroundinganimperfectaiautograder.In:Proceedingsofthe2021CHIconference on human factors in computing systems. pp. 1–15 (2021)

    2021

  14. [21]

    Impey, C., Wenger, M., Garuda, N., Golchin, S., Stamer, S.: Using large language models for automated grading of student writing about science (2024)

    2024

  15. [23]

    System44, 66–78 (2014)

    Li, Z., Link, S., Ma, H., Yang, H., Hegelheimer, V.: The role of automated writing evaluation holistic scores in the esl classroom. System44, 66–78 (2014)

    2014

  16. [24]

    In: 2019 IEEE/ACM 41st Interna- tional Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET)

    Liu, X., Wang, S., Wang, P., Wu, D.: Automatic grading of programming assign- ments: an approach based on formal semantics. In: 2019 IEEE/ACM 41st Interna- tional Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). pp. 126–137. IEEE (2019)

    2019

  17. [25]

    arXiv preprint arXiv:2308.02439 (2023)

    Matelsky, J.K., Parodi, F., Liu, T., Lange, R.D., Kording, K.P.: A large language model-assisted education tool to provide feedback on open-ended responses. arXiv preprint arXiv:2308.02439 (2023)

    2023 arXiv

  18. [26]

    Messer, M., Brown, N.C., Kölling, M., Shi, M.: Automated grading and feedback toolsforprogrammingeducation:Asystematicreview.ACMTransactionsonCom- puting Education 24(1), 1–43 (2024)

    2024

  19. [27]

    Research Methods in Applied Linguistics2(2), 100050 (2023)

    Mizumoto, A., Eguchi, M.: Exploring the potential of using an ai language model for automated essay scoring. Research Methods in Applied Linguistics2(2), 100050 (2023)

    2023

  20. [28]

    In: Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications

    Mizumoto, T., Ouchi, H., Isobe, Y., Reisert, P., Nagata, R., Sekine, S., Inui, K.: Analytic score prediction and justification identification in automated short answer scoring. In: Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applica...

    2019

  21. [29]

    Page,E.B.:Theimminenceof...gradingessaysbycomputer.ThePhiDeltaKappan 47(5), 238–243 (1966)

    1966

  22. [30]

    OpenAI blog1(8), 9 (2019) Automatic Assignment Grading in Higher Education 15

    Radford,A.,Wu,J.,Child,R.,Luan,D.,Amodei,D.,Sutskever,I.,etal.:Language models are unsupervised multitask learners. OpenAI blog1(8), 9 (2019) Automatic Assignment Grading in Higher Education 15

    2019

  23. [31]

    In: Proceedings of the 12th workshop on in- novative use of NLP for building educational applications

    Riordan, B., Horbach, A., Cahill, A., Zesch, T., Lee, C.: Investigating neural ar- chitectures for short answer scoring. In: Proceedings of the 12th workshop on in- novative use of NLP for building educational applications. pp. 159–168 (2017)

    2017

  24. [32]

    Language Testing38(2), 247–272 (2021)

    Shin, J., Gierl, M.J.: More efficient processes for creating automated essay scoring frameworks: A demonstration of two algorithms. Language Testing38(2), 247–272 (2021)

    2021

  25. [33]

    Sung, C., Dhamecha, T., Saha, S., Ma, T., Reddy, V., Arora, R.: Pre-training bert on domain resources for short answer grading. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural La...

    2019

  26. [34]

    arXiv preprint arXiv:2403.05530 (2024)

    Team, G., Georgiev, P., Lei, V.I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vin- cent,D.,Pan,Z.,Wang,S.,etal.:Gemini1.5:Unlockingmultimodalunderstanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 (2024)

    2024 arXiv

  27. [35]

    In: Proceedings of the 50th ACM Technical Symposium on Computer Science Ed- ucation

    Ureel II, L.C., Wallace, C.: Automated critique of early programming antipatterns. In: Proceedings of the 50th ACM Technical Symposium on Computer Science Ed- ucation. pp. 738–744 (2019)

    2019

  28. [36]

    arXiv preprint arXiv:2109.01652 (2021)

    Wei, J., Bosma, M., Zhao, V.Y., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., Le, Q.V.: Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021)

    2021 arXiv

  29. [37]

    arXiv preprint arXiv:2401.06431 (2024)

    Xiao, C., Ma, W., Xu, S.X., Zhang, K., Wang, Y., Fu, Q.: From automation to augmentation: Large language models elevating essay scoring landscape. arXiv preprint arXiv:2401.06431 (2024)

    2024 arXiv

  30. [38]

    5 technical report

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al.: Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  31. [39]

    In: International Conference on Artificial Intelligence in Education

    Zeng, Z., Li, L., Guan, Q., Gašević, D., Chen, G.: Generalizable automatic short answer scoring via prototypical neural network. In: International Conference on Artificial Intelligence in Education. pp. 438–449. Springer (2023) 16 C. Yeung et al. A Hypothesis Testing for Surve...

    2023

  32. [40]

    – Alternative Hypothesis (H1): The median response for each question (Q1 to Q9) is greater than 3

    Wilcoxon Signed-Rank Test(for Q1 to Q9) – Null Hypothesis (H0): The median response for each question (Q1 to Q9) is less then or equal to 3. – Alternative Hypothesis (H1): The median response for each question (Q1 to Q9) is greater than 3. 2.Binomial Test(for Q10) – Null Hypot...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.