Pith. sign in

REVIEW 5 major objections 5 minor 49 references

Do Students Rely on AI? Analysis of Student-ChatGPT Conversations from a Field Study

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In real STEM quizzes, students followed ChatGPT's advice in about 52% of interactions, yet only about half of those AI-guided attempts produced a correct answer.

desk verdict Real field data and a useful taxonomy, but the 'no over-reliance' claim is less secure than it looks; the follow label is too thinly validated and the coding scheme hides the canonical over-reliance case. read the letter →

arxiv 2508.20244 v1 pith:JSSW2SU7 submitted 2025-08-27 cs.AI

Jiayu Zheng , Lingxin Hao , Kelun Lu , Ashi Garg , Mike Reese , Melo-Jean Yap , I-Jeng Wang , Xingyun Wu
show 8 more authors
Wenrui Huang Jenna Hoffman Ariane Kelly My Le Ryan Zhang Yanyu Lin Muhammad Faayez Anqi Liu
This is my paper
classification cs.AI
keywords AIreliancehuman-AIinteractiongenerativeineducationtaxonomystudent-ChatGPTconversationsfieldstudybehavioralpredictionappropriate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish what college students actually do when a ChatGPT-4 quiz assistant is placed inside real STEM classrooms: do they follow its advice, ignore it, or misuse it? Analyzing 315 recorded conversations from 182 students, it reports that overall reliance was low and miscalibrated — students followed the AI's recommendations in about 52% of interactions, and only about half of those AI-guided attempts ended with a correct answer. The most common single scenario (about 21%) was 'failed application,' in which students took correct, relevant AI advice and still submitted a wrong answer; the authors read this, together with the persistence of negative patterns across a student's successive attempts, as evidence that many students could not effectively use AI for learning during a short, unfamiliar exposure. As a reusable contribution, the paper offers a four-stage reliance taxonomy that measures reliance without needing to observe students' private initial answers, and it shows that a few observable behaviors — how closely the prompt paraphrases the quiz text, how long the conversation lasts, and how complex the language is — predict AI adoption with up to 0.778 accuracy.

What carries the argument

The load-bearing object is the four-stage reliance taxonomy, which compresses each student–AI conversation into a four-digit code (AI correctness, AI relevance, student adoption, final-answer correctness) mapped to twelve named scenarios such as 'Appropriate reliance' (1-1-1-1), 'Failed application' (1-1-1-0), and 'Inappropriate self-reliance' (1-1-0-0). Every headline result — the 52/48 follow split, the dominance of failed application, the claim of no over-reliance — is a distribution statement over these codes, and the taxonomy's distinctive move is measuring reliance without observing any initial human decision. The supporting machinery is the prediction pipeline: behavioral features (ti

What would settle it

Recode the same 315 conversations with AI relevance scored independently of AI correctness, so that an on-topic-but-wrong AI answer can be 'relevant.' If the count of students who followed wrong-but-relevant advice is comparable to the 46 'inappropriate self-reliance' cases — rather than the 14 'inappropriate reliance' cases now reported — the paper's claim that students did not over-rely on AI collapses. This is a coding change only; no new data are needed.

Watch

Extended reading notes

Core claim

Coding 315 student–ChatGPT quiz conversations with a four-stage taxonomy — AI answer correct? relevant? did the student follow it? was the final answer correct? — the paper's central claim is that student reliance on AI in authentic classrooms was miscalibrated, not runaway. Students split almost evenly between following the AI (~52%) and trusting their own reasoning, and only about half of the AI-guided attempts ended correctly. The largest scenario (66 of 315) was 'failed application': students adopted correct, relevant AI advice and still answered wrong. More students ignored good advice and failed (46) than followed bad advice and failed (14), which the authors read as evidence against a

Load-bearing premise

The 'students did not over-rely on AI' conclusion rests on the taxonomy's coding rule that a factually wrong AI answer is automatically labeled irrelevant, which makes it impossible to count the classic case of a student following wrong but on-topic AI advice; if relevance were measured independently of correctness, the over-reliance picture could look different.

Editorial extensions

If this is right

  • If reliance is miscalibrated rather than runaway, the right intervention is calibration — onboarding and real-time feedback — rather than restricting or banning AI tools.
  • The prevalence of 'failed application' (following correct advice and still failing) means educators cannot equate 'used AI' with 'got the answer from AI'; the failure mode itself should shape assessment design.
  • Because prompt-quiz similarity, interaction time, and language complexity predict adoption, an AI interface could detect when a student is about to follow advice and prompt verification — a mechanism the paper explicitly suggests for future design.
  • The persistence of negative trajectories implies that one-shot orientation is insufficient; students need repeated, strategy-level feedback across several interactions to shift from failed to appropriate reliance.
  • The taxonomy transfers to other learning contexts because it does not require knowing a student's pre-AI answer, which the paper argues most lab-based reliance measures cannot do.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the taxonomy defines an incorrect AI answer as automatically irrelevant, the classic over-reliance case — following on-topic but wrong AI advice — is unmeasurable in this data; coding relevance independently of correctness would likely shift the reported 46-vs-14 balance and could weaken the 'no over-reliance' reading.
  • The same behavioral features that predict AI adoption could be repurposed as an audit trail — flagging which submitted quiz answers likely originated from AI contact — a use the paper does not pursue.
  • The study measured students in weeks 9–11 of Fall 2023, when ChatGPT was still new; the low-reliance distribution may be period-specific, and a replication after students gained familiarity could show opposite errors (complacent adoption), which would test whether calibration problems are intrinsic or just an onboarding artifact.
  • A turn-level version of the taxonomy could localize where within a conversation students go astray — e.g., whether 'failed application' stems from an early misinterpretation or a late arithmetic slip; the paper only analyzes at conversation granularity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper reports a field study of 315 conversations between 182 college students and ChatGPT-4 during STEM quizzes. The authors introduce a four-stage reliance taxonomy (AI correctness, relevance, student adoption, final answer correctness) that collapses to 12 scenarios, and report that the most frequent scenarios were Failed application (66, 20.9%) and Appropriate reliance (65, 20.6%); students followed AI advice in about 52% of cases. They then construct individual-level reliance trajectories and train classifiers to predict adoption, reporting best test accuracy 0.778 with behavioral+text features. The conclusions emphasize low and frequently ineffective reliance, persistence of negative patterns, and behavioral predictors of AI adoption.

Significance. If the measurement claims are valid, the study provides a rare observational dataset on spontaneous GenAI use in authentic coursework and a taxonomy that does not depend on an elicited initial human decision, which is a useful contribution. The trajectory analysis and behavioral-feature analysis are also valuable. However, the paper's headline conclusions are contingent on two load-bearing methodological choices: the definitional nesting of relevance under correctness, and the GPT-4o-derived 'follow' label. The manuscript's own appendix concedes the follow construct is hard to observe, and the reported validation is not sufficient to secure the 52/48 split. The taxonomy's potential is real, but the empirical findings as stated are not fully supported. Credit is due for internally consistent descriptive counts (table rows sum to 315), transparent prompt templates in the appendix, and a clearly defined prediction target.

major comments (5)
  1. [Measurements, Reliance Taxonomy; Table 1] The taxonomy defines relevance conditional on correctness: 'an incorrect response cannot be relevant by definition.' This removes four 0-1-*-* codes from the 16-code space, including the case of a student following on-topic but wrong AI advice. The paper's evidence against over-reliance is the larger count of Inappropriate self-reliance (46) than the two Inappropriate reliance categories combined (1+13=14), but both Inappropriate reliance rows in Table 1 have relevance=0. An incorrect but on-topic ChatGPT answer that is followed and leads to an incorrect final answer is not represented anywhere. The comparison therefore reflects the coding convention, not a free empirical measurement. Please re-code relevance independently of correctness and show how the scenario counts and the over-reliance conclusion change, or justify the nesting explicitly and report the suppressed cells.
  2. [Methods, Constructing Reliance Taxonomy; Appendix follow prompt] The outcome that drives every reliance count is the GPT-4o follow label. Validation is described only as 'alignment rate ... exceeds 85%' over the remaining annotated cases (~25 cases), aggregated across the three stages; no per-stage accuracy, confidence interval, or human inter-annotator agreement is reported. The follow prompt itself instructs the model that 'it is hard to identify whether students follow GPT's idea without students' thinking process. Just try your best to extract any signals.' This is an explicit limitation in the manuscript and should be treated as such. If follow labels are noisy or biased, the 52/48 split and all scenario counts shift. Please report per-stage alignment with confidence intervals on a larger validation set, and provide a qualitative error analysis for the follow stage.
  3. [Figure 7, Example 1] The illustrative failed-application example contradicts Table 1. The explanation says 'The chatbot gave a correct and relevant answer, but the student did not follow it and obtained an incorrect result,' which corresponds to code 1-1-0-0 (Inappropriate self-reliance), not 1-1-1-0 (Failed application). If an example selected to demonstrate the coding can be misclassified, the reliability of the Failed-application count (N=66) is not assured. Please correct the example or its explanation and add a consistency check in the annotation protocol.
  4. [Methods, Predicting AI-adoption; Table 4] The train/test split is at the level of student–quiz pairs rather than students. A student can appear in both training and test sets, allowing the models to exploit individual-specific patterns and inflating the reported accuracy. In addition, the 0.778 best accuracy is reported without confidence intervals or significance tests against the 0.52 majority baseline. Please use a student-level split or repeated cross-validation, and report per-fold variability and a baseline comparison.
  5. [Results, Table 3] The claim that negative reliance patterns 'often persisted' is based on very small counts: Failed application–Failed application (N=3), Inappropriate self-reliance–Failed application (N=5), Appropriate reliance–Failed application (N=4), etc. Among the 95 students with two or more interactions, these are small minorities. Please report the proportion of multi-interaction students who exhibit repeated negative trajectories, and avoid 'often' unless that proportion is large enough to support it.
minor comments (5)
  1. [Abstract; Results] The abstract says students exhibited 'overall low reliance on AI,' but the Results report that students followed AI advice in about 52% of interactions. Please clarify that the intended claim is low appropriate reliance or low effective use, not low follow.
  2. [Table 2 notes] The note says category (1) 'AI is correct but irrelevant, students follow advice and get the wrong answer (46 cases),' but Table 2 lists 1 case for that category; 46 is the count for Inappropriate self-reliance. The note should be corrected.
  3. [Table 1, Serendipitous success] The 0-0-1-1 code defines 'Serendipitous success' as following a wrong and off-topic answer yet arriving at a correct final answer. The mechanism is not explained; this code needs a concrete illustration or should be merged/excluded.
  4. [Throughout] Please correct typos: 'simialrity' (similarity), 'inetracting' (interacting), 'stduents' (students), 'adpotion' (adoption), 'predicators' (predictors), 'problen' (problem).
  5. [Figure 1] The figure is linked to an external Canva URL. The figure should be embedded in the manuscript so the published version is self-contained.

Circularity Check

1 steps flagged · score 3.0 of 10

One definitional nesting (relevance forced to 0 when GPT is incorrect) partially constrains the reliance taxonomy, but the central empirical counts and the ML prediction are not circularly derived.

  1. self definitional [Measurements, Reliance Taxonomy; Table 1; Figure 1 caption]
    "Next, we assess whether a correct response actually addresses the student’s question (an incorrect response cannot be relevant by definition)."

    The relevance label is defined as contingent on the correctness label: whenever correctness=0, relevance is forced to 0. This collapses the 16 possible four-bit reliance codes to 12 and makes the theoretically important category 'GPT answer wrong but on-topic' unobservable. All scenario counts and the statement that students 'did not over-rely on AI' are computed under this constraint; cases of following wrong but relevant AI advice are relabeled as 'irrelevant' (Table 1: 0-0-1-x). The taxonomy's relevance stage is therefore not an independent measurement but a deterministic function of correctness, so reliance-scenario distributions are partly built into the coding scheme rather than freely observed.

full rationale

The paper's central descriptive claims are empirical codings of 315 conversations, not derivations, so most of the analysis is not circular in the strong sense. The one structural circularity is the definitional nesting of the relevance stage: relevance is forced to 0 whenever correctness is 0, collapsing 16 codes to 12 and making wrong-but-on-topic AI advice unobservable. This does not, by itself, force the headline 52/48 follow split or the 46-vs-14 comparison (those counts are driven by the follow/final-answer labels), but it does shape the scenario labels and the reported 'relevance' distribution (32.7% irrelevant), so the taxonomy's four stages are not four independent measurements. Separately, the load-bearing 'student follow' label is produced by GPT-4o under a prompt that concedes 'it is hard to identify whether students follow GPT's idea without students' thinking process. Just try your best to extract any signals' (Appendix). This is a measurement-validity limitation, not a circular reduction: the GPT-4o annotator does not see the computed behavioral features used in the prediction task, and the ML accuracy of 0.778 is a genuine out-of-sample result. The paper's self-citations (e.g., Cao, Liu, and Huang 2024) are background references and not load-bearing. Overall score 3: one definitional step partially constrains the taxonomy, but the central empirical claims retain independent content.

Assumptions & free parameters 4 free parameters · 6 assumptions · 1 invented entities

The descriptive findings rest on an LLM-judge labeling pipeline with a stated definitional constraint (relevance nested in correctness) and an unreported-size validation set; the predictive and clustering findings rest on hand-chosen parameters (PCA k=8, edit costs, dendrogram cuts) and unreported hyperparameters. No external benchmark validates the twelve scenario labels; the final-answer grade that anchors stage 4 is part of the coding itself. This is an empirical study, so the axioms are measurement assumptions rather than mathematical postulates.

free parameters (4)
  • PCA dimensionality (k=8) for text embeddings = 8 components explaining 43% of variance
    Hand-chosen reduction of the 1,042-dimension embedding; the modest explained variance and the absence of a selection criterion make the choice ad hoc.
  • Edit-distance costs for sequence clustering = insert/delete = 1, substitution = 2
    Hand-set so substitutions are penalized more than gaps; no sensitivity analysis is reported for the resulting clusters.
  • Number of conversation clusters = 4 for question-type sequences, 3 for Bloom-coded sequences
    Chosen by cutting dendrograms; the cluster counts determine the reported archetypes (conceptual/procedural/calculation/balanced; remember/understand/apply).
  • Classifier hyperparameters (DT, RF, XGB, SVM, LR) = not reported
    Tuned via cross-validation on the training split, but exact settings are omitted, so the reported accuracies are fit-dependent.
assumptions (6)
  • ad hoc to paper An incorrect response cannot be relevant by definition
    Measurements, Reliance Taxonomy: 'an incorrect response cannot be relevant by definition'. This nesting couples stages 1 and 2, collapses 16 codes to 12, and makes 'followed on-topic wrong advice' unmeasurable.
  • domain assumption GPT-4o labels approximate human gold standard at >85% alignment
    Methods, Constructing Reliance Taxonomy: alignment 'exceeds 85%' on an unspecified held-out subset; the full 315-conversation labels rest on this.
  • domain assumption Student 'follow' behavior is inferable from conversation text and the submitted answer
    Appendix, follow-label prompt concedes: 'it is hard to identify whether students follow GPT's idea without students' thinking process. Just try your best.'
  • domain assumption Flesch-Kincaid grade level and all-distilroberta-v1 similarity operationalize complexity and copying
    Measurements, Behavioral and Text Features: no validation of these metrics against human judgments of copying or complexity for this data.
  • domain assumption TA grading per instructor instructions is ground truth for final-answer correctness
    Data section: 'The quiz answers were graded by the teaching assistants according to the grading instructions provided by the instructors.'
  • standard math Edit-distance costs with Ward linkage preserve meaningful temporal conversation patterns
    Methods, sequence clustering: standard agglomerative clustering is applied; the specific cost choice is a hand-set parameter (see free parameters).
invented entities (1)
  • Four-stage reliance taxonomy with 12 named scenarios (Appropriate reliance, Failed application, etc.)
    purpose: Conversation-level classification of student-AI reliance without requiring an initial human decision
    The scenario labels are assigned by the same pipeline that defines them; the final-answer grade anchoring stage 4 is part of the coding, and the stages are not validated against external measures such as learning gains, student self-reports, or think-aloud protocols.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do Students Rely on AI? Analysis of Student-ChatGPT Conversations from a Field Study." pith.science (2026). https://pith.science/paper/JSSW2SU7

@misc{pith2026250820244,
  author       = {Pith},
  title        = {Pith review of: Do Students Rely on AI? Analysis of Student-ChatGPT Conversations from a Field Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JSSW2SU7}},
  note         = {Machine review of arXiv:2508.20244}
}
read the original abstract

This study explores how college students interact with generative AI (ChatGPT-4) during educational quizzes, focusing on reliance and predictors of AI adoption. Conducted at the early stages of ChatGPT implementation, when students had limited familiarity with the tool, this field study analyzed 315 student-AI conversations during a brief, quiz-based scenario across various STEM courses. A novel four-stage reliance taxonomy was introduced to capture students' reliance patterns, distinguishing AI competence, relevance, adoption, and students' final answer correctness. Three findings emerged. First, students exhibited overall low reliance on AI and many of them could not effectively use AI for learning. Second, negative reliance patterns often persisted across interactions, highlighting students' difficulty in effectively shifting strategies after unsuccessful initial experiences. Third, certain behavioral metrics strongly predicted AI reliance, highlighting potential behavioral mechanisms to explain AI adoption. The study's findings underline critical implications for ethical AI integration in education and the broader field. It emphasizes the need for enhanced onboarding processes to improve student's familiarity and effective use of AI tools. Furthermore, AI interfaces should be designed with reliance-calibration mechanisms to enhance appropriate reliance. Ultimately, this research advances understanding of AI reliance dynamics, providing foundational insights for ethically sound and cognitively enriching AI practices.

Figures

Figures reproduced from arXiv: 2508.20244 by the authors.

Figure 1
Figure 1. Visualizing Reliance Taxonomy. Based on our identification of the 4 labels as the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 4
Figure 4. Remember-driven Cluster [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Understand-driven Cluster with Ward’s linkage to these distance matrices (Solomon et al. 2022). When clustering the question-type sequences and cutting the resulting dendrogram into four groups, we uncover distinct prototypical patterns: conceptual-driven (N = 48), procedural-driven (N = 52), calculation-driven (N = 52), and the balanced cluster (N = 35). Applying the same pipeline to Bloom-coded sequences yields th… view at source ↗
Figures from the paper (4 more)
Figure 6
Figure 6. Figure 6: Apply-driven Cluster A100 GPU (Muennighoff et al. 2022). Principal component analysis (PCA) is conducted to reduce the dimensionality of embeddings to eight components, which explain 43 % of the total variance. Methods To investigate how do students rely on AI and expl…
Figure 7
Figure 7. Figure 7: Two illustrative examples of failed application [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: SHAP Bar Plot of XGBoost [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: SHAP Beeswarm Plot of XGBoost around zero confirms that once we account for these rich semantic and behavioral signals, no single question type or Bloom’s Taxonomy systematically predisposes a student to either trust or disregard the AI. Conclusions and Discussions In …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 40 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    G.; Arbul \'u Ballesteros, M

    Acosta-Enriquez, B. G.; Arbul \'u Ballesteros, M. A.; Arbulu Perez Vargas, C. G.; Orellana Ulloa, M. N.; Guti \'e rrez Ulloa, C. R.; Pizarro Romero, J. M.; Guti \'e rrez Jaramillo, N. D.; Cuenca Orellana, H. U.; Ayala Anzo \'a tegui, D. X.; and L \'o pez Roca, C. 2024. Knowledge, attitudes, and perceived Ethics regarding the use of ChatGPT among generatio...

    work page 2024

  4. [4]

    F.; Han, H.; Alam, M

    Ahmad, S. F.; Han, H.; Alam, M. M.; Rehmat, M.; Irshad, M.; Arra \ n o-Mu \ n oz, M.; Ariza-Montes, A.; et al. 2023. Impact of artificial intelligence on human loss in decision making, laziness and safety in education. Humanities and Social Sciences Communications, 10(1): 1--14

    work page 2023

  5. [5]

    Ammari, T.; Chen, M.; Zaman, S.; and Garimella, K. 2025. How Students (Really) Use ChatGPT: Uncovering Experiences Among Undergraduate Students. arXiv preprint arXiv:2505.24126

    arXiv 2025

  6. [6]

    T.; and Weld, D

    Bansal, G.; Wu, T.; Zhou, J.; Fok, R.; Nushi, B.; Kamar, E.; Ribeiro, M. T.; and Weld, D. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI conference on human factors in computing systems, 1--16

    work page 2021

  7. [7]

    F.; Bowe, K

    Bauer, C. F.; Bowe, K. A.; Meredith, D. C.; and Kustina, J. 2025. Students as Partners in Teaching, Managing, and Developing Peer Learning in STEM Gateway Courses. Journal of College Science Teaching, 1--9

    work page 2025

  8. [8]

    Cao, S.; Liu, A.; and Huang, C.-M. 2024. Designing for appropriate reliance: The roles of ai uncertainty presentation, initial user decision, and user demographics in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 8(CSCW1): 1--32

    work page 2024

Show all 49 references
  1. [9]

    R.; Lima, N.; Viegas, C.; and Caldeira, A

    Costa, A. R.; Lima, N.; Viegas, C.; and Caldeira, A. 2024. Critical minds: enhancing education with ChatGPT. Cogent Education, 11(1): 2415286

    2024

  2. [10]

    Dai, Y.; Xiao, J.-Y.; Huang, Y.; Zhai, X.; Wai, F.-C.; and Zhang, M. 2025. How generative AI enables an online project-based learning platform: An applied study of learning behavior analysis in undergraduate students. Applied Sciences, 15(5): 2369

    2025

  3. [11]

    Ding, L.; Li, T.; Jiang, S.; and Gapud, A. 2023. Students’ perceptions of using ChatGPT in a physics class as a virtual tutor. International Journal of Educational Technology in Higher Education, 20(1): 63

    2023

  4. [12]

    Eckhardt, S.; K \"u hl, N.; Dolata, M.; and Schwabe, G. 2024. A Survey of AI Reliance. arXiv preprint arXiv:2408.03948

    2024 arXiv

  5. [13]

    Fan, Y.; Tang, L.; Le, H.; Shen, K.; Tan, S.; Zhao, Y.; Shen, Y.; Li, X.; and Ga s evi \'c , D. 2025. Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, ...

    2025

  6. [14]

    F rat, M. 2025. A global triangulation approach to ChatGPT integration in education. Educational Technology Quarterly

    2025

  7. [15]

    Galdames, I. S. 2024. Impact of Artificial Intelligence on Higher Education: A Literature Review. In International Conference in Information Technology and Education, 373--392. Springer

    2024

  8. [16]

    Gerlich, M. 2025. AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies, 15(1): 6

    2025

  9. [17]

    Grassini, S. 2023. Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in educational settings. Education sciences, 13(7): 692

    2023

  10. [18]

    D.; Fricticarani, A.; Dayurni, P.; Rahmadani, K.; Astiti, A

    Hakiki, M.; Fadli, R.; Samala, A. D.; Fricticarani, A.; Dayurni, P.; Rahmadani, K.; Astiti, A. D.; and Sabir, A. 2023. Exploring the impact of using Chat-GPT on student learning outcomes in technology learning: The comprehensive experiment. Advances in Mobile Learning Educatio...

    2023

  11. [19]

    Handa, K.; Bent, D.; Tamkin, A.; McCain, M.; Durmus, E.; Stern, M.; Schiraldi, M.; Huang, S.; Ritchie, S.; Syverud, S.; Jagadish, K.; Vo, M.; Bell, M.; and Ganguli, D. 2025. Anthropic Education Report: How University Students Use Claude

    2025

  12. [20]

    M.; and Sobaih, A

    Hasanein, A. M.; and Sobaih, A. E. E. 2023. Drivers and consequences of ChatGPT use in higher education: Key stakeholder perspectives. European journal of investigation in health, psychology and education, 13(11): 2599--2614

    2023

  13. [21]

    K.; Rooks, G.; Willemsen, M

    Kahr, P. K.; Rooks, G.; Willemsen, M. C.; and Snijders, C. C. 2024. Understanding trust and reliance development in ai advice: Assessing model accuracy, model explanations, and experiences from previous interactions. ACM Transactions on Interactive Intelligent Systems, 14(4): 1--30

    2024

  14. [22]

    P.; Fishburne Jr, R

    Kincaid, J. P.; Fishburne Jr, R. P.; Rogers, R. L.; and Chissom, B. S. 1975. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel

    1975

  15. [23]

    D.; and Mernik, M

    Kosar, T.; Ostoji \'c , D.; Liu, Y. D.; and Mernik, M. 2024. Computer science education in chatgpt era: Experiences from an experiment in a programming course for novice programmers. Mathematics, 12(5): 629

    2024

  16. [24]

    Krathwohl, D. R. 2002. A revision of Bloom's taxonomy: An overview. Theory into practice, 41(4): 212--218

    2002

  17. [25]

    E.; Lukowicz, P.; Kuhn, J.; K \"u chemann, S.; and Karolus, J

    Krupp, L.; Steinert, S.; Kiefer-Emmanouilidis, M.; Avila, K. E.; Lukowicz, P.; Kuhn, J.; K \"u chemann, S.; and Karolus, J. 2024. Unreflected acceptance--investigating the negative consequences of chatgpt-assisted problem solving in physics education. In HHAI 2024: Hybrid Huma...

    2024

  18. [26]

    u per, A.; Lodde, G. C.; Livingstone, E.; Schadendorf, D.; and Kr \

    K \"u per, A.; Lodde, G. C.; Livingstone, E.; Schadendorf, D.; and Kr \"a mer, N. 2025. Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists. Journal of Medical Internet Resear...

    2025

  19. [27]

    Lai, V.; and Tan, C. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency, 29--38

    2019

  20. [28]

    Lau, R. 2023. BloomBERT: A Task Complexity Classifier. https://github.com/RyanLauQF/BloomBERT

    2023

  21. [29]

    Li, Z.; Lu, Z.; and Yin, M. 2023. Modeling human trust and reliance in ai-assisted decision making: A markovian approach. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 6056--6064

    2023

  22. [30]

    M.; and Lee, S.-I

    Lundberg, S. M.; and Lee, S.-I. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30

    2017

  23. [31]

    McNichols, H.; and Lan, A. 2025. The StudyChat Dataset: Student Dialogues With ChatGPT in an Artificial Intelligence Course. arXiv preprint arXiv:2503.07928

    2025

  24. [32]

    Muennighoff, N.; Tazi, N.; Magne, L.; and Reimers, N. 2022. MTEB: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316

    2022 arXiv

  25. [33]

    Narayanan, S.; Yu, G.; Ho, C.-J.; and Yin, M. 2023. How does value similarity affect human reliance in AI-assisted ethical decision making? In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, 49--57

    2023

  26. [34]

    Ouyang, F.; and Jiao, P. 2021. Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, 2: 100020

    2021

  27. [35]

    Paladines, J.; and Ramirez, J. 2020. A systematic literature review of intelligent tutoring systems with dialogue in natural language. IEEE Access, 8: 164246--164267

    2020

  28. [36]

    Santos, P.; Urgel, K.; and Moreno, V. 2024. Generative Artificial Intelligence in teaching and learning of ICT engineering education: A literature review and illustrative scenarios. In 2024 47th MIPRO ICT and Electronics Convention (MIPRO), 1338--1343. IEEE

    2024

  29. [37]

    Schaschek, M.; Spatscheck, N.; and Winkelmann, A. 2024. For Those About to Rely—A Taxonomy of Experimental Studies on AI Reliance

    2024

  30. [38]

    Schemmer, M.; Kuehl, N.; Benz, C.; Bartos, A.; and Satzger, G. 2023. Appropriate reliance on AI advice: Conceptualization and the effect of explanations. In Proceedings of the 28th International Conference on Intelligent User Interfaces, 410--422

    2023

  31. [39]

    Singh, H.; Tayarani-Najaran, M.-H.; and Yaqoob, M. 2023. Exploring computer science students’ perception of ChatGPT in higher education: A descriptive and correlation study. Education Sciences, 13(9): 924

    2023

  32. [40]

    H.; Jones, S.; Brinberg, M.; Bodie, G

    Solomon, D. H.; Jones, S.; Brinberg, M.; Bodie, G. D.; and Ram, N. 2022. Using sequence analysis to identify conversational motifs in supportive interactions. Journal of social and Personal Relationships, 39(10): 3155--3179

    2022

  33. [41]

    Sun, T.; Zhao, K.; and Chen, M. 2024. Human-AI Interaction: Human Behavior Routineness Shapes AI Performance. IEEE Transactions on Knowledge and Data Engineering

    2024

  34. [42]

    Tan, X. 2023. The impact of ChatGPT on education and future prospects. Highlights in science, engineering and technology, 61: 138--143

    2023

  35. [43]

    Wang, S.; Wang, F.; Zhu, Z.; Wang, J.; Tran, T.; and Du, Z. 2024. Artificial intelligence in education: A systematic literature review. Expert Systems with Applications, 252: 124167

    2024

  36. [44]

    Wang, Y.; Pan, Y.; Yan, M.; Su, Z.; and Luan, T. H. 2023. A survey on ChatGPT: AI--generated contents, challenges, and solutions. IEEE Open Journal of the Computer Society, 4: 280--302

    2023

  37. [45]

    Wu, F.; Dang, Y.; and Li, M. 2025. A Systematic Review of Responses, Attitudes, and Utilization Behaviors on Generative AI for Teaching and Learning in Higher Education. Behavioral Sciences, 15(4): 467

    2025

  38. [46]

    Yin, M.; Wortman Vaughan, J.; and Wallach, H. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems, 1--12

    2019

  39. [47]

    Zhai, C.; Wibowo, S.; and Li, L. D. 2024. The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review. Smart Learning Environments, 11(1): 28

    2024

  40. [48]

    Zhang, D.; Li, J.; Zeng, Z.; and Wang, F. 2025. Jasper and Stella: distillation of SOTA embedding models. arXiv:2412.19048

    2025 arXiv

  41. [49]

    D.; Ren, X.; Dziri, N.; Jurafsky, D.; and Sap, M

    Zhou, K.; Hwang, J. D.; Ren, X.; Dziri, N.; Jurafsky, D.; and Sap, M. 2024. Rel-AI: An Interaction-Centered Approach To Measuring Human-LM Reliance. arXiv preprint arXiv:2407.07950

    2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.