Analyzing the Impact of AI Tools on Student Study Habits and Academic Performance

Angela Guercio; Ben Ward; Deepshikha Bhati; Fnu Neha

REVIEW 3 major objections 3 minor 2 cited by

Analyzing the Impact of AI Tools on Student Study Habits and Academic Performance

T0 review · 3 major / 3 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper reports that students perceive AI tools as improving academic performance, with 83% reporting improvement and a claimed drop in study time alongside higher GPA, based on a 71-student survey.

desk verdict The abstract's central result—fewer study hours, higher GPA—has no supporting measurement or statistic anywhere in the paper; this is a descriptive attitude survey with an overstated headline. read the letter →

arxiv 2412.02166 v1 pith:RDRFULJD submitted 2024-12-03 cs.AI

Ben Ward , Deepshikha Bhati , Fnu Neha , Angela Guercio This is my paper · ORCID

classification cs.AI

keywords AIineducationstudyhabitsacademicperformanceperceivedeffectivenessself-reportsurveyadaptivelearningstudentmotivationadoption

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that AI study tools improve student outcomes: better time management, faster feedback, and more personalized learning, with students reporting a significant reduction in study hours alongside an increase in GPA. The evidence is a survey of 71 U.S. university students, mostly in STEM, who were asked about their AI usage, comfort, motivation, and perceived academic improvement. The headline result is that 83% of respondents reported at least some academic improvement since adopting AI tools, while a large majority said AI had a positive effect on their study routines and motivation. The paper concludes that AI should complement, not replace, traditional teaching, and that developers should address privacy, over-reliance, and integration challenges. A sympathetic reader would take the study as an early, perception-based signal that AI tools can help, not as a controlled measurement of effect.

What carries the argument

The analytical engine is a mixed-methods survey: a Likert-scale questionnaire plus follow-up interviews, analyzed with descriptive statistics, t-tests/ANOVA, and regression, with thematic analysis of open-ended responses. The load-bearing object is the self-reported perception of academic improvement, captured in a single bar chart where 83% of respondents chose 'significant' or 'slight' improvement. That perception index is what connects AI usage to the paper's conclusions about study hours and GPA.

What would settle it

Take a cohort of students who begin using an AI study tool, log their actual study time and GPA for a semester, and compare them with a matched group that does not use the tool; the central claim falls if the AI group does not show both lower logged study time and equal or higher objective GPA.

Watch

Extended reading notes

Core claim

The authors' central claim is that AI-powered study tools improve academic performance by making study time more efficient: students report spending fewer hours studying while earning higher GPAs. On the survey's own numbers, 48% of respondents said their academic performance had 'significantly improved' and 35% said 'slightly improved' since they began using AI tools; 78% used AI tools often or sometimes; and the average self-rated impact on study routines was 4.37 out of 5. The authors read these patterns as evidence that AI supports personalized learning, adaptive test adjustments, and real-time feedback, and they frame the main remaining problems as over-reliance and difficult integration with conventional teaching.

Load-bearing premise

The load-bearing premise is that students' self-reports of perceived improvement, recalled study-time changes, and GPA gains accurately reflect real academic performance; the survey has no pre-AI baseline, no objective grade records, and no control group.

Editorial extensions

If this is right

If students really do maintain or improve grades while studying less, AI tools would be a cost-effective lever for academic efficiency, worth integrating into course design.
Developers would be justified in prioritizing adaptive learning paths, personalized test difficulty, and real-time classroom analytics, since these are the features students say they want.
Educators could treat AI as a complement to traditional instruction rather than a replacement, using it for tutoring, planning, and feedback while guarding against over-reliance.
Institutions should invest in privacy and transparency safeguards, such as GDPR and FERPA compliance, because students named data security as a condition for continued AI use.
Adoption efforts should target non-STEM fields and lower-division students, where the survey suggests AI use and awareness are lower.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

My inference: the paper's own data cannot distinguish 'AI made me study less' from 'I used AI instead of studying,' so the reported study-hour drop may reflect substitution rather than efficiency; a time-diary or log-based study would separate these.
My inference: the sample is 71 students, 90% from U.S. institutions and 70.5% STEM, so the findings are most plausibly about tech-comfortable undergraduates; extending them to other populations is a testable leap.
My inference: a natural next test is to compare exam scores or assignment quality across AI-usage frequency groups; the current data do not include those objective outcomes.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Read the paper. The short version: the abstract promises a significant reduction in study hours and an increase in GPA, and the conclusion repeats it. The body never presents either outcome. There is no GPA data, no before/after study-hour measurement, no t-test, ANOVA, or regression output on those variables. Section III.C says these analyses were performed, but the results don't appear. That's not a minor gap; it's the central claim.

What the paper actually contains is a 71-person convenience survey, mostly US and STEM, asking about AI usage, perceived improvement, comfort, and motivation. The descriptive results (48% "significant improvement," comfort 4.31/5, etc.) are consistent with prior surveys in the same literature. The genuinely useful part is the open-ended feedback in Section XV—students' feature requests like adaptive learning paths and real-time classroom analysis, plus cautions about over-reliance. That qualitative material could inform designers, but it doesn't support the causal headline.

The paper handles its limitations in places: it acknowledges the US-centric sample and the STEM skew. But it never flags that the outcome variables are entirely self-reported perceptions, with no baseline and no control group. The "significant improvement" is a Likert item about perceived improvement, not measured academic performance. The abstract converts that into a causal claim.

There are also small presentation problems—the text calls Figure 1 an age distribution while the caption says gender, and the flow of sections gets repetitive. Not worth dwelling on.

Is the paper coherent on its own terms as a descriptive pilot? Yes. As a report of AI's impact on GPA and study time, no. If it were revised to state only what it measured, it could be a minor venue note. As it stands, the abstract and conclusion need to change, and even then the contribution is modest.

Who is this for? Someone cataloging student attitudes or looking for user-request features in AI education tools. Not for someone asking whether AI improves outcomes. I wouldn't cite it, and I wouldn't send it out for full peer review in this state; a desk editor could catch the mismatch, or an editor could ask for a major revision removing the unsupported claims. Either way, the current version is not publishable as-is.

Referee Report

3 major / 3 minor

Summary. This paper reports a survey-based study of 71 university students on their use of AI tools, with sections presenting descriptive statistics on demographics, AI usage frequency, types of tools, perceived academic improvement, study-time allocation, comfort, and motivation, alongside qualitative feedback on desired features and concerns. The abstract and conclusion claim a significant reduction in study hours and an increase in GPA as a result of AI tool use, based on a mixed-methods design with Likert-scale items and follow-up interviews.

Significance. If its central claims were supported, the paper would offer a useful data point on how AI tools affect study habits and academic outcomes in higher education. The paper does provide a descriptive snapshot of AI adoption patterns among a STEM-heavy, U.S.-centric student sample, and the qualitative feedback on desired features and perceived benefits is potentially useful for tool designers. However, the headline quantitative claims about reduced study hours and increased GPA are not backed by any reported measurement or statistical analysis, and the outcome variables are entirely self-reported. The paper is best read as a descriptive pilot study of student attitudes, not as evidence of academic-performance effects.

major comments (3)

[Abstract and Section XVI] The central claim of a "significant reduction in study hours alongside an increase in GPA" is asserted in the abstract and the conclusion but is never measured or reported anywhere in the body. No survey item, table, figure, or statistical test in Sections III–XV presents study hours before versus after AI adoption or any GPA distribution or GPA change. The paper therefore does not substantiate its headline result.
[Section VIII] The only outcome resembling academic performance is "Perceived Academic Improvement," a single self-report item in which 48% of students said they experienced "Significant Improvement" and 35% "Slight Improvement." This is not a measure of GPA or any objective academic outcome, and the conclusion that AI tools increase GPA reduces to participants stating that they improved. Without pre-AI GPA data, a control group, or institutional records, the causal claim in the abstract is unsupported.
[Section III.C] The Methods section states that t-tests, ANOVA, and regression analysis were performed, but the results of these analyses never appear in the paper. There are no test statistics, p-values, effect sizes, confidence intervals, regression coefficients, or model summaries anywhere in Sections IV–XV. As written, the inferential-statistics claim in III.C is unverifiable and does not support any finding.

minor comments (3)

[Section III.A and Section IV] The figure references are inconsistent: Section III.A says the age distribution is plotted in Figure 1, but Figure 1 is captioned "Gender distribution of survey respondents," and Section IV says Figure 1 displays the age distribution. The reader must infer which figure actually corresponds to age and gender; please correct the cross-references.
[Section II.C and References] The in-text citations for [14] and [15] do not match the reference list: the text cites Zawacki-Richter et al. (2019) for [14] and Holmes et al. (2019) for [15], but reference [14] is Martin et al. and reference [15] is Baker (2019). Please reconcile the citations with the bibliography.
[Abstract] The abstract mentions "follow-up interviews" as part of the data collection, but no interview protocol, participant counts, or thematic analysis of interview transcripts is reported in the paper. Please clarify whether interviews were actually conducted and, if so, present their findings or remove the mention.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the headline GPA/study-hours claim is unsupported by data rather than derived circularly.

full rationale

The paper's central quantitative claim—'the study found a significant reduction in study hours alongside an increase in GPA' (Abstract)—is asserted but never operationalized or measured in the reported survey. Section VIII ('Perceived Academic Improvement') reports only self-reported perceptions (48% 'Significant Improvement'), and no figure or table in the manuscript reports before/after GPA, total study-hour changes, or an inferential test of either outcome. This is a serious evidentiary and construct-validity gap, but it is not circularity under the definitions used here: the paper does not define GPA or study-hour reduction in terms of the self-report item, and the conclusion is not entailed by the reported survey data. It is unsupported rather than equivalent to its inputs. The only self-citations ([8]–[10] in Related Work) support background claims about AI tools for ASD and explainable AI; they are not load-bearing for the main survey result. No equation, fitted parameter, uniqueness theorem, or 'prediction' is shown to reduce by construction to its own input. Therefore no specific circular step can be exhibited, and the circularity score is 0. The abstract/conclusion overstatement should be handled as a correctness and reporting-risk issue, not circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim relies on three unvalidated domain assumptions: self-reports are accurate, the sample is representative, and observed improvements are caused by AI. No free parameters are fitted because no model is actually estimated. No invented entities are introduced.

assumptions (3)

domain assumption Self-reported perceived improvement and recalled study-hour changes accurately reflect actual academic performance.
The central claim of GPA increase and study-hour reduction rests entirely on self-report data from Section VIII and Section XI; no objective records or baseline are provided.
domain assumption The 71-student convenience sample is representative enough to support general claims about AI's impact on students.
The paper generalizes beyond the sample despite 70.5% STEM and 90% US respondents (Sections V and VII).
domain assumption Observed improvements are caused by AI tool use rather than by other factors.
The conclusion attributes changes to AI tools with no control group or causal design, as seen in Section XVI.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Analyzing the Impact of AI Tools on Student Study Habits and Academic Performance." pith.science (2026). https://pith.science/paper/RDRFULJD

@misc{pith2026241202166,
  author       = {Pith},
  title        = {Pith review of: Analyzing the Impact of AI Tools on Student Study Habits and Academic Performance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RDRFULJD}},
  note         = {Machine review of arXiv:2412.02166}
}

read the original abstract

This study explores the effectiveness of AI tools in enhancing student learning, specifically in improving study habits, time management, and feedback mechanisms. The research focuses on how AI tools can support personalized learning, adaptive test adjustments, and provide real-time classroom analysis. Student feedback revealed strong support for these features, and the study found a significant reduction in study hours alongside an increase in GPA, suggesting positive academic outcomes. Despite these benefits, challenges such as over-reliance on AI and difficulties in integrating AI with traditional teaching methods were also identified, emphasizing the need for AI tools to complement conventional educational strategies rather than replace them. Data were collected through a survey with a Likert scale and follow-up interviews, providing both quantitative and qualitative insights. The analysis involved descriptive statistics to summarize demographic data, AI usage patterns, and perceived effectiveness, as well as inferential statistics (T-tests, ANOVA) to examine the impact of demographic factors on AI adoption. Regression analysis identified predictors of AI adoption, and qualitative responses were thematically analyzed to understand students' perspectives on the future of AI in education. This mixed-methods approach provided a comprehensive view of AI's role in education and highlighted the importance of privacy, transparency, and continuous refinement of AI features to maximize their educational benefits.

Figures

Figures reproduced from arXiv: 2412.02166 by the authors.

**Figure 2.** Age distribution of survey respondents B. Survey Design The survey was structured to gather comprehensive data on AI tool usage, perceptions, and effectiveness across key sections: • Demographics: Collected information on age, gender, major, grade level, and university to analyze variations in AI usage based on these attributes. • AI Usage: Focused on types and frequency of AI tool use (e.g., note-taking apps, AI tu… view at source ↗

**Figure 4.** The distribution of survey respondents by their academic grade level [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (3 more)

**Figure 7.** Figure 7: Pie chart displaying the frequency with which students use AI-powered [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]

**Figure 9.** Figure 9: Pie chart illustrating the percentage of study time students spend using [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]

**Figure 10.** Figure 10: Radar chart displaying students’ comfort levels with AI technology [PITH_FULL_IMAGE:figures/full_fig_p006_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

How Adding Metacognitive Requirements in Support of AI Feedback in Practice Exams Transforms Student Learning Behaviors
cs.HC 2025-05 conditional novelty 5.0 of 10

In a 1,002-student biology course, AI feedback type did not affect midterm performance, while required confidence ratings and explanations were self-reported as the main driver of changed study behavior.
Inclusive Education with AI: Supporting Special Needs and Tackling Language Barriers
cs.CY 2025-04 unverdicted novelty 1.0 of 10

A narrative review concludes that AI tools can support inclusive early education when implemented with ethical safeguards, but the evidence base it relies on is not rigorously appraised.

Reference graph

Works this paper leans on

16 extracted references · 15 canonical work pages · cited by 2 Pith papers

[1]

Generative AI in education: A study of educators’ awareness, sentiments, and influencing factors,

A. Ghimire, J. Prather, and J. Edwards, “Generative AI in education: A study of educators’ awareness, sentiments, and influencing factors,” arXiv preprint arXiv:2403.15586 , 2024

arXiv 2024
[2]

Shaping the future of learning: The role of ai in education 4.0,

G. Elhussein, E. Hasselaar, and O. Lutsyshyn, “Shaping the future of learning: The role of ai in education 4.0,” in Proceedings of the World Economic Forum Annual Meeting . World Economic Forum, 2024

work page 2024
[3]

Artificial intelligence and the future of teaching and learning. US department of education, office of educational technology,

M. Cardona, R. Rodr ´ıguez, and K. Ishmael, “Artificial intelligence and the future of teaching and learning. US department of education, office of educational technology,” 2023

work page 2023
[4]

Artificial intelligence in education: Ad- dressing ethical challenges in K-12 settings,

S. Akgun and C. Greenhow, “Artificial intelligence in education: Ad- dressing ethical challenges in K-12 settings,” AI and Ethics, vol. 2, no. 3, pp. 431–440, 2022

work page 2022
[5]

Integrating AI in education: Opportunities, challenges, and ethical considerations,

C. A. Eden, O. N. Chisom, I. S. Adeniyi et al. , “Integrating AI in education: Opportunities, challenges, and ethical considerations,” Magna Scientia Advanced Research and Reviews , vol. 10, no. 2, pp. 006–013, 2024

work page 2024
[6]

Education 4.0 made simple: Ideas for teaching,

A. A. Hussin, “Education 4.0 made simple: Ideas for teaching,” Inter- national Journal of Education and Literacy Studies , vol. 6, no. 3, pp. 92–98, 2018

work page 2018
[7]

Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review,

C.-C. Lin, A. Y . Huang, and O. H. Lu, “Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review,” Smart Learning Environments, vol. 10, no. 1, p. 41, 2023

work page 2023
[8]

Bookmate: Lever- aging deep learning to empower caregivers of people with ASD in generation of social stories,

D. Bhati, A. Guercio, V . Rossano, and R. Francese, “Bookmate: Lever- aging deep learning to empower caregivers of people with ASD in generation of social stories,” in 2023 27th International Conference Information Visualisation (IV) . IEEE, 2023, pp. 403–408

work page 2023

Show all 16 references

[9]

A multimodal conversational interface to support the creation of customized social stories for people with ASD,

R. Francese, A. Guercio, V . Rossano, and D. Bhati, “A multimodal conversational interface to support the creation of customized social stories for people with ASD,” in Proceedings of the 2022 International Conference on Advanced Visual Interfaces , 2022, pp. 1–5

2022
[10]

A survey on explainable artificial intelligence (xai) techniques for visualizing deep learning models in medical imaging,

D. Bhati, F. Neha, and M. Amiruzzaman, “A survey on explainable artificial intelligence (xai) techniques for visualizing deep learning models in medical imaging,” Journal of Imaging , vol. 10, no. 10, p. 239, 2024

2024
[11]

Transforming edu- cation: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis,

Z. Bahroun, C. Anane, V . Ahmed, and A. Zacca, “Transforming edu- cation: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis,” Sustain- ability, vol. 15, no. 17, p. 12983, 2023

2023
[12]

Evaluating the evaluators: A comparative study of ai and teacher assessments in higher education,

A. Alper et al. , “Evaluating the evaluators: A comparative study of ai and teacher assessments in higher education,” Digital Education Review, no. 45, pp. 124–140, 2024

2024
[13]

AI vs. traditional education: The battle for the classroom of the future,

I. Asarsh, “AI vs. traditional education: The battle for the classroom of the future,” 2023

2023
[14]

Systematic review of research on artificial intelligence in K-12 education (2017–2022),

F. Martin, M. Zhuang, and D. Schaefer, “Systematic review of research on artificial intelligence in K-12 education (2017–2022),” Computers and Education: Artificial Intelligence , p. 100195, 2023

2017
[15]

AI in education: Promises and implications for teaching and learning,

R. Baker, “AI in education: Promises and implications for teaching and learning,” Technology-Based Assessment for 21st Century Skills: Theoretical and Practical Implications from Modern Research, pp. 253– 268, 2019

2019
[16]

Selwyn, Should robots replace teachers?: AI and the future of education

N. Selwyn, Should robots replace teachers?: AI and the future of education. John Wiley & Sons, 2019

2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Generative AI in education: A study of educators’ awareness, sentiments, and influencing factors,

A. Ghimire, J. Prather, and J. Edwards, “Generative AI in education: A study of educators’ awareness, sentiments, and influencing factors,” arXiv preprint arXiv:2403.15586 , 2024

arXiv 2024

[2] [2]

Shaping the future of learning: The role of ai in education 4.0,

G. Elhussein, E. Hasselaar, and O. Lutsyshyn, “Shaping the future of learning: The role of ai in education 4.0,” in Proceedings of the World Economic Forum Annual Meeting . World Economic Forum, 2024

work page 2024

[3] [3]

Artificial intelligence and the future of teaching and learning. US department of education, office of educational technology,

M. Cardona, R. Rodr ´ıguez, and K. Ishmael, “Artificial intelligence and the future of teaching and learning. US department of education, office of educational technology,” 2023

work page 2023

[4] [4]

Artificial intelligence in education: Ad- dressing ethical challenges in K-12 settings,

S. Akgun and C. Greenhow, “Artificial intelligence in education: Ad- dressing ethical challenges in K-12 settings,” AI and Ethics, vol. 2, no. 3, pp. 431–440, 2022

work page 2022

[5] [5]

Integrating AI in education: Opportunities, challenges, and ethical considerations,

C. A. Eden, O. N. Chisom, I. S. Adeniyi et al. , “Integrating AI in education: Opportunities, challenges, and ethical considerations,” Magna Scientia Advanced Research and Reviews , vol. 10, no. 2, pp. 006–013, 2024

work page 2024

[6] [6]

Education 4.0 made simple: Ideas for teaching,

A. A. Hussin, “Education 4.0 made simple: Ideas for teaching,” Inter- national Journal of Education and Literacy Studies , vol. 6, no. 3, pp. 92–98, 2018

work page 2018

[7] [7]

Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review,

C.-C. Lin, A. Y . Huang, and O. H. Lu, “Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review,” Smart Learning Environments, vol. 10, no. 1, p. 41, 2023

work page 2023

[8] [8]

Bookmate: Lever- aging deep learning to empower caregivers of people with ASD in generation of social stories,

D. Bhati, A. Guercio, V . Rossano, and R. Francese, “Bookmate: Lever- aging deep learning to empower caregivers of people with ASD in generation of social stories,” in 2023 27th International Conference Information Visualisation (IV) . IEEE, 2023, pp. 403–408

work page 2023