{"paper":{"title":"From Heuristics to Analytics: Forecasting Effort and Progress in Online Learning","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Feature-based models forecast weekly student effort and progress in tutoring systems with 22-33 percent lower error than heuristic rules.","cross_cats":["cs.CY"],"primary_cat":"cs.LG","authors_text":"Boyuan Guo, Conrad Borchers, Danielle R. Thomas, Eric S. Qiu, Vincent Aleven","submitted_at":"2026-05-12T22:04:06Z","abstract_excerpt":"Sustained effort is essential for realizing the benefits of intelligent tutoring systems (ITS), yet many learners disengage or underuse available practice time. We introduce engagement forecasting as a supervised prediction task based on ITS logs, targeting two outcomes central to effort and learning progress: minutes practiced per week and new skills mastered per week. Using interaction log data from 425 middle-school students over a school year, we benchmark fifteen predictors including regressions, decision trees, and neural networks. We show that these feature-based models reduce mean abso"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"feature-based models reduce mean absolute error (MAE) by 22-33% relative to heuristic baselines, including fixed-percentile rules adapted from prior work","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The 425-student log dataset and the chosen features are representative enough for the models to generalize to new students and new weeks without substantial distribution shift.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Feature-based ML models forecast weekly student effort and progress in ITS with 22-33% lower MAE than percentile heuristics on data from 425 middle-school students.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Feature-based models forecast weekly student effort and progress in tutoring systems with 22-33 percent lower error than heuristic rules.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"71c4411a1de2641aae68c6116713e6f38bf586cd342fc6dd2fa456634ad6c426"},"source":{"id":"2605.12788","kind":"arxiv","version":1},"verdict":{"id":"8b894eea-8e98-4cee-84c1-97cfd3b5ab56","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-14T20:27:28.547087Z","strongest_claim":"feature-based models reduce mean absolute error (MAE) by 22-33% relative to heuristic baselines, including fixed-percentile rules adapted from prior work","one_line_summary":"Feature-based ML models forecast weekly student effort and progress in ITS with 22-33% lower MAE than percentile heuristics on data from 425 middle-school students.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The 425-student log dataset and the chosen features are representative enough for the models to generalize to new students and new weeks without substantial distribution shift.","pith_extraction_headline":"Feature-based models forecast weekly student effort and progress in tutoring systems with 22-33 percent lower error than heuristic rules."},"references":{"count":57,"sample":[{"doi":"","year":2017,"title":"M. A. Adams, J. C. Hurley, M. Todd, N. Bhuiyan, C. L. Jarrett, W. J. Tucker, K. E. Hollingshead, and S. S. Angadi. Adaptive goal setting and financial incentives: a 2×2 factorial randomized controlled","work_id":"ee336a2b-af38-4d36-949a-1fafb7b8d61a","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2021,"title":"B. Albreiki, N. Zaki, and H. Alashwal. A systematic literature review of student’ performance prediction us- ing machine learning techniques.Education Sciences, 11(9):1–27, 2021","work_id":"b903e8bd-64b2-429f-9f9d-b60ba909d806","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"V. Aleven, C. Borchers, Y. Huang, T. Nagashima, B. McLaren, P. Carvalho, O. Popescu, J. Sewall, and K. Koedinger. An integrated platform for studying learning with intelligent tutoring systems: Ctat+t","work_id":"681ea6d0-c31f-45ac-828f-ab80640a7471","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2010,"title":"I. Arroyo, H. Meheranian, and B. P. Woolf. Effort-based tutoring: An empirical approach to intelligent tutoring. InProceedings of the 3rd International Conference on Educational Data Mining (EDM), pag","work_id":"6d5a64bd-0314-459f-8bdf-d2a6f8e3d63f","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2004,"title":"R. S. Baker, A. T. Corbett, and K. R. Koedinger. Detecting student misuse of intelligent tutoring sys- tems. InProceedings of the 7th International Confer- ence on Intelligent Tutoring Systems (ITS), ","work_id":"ed3228c9-3110-46bd-95f9-4f08ebabc543","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":57,"snapshot_sha256":"6436e51d9feaaf741c74aece5c83d55cd72a55be265fdcf515c282522a839228","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}