Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Chatting with a Learning Analytics Dashboard: The Role of Generative AI Literacy on Learner Interaction with Conventional and Scaffolding Chatbots

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that adding either a conventional or a scaffolding generative-AI chatbot to a learning analytics dashboard significantly improves learners' comprehension of complex visualisations, and that a learner's generative-AI…

desk verdict Useful empirical comparison of chatbot designs in a learning analytics dashboard, but the central causal claim is weakened by the lack of a no-chatbot control and non-counterbalanced question order. read the letter →

arxiv 2411.15597 v1 pith:BWSD3NU3 submitted 2024-11-23 cs.HC

Yueqiao Jin , Kaixun Yang , Lixiang Yan , Vanessa Echeverria , Linxuan Zhao , Riordan Alfredo , Mikaela Milesi , Jie Fan
show 3 more authors
Xinyu Li Dragan Gašević Roberto Martinez-Maldonado
This is my paper · ORCID
classification cs.HC
keywords learninganalyticsdashboardgenerativeAIliteracychatbotsdatavisualisationscaffoldinghuman-computerinteractionepistemicnetworkanalysismultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Learning analytics dashboards present complex data that learners often struggle to interpret. This paper asks whether adding a generative-AI chatbot—either one that simply answers questions or one that proactively guides users with scaffolding questions—makes the insights in those dashboards more comprehensible, and whether a learner's generative-AI literacy decides how much they benefit. In a study of 81 medical and nursing students, both chatbot types raised comprehension scores from baseline to intervention, with large effects, and higher generative-AI literacy predicted larger improvements. The effect of literacy was more than twice as strong with the conventional chatbot as with the scaffolding chatbot, suggesting that proactive guidance can partly compensate for low literacy. If correct, the result gives designers of educational dashboards a concrete reason to build scaffolding into AI assistants and to measure AI literacy when deploying them.

What carries the argument

The central mechanism is the contrast between the two chatbot designs built on the VizChat prototype: a conventional chatbot that reactively answers user queries and a scaffolding chatbot that proactively poses expert-designed guiding questions and gives step-by-step feedback on visualisation elements. Comprehension is measured as an improvement score, the difference between a six-question multiple-choice test taken before and after chatbot use, with questions based on Bloom's taxonomy levels 1 and 2 and a correction-for-guessing formula. Generative-AI literacy is measured with the 20-item Generative AI Literacy Assessment Test (GLAT). The statistical core is an OLS regression of improvement on baseline score and literacy, with a tested interaction term, run separately for each condition, and the cognitive analysis uses epistemic network analysis of coded utterances. The competing designs do the argumentative work: the literacy coefficient's difference between conditions is what supports the claim that scaffolding lowers the literacy requirement.

What would settle it

Run the same two-phase design with a third condition in which participants get no chatbot at all (or only the written context again) while keeping identical visualisations and counterbalancing the question sets; if the no-chatbot condition shows the same baseline-to-intervention gain, then the improvement attributed to the chatbots is not theirs alone.

Watch

Extended reading notes

Core claim

On its own terms, the paper reports three empirical findings. First, within-subject comprehension scores on six multiple-choice questions improved significantly after interacting with either chatbot (conventional group median from 3 to 4, scaffolding group from 3 to 5), with large rank-biserial effect sizes (0.87 and 0.94). Second, regression analyses of improvement scores found a positive association with generative-AI literacy in both conditions, but the coefficient for literacy was $0.19$ points per literacy point in the conventional group versus $0.08$ in the scaffolding group, and only the conventional model needed a baseline-by-literacy interaction term. Third, epistemic network analysis of user-chatbot utterances showed that high-literacy learners engaged in more information integration and reflection with the conventional chatbot, while low-literacy learners relied on clarification and commands; with the scaffolding chatbot, low-literacy learners showed more exploratory, iterative interaction patterns. The authors interpret these results as evidence that chatbot-assisted dashboards support comprehension and that scaffolding can reduce the literacy gap.

Load-bearing premise

The result rests on the assumption that the comprehension gains measured after chatbot interaction are caused by the chatbot itself; because every participant did the baseline before the intervention with no counterbalancing and no no-chatbot control, practice effects, growing familiarity with the visualisations, or easier question sets could explain the improvement instead.

Editorial extensions

If this is right

  • If replication holds, adding either reactive or proactive chatbots to existing dashboards can be expected to raise comprehension of complex visualisations in similar populations, with no significant difference in mean improvement between the two designs.
  • The literacy effect implies that when a conventional, unguided chatbot is deployed, learners with higher generative-AI literacy will benefit considerably more, so interface designers should not assume equal benefit without measuring literacy.
  • Scaffolding chatbots' smaller literacy coefficient suggests proactive guidance can serve as an equity mechanism, letting lower-literacy learners approach the gains of higher-literacy peers.
  • The epistemic network analysis patterns imply that low-literacy learners' interactions focus on clarification and commands, so logging such patterns could help a dashboard detect users who need more supportive prompting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave untested is that a no-chatbot control group might show a similar improvement due to task practice alone; the fixed baseline-then-intervention order without counterbalancing makes this the key confound.
  • If the mechanism is that scaffolding lowers prompt-crafting demands, the same conventional-versus-scaffolding contrast should replicate in other data-rich domains such as financial or health literacy dashboards.
  • The measured literacy score could be used as an adaptive routing signal: assign low-literacy learners to scaffolding chatbots automatically, and reserve conventional chatbots for high-literacy users.
  • The epistemic network analysis differences also suggest a hybrid design: start all learners with scaffolding prompts, then fade them as the user's interaction patterns show integration and reflection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports a between-subject experiment with 81 medical/nursing students who completed a baseline analytical writing task and six comprehension questions, were then assigned to either a conventional (reactive) or scaffolding (proactive) GenAI chatbot in a learning analytics dashboard, and completed a second, parallel writing task and question set with chatbot support. GenAI literacy was measured with the GLAT instrument, and learner-chatbot interactions were analysed with epistemic network analysis. The paper reports that both chatbots significantly improved comprehension (RQ1), that higher GenAI literacy was associated with larger comprehension gains, especially for the conventional chatbot (RQ2), and that learners with different literacy levels exhibited different interaction patterns (RQ3). The authors conclude that scaffolding chatbots reduce reliance on learners' GenAI literacy and offer a more equitable design.

Significance. If the causal claims are correct, the study provides useful empirical evidence for the value of GenAI chatbots in learning analytics dashboards and for the importance of GenAI literacy as a learner characteristic. The strengths include an a priori power analysis, random assignment to the two chatbot conditions, a mixed-methods design combining quantitative comprehension gains with epistemic network analysis, and an open repository with materials and the coding scheme. The main limitation is that the headline comprehension gains rest entirely on a within-subject baseline-to-intervention contrast without a no-chatbot control condition or counterbalancing of question sets; this directly threatens the causal interpretation. The moderation claim about scaffolding reducing reliance on GenAI literacy is also not supported by a formal statistical test of the interaction between condition and literacy. These issues are fixable by re-analysis and by softening or re-scoping the causal claims, rather than by adding new data, so the work has a defensible core that merits revision.

major comments (4)
  1. [§3.3.3 and §4.2] The central RQ1 claim that the chatbots caused comprehension improvement is not supported by the design as described. Every participant first completed the baseline task and then the intervention task in a fixed order, with the same visualisation types and similar question formats, and there was no no-chatbot control condition. The observed Baseline_score to Intervention_score gains could therefore be driven by practice with the visualisations, familiarisation with the question format, or differences in difficulty between the two question sets. The authors' statement in Section 3.3 that participants had no prior knowledge of the learning context addresses prior content knowledge, not task practice. To support the causal wording in Section 5, the authors should add a no-chatbot control arm and/or counterbalance the two question sets across phases, or explicitly reframe the result as an observed improvement that is associated with chatbot use rather than caused by it.
  2. [§3.5.3 and §4.3] The conclusion that scaffolding chatbots reduce reliance on GenAI literacy is not formally tested. The authors compare separate OLS coefficients for GenAI_literacy in the conventional group (β = 0.19) and the scaffolding group (β = 0.08), but no test of the condition-by-literacy interaction or of the equality of the coefficients is reported. A more direct test would be a single regression on the Intervention_score with Baseline_score, GenAI_literacy, condition, and the GenAI_literacy × condition interaction. In addition, the current gain-score regression uses Improvement_score = Intervention_score − Baseline_score as the dependent variable while including Baseline_score as a regressor; this creates a mechanical negative association between baseline and improvement that is not fully addressed by the ceiling-effect interpretation. The moderation claim should be supported by an explicit interaction test, or the authors should temper the claim.
  3. [§3.5.4 and §4.4] The ENA significance tests appear to be inconsistent with the stated Bonferroni correction. Section 3.5.4 says Bonferroni correction was applied with an initial alpha of 0.05, but Section 4.4 reports a p-value of 0.04 for the conventional group comparison along the X-axis and calls it statistically significant with alpha 0.05. With multiple comparisons across axes and conditions, the corrected threshold would be substantially smaller than 0.05, so p = 0.04 would not be significant. The authors should report corrected p-values, state the exact number of comparisons, or explicitly justify why no correction is needed for the specific RQ3 contrasts.
  4. [§4.2] The within-subject comparisons in Section 4.2 are labelled as 'Mann-Whitney U test' but report a W statistic and compare paired baseline and intervention scores. These should be Wilcoxon signed-rank tests, as stated in Section 3.5.2. If the Mann-Whitney U test was actually misapplied to paired data, the analyses need to be redone; if it is simply a terminological error, the text and table labels should be corrected so that the reported effect sizes and p-values are unambiguously associated with the correct test.
minor comments (5)
  1. [§3.5] The correction-for-guessing formula is mentioned for both the comprehension scores and the GenAI literacy score, but the paper does not report whether the descriptive statistics in Section 4.2 are corrected or raw scores. Since the correction can change the scale and possibly produce negative values, please clarify the formula and state whether medians, IQRs, and regression inputs use corrected or raw scores.
  2. [§4.4] There is a typo in the scaffolding condition paragraph: 'Chabot.Information' should be 'Chatbot.Information'.
  3. [§3.3.3] The paper says participants were randomly assigned to one of two intervention groups, but no details are given about the randomisation method or whether allocation was concealed. A sentence describing the allocation procedure would strengthen the report.
  4. [§3.5.4] The median-split categorisation of low and high GenAI literacy is sample-dependent and yields a small high-literacy subgroup in the conventional condition (n = 11). Please acknowledge this as a limitation and, if feasible, report sensitivity of the ENA results to alternative cut points.
  5. [Figure 3] The phrase 'left–below in Section 4.4' is unclear; please refer to panels of the figure explicitly, e.g., 'left panel, described in Section 4.4'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the comprehension-improvement claim rests on separate baseline and intervention measurements, not on fitted parameters or self-referential definitions.

full rationale

This paper is an empirical user study rather than a derivation chain, and no central claim reduces to its own inputs. RQ1's conclusion that both chatbots improved comprehension is based on Wilcoxon signed-rank tests comparing Baseline_score to Intervention_score (Sections 3.5.2 and 4.2), which are two separately administered six-question comprehension sets. The improvement score is not constructed from the chatbot design, and no parameter fitted to the intervention outcomes is later relabeled as a prediction. RQ2's OLS regression of Improvement_score on Baseline_score and GenAI_literacy is a standard associational model; the GenAI_literacy measure (GLAT) is an independently scored 20-item test whose validation, although cited to the authors' prior work, was conducted on a separate sample and does not include or encode the comprehension outcome, so the reported association is not true by definition. The self-citations for the LAD itself ([42]) and the VizChat prototype ([71]) establish provenance of the materials, but the effectiveness claim is not derived from those citations. The absence of a no-chatbot control and the fixed baseline-to-intervention order (Section 3.3.3) is a genuine internal-validity threat that could explain the gains through practice effects, but that is a confounding-variable concern, not a circularity in which the conclusion is equivalent to an input. No step in the paper defines the outcome in terms of the predictor, fits a parameter and calls it a prediction, or imports a uniqueness result from the authors' prior work to forbid alternatives.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The study is empirical, so the ledger records measurement and design assumptions rather than mathematical axioms. The main inputs are the validity of GLAT, the comparability of baseline and intervention tasks, the assumption that the two chatbot conditions differ only in scaffolding, and the coding assumption that utterances reflect cognitive processing. The median split for the ENA is a data-dependent modeling choice.

free parameters (1)
  • GenAI literacy group split threshold = upper 50% versus lower 50% (median split)
    Used in Section 3.5.4 to dichotomize participants into low and high GenAI literacy for the ENA; the cut point is data-dependent and can affect the reported network differences.
assumptions (5)
  • domain assumption The six researcher-written multiple-choice questions validly measure comprehension of the dashboard visualisations.
    Section 3.3.3 states the questions are based on Bloom's taxonomy levels 1 and 2 and were validated by two researchers with a third checking applicability, but this is not an externally validated measurement instrument.
  • domain assumption Baseline and intervention visualisation sets and question sets are comparable, so score changes reflect the intervention rather than practice or ordering effects.
    Section 3.3.3 says the visualisation types remain consistent but the specific data and insights vary; no counterbalancing or no-chatbot control is used.
  • domain assumption The GLAT instrument validly measures GenAI literacy.
    Section 3.3.2 reports internal validity statistics from a 204-person validation, but the instrument is an author-developed preprint with no independent replication.
  • domain assumption The conventional and scaffolding chatbot conditions differ only in scaffolding features, not in retrieval quality, underlying LLM behavior, or other confounds.
    Section 3.2 describes common RAG and GPT-4o architecture, but the actual response quality and prompting behavior are not audited for equivalence.
  • domain assumption Coded utterances in the chat logs reflect cognitive processing stages from Information Processing Theory.
    Section 3.5.4 defines the coding scheme, but the mapping from dialogue behaviour to cognitive constructs is interpretive and relies on inter-rater reliability rather than ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chatting with a Learning Analytics Dashboard: The Role of Generative AI Literacy on Learner Interaction with Conventional and Scaffolding Chatbots." pith.science (2026). https://pith.science/paper/BWSD3NU3

@misc{pith2026241115597,
  author       = {Pith},
  title        = {Pith review of: Chatting with a Learning Analytics Dashboard: The Role of Generative AI Literacy on Learner Interaction with Conventional and Scaffolding Chatbots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWSD3NU3}},
  note         = {Machine review of arXiv:2411.15597}
}
read the original abstract

Learning analytics dashboards (LADs) simplify complex learner data into accessible visualisations, providing actionable insights for educators and students. However, their educational effectiveness has not always matched the sophistication of the technology behind them. Explanatory and interactive LADs, enhanced by generative AI (GenAI) chatbots, hold promise by enabling dynamic, dialogue-based interactions with data visualisations and offering personalised feedback through text. Yet, the effectiveness of these tools may be limited by learners' varying levels of GenAI literacy, a factor that remains underexplored in current research. This study investigates the role of GenAI literacy in learner interactions with conventional (reactive) versus scaffolding (proactive) chatbot-assisted LADs. Through a comparative analysis of 81 participants, we examine how GenAI literacy is associated with learners' ability to interpret complex visualisations and their cognitive processes during interactions with chatbot-assisted LADs. Results show that while both chatbots significantly improved learner comprehension, those with higher GenAI literacy benefited the most, particularly with conventional chatbots, demonstrating diverse prompting strategies. Findings highlight the importance of considering learners' GenAI literacy when integrating GenAI chatbots in LADs and educational technologies. Incorporating scaffolding techniques within GenAI chatbots can be an effective strategy, offering a more guided experience that reduces reliance on learners' GenAI literacy.

Figures

Figures reproduced from arXiv: 2411.15597 by the authors.

Figure 1
Figure 1. Three LAD visualisations, representing the activity of two nurses in the first phase of their simulation, used in the current study [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. System design of conventional and scaffolding Generative AI (GenAI) chatbots, highlighting four main components: a) [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Comparison plots for the preliminary analysis (left), RQ3 conventional (mid), and scaffolding chatbots (right). [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. "We need to avail ourselves of GenAI to enhance knowledge distribution": Empowering Older Adults through GenAI Literacy

    cs.HC 2025-06 conditional novelty 5.0 of 10

    A pilot study of a GenAI literacy chatbot for older adults found positive qualitative feedback and a non-significant trend in self-reported AI literacy.

Reference graph

Works this paper leans on

77 extracted references · 58 canonical work pages · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Ahmad Al-Doulat, Nasheen Nur, Alireza Karduni, Aileen Benedict, Erfan Al-Hossami, Mary Lou Maher, Wenwen Dou, Mohsen Dorodchi, and Xi Niu. 2020. Making Sense of Student Success and Risk Through Unsupervised Machine Learning and Interactive Storytelling. In Artificial Intelligence in Education. Springer International Publishing, Cham, 3–15

    work page 2020

  3. [3]

    Ali Abbas Falah Alzubi. 2024. Generative Artificial Intelligence in the EFL Writing Context: Students’ Literacy in Perspective. Qubahan Academic Journal 4, 2 (May 2024), 59–69. https://doi.org/10.48161/qaj.v4n2a506

    work page doi:10.48161/qaj.v4n2a506 2024

  4. [4]

    Ravinithesh Annapureddy, Alessandro Fornaroli, and Daniel Gatica-Perez. 2024. Generative AI Literacy: Twelve Defining Competencies. Digital Government: Research and Practice (Aug. 2024). https://doi.org/10.1145/3685680

    doi:10.1145/3685680 2024

  5. [5]

    American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. 2014. Standards for Educational and Psychological Testing . AERA, Washington, DC

    work page 2014

  6. [6]

    Richard C Atkinson. 1968. Human memory: A proposed system and its control processes. The psychology of learning and motivation 2 (1968)

    work page 1968

  7. [7]

    Zied Bahroun, Chiraz Anane, Vian Ahmed, and Andrew Zacca. 2023. Transforming education: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis. Sustainability 15, 17 (2023), 12983

    work page 2023

  8. [8]

    Benjamin S Bloom, David R Krathwohl, Bertram B Masia, et al. 1984. Bloom taxonomy of educational objectives. In Allyn and Bacon. Pearson Education London

    work page 1984

Show all 77 references
  1. [9]

    Dale Bowman, Zachari Swiecki, Zhiqiang Cai, Yeyu Wang, Brendan Eagan, Jeff Linderoth, and David Williamson Shaffer. 2021. The Mathematical Foundations of Epistemic Network Analysis. InAdvances in Quantitative Ethnography, Andrew R. Ruis and Seung B. Lee (Eds.). Springer Intern...

    2021

  2. [10]

    Aras Bozkurt. 2024. Why Generative AI Literacy, Why Now and Why it Matters in the Educational Landscape? Kings, Queens and GenAI Dragons. Open Praxis 16, 3 (2024), 283–290

    2024

  3. [11]

    Gary Charness, Uri Gneezy, and Michael A Kuhn. 2012. Experimental methods: Between-subject and within-subject design. Journal of economic behavior & organization 81, 1 (2012), 1–8

    2012

  4. [12]

    Thomas K.F. Chiu. 2024. Future research recommendations for transforming higher education with generative AI. Computers and Education: Artificial Intelligence 6 (June 2024), 100197

    2024

  5. [13]

    Linda Corrin. 2018. Evaluating students’ interpretation of feedback in interactive dashboards. Score reporting research and applications (2018), 145–159

    2018

  6. [14]

    Mutlu Cukurova. 2024. The interplay of learning, analytics and artificial intelligence in education: A vision for hybrid intelligence. BJET (2024)

    2024

  7. [15]

    Vanessa Echeverria, Roberto Martinez-Maldonado, Roger Granda, Katherine Chiluiza, Cristina Conati, and Simon Buckingham Shum. 2018. Driving data storytelling from learning design. In Proceedings of the 8th international conference on learning analytics and knowledge . 131–140

    2018

  8. [16]

    Vanessa Echeverria, Roberto Martinez-Maldonado, Simon Buckingham Shum, Katherine Chiluiza, Roger Granda, and Cristina Conati. 2018. Exploratory versus explanatory visual learning analytics: Driving teachers’ attention through educational data storytelling. JLA 5, 3 (2018), 73–97

    2018

  9. [17]

    Yizhou Fan, Yuanru Tan, Mladen Raković, Yeyu Wang, Zhiqiang Cai, David Williamson Shaffer, and Dragan Gašević. 2022. Dissecting Learning Tactics in MOOC using Ordered Network Analysis. Journal of Computer Assisted Learning (2022), in press

    2022

  10. [18]

    Gloria Milena Fernandez Nieto, Kirsty Kitto, Simon Buckingham Shum, and Roberto Martínez-Maldonado. 2022. Beyond the learning analytics dashboard: Alternative ways to communicate student data insights combining visualisation, narrative and storytelling. In LAK ’22. 219–229

    2022

  11. [19]

    Gloria Milena Fernandez-Nieto, Roberto Martinez-Maldonado, Vanessa Echeverria, Kirsty Kitto, Dragan Gašević, and Simon Buckingham Shum

  12. [20]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 (2023)

    2023 arXiv

  13. [21]

    Pauline Gibbons. 2002. Scaffolding language, scaffolding learning . Heinemann Portsmouth, NH

    2002

  14. [22]

    Kirk Goldsberry. 2012. Courtvision: New visual and spatial analytics for the nba. In 2012 MIT Sloan sports analytics conference , Vol. 9. 12–15

    2012

  15. [23]

    Lucas Rafael Stefanel Gris, Ricardo Marcacini, Arnaldo Candido Junior, Edresson Casanova, Anderson Soares, and Sandra Maria Aluísio. 2023. Evaluating OpenAI’s Whisper ASR for Punctuation Prediction and Topic Modeling of life histories of the Museum of the Person. arXiv preprin...

    2023 arXiv

  16. [24]

    Sara Hennessy, Mutlu Cukurova, Cathy Lewin, Manolis Mavrikis, and Louis Major. 2024. BJET Editorial 2024: A call for research rigour. , 5–9 pages. Manuscript submitted to ACM 16 Jin et al

    2024

  17. [25]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys 55, 12 (2023), 1–38

    2023

  18. [26]

    Yueqiao Jin, Roberto Martinez-Maldonado, Dragan Gašević, and Lixiang Yan. 2024. GLAT: The Generative AI Literacy Assessment Test. arXiv preprint arXiv:2411.00283 (2024)

    2024 arXiv

  19. [27]

    Yueqiao Jin, Lixiang Yan, Vanessa Echeverria, Dragan Gašević, and Roberto Martinez-Maldonado. 2024. Generative AI in Higher Education: A Global Perspective of Institutional Adoption Policies and Guidelines. arXiv preprint arXiv:2405.11800 (2024)

    2024 arXiv

  20. [28]

    Rogers Kaliisa, Kamila Misiejuk, Sonsoles López-Pernas, Mohammad Khalil, and Mohammed Saqr. 2024. Have Learning Analytics Dashboards Lived Up to the Hype? A Systematic Review of Impact on Students’ Achievement, Motivation, Participation and Attitude. In LAK ’24 (LAK ’24) . ACM

    2024

  21. [29]

    Rogers Kaliisa, Anders Mørch, and Anders Kluge. 2022. ‘My Point of Departure for Analytics is Extreme Skepticism’: Implications Derived from An Investigation of University Teachers’ Learning Analytics Perspectives and Design Practices. Technology, Knowledge and Learning 27 (06 2022)

    2022

  22. [30]

    Hassan Khosravi, Olga Viberg, Vitomir Kovanovic, and Rebecca Ferguson. 2023. Generative AI and Learning Analytics. JLA 10, 3 (2023), 1–6

    2023

  23. [31]

    Jeonghyun Kim, Il-Hyun Jo, and Yeonjeong Park. 2016. Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement. Asia Pacific Education Review 17 (2016), 13–24

    2016

  24. [32]

    Nam Ju Kim, Brian R Belland, and Andrew E Walker. 2018. Effectiveness of computer-based scaffolding in the context of problem-based learning for STEM education: Bayesian meta-analysis. Educational Psychology Review 30 (2018), 397–429

    2018

  25. [33]

    Mohammad Amin Kuhail, Nazik Alturki, Salwa Alramlawi, and Kholood Alhejori. 2022. Interacting with educational chatbots: A systematic review. Education and Information Technologies 28, 1 (July 2022), 973–1018. https://doi.org/10.1007/s10639-022-11177-3

    2022 doi

  26. [34]

    Sukwon Lee, Sung-Hee Kim, and Bum Chul Kwon. 2016. Vlat: Development of a visualization literacy assessment test. IEEE transactions on visualization and computer graphics 23, 1 (2016), 551–560

    2016

  27. [35]

    Unggi Lee, Ariel Han, Jeongjin Lee, Eunseo Lee, Jiwon Kim, Hyeoncheol Kim, and Cheolil Lim. 2023. Prompt Aloud!: Incorporating image-generative AI into STEAM class with learning analytics using prompt data. Education and Information Technologies (2023), 1–31

    2023

  28. [36]

    Haotian Li, Yun Wang, and Huamin Qu. 2024. Where are we so far? understanding data storytelling tools from the perspective of human-ai collaboration. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–19

    2024

  29. [37]

    Lyn Lim, Maria Bannert, Joep van der Graaf, Shaveen Singh, Yizhou Fan, Surya Surendrannair, Mladen Rakovic, Inge Molenaar, Johanna Moore, and Dragan Gašević. 2023. Effects of real-time analytics-based personalized scaffolds on students’ self-regulated learning. Computers in Hu...

    2023

  30. [38]

    Duri Long and Brian Magerko. 2020. What is AI Literacy? Competencies and Design Considerations. In CHI ’20. ACM

    2020

  31. [39]

    Wenhan Lyu, Yimeng Wang, Tingting (Rachel) Chung, Yifan Sun, and Yixuan Zhang. 2024. Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study. In L@S ’24. ACM, 63–74

    2024

  32. [40]

    Pingchuan Ma, Rui Ding, Shuai Wang, Shi Han, and Dongmei Zhang. 2023. Demonstration of InsightPilot: An LLM-empowered automated data exploration system. arXiv preprint arXiv:2304.00477 (2023)

    2023 arXiv

  33. [41]

    Roberto Martinez-Maldonado, Vanessa Echeverria, Gloria Fernandez Nieto, and Simon Buckingham Shum. 2020. From data to insights: A layered storytelling approach for multimodal learning analytics. In Proceedings of the 2020 chi conference on human factors in computing systems . 1–15

    2020

  34. [42]

    Roberto Martinez-Maldonado, Vanessa Echeverria, Gloria Fernandez-Nieto, Lixiang Yan, Linxuan Zhao, Riordan Alfredo, Xinyu Li, Samantha Dix, Hollie Jaggard, Rosie Wotherspoon, et al. 2023. Lessons learnt from a multimodal learning analytics deployment in-the-wild. ACM Transacti...

    2023

  35. [43]

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems 26 (2013)

    2013

  36. [44]

    Davy Tsz Kit Ng, Jac Ka Lok Leung, Samuel Kai Wah Chu, and Maggie Shen Qiao. 2021. Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence 2 (2021), 100041. https://doi.org/10.1016/j.caeai.2021.100041

    2021

  37. [45]

    Xavier Ochoa. 2022. Multimodal Learning Analytics - Rationale, Process, Examples, and Direction. In The Handbook of Learning Analytics (2 ed.), Charles Lang, George Siemens, Alyssa Friend Wise, Dragan Gašević, and Agathe Merceron (Eds.). SoLAR, 54–65

    2022

  38. [46]

    Chinedu Wilfred Okonkwo and Abejide Ade-Ibijola. 2021. Chatbots applications in education: A systematic review. Comput. Educ.: Artif. Intell. 2 (2021), 100033

    2021

  39. [47]

    Keng-Boon Ooi, Garry Wei-Han Tan, Mostafa Al-Emran, Mohammed A Al-Sharafi, Alexandru Capatina, Amrita Chakraborty, Yogesh K Dwivedi, Tzu-Ling Huang, Arpan Kumar Kar, Voon-Hsien Lee, et al. 2023. The potential of generative artificial intelligence across disciplines: Perspectiv...

    2023

  40. [48]

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology . 1–22

    2023

  41. [49]

    Adriano Pinargote, Eddy Calderón, Kevin Cevallos, Gladys Carrillo, Katherine Chiluiza, and Vanessa Echeverría. 2024. Automating Data Narratives in Learning Analytics Dashboards using GenAI.. In LAK Workshops. 150–161

    2024

  42. [50]

    Jeevan Pokhrel and Aruna Awasthi. 2021. Effectiveness of Dashboard and Intervention Design . Springer International Publishing, 93–116

    2021

  43. [51]

    Stanislav Pozdniakov, Roberto Martinez-Maldonado, Yi-Shan Tsai, Vanessa Echeverria, Namrata Srivastava, and Dragan Gasevic. 2023. How Do Teachers Use Dashboards Enhanced with Data Storytelling Elements According to their Data Visualisation Literacy Skills?. In LAK’23. 89–99

    2023

  44. [52]

    Gomathy Ramaswami, Teo Susnjak, Anuradha Mathrani, and Rahila Umer. 2022. Use of Predictive Analytics within Learning Analytics Dashboards: A Review of Case Studies. Technology, Knowledge and Learning 28, 3 (Aug. 2022), 959–980. https://doi.org/10.1007/s10758-022-09613-x Manus...

    2022 doi

  45. [53]

    Wenqi Ren, Yang Tang, Qiyu Sun, Chaoqiang Zhao, and Qing-Long Han. 2023. Visual semantic segmentation based on few/zero-shot learning: An overview. IEEE/CAA Journal of Automatica Sinica (2023)

    2023

  46. [54]

    Muhittin Sahin and Dirk Ifenthaler. 2021. Visualizations and dashboards for learning analytics: A systematic literature review. Visualizations and dashboards for learning analytics (2021), 3–22

    2021

  47. [55]

    David Williamson Shaffer, Wesley Collier, and Andrew R Ruis. 2016. A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. JLA 3, 3 (2016), 9–45

    2016

  48. [56]

    Hongbo Shao, Roberto Martinez-Maldonado, Vanessa Echeverria, Lixiang Yan, and Dragan Gasevic. 2024. Data Storytelling in Data Visualisation: Does it Enhance the Efficiency and Effectiveness of Information Retrieval and Insights Comprehension?. In CHI’24. 1–21

    2024

  49. [57]

    Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. arXiv preprint arXiv:2104.07567 (2021)

    2021 arXiv

  50. [58]

    Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (RAG) models for open domain question answering. Transactions of the Association for Comp...

    2023

  51. [59]

    Teo Susnjak, Gomathy Suganya Ramaswami, and Anuradha Mathrani. 2022. Learning analytics dashboard: a tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education 19, 1 (Feb. 2022). https://doi.org/10.1186/s41239-021-00313-7

    2022 doi

  52. [60]

    Robert M Thorndike, George K Cunningham, Robert Ladd Thorndike, and Elizabeth P Hagen. 1991. Measurement and evaluation in psychology and education. Macmillan Publishing Co, Inc

    1991

  53. [61]

    Katrien Verbert, Xavier Ochoa, Robin De Croon, Raphael A Dourado, and Tinne De Laet. 2020. Learning analytics dashboards: the past, the present and the future. In Proceedings of the 10th Learning Analytics and Knowledge Conference . 35–40

    2020

  54. [62]

    Cai-Ting Wen, Chen-Chung Liu, Ching-Yuan Li, Ming-Hua Chang, Shih-Hsun Fan Chiang, Hung-Ming Lin, Fu-Kwun Hwang, and Gautam Biswas

  55. [63]

    Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt

  56. [64]

    Computers & Education 215 (2024), 105043

    The learning analytics of computational scientific modeling with self-explanation for subgoals and demonstration scaffolding. Computers & Education 215 (2024), 105043

    2024

  57. [65]

    GE Xun and Susan M Land. 2004. A conceptual framework for scaffolding III-structured problem-solving processes using question prompts and peer interactions. Educational technology research and development 52, 2 (2004), 5–22

    2004

  58. [66]

    Lixiang Yan, Vanessa Echeverria, Yueqiao Jin, Gloria Fernandez-Nieto, Linxuan Zhao, Xinyu Li, Riordan Alfredo, Zachari Swiecki, Dragan Gašević, and Roberto Martinez-Maldonado. 2024. Evidence-based multimodal learning analytics for feedback and reflection in collaborative learn...

    2024

  59. [67]

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155 (2023)

    2023 arXiv

  60. [68]

    Lixiang Yan, Roberto Martinez-Maldonado, and Dragan Gasevic. 2024. Generative Artificial Intelligence in Learning Analytics: Contextualising Opportunities and Challenges through the Learning Analytics Cycle. In LAK ’24. 101–111

    2024

  61. [69]

    Lixiang Yan, Roberto Martinez-Maldonado, Linxuan Zhao, Samantha Dix, Hollie Jaggard, Rosie Wotherspoon, Xinyu Li, and Dragan Gašević. 2023. The role of indoor positioning analytics in assessment of simulation-based learning. BJET 54, 1 (2023), 267–292

    2023

  62. [70]

    Lixiang Yan, Samuel Greiff, Ziwen Teuber, and Dragan Gašević. 2024. Promises and challenges of generative artificial intelligence for human learning. Nature Human Behaviour 8, 10 (2024), 1839–1850

    2024

  63. [71]

    Lixiang Yan, Linxuan Zhao, Vanessa Echeverria, Yueqiao Jin, Riordan Alfredo, Xinyu Li, Dragan Gaševi’c, and Roberto Martinez-Maldonado. 2024. VizChat: Enhancing Learning Analytics Dashboards with Contextualised Explanations Using Multimodal Generative AI Chatbots. In Internati...

    2024

  64. [72]

    Ahmed Mohamed Fahmy Yousef and Ahmed Ramadan Khatiry. 2021. Cognitive versus behavioral learning analytics dashboards for supporting learner’s awareness, reflection, and learning process. Interactive Learning Environments 31, 9 (Dec. 2021), 5460–5476

    2021

  65. [73]

    Lixiang Yan, Lele Sha, Linxuan Zhao, Yuheng Li, Roberto Martinez-Maldonado, Guanliang Chen, Xinyu Li, Yueqiao Jin, and Dragan Gašević. 2024. Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Tech...

    2024

  66. [74]

    Xin Zhao, Andrew Cox, and Liang Cai. 2024. ChatGPT and the digitisation of writing. Humanit. Soc. Sci. Commun. 11, 1 (April 2024). Manuscript submitted to ACM

    2024

  67. [76]

    Linxuan Zhao, Zachari Swiecki, Dragan Gašević, Lixiang Yan, Samantha Dix, Hollie Jaggard, Rosie Wotherspoon, Abra Osborne, Xinyu Li, Riordan Alfredo, and Roberto Martinez-Maldonado. 2023. METS: Multimodal Learning Analytics of Embodied Teamwork Learning. In LAK ’23. 186–196

    2023

  68. [2023]

    arXiv preprint arXiv:2302.11382 (2023)

    A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023)

    2023 arXiv

  69. [2024]

    In LAK ’24

    Data storytelling editor: A teacher-centred tool for customising learning analytics dashboard narratives. In LAK ’24. 678–689

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.