REVIEW 4 major objections 5 minor 47 references
ExplorAR: Assisting Older Adults to Learn Smartphone Apps through AR-powered Trial-and-Error with Interactive Guidance
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read AR-based trial-and-error with instant feedback enables older adults to learn smartphone apps more effectively than video tutorials.
desk verdict Genuinely useful AR trial-and-error system with a confounded evaluation; the current study can't support the headline claim without a redesign or much more careful statistics. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is ExplorTree, a hierarchical structure of ExplorNodes, where each node stores a page ID, a screenshot of the interface, and the list of actions (tap, swipe, or text input with coordinates) leading to subsequent nodes. The system matches the phone's current screen to a node, displays an enlarged screenshot in the AR space, and compares the user's tap position against the action's area: a correct action turns the border green, a wrong action turns it red, and after a wrong page or 20 seconds of inactivity it offers visual guidance that highlights the element leading to the next step. This gives older adults a safe space to try, error, and recover without losing their place, which the paper argues is what drives the retention benefit.
What would settle it
Re-run the comparison with the same app learned under all three conditions, counterbalancing which app is paired with which method across participants; if ExplorAR no longer yields the shortest independent completion time and fewest mistakes, the claimed advantage is an artifact of task difficulty.
Extended reading notes
Core claim
The central claim is that AR-supported trial-and-error, implemented as ExplorAR, improves both the outcome and the experience of older adults learning unfamiliar smartphone apps, compared with a conventional video tutorial and with AR step-by-step instructions that allow no self-correction. The objective results back this: after learning, the ExplorAR group finished the analogous transfer tasks fastest (mean 18.3 s vs 27.7 s for video and 34.4 s for AR instructions) and accumulated the fewest mistakes (7 vs 13 and 18). Subjectively, older adults rated ExplorAR higher on pragmatic quality, hedonic quality, and overall user experience, and in interviews reported reduced fear of making mistakes and increased confidence to explore.
Load-bearing premise
The study assumes the three apps and their tasks are equally difficult, because each learning method was paired with a different app and the app order was fixed, so easier tasks rather than the method itself could explain ExplorAR's faster completion and fewer mistakes.
Editorial extensions
If this is right
- If the claim holds, AR-based trial-and-error can replace passive video tutorials as the default support for older adults learning new apps, reducing reliance on family or friends for help.
- The error-recovery design can be extended to riskier operations such as payments or sensitive data entry, where the safe AR environment lets users practice without real consequences.
- The ExplorTree representation can be reused to auto-generate interactive guidance for other apps and devices, cutting the cost of producing accessible tutorials.
- Learning by correcting mistakes should produce longer-lasting procedural memory than following instructions, which the paper's transfer-task results begin to support.
Reading between the lines
- Editorial inference: the study does not isolate whether the benefit comes from trial-and-error itself or from its AR delivery; a non-AR trial-and-error condition, such as on-phone prompts with the same feedback logic, would separate the two.
- Editorial inference: because each learning method was paired with a different app, a rotated app-method assignment across participants would test whether part of the video's disadvantage reflects interface unfamiliarity rather than the delivery medium.
- Editorial inference: the same guided-mistake mechanism could plausibly help other groups, such as low-literacy users or people with cognitive impairments, but that extension is untested in this paper.
- Editorial inference: the objective gains are large enough that a replication with a larger sample and within-app counterbalancing would decide whether the method, not the task pairing, is responsible for the advantage.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents ExplorAR, an AR-based trial-and-error system intended to help older adults learn smartphone applications. The authors describe the system's architecture (ExplorNode, ExplorTree, and a HoloLens 2 application), its design rationale grounded in prior work, and a within-subjects user study with 18 older adults comparing ExplorAR against video tutorials and AR-based step-by-step instructions. Objective measures (learning time, task completion time, mistake counts) and subjective measures (UEQ-S, interviews) are reported. The paper claims that ExplorAR improves learning effectiveness and confidence, and it offers four design implications.
Significance. If the reported findings are robust, the work would be a meaningful contribution to accessible technology for older adults, demonstrating a concrete way to combine augmented reality with trial-and-error learning and error recovery. The system implementation, automation of instructional content via AppAgent, and the qualitative insights from the user study are valuable. However, the central quantitative claims are not currently supported by appropriate statistical analysis, and one claim in the Introduction is contradicted by the paper's own data. The significance of the contribution depends on resolving these issues.
major comments (4)
- [Section 5.5, Table 2] The paper claims "significant difference" and "significantly less time" (e.g., for task completion time) but reports no inferential statistics: no test names, p-values, effect sizes, or confidence intervals are given for learning time, task completion time, or mistake counts. The mistake counts are pooled totals across participants, which cannot support the claim that ExplorAR produced the "fewest mistakes" without per-participant distributions. Please provide a repeated-measures ANOVA or mixed-effects model with condition as a fixed factor, participant as a random effect, and order and app/task as covariates or factors, and report effect sizes and confidence intervals. For mistake counts, use a per-participant count model (e.g., Poisson or negative binomial mixed model).
- [Section 1 vs. Table 2] The Introduction states that "both AR-based methods enable participants to learn and complete tasks more effectively than the video tutorial," but Table 2 shows that AR instruction had a longer mean task completion time (34.4s vs. 27.7s) and more mistakes (18 vs. 13) than video tutorial. This is an internal inconsistency that must be resolved; either correct the claim or provide a justification based on other measures (e.g., learning time) that supports the statement.
- [Section 5.2] The study design description is ambiguous. The text says "we prerecorded tutorials with three different apps corresponding to three conditions" and "We fixed the order of the apps," followed by "assigned each participant to one of six combination condition status orders, following a balanced Latin square." Please state explicitly whether app-condition pairing is counterbalanced across participants. If the app order is fixed and the condition order is varied via the Latin square, then each app is paired with each condition equally often across participants; in that case, describe how app and order effects are included in the analysis. If the pairing is not counterbalanced, the objective comparisons would be confounded by task difficulty. Also clarify whether the three tasks (e.g., recharging phone credits, buying on a shopping app, etc.) were matched in difficulty, and whether "never used them before" refers to the apps or the tasks.
- [Section 5.6.1 and Figure 5] The subjective UEQ-S results are reported as means (e.g., pragmatic quality, hedonic quality, overall) without any inferential statistics. Claims such as "participants rate higher scores for both two modes of the AR-based learning methods" and "ExplorAR has a slight advantage" need statistical support, for example Wilcoxon signed-rank tests or repeated-measures ANOVA with appropriate post-hoc comparisons. Please add the test statistics, p-values, and effect sizes.
minor comments (5)
- [Section 4.2.2] The sentence "If user clicks wrong position to the page does not match the right page" is grammatically incorrect and should be rephrased, for example: "If the user clicks the wrong position and navigates to a page that does not match the correct page."
- [Section 4.1] The phrase "we encourage LLM to interact with smartphone apps" should be revised to "we instruct an LLM to interact" or "we use an LLM to interact," since the intended meaning is that the LLM is employed to record interactions, not encouraged in an emotional sense.
- [Section 3] The sentence "This three components comprises" contains a subject-verb agreement error; it should be "These three components comprise."
- [Section 5.2] In the sentence "In the choice of tutorial tasks, we made sure participants had never used them before the experiment," the pronoun "them" is ambiguous; please specify whether it refers to the apps or the tasks.
- [Overall] The paper would benefit from a thorough proofread to fix several grammatical errors and typos, such as "intergrate" (Section 3), "AR instrucrion" (Table 1), and missing spaces before parentheses (e.g., "SD = 4.23) Participants").
Circularity Check
No significant circularity: ExplorAR is evaluated by an external user study against video and AR baselines, not by a derivation that reduces to its own inputs.
full rationale
The paper's central claim is empirical: an 18-participant user study compares ExplorAR with a video tutorial and an AR step-by-step condition, measuring learning time, independent task completion time, and mistake counts (Table 2). These outcomes are observed behaviors, not quantities defined by the system or fitted from the data being 'predicted.' The ExplorTree representation and AR guidance are engineering constructions, and the paper does not present a mathematical derivation in which a target result equals an input by construction. The design considerations are said to be 'derived from prior work,' including the authors' own earlier papers (e.g., Synapse [15] and the CHI 2024 AR study [16]), but those citations inform design choices; the effectiveness claim is tested against external baselines and does not require the cited prior results to be true in order for the measured comparison to stand. There is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors' own prior work to force a choice, and no ansatz smuggled in by self-citation. The app-condition confound noted in Section 5.2 is a genuine methodological validity concern about whether the compared apps/tasks are equivalent, but a confound is not circularity: the measured advantage is not equivalent to the study's inputs by definition. Overall, the derivation chain is self-contained and externally benchmarked, so the appropriate circularity score is 0.
Assumptions & free parameters
assumptions (4)
- domain assumption Three selected apps (finance, shopping, public transportation) and their assigned learning tasks are comparable in difficulty across the three conditions.
- domain assumption AppAgent's LLM-generated interaction documentation, after manual validation, accurately captures all navigation steps needed to build ExplorTree.
- domain assumption Vuforia image recognition reliably matches the smartphone's current screen to the corresponding ExplorNode PageImage in real time.
- domain assumption The recruited sample represents the broader population of community-dwelling older adults learning smartphone apps.
Cite this review
Pith. "Pith review of ExplorAR: Assisting Older Adults to Learn Smartphone Apps through AR-powered Trial-and-Error with Interactive Guidance." pith.science (2026). https://pith.science/paper/P645H2C2
@misc{pith2026250801282,
author = {Pith},
title = {Pith review of: ExplorAR: Assisting Older Adults to Learn Smartphone Apps through AR-powered Trial-and-Error with Interactive Guidance},
year = {2026},
howpublished = {\url{https://pith.science/paper/P645H2C2}},
note = {Machine review of arXiv:2508.01282}
}
read the original abstract
Older adults tend to encounter challenges when learning to use new smartphone apps due to age-related cognitive and physical changes. Compared to traditional support methods such as video tutorials, trial-and-error allows older adults to learn to use smartphone apps by making and correcting mistakes. However, it remains unknown how trial-and-error should be designed to empower older adults to use smartphone apps and how well it would work for older adults. Informed by the guidelines derived from prior work, we designed and implemented ExplorAR, an AR-based trial-and-error system that offers real-time and situated visual guidance in the augmented space around the smartphone to empower older adults to explore and correct mistakes independently. We conducted a user study with 18 older adults to compare ExplorAR with traditional video tutorials and a simplified version of ExplorAR. Results show that the AR-supported trial-and-error method enhanced older adults' learning experience by fostering deeper cognitive engagement and improving confidence in exploring unknown operations.
Reference graph
Works this paper leans on
-
[1]
Monica Anderson and Andrew Perrin. 2017. Technology Use Among Seniors. Washington, DC: Pew Research Center for Internet & Technology (2017)
work page 2017
-
[2]
Yvonne Barnard, Mike D Bradley, Frances Hodgson, and Ashley D Lloyd. 2013. Learning to use new technologies by older adults: Perceived difficulties, experi- mentation behaviour and usability. Computers in human behavior 29, 4 (2013), 1715–1724
work page 2013
-
[3]
Anna Cornelia Maria Bauer and Gerda Andringa. 2020. The Potential of Immer- sive Virtual Reality for Cognitive Training in Elderly. Gerontology 66, 6 (2020), 614–623
work page 2020
-
[4]
Jonas Blattgerste, Patrick Renner, and Thies Pfeiffer. 2019. Augmented reality action assistance and learning for cognitively impaired people: a systematic literature review. In Proceedings of the 12th ACM international conference on pervasive technologies related to assistive environments . 270–279
work page 2019
-
[5]
Statistics Bureau. 2010. Ministry of internal affairs and communications, Japan. Annual Report on Current Population Estimates (2010)
work page 2010
-
[6]
Ke Chen and Alan Hoi Shou Chan. 2014. Gerontechnology acceptance by elderly Hong Kong Chinese: a senior technology acceptance model (STAM). Ergonomics 57, 5 (2014), 635–652
work page 2014
-
[7]
Sho Conte and Cosmin Munteanu. 2019. Help! I’m Stuck, and there’s no F1 Key on My Tablet!. InProceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (Taipei, Taiwan) (MobileHCI ’19). Association for Computing Machinery, New York, NY, USA, Article 10, 11 pages. doi:10.1145/3338286.3340121
arXiv 2019
-
[8]
Teresa Correa, Joseph D Straubhaar, Wenhong Chen, and Jeremiah Spence. 2015. Brokering new technologies: The role of children in their parents’ usage of the internet. New Media & Society 17, 4 (2015), 483–500
work page 2015
Show all 47 references
-
[9]
Coursera. 2012. Coursera. https://www.coursera.org/. Accessed: 2024-09-02
2012
-
[10]
Ernst & Young. 2019. Global FinTech Adoption Index 2019. https: //assets.ey.com/content/dam/ey-sites/ey-com/en_gl/topics/banking-and- capital-markets/ey-global-fintech-adoption-index.pdf. Accessed: 2024-09-02
2019
-
[11]
Mingming Fan and Khai N Truong. 2018. Guidelines for creating senior-friendly product instructions. ACM Transactions on Accessible Computing (TACCESS) 11, 2 (2018), 1–35
2018
-
[12]
Shihan Fu, Jianhao Chen, Emily Kuang, and Mingming Fan. 2024. Bridging the Literacy Gap for Adults: Streaming and Engaging in Adult Literacy Education through Livestreaming. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24...
2024 doi
-
[13]
Hari Chandana, Nazeer Shaik, and P
B. Hari Chandana, Nazeer Shaik, and P. Chitralingappa. 2023. Exploring the Frontiers of User Experience Design: VR, AR, and the Future of Interaction. In 2023 International Conference on Computer Science and Emerging Technologies (CSET). 1–6. doi:10.1109/CSET58993.2023.10346724
2023
-
[14]
Jeremy Hartmann, Aakar Gupta, and Daniel Vogel. 2020. Extend, Push, Pull: Smartphone Mediated Interaction in Spatial Augmented Reality via Intuitive Mode Switching. In Proceedings of the 2020 ACM Symposium on Spatial User Inter- action (Virtual Event, Canada) (SUI ’20). Associ...
2020
-
[15]
Xiaofu Jin, Xiaozhu Hu, Xiaoying Wei, and Mingming Fan. 2022. Synapse: Interactive Guidance by Demonstration with Trial-and-Error Support for Older Adults to Use Smartphone Apps. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 3, Article 121 (Sept. 2022), 24 pages. do...
2022 doi
-
[16]
Xiaofu Jin, Wai Tong, Xiaoying Wei, Xian Wang, Emily Kuang, Xiaoyu Mo, Huamin Qu, and Mingming Fan. 2024. Exploring the Opportunity of Augmented Reality (AR) in Supporting Older Adults to Explore and Learn Smartphone Ap- plications (CHI ’24). Association for Computing Machiner...
2024
-
[17]
E. Klopfer. 2008. Augmented Learning: Research and Design of Mobile Educational Games. Penguin Random House LLC. https://books.google.com/books?id= I0kaFNaK704C
2008
-
[18]
Sri Kurniawan. 2008. Older people and mobile phones: A multi-method investi- gation. International Journal of Human-Computer Studies 66, 12 (2008), 889–901
2008
-
[19]
Li Na Lee, Mi Jeong Kim, and Won Ju Hwang. 2019. Potential of Augmented Reality and Virtual Reality Technologies to Promote Wellbeing in Older Adults. Applied Sciences 9, 17 (2019). doi:10.3390/app9173556
2019 doi
-
[20]
Rock Leung, Charlotte Tang, Shathel Haddad, Joanna Mcgrenere, Peter Graf, and Vilia Ingriany. 2012. How Older Adults Learn to Use Mobile Devices: Survey and Field Investigations. ACM Trans. Access. Comput. 4, 3, Article 11 (Dec. 2012), 33 pages. doi:10.1145/2399193.2399195
2012
-
[21]
Ruohao Li, Jiawei Li, Jia Sun, Zhiqing Wu, Zisu Li, Ziyan Wang, Ge Lin Kan, and Mingming Fan. 2025. RemVerse: Supporting Reminiscence Activities for Older Adults through AI-Assisted Virtual Reality. arXiv preprint arXiv:2507.13247 (2025)
2025 arXiv
-
[22]
Zisu Li, Li Feng, Chen Liang, Yuru Huang, and Mingming Fan. 2023. Exploring the Opportunities of AR for Enriching Storytelling with Family Photos between Grandparents and Grandchildren. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 3 (2...
2023
-
[23]
Zisu Li, Jiawei Li, Zeyu Xiong, Shumeng Zhang, Faraz Faruqi, Stefanie Mueller, Chen Liang, Xiaojuan Ma, and Mingming Fan. 2025. InteRecon: Towards Re- constructing Interactivity of Personal Memorable Items in Mixed Reality. arXiv preprint arXiv:2502.09973 (2025)
2025 arXiv
-
[24]
Tracy L Mitzner, Julie B Boron, Cara Bailey Fausset, Anne E Adams, Neil Charness, Sara J Czaja, Katinka Dijkstra, Arthur D Fisk, Wendy A Rogers, and Joseph Sharit
-
[25]
Sarah Morélot, Alain Garrigou, Julie Dedieu, and Bernard N’Kaoua. 2021. Virtual reality for fire safety training: Influence of immersion and sense of presence on conceptual and procedural acquisition. Computers & Education 166 (2021), 104145
2021
-
[26]
Christian Moro, James Birt, Zane Stromberga, Charlotte Phelps, Justin Clark, Paul Glasziou, and Anna Mae Scott. 2021. Virtual and augmented reality enhancements to medical and science student physiology and anatomy test performance: A systematic review and meta-analysis. Anato...
2021
-
[27]
Arthur Nishimoto and Andrew E Johnson. 2019. Extending Virtual Reality Display Wall Environments Using Augmented Reality. In Symposium on Spatial User Interaction (New Orleans, LA, USA) (SUI ’19). Association for Computing Machinery, New York, NY, USA, Article 7, 5 pages. doi:...
2019
-
[28]
El-Glaly
Shraddha Pandya and Yasmine N. El-Glaly. 2018. TapTag: Assistive Gestural Interactions in Social Media on Touchscreens for Older Adults. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulder, CO, USA) (ICMI ’18). Association for Computing M...
2018
-
[29]
Carolyn Pang, Zhiqin Collin Wang, Joanna McGrenere, Rock Leung, Jiamin Dai, and Karyn Moffatt. 2021. Technology Adoption and Learning Preferences for Older Adults: Evolving Perceptions, Ongoing Challenges, and Emerging Design Opportunities. In Proceedings of the 2021 CHI Confe...
2021 doi
-
[30]
Patrick Reipschläger, Severin Engert, and Raimund Dachselt. 2020. Augmented Displays: Seamlessly Extending Interactive Surfaces With Head-Mounted Aug- mented Reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CH...
2020
-
[31]
Lena Rosenberg, Anders Kottorp, Bengt Winblad, and Louise Nygård. 2009. Per- ceived difficulty in everyday technology use among older adults with or without cognitive deficits. Scandinavian journal of occupational therapy 16, 4 (2009), 216–226
2009
-
[32]
Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2017. Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence 4 (01 2017), 103. doi:10.9781/ijimai.2017.09.001
2017 doi
-
[33]
Alexander Seifert and Anna Schlomann. 2021. The Use of Virtual and Augmented Reality by Older Adults: Potentials and Challenges. Frontiers in Virtual Reality 2 (2021). doi:10.3389/frvir.2021.639718
2021
-
[34]
Mahmud Akhter Shareef, Vinod Kumar, Uma Kumar, and Yogesh K. Dwivedi
-
[35]
Froehlich
Lee Stearns, Leah Findlater, and Jon E. Froehlich. 2018. Design of an Augmented Reality Magnification Aid for Low Vision Users. In Proceedings of the 20th Inter- national ACM SIGACCESS Conference on Computers and Accessibility (Galway, Ireland) (ASSETS ’18). Association for Co...
2018
-
[36]
Constantine Stephanidis. 2021. Design for All In Digital Technologies . John Wiley & Sons, Ltd, Chapter 46, 1187–1215. doi:10.1002/9781119636113.ch46 MM ’25, October 27–31, 2025, Dublin, Ireland. Jiawei Li, Linjie Qiu, Zhiqing Wu, Qiongyan Chen, Ziyan Wang, and Mingming Fan
2021 doi
-
[37]
Christian Stößel, Hartmut Wandke, and Lucienne Blessing. 2009. An evaluation of finger-gesture interaction on mobile devices for elderly users. Prospektive Gestaltung von Mensch-Technik-Interaktion 8 (2009), 470–475
2009
-
[38]
Robin E Sülter, Paul E Ketelaar, and Wolf-Gero Lange. 2022. SpeakApp-Kids! Virtual reality training to reduce fear of public speaking in children–A proof of concept. Computers & Education 178 (2022), 104384
2022
-
[39]
Ryo Suzuki, Adnan Karim, Tian Xia, Hooman Hedayati, and Nicolai Marquardt
-
[40]
Eleftheria Vaportzis, Maria Giatsi Clausen, and Alan J. Gow. 2017. Older Adults Perceptions of Technology and Barriers to Interacting with Tablet Computers: A Focus Group Study. Frontiers in Psychology Volume 8 - 2017 (2017). doi:10.3389/ fpsyg.2017.01687
2017
-
[41]
Qianjie Wei, Jingling Zhang, Pengqi Wang, Xiaofu Jin, and Mingming Fan. 2024. Augmented Library: Toward Enriching Physical Library Experience Using HMD- Based Augmented Reality. In Proceedings of the 17th International Symposium on Visual Information Communication and Interact...
2024
-
[42]
Zhiqing Wu, Duotun Wang, Shumeng Zhang, Yuru Huang, Zeyu Wang, and Mingming Fan. 2024. Toward Making Virtual Reality (VR) More Inclusive for Older Adults: Investigating Aging Effect on Target Selection and Manipulation Tasks in VR. In Proceedings of the CHI Conference on Human...
2024
-
[43]
Chi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2025. AppAgent: Multimodal Agents as Smartphone Users. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Ma...
2025
-
[44]
Xiaowen Zhu and Xianping Cheng. 2024. Staying connected: smartphone accep- tance and use level differences of older adults in China. Universal Access in the Information Society 23, 1 (2024), 203–212
2024
-
[2010]
Computers in human behavior 26, 6 (2010), 1710–1721
Older adults talk technology: Technology usage and attitudes. Computers in human behavior 26, 6 (2010), 1710–1721
2010
-
[2011]
Government Information Quarterly 28, 1 (2011), 17–35
e-Government Adoption Model (GAM): Differing service maturity levels. Government Information Quarterly 28, 1 (2011), 17–35. doi:10.1016/j.giq.2010.05. 006
2011 doi
-
[2022]
In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22)
Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York...
2022
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.