Pith. sign in

REVIEW 7 cited by

Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI Testing

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.09434 v1 pith:7TQI35EN submitted 2023-05-16 cs.SE

Zhe Liu , Chunyang Chen , Junjie Wang , Mengzhuo Chen , Boyu Wu , Xing Che , Dandan Wang , Qing Wang This is my paper · ORCID
classification cs.SE
keywords testingmobileappsgptdroidautomatedhuman-liketestactions
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Mobile apps are indispensable for people's daily life, and automated GUI (Graphical User Interface) testing is widely used for app quality assurance. There is a growing interest in using learning-based techniques for automated GUI testing which aims at generating human-like actions and interactions. However, the limitations such as low testing coverage, weak generalization, and heavy reliance on training data, make an urgent need for a more effective approach to generate human-like actions to thoroughly test mobile apps. Inspired by the success of the Large Language Model (LLM), e.g., GPT-3 and ChatGPT, in natural language understanding and question answering, we formulate the mobile GUI testing problem as a Q&A task. We propose GPTDroid, asking LLM to chat with the mobile apps by passing the GUI page information to LLM to elicit testing scripts, and executing them to keep passing the app feedback to LLM, iterating the whole process. Within it, we extract the static context of the GUI page and the dynamic context of the iterative testing process, design prompts for inputting this information to LLM, and develop a neural matching network to decode the LLM's output into actionable steps to execute the app. We evaluate GPTDroid on 86 apps from Google Play, and its activity coverage is 71%, with 32% higher than the best baseline, and can detect 36% more bugs with faster speed than the best baseline. GPTDroid also detects 48 new bugs on the Google Play with 25 of them being confirmed/fixed. We further summarize the capabilities of GPTDroid behind the superior performance, including semantic text input, compound action, long meaningful test trace, and test case prioritization.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DragonCrawl: A Generative, Intent-Based Framework for Scalable Mobile End-to-End Testing

    cs.SE 2026-07 conditional novelty 6.0 of 10

    DragonCrawl uses GPT-4o-based intent reasoning, multimodal end-state detection, and backend tool calling to run 1,013 mobile regression tests in CI/CD, achieving over 91% pass rates and cutting test maintenance effort.

  2. CityRAG: Stepping Into a City via Spatially-Grounded Video Generation

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    PlayCoder combines a repository-aware coding agent with a vision-based GUI testing agent and an automated program repair loop to detect and fix silent logic errors in LLM-generated interactive application code.

  3. LogiDroid: Individual Functional Test Generation via Business Logic Extraction and Adaptation

    cs.SE 2026-02 conditional novelty 6.0 of 10

    LogiDroid generates functional Android test cases with verification assertions by retrieving similar test cases, fusing their business logic, and adapting it to the target app's real-time GUI state.

  4. Synthetic Heuristic Evaluation: A Comparison between AI- and Human-Powered Usability Evaluation

    cs.HC 2025-07 reject novelty 6.0 of 10

    An LLM prompted to conduct heuristic evaluation reported more usability issues on two apps than five human experts, but the ground truth included the LLM's own findings.

  5. VISCA: Inferring Component Abstractions for Automated End-to-End Testing

    cs.SE 2025-06 conditional novelty 6.0 of 10

    VISCA converts webpages into a semantic component abstraction and uses it as context for LLM-generated end-to-end tests, reporting 92% average feature coverage on E2EBench.

  6. Data-driven Progressive Discovery of Physical Laws

    cs.LG 2026-03 unverdicted novelty 5.0 of 10

    CoSR discovers physical laws via progressive chains of symbolic knowledge units, recovering Kepler-to-Newton and improving scaling laws in convection, pipe flow, laser-metal interaction, and aircraft aerodynamics.

  7. Evaluation and Benchmarking of LLM Agents: A Survey

    cs.LG 2025-07 conditional novelty 4.0 of 10

    A review that proposes a two-dimensional taxonomy for evaluating LLM agents and highlights enterprise-specific evaluation gaps.

Pith tools