Pith. sign in

REVIEW 8 cited by

AutoDroid: LLM-powered Task Automation in Android

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2308.15272 v4 pith:BMXJH5F4 submitted 2023-08-29 cs.AI cs.SE

Hao Wen , Yuanchun Li , Guohong Liu , Shanhui Zhao , Tao Yu , Toby Jia-Jun Li , Shiqi Jiang , Yunhao Liu
show 2 more authors
Yaqin Zhang Yunxin Liu
This is my paper
classification cs.AIcs.SE
keywords autodroidtaskautomationlanguageandroidknowledgellmstasks
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Mobile task automation is an attractive technique that aims to enable voice-based hands-free user interaction with smartphones. However, existing approaches suffer from poor scalability due to the limited language understanding ability and the non-trivial manual efforts required from developers or end-users. The recent advance of large language models (LLMs) in language understanding and reasoning inspires us to rethink the problem from a model-centric perspective, where task preparation, comprehension, and execution are handled by a unified language model. In this work, we introduce AutoDroid, a mobile task automation system capable of handling arbitrary tasks on any Android application without manual efforts. The key insight is to combine the commonsense knowledge of LLMs and domain-specific knowledge of apps through automated dynamic analysis. The main components include a functionality-aware UI representation method that bridges the UI with the LLM, exploration-based memory injection techniques that augment the app-specific domain knowledge of LLM, and a multi-granularity query optimization module that reduces the cost of model inference. We integrate AutoDroid with off-the-shelf LLMs including online GPT-4/GPT-3.5 and on-device Vicuna, and evaluate its performance on a new benchmark for memory-augmented Android task automation with 158 common tasks. The results demonstrated that AutoDroid is able to precisely generate actions with an accuracy of 90.9%, and complete tasks with a success rate of 71.3%, outperforming the GPT-4-powered baselines by 36.4% and 39.7%. The demo, benchmark suites, and source code of AutoDroid will be released at url{https://autodroid-sys.github.io/}.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 12 citations worldwide. Full citation record

  1. MobiBench: Multi-Branch, Modular Benchmark for Mobile GUI Agents

    cs.AI 2025-12 conditional novelty 8.0 of 10

    MobiBench reaches near-human offline evaluation fidelity for mobile GUI agents by accepting any valid action at each step, and enables modular attribution of performance to agent components.

  2. A Comprehensive Study of Implementation Bugs in Multi-modal Agents

    cs.SE 2026-07 accept novelty 7.0 of 10

    First systematic taxonomy of 158 multi-modal agent bugs plus a runtime analyzer that recovers most open issues and surfaces 31 new ones.

  3. FuncDroid: Towards Inter-Functional Flows for Comprehensive Mobile App GUI Testing

    cs.SE 2026-02 conditional novelty 6.0 of 10

    An LLM-guided Android GUI tester that explicitly models inter-functional flows finds more bugs than coverage- or single-functionality-oriented baselines.

  4. Screen2AX: Vision-Based Approach for Automatic macOS Accessibility Generation

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Screen2AX generates hierarchical macOS accessibility metadata from a screenshot and reports improved GPT-4 UI task success compared with native accessibility and OmniParser V2.

  5. Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent

    cs.AI 2025-05 conditional novelty 5.0 of 10

    A hand-authored EFSM planning module boosts Qwen2.5-VL-72B on AndroidWorld from 35.0% to 63.8% task success.

  6. AppAgent-Claw: CLI Is All You Need for GUI Automation

    cs.HC 2026-04 conditional novelty 4.0 of 10

    A record-once, replay-many system converts demonstrated GUI workflows into reliable OpenClaw skills via layered visual localization and post-action validation, without runtime LLM inference.

  7. Cybernaut: Towards Reliable Web Automation

    cs.SE 2025-08 reject novelty 4.0 of 10

    A demonstration-to-SOP framework plus robust element identification and a trace similarity metric improves enterprise web automation success rates on an internal benchmark, with a fine-tuned consistency classifier rea...

  8. Uncertainty-Aware GUI Agent: Adaptive Perception through Component Recommendation and Human-in-the-Loop Refinement

    cs.AI 2025-08 conditional novelty 4.0 of 10

    A GUI agent that trims UI input with a recommendation module and asks users when decisions are ambiguous reports state-of-the-art success rates, though the interaction module is not benchmarked.

Pith tools