REVIEW 18 cited by
GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Autonomous Graphical User Interface (GUI) navigation agents can enhance user experience in communication, entertainment, and productivity by streamlining workflows and reducing manual intervention. However, prior GUI agents often trained with datasets comprising tasks that can be completed within a single app, leading to poor performance in cross-app navigation. To address this problem, we present GUIOdyssey, a comprehensive dataset for cross-app mobile GUI navigation. GUIOdyssey comprises 8,334 episodes with an average of 15.3 steps per episode, covering 6 mobile devices, 212 distinct apps, and 1,357 app combinations. Each step is enriched with detailed semantic reasoning annotations, which aid the model in building cognitive processes and enhancing its reasoning abilities for complex cross-app tasks. Building on GUIOdyssey, we develop OdysseyAgent, an exploratory multimodal agent for long-step cross-app navigation equipped with a history resampler module that efficiently attends to historical screenshot tokens, balancing performance and inference speed. Extensive experiments conducted in both in-domain and out-of-domain scenarios validate the effectiveness of our approach. Moreover, we demonstrate that historial information involving actions, screenshots and context in our dataset can significantly enhances OdysseyAgent's performance on complex cross-app tasks.
Forward citations
Cited by 18 Pith papers
-
TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarios
TSHA is a new 80,000-pair benchmark for indoor safety hazard assessment; current vision-language models score roughly 45-85, and fine-tuning on TSHA raised Qwen2.5-VL-3B by 18.3 points on TSHA's test set.
-
GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL
Action-weighted SFT plus KL-regularized GRPO on a curated 81K reasoning dataset lifts open-source GUI agents by 11–22 points on online task-completion benchmarks.
-
MobiAgent: A Systematic Framework for Customizable Mobile Agents
A full-stack mobile agent framework reports state-of-the-art task completion on its own DAG-based benchmark and 2-3x speedups from replaying recorded trajectories.
-
UItron: Foundational GUI Agent with Advanced Perception and Planning
UItron, trained with multi-stage SFT and curriculum RL on a new 1M-step Chinese app dataset, beats prior GUI agents on perception, grounding, planning, and especially Chinese mobile app scenarios.
-
PG-Agent: An Agent Powered by Page Graph
An MLLM GUI agent that stores past episodes as a page graph and retrieves action guidelines from it improves step success on three benchmarks.
-
SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience
A self-evolving computer-use agent trained with full-trajectory state judging and curriculum task generation goes from 11.3% to 34.5% average success on five OSWorld apps, and a specialist-to-generalist variant beats ...
-
MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents
MMBench-GUI provides a multi-platform, four-level benchmark and an efficiency-aware metric, and its experiments indicate visual grounding is the main bottleneck for current GUI agents.
-
Chain-of-Memory: Enhancing GUI Agents for Cross-Application Navigation
Explicitly storing short-term and long-term text memories, instead of raw screenshots, improves GUI agent accuracy on cross-app tasks, and a new annotated dataset helps 7B models approach 72B-level memory use.
-
GUI-Robust: A Comprehensive Dataset for Testing GUI Agent Robustness in Real-World Anomalies
A new benchmark with 5,318 GUI tasks, including 200 abnormal ones, shows that state-of-the-art GUI agents degrade sharply when real-world anomalies appear.
-
What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilities
A self-generating graph benchmark produces 36k GUI agent tasks with controllable complexity and ten capability scores, and fine-tuning on its trajectories gives small gains on AndroidControl and OmniAct.
-
GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents
An attention-based action head with multi-patch supervision outperforms coordinate-generation baselines on GUI grounding, and a verifier further improves accuracy.
-
ZeroGUI: Automating Online GUI Learning at Zero Human Cost
ZeroGUI uses VLM-generated tasks and VLM-estimated rewards with two-stage GRPO to improve GUI agent success rates on OSWorld and AndroidLab without human annotations.
-
TransBench: Breaking Barriers for Transferable Graphical User Interface Agents in Dynamic Digital Environments
TransBench is a new benchmark of 1,459 screenshots and 22,000 grounding instructions for measuring how well GUI agents transfer across app versions, platforms, and applications.
-
Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents
A real-device-centric foundation GUI agent with hybrid GUI+CLI batched actions, AutoResearch data flywheel, online RL, and a proactive harness reaches SOTA mobile and competitive desktop/web scores.
-
Qwen-Audio-VAE Technical Report
A 12.5 Hz continuous audio VAE reconstructs speech, music, and sound well while encoding 64×30s clips in 541 ms after latency-aware encoder pruning.
-
Atomic-to-Compositional Generalization for Mobile Agents with A New Benchmark and Scheduling System
AI phone agents succeed on single-step commands but fail on multi-step compositions; a new benchmark measures this gap and a scheduler that decomposes tasks recovers most of it.
-
GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behavior
GUI-Reflection trains an 8B multimodal GUI agent to recognize mistakes, undo incorrect actions, and retry, improving AndroidWorld success rate from 14.58% (filtered BC baseline) to 34.72% with reflection data and onli...
-
SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control
A multi-agent RL workflow that interleaves single-agent updates, applied to mobile GUI control, achieves SOTA zero-shot performance and a +14.8 MATH500 gain.
Discussion (0). Sign in to comment.