REVIEW 7 cited by
CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Multimodal large language models (MLLMs) have shown remarkable potential as human-like autonomous language agents to interact with real-world environments, especially for graphical user interface (GUI) automation. However, those GUI agents require comprehensive cognition ability including exhaustive perception and reliable action response. We propose a Comprehensive Cognitive LLM Agent, CoCo-Agent, with two novel approaches, comprehensive environment perception (CEP) and conditional action prediction (CAP), to systematically improve the GUI automation performance. First, CEP facilitates the GUI perception through different aspects and granularity, including screenshots and complementary detailed layouts for the visual channel and historical actions for the textual channel. Second, CAP decomposes the action prediction into sub-problems: action type prediction and action target conditioned on the action type. With our technical design, our agent achieves new state-of-the-art performance on AITW and META-GUI benchmarks, showing promising abilities in realistic scenarios. Code is available at https://github.com/xbmxb/CoCo-Agent.
Forward citations
Cited by 7 Pith papers
-
MobiBench: Multi-Branch, Modular Benchmark for Mobile GUI Agents
MobiBench reaches near-human offline evaluation fidelity for mobile GUI agents by accepting any valid action at each step, and enables modular attribution of performance to agent components.
-
SlowBA: An efficiency backdoor attack towards VLM-based GUI agents
SlowBA uses two-stage reward-level injection and pop-up triggers to make VLM GUI agents produce much longer, slower responses under attack while largely preserving task accuracy.
-
SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agents
A history-aware guard model with an LLM judge is reported to cut jailbreak success on mobile agent tasks from 86.1% to 8.4% while keeping task completion unchanged at 77.8%.
-
RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
Multimodal computer-use agents show risky intent in about 85% of tasks and complete risky actions in about 60%, measured on RiOSWorld, a new 492-task virtual-machine benchmark.
-
Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent
A hand-authored EFSM planning module boosts Qwen2.5-VL-72B on AndroidWorld from 35.0% to 63.8% task success.
-
VLM-3D:End-to-End Vision-Language Models for Open-World 3D Perception
The paper promises VLM-3D but the body text is entirely the MVISU-Bench mobile-agent benchmark paper, so the stated result is unsupported.
-
AppVLM: A Lightweight Vision Language Model for Online App Control
A 3B VLM fine-tuned with SFT on AndroidControl and ReST-style iterations on AndroidWorld achieves competitive AndroidWorld success rates with GPT-4o while running about ten times faster.
Discussion (0). Continue with ORCID to comment.