REVIEW 10 cited by
Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
An elusive goal in navigation research is to build an intelligent agent that can understand multimodal instructions including natural language and image, and perform useful navigation. To achieve this, we study a widely useful category of navigation tasks we call Multimodal Instruction Navigation with demonstration Tours (MINT), in which the environment prior is provided through a previously recorded demonstration video. Recent advances in Vision Language Models (VLMs) have shown a promising path in achieving this goal as it demonstrates capabilities in perceiving and reasoning about multimodal inputs. However, VLMs are typically trained to predict textual output and it is an open research question about how to best utilize them in navigation. To solve MINT, we present Mobility VLA, a hierarchical Vision-Language-Action (VLA) navigation policy that combines the environment understanding and common sense reasoning power of long-context VLMs and a robust low-level navigation policy based on topological graphs. The high-level policy consists of a long-context VLM that takes the demonstration tour video and the multimodal user instruction as input to find the goal frame in the tour video. Next, a low-level policy uses the goal frame and an offline constructed topological graph to generate robot actions at every timestep. We evaluated Mobility VLA in a 836m^2 real world environment and show that Mobility VLA has a high end-to-end success rates on previously unsolved multimodal instructions such as "Where should I return this?" while holding a plastic bin. A video demonstrating Mobility VLA can be found here: https://youtu.be/-Tof__Q8_5s
Forward citations
Cited by 10 Pith papers
-
ABot-N1: Toward a General Visual Language Navigation Foundation Model
A slow–fast VLN model that routes five navigation tasks through CoT plus image-space pixel goals reaches SOTA on established and new urban benchmarks, including 77.3% POI arrival.
-
PLanAR: Planning-Language-Grounded Agentic Reasoning for Robot Manipulation
AgenticLab's closed-loop planning-language pipeline lets different vision-language models drive a real robot, and benchmark tests show action-verification quality, not planning, determines long-horizon success.
-
TANGO: Traversability-Aware Navigation with Local Metric Control for Topological Goals
A navigation pipeline that bridges object-level global planning with traversability-aware local control, using only RGB images and pretrained models, improves success over prior zero-shot and learned baselines in simulation.
-
CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models
Counterfactual language-action relabeling raises instruction-following success from about 26% to 53% in real-world navigation tests.
-
Adversarial Attacks on Robotic Vision Language Action Models
Text-based adversarial suffixes can make OpenVLA robot policies elicit chosen target actions with over 90% success on one-hot targets and persist across rollout steps.
-
GraphPad: Inference-Time 3D Scene Graph Updates for Embodied Question Answering
Allowing a vision-language model to edit its own 3D scene graph during inference improves embodied question answering from 52.3% to 55.3% on OpenEQA.
-
Green for Go, Red for No: Visual Grounding via Semantic Segmentation for VLA Navigation Policies
Real-time SegFormer green/red overlays reduce OmniVLA far-waypoint error 27-44% on Grand Tour language goals mainly by shortening trajectories, with little help for image goals.
-
PixelNav: Towards Model-based Vision-Only Navigation with Topological Graphs
A camera-only navigation system combining visual place recognition, traversability segmentation, and model predictive control over a topological graph, evaluated on a real robot.
-
ReFineVLA: Reasoning-Aware Teacher-Guided Transfer Fine-Tuning
Fine-tuning a vision-language-action robot model on teacher-generated reasoning rationales raises average simulated manipulation success by up to 8.6 percentage points over the SpatialVLA baseline.
-
Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
A review that categorizes large-model-empowered embodied AI into hierarchical and end-to-end decision-making, imitation and reinforcement learning, and world models.
Discussion (0). Continue with ORCID to comment.