REVIEW 11 cited by
MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Multimodal Large Language Models (MLLMs) excel in understanding complex language and visual data, enabling generalist robotic systems to interpret instructions and perform embodied tasks. Nevertheless, their real-world deployment is hindered by substantial computational and storage demands. Recent insights into the homogeneous patterns in the LLM layer have inspired sparsification techniques to address these challenges, such as early exit and token pruning. However, these methods often neglect the critical role of the final layers that encode the semantic information most relevant to downstream robotic tasks. Aligning with the recent breakthrough of the Shallow Brain Hypothesis (SBH) in neuroscience and the mixture of experts in model sparsification, we conceptualize each LLM layer as an expert and propose a Mixture-of-Layers Vision-Language-Action model (MoLe-VLA, or simply MoLe) architecture for dynamic LLM layer activation. We introduce a Spatial-Temporal Aware Router (STAR) for MoLe to selectively activate only parts of the layers based on the robot's current state, mimicking the brain's distinct signal pathways specialized for cognition and causal reasoning. Additionally, to compensate for the cognitive ability of LLMs lost in MoLe, we devise a Cognition Self-Knowledge Distillation (CogKD) framework. CogKD enhances the understanding of task demands and improves the generation of task-relevant action sequences by leveraging cognitive features. Extensive experiments conducted in both RLBench simulation and real-world environments demonstrate the superiority of MoLe-VLA in both efficiency and performance. Specifically, MoLe-VLA achieves an 8% improvement in the mean success rate across ten tasks while reducing computational costs by up to x5.6 compared to standard LLMs.
Forward citations
Cited by 11 Pith papers
-
ActionCache: Training-Free Acceleration for Vision-Language-Action Models with Action Caching and Refinement
A training-free cache of intermediate actions, retrieved by random-projected VLM embeddings, cuts flow-based VLA action-head latency up to 40× in tests while keeping success rates near base-model levels.
-
SAFE-Pruner: Semantic Attention-Guided Future-Aware Token Pruning for Efficient Vision-Language-Action Manipulation
SAFE-Pruner forecasts deep-layer visual-token saliency from historical attention maps and refreshes at subtask boundaries, enabling up to 1.89x faster VLA inference with minimal success-rate drop.
-
TIDAL: Temporally Interleaved Diffusion and Action Loop for High-Frequency VLA Control
TIDAL raises VLA control feedback from ~2.4 Hz to ~9 Hz by caching semantic intent and interleaving one-step flow generation with execution, doubling dynamic interception success in simulation.
-
CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification
CogVLA pairs instruction-conditioned visual-token aggregation (EFA-Routing) with transformer-layer pruning (LFP-Routing) and bidirectional action decoding (CAtten), reporting LIBERO 97.4%, real-world 70.0%, 2.5x less ...
-
ROSA: Harnessing Robot States for Vision-Language and Action Alignment
ROSA trains a VLA model jointly on expert actions and automatically recorded robot states, improving success rates and generalization, particularly with few demonstrations.
-
Towards a Generalizable Bimanual Foundation Policy via Flow-based Video Prediction
CogRobot uses optical flow as an intermediate variable to fine-tune a text-to-video model for predicting bimanual robot trajectories, then maps those predictions to actions with a goal-conditioned diffusion policy.
-
The Latent Color Subspace: Emergent Order in High-Dimensional Chaos
FLUX.1’s VAE latent space contains an interpretable Hue–Saturation–Lightness structure that enables training-free color prediction and control via closed-form latent edits.
-
Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance
A plug-and-play fine-tuning method using two VAEs and a latent-distance guidance loss improves cross-embodiment and cross-task success rates of diffusion- and flow-based VLA policies.
-
EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models
EfficientVLA combines LLM layer pruning, task-aware visual token selection, and diffusion-head feature caching to cut CogACT's inference cost to 28.9% of baseline FLOPs with a 0.6% SIMPLER success drop.
-
Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey
A survey that groups VLA efficiency techniques into four categories: model architecture, perception features, action generation, and training/inference strategies.
-
Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
A review that categorizes large-model-empowered embodied AI into hierarchical and end-to-end decision-making, imitation and reinforcement learning, and world models.
Discussion (0). Sign in to comment.