REVIEW 16 cited by
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Building general-purpose robots that operate seamlessly in any environment, with any object, and utilizing various skills to complete diverse tasks has been a long-standing goal in Artificial Intelligence. However, as a community, we have been constraining most robotic systems by designing them for specific tasks, training them on specific datasets, and deploying them within specific environments. These systems require extensively-labeled data and task-specific models. When deployed in real-world scenarios, such systems face several generalization issues and struggle to remain robust to distribution shifts. Motivated by the impressive open-set performance and content generation capabilities of web-scale, large-capacity pre-trained models (i.e., foundation models) in research fields such as Natural Language Processing (NLP) and Computer Vision (CV), we devote this survey to exploring (i) how these existing foundation models from NLP and CV can be applied to the field of general-purpose robotics, and also exploring (ii) what a robotics-specific foundation model would look like. We begin by providing a generalized formulation of how foundation models are used in robotics, and the fundamental barriers to making generalist robots universally applicable. Next, we establish a taxonomy to discuss current work exploring ways to leverage existing foundation models for robotics and develop ones catered to robotics. Finally, we discuss key challenges and promising future directions in using foundation models for enabling general-purpose robotic systems. We encourage readers to view our living GitHub repository 2 of resources, including papers reviewed in this survey, as well as related projects and repositories for developing foundation models for robotics.
Forward citations
Cited by 16 Pith papers
-
Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning
A lifelong robot learning method that mixes a growing library of LoRA-style experts with a context router and replays router coefficients to achieve forward transfer with near-zero forgetting.
-
A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference
Motion-aware vector quantization plus centroid-reuse GEMM and a custom accelerator yield multi-fold VLA speedups over GPU and prior accelerators with ~2.5% average success-rate drop.
-
FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation
FELT predicts finger pressure maps from RGB images and uses them or their learned features to improve manipulation policies without real tactile sensors at deployment.
-
Argus: Metric Panoramic 3D Reconstruction for Indoor Scenes
Argus plus Realsee3D deliver state-of-the-art metric camera pose, depth, and point-cloud reconstruction from unordered indoor panoramas via learned covisibility anchoring and geometric factorization.
-
AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models
Structured prompting plus a discrete skill library lets a frozen VLM direct aerial manipulation, reaching 87.5% simulated and 80% hardware success in pick-and-place tasks.
-
INSIGHT: INference-time Sequence Introspection for Generating Help Triggers in Vision-Language-Action Models
Token-level uncertainty sequences from a VLA policy, classified by a small transformer, predict when a robot should request human help better than static uncertainty scores.
-
T-Rex: Task-Adaptive Spatial Representation Extraction for Robotic Manipulation with Vision-Language Models
A zero-training framework that adaptively selects spatial representation extractors per object and per task stage improves real-world robot manipulation success and efficiency over fixed-representation baselines.
-
UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation
UAD distills affordance knowledge from vision-language models and DINOv2 features into a lightweight task-conditioned model that predicts pixel-level manipulation regions and improves few-shot imitation learning gener...
-
Beyond Flat Policies: Hierarchical Post-Training for Embodied Agents in Robotic Manipulation
HiRoC uses a language planner to split manipulation tasks into subgoals and a reinforcement-learned executor to follow them, reporting 93.5% average success on LIBERO.
-
Enhancing Reliability in LLM-Integrated Robotic Systems: A Unified Approach to Security and Safety
A unified framework of secure prompting, state memory, and rule-based safety validation improves LLM-driven robot navigation under prompt injection attacks and obstacle-heavy environments, with modest real-robot verification.
-
EndoVLA: Dual-Phase Vision-Language-Action Model for Autonomous Tracking in Endoscopy
A vision-language-action model trained with supervised and reinforcement learning tracks endoscopic targets and simple objects on a robotic endoscope.
-
Position: Intelligent Science Laboratory Requires the Integration of Cognitive and Embodied AI
The paper argues that integrating cognitive AI and embodied robots into closed-loop Intelligent Science Laboratories is essential for the next leap in automated scientific discovery.
-
Towards Tangible Immersion for Cobot Programming-by-Demonstration: Visual, Tactile and Haptic Interfaces for Mixed-Reality Cobot Automation in Semiconductor Manufacturing
A mixed-reality and haptics-augmented programming-by-demonstration pipeline is proposed to let non-experts program collaborative robots for semiconductor handling through demonstrated primitives, but without evaluation data.
-
Reducing Latency in LLM-Based Natural Language Commands Processing for Robot Navigation
A direct ChatGPT-to-ROS2 voice command pipeline for robot navigation is reported to cut latency by 7.01%, but the paper's own data do not reproduce that number.
-
Foundation Model Driven Robotics: A Comprehensive Review
A review of foundation-model-driven robotics that synthesizes recent work across perception, planning, control, HRI, simulation, and sim-to-real transfer, and highlights open challenges.
- CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity
Discussion (0). Continue with ORCID to comment.