REVIEW 5 cited by
RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The grand aim of having a single robot that can manipulate arbitrary objects in diverse settings is at odds with the paucity of robotics datasets. Acquiring and growing such datasets is strenuous due to manual efforts, operational costs, and safety challenges. A path toward such an universal agent would require a structured framework capable of wide generalization but trained within a reasonable data budget. In this paper, we develop an efficient system (RoboAgent) for training universal agents capable of multi-task manipulation skills using (a) semantic augmentations that can rapidly multiply existing datasets and (b) action representations that can extract performant policies with small yet diverse multi-modal datasets without overfitting. In addition, reliable task conditioning and an expressive policy architecture enable our agent to exhibit a diverse repertoire of skills in novel situations specified using language commands. Using merely 7500 demonstrations, we are able to train a single agent capable of 12 unique skills, and demonstrate its generalization over 38 tasks spread across common daily activities in diverse kitchen scenes. On average, RoboAgent outperforms prior methods by over 40% in unseen situations while being more sample efficient and being amenable to capability improvements and extensions through fine-tuning. Videos at https://robopen.github.io/
Forward citations
Cited by 5 Pith papers
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
EquiBim: Learning Symmetry-Equivariant Policy for Bimanual Manipulation
Adding a loss that enforces left-right equivariance between observations and actions improves average bimanual imitation policy success by +2.7 to +9.5 points across four observation/action settings.
-
RoboBERT: An End-to-end Multimodal Robotic Manipulation Model
A two-stage trained vision-language-action diffusion policy with carefully selected data augmentations reaches mean episode lengths of 4.52 (ABCD to D) and 3.79 (ABC to D) on CALVIN.
-
Imitation Learning Based on Disentangled Representation Learning of Behavioral Characteristics
A weakly-supervised CVAE with action chunking lets a robot change wiping speed online from instruction labels, but the same mechanism fails to disentangle wiping force and fails on spatial pick-and-place directives.
-
Data Pyramid for Embodied Manipulation
Embodied training data form a five-layer pyramid—real-robot, UMI, ego/exo, simulation, general V–L—ordered by the trade-off between scale and robot alignment, and model capabilities track how those layers are mixed.
Discussion (0). Continue with ORCID to comment.