Pith. sign in

REVIEW 11 cited by

Improving Vision-Language-Action Model with Online Reinforcement Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.16664 v1 pith:KHJIRD3M submitted 2025-01-28 cs.RO cs.CVcs.LG

Yanjiang Guo , Jianke Zhang , Xiaoyu Chen , Xiang Ji , Yen-Jen Wang , Yucheng Hu , Jianyu Chen This is my paper
classification cs.ROcs.CVcs.LG
keywords modelslargelearningimprovereinforcementsupervisedchallengesfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent studies have successfully integrated large vision-language models (VLMs) into low-level robotic control by supervised fine-tuning (SFT) with expert robotic datasets, resulting in what we term vision-language-action (VLA) models. Although the VLA models are powerful, how to improve these large models during interaction with environments remains an open question. In this paper, we explore how to further improve these VLA models via Reinforcement Learning (RL), a commonly used fine-tuning technique for large models. However, we find that directly applying online RL to large VLA models presents significant challenges, including training instability that severely impacts the performance of large models, and computing burdens that exceed the capabilities of most local machines. To address these challenges, we propose iRe-VLA framework, which iterates between Reinforcement Learning and Supervised Learning to effectively improve VLA models, leveraging the exploratory benefits of RL while maintaining the stability of supervised learning. Experiments in two simulated benchmarks and a real-world manipulation suite validate the effectiveness of our method.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

    cs.CV 2026-01 conditional novelty 7.0 of 10

    Using a simple action-token adapter, nine VLMs are compared as robot policy backbones, showing general VLM ability transfers poorly to control and the vision encoder is the key bottleneck.

  2. RL Bootstrapping of OpenVLA-OFT for a Novel Robot Embodiment

    cs.RO 2026-08 conditional novelty 6.0 of 10

    RL-only fine-tuning with dense geometric rewards aligns OpenVLA-OFT to a novel cable-driven robot, reaching 53.5% average success on four directional commands without any demonstration data.

  3. VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models

    cs.RO 2025-12 conditional novelty 6.0 of 10

    An open benchmark with 170 graded manipulation tasks shows current VLA robot policies memorize their training settings, degrade sharply under visual shifts, ignore safety constraints, and fail to compose long-horizon skills.

  4. Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models

    cs.LG 2025-10 conditional novelty 6.0 of 10

    FPO fine-tunes flow-matching vision-language-action policies with a PPO-style objective that replaces intractable policy ratios with per-sample conditional flow-matching loss differences, reaching 87.2% average succes...

  5. Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models

    cs.RO 2025-09 conditional novelty 5.0 of 10

    ARFM adaptively adjusts a scaling factor in the flow-matching loss so that offline RL advantage signals are preserved while gradient variance is controlled, improving VLA robot policy fine-tuning.

  6. Source Component Shift Adaptation via Offline Decomposition and Online Mixing Approach

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    The claimed result is that source-component-shift adaptation splits cleanly into offline component learning via EM and online mixing-weight updates, cutting cumulative test loss by up to 67.4%.

  7. SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training

    cs.RO 2025-07 conditional novelty 5.0 of 10

    Simulation-pretrained policies, with digital-twin demos for critic bootstrapping and action proposals, cut real-world RL training time while reaching near-perfect success on three manipulation tasks.

  8. Leveraging OS-Level Primitives for Robotic Action Management

    cs.OS 2025-08 conditional novelty 4.0 of 10

    Applying OS-style exception handling, context caching, and replay to robotic action slices raises success rates 7x to 24x and cuts execution steps up to 74% for repetitive manipulation tasks, without retraining the VLA model.

  9. Continual Learning for Generative AI: From LLMs to MLLMs and Beyond

    cs.LG 2025-06 conditional novelty 4.0 of 10

    A survey that categorizes continual learning methods for generative models into architecture-based, regularization-based, and replay-based paradigms across four model families.

  10. Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimization

    cs.LG 2025-09 unverdicted

    A survey categorizing deep reinforcement learning and direct preference optimization methods for aligning large vision-language models, with tables of studies and datasets and no new experimental result.

  11. RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models

    cs.RO 2025-06

Pith tools