pith. sign in

arxiv: 2605.21854 · v2 · pith:SA6ONG4Wnew · submitted 2026-05-21 · 💻 cs.CV · cs.AI

CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models

Zhi Liu This is my paper

Pith reviewed 2026-06-30 17:45 UTC · model grok-4.3

classification 💻 cs.CV cs.AI
keywords vision-language-actiondirect preference optimizationflow matchingDoRALIBERO benchmarkpost-traininginference optimizationparameter-efficient fine-tuning
0
0 comments X

The pith

A surrogate flow-matching log-probability estimator lets DPO operate on continuous-action VLA backbones without ODE integration.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes a way to apply Direct Preference Optimization, long standard in language models, to continuous-action flow-matching vision-language-action models. It introduces a surrogate estimator that approximates the needed log-probabilities directly from the flow-matching process. Head-to-head tests on the LIBERO benchmark show DoRA fine-tuning lifting success rates by a mean 10.4 percentage points over supervised fine-tuning baselines, with gains reaching 20 points on object-centric tasks. The study also maps inference costs, revealing that the denoising loop consumes nearly 80 percent of sample time and that standard caching approaches yield at most 21 percent speedup while often harming reliability. A separate multi-view temporal projection head pretrained on LIBERO frames reaches 99.5 percent k-NN recall for task retrieval and is released for downstream use.

Core claim

CrossVLA shows that a surrogate flow-matching log-probability estimator enables Direct Preference Optimization on continuous-action backbones without probability-flow ODE integration, and that DoRA as the parameter-efficient adapter produces mean success-rate gains of 10.4 percentage points over OpenVLA supervised fine-tuning across the four LIBERO suites.

What carries the argument

The surrogate flow-matching log-probability estimator that supplies the log-probability term for the DPO loss directly from the flow-matching objective.

If this is right

  • DPO becomes applicable to flow-matching VLA architectures without custom ODE solvers at training time.
  • DoRA yields larger and more consistent gains than LoRA or plain SFT on LIBERO manipulation suites.
  • The denoising loop dominates inference latency in flow-matching VLAs, limiting the upside of token-level optimizations.
  • Prefix and chunk caching strategies for VLA inference improve speed by at most 21 percent and can drop task success to between 0 and 80 percent.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The released multi-view temporal projection head could be tested as a drop-in initialization for other multi-camera VLA training regimes.
  • The surrogate estimator may extend to other continuous generative policies in robotics where exact likelihoods are intractable.
  • Inference optimizations for VLAs may need to be co-designed with the generative paradigm rather than borrowed from autoregressive language models.

Load-bearing premise

The surrogate estimator approximates the true log-probability closely enough that the resulting DPO gradients remain useful for policy improvement.

What would settle it

Compare the policy success rates or effective preference rankings obtained by running DPO with the surrogate estimator against the same procedure using exact log-probabilities obtained via full ODE integration on a small continuous VLA model.

Figures

Figures reproduced from arXiv: 2605.21854 by Zhi Liu.

Figure 1
Figure 1. Figure 1: LIBERO 4-suite: SFT, LoRA single-seed, LoRA multiseed pool (where available), DoRA [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Left: π0.5 sample_actions latency breakdown. Right: cache strategy benchmark on LIBERO Spatial × 50 trials. Strategy 1: Chunk-level Cache Naive: cache the whole 10-step action chunk; on the next env step, if visual signature is similar (cosine ≥ 0.95) to the cached observation, reuse cached chunk’s i-th action. The cache mechanism worked (82% reuse), yet the run was both +30% slower and −20% in success rat… view at source ↗
Figure 3
Figure 3. Figure 3: Multi-view + temporal contrastive training on real LIBERO RLDS with frozen SigLIP [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative k-NN retrieval examples: 6 query frames [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
read the original abstract

Vision-Language-Action (VLA) models have rapidly converged on a small set of architectural patterns: discrete-token autoregression (e.g. OpenVLA) and continuous-action flow-matching (e.g. pi-0.5). Yet preference alignment via Direct Preference Optimisation (DPO) -- the de-facto post-training step in language models -- has been studied almost exclusively on autoregressive VLAs. We present CrossVLA, an empirical study of cross-paradigm VLA post-training. Three contributions: (i) a surrogate flow-matching log-probability estimator that lets DPO operate on continuous-action backbones without probability-flow ODE integration; (ii) a head-to-head comparison of LoRA and DoRA as the parameter-efficient layer for VLA DPO, finding DoRA improves over OpenVLA SFT by a mean +10.4 pp across LIBERO 4-suite (600 trials, 3 seeds) -- per-suite +20.0 Object, +11.0 Long-horizon, +8.0 Goal, +2.7 Spatial -- with zero seed variance on Object (38/50 on each of 3 seeds); (iii) an inference-time anatomy showing the denoise loop dominates 78.6% of sample_actions latency and prefix-K/V caching a la VLA-Cache caps at a 21% acceleration ceiling -- both chunk-level and token-level cache strategies degrade success rate to 0-80% in our benchmarks. We further pretrain a multi-view + temporal projection head on 6000 LIBERO frames, achieving 99.5% k-NN recall@1 for same-task retrieval (36x over random), available as a downstream initialisation. All code, ckpts, training logs, and reproduction scripts are open at https://github.com/lz-googlefycy/vla-lab.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The paper presents CrossVLA as an empirical study of cross-paradigm post-training for Vision-Language-Action models. It introduces (i) a surrogate flow-matching log-probability estimator to enable Direct Preference Optimization (DPO) on continuous-action flow-matching backbones without probability-flow ODE integration; (ii) a comparison showing DoRA yields a mean +10.4 pp success-rate improvement over OpenVLA SFT across the LIBERO 4-suite (600 trials, 3 seeds), with per-suite gains up to +20.0 pp and zero seed variance on one suite; (iii) an inference-time breakdown identifying the denoise loop as 78.6% of latency and showing that prefix K/V caching yields at most 21% acceleration while chunk- and token-level strategies degrade success rates; and a pretrained multi-view temporal projection head achieving 99.5% k-NN recall@1. All artifacts are released openly.

Significance. If the surrogate estimator is shown to preserve the relative log-probability ratios required by the DPO loss, the work would provide a practical route for preference alignment on continuous-action VLAs and demonstrate clear gains from DoRA over standard LoRA/SFT on public benchmarks. The multi-seed empirical protocol, open code, and inference analysis are strengths that would support adoption if the approximation quality is verified. The absence of such verification currently limits the reliability of the DPO-related claims.

major comments (2)
  1. [Abstract] Abstract, contribution (i): the surrogate flow-matching log-probability estimator is presented as the enabling mechanism for DPO on continuous-action models, yet the manuscript supplies no quantification of approximation error (e.g., mean absolute deviation, rank correlation, or held-out trajectory comparison) against exact ODE-integrated log-probabilities, nor an ablation replacing the surrogate with exact integration on a tractable subset. This directly affects the validity of the DPO gradients and the reported DoRA gains in contribution (ii).
  2. [Abstract / DPO experiments] Abstract / § on DPO experiments: the claim that the surrogate enables valid DPO rests on the unverified assumption that it preserves ordering and magnitude sufficiently for the preference loss; without explicit validation metrics or sensitivity analysis, the +10.4 pp mean improvement cannot be confidently attributed to the cross-paradigm method rather than benchmark-specific tuning or implementation details.
minor comments (3)
  1. [Methods] The manuscript would benefit from an explicit equation or pseudocode block defining the surrogate estimator (currently described only at high level in the abstract).
  2. [Results] Table or figure reporting per-seed success rates should be added to substantiate the "zero seed variance" claim on the Object suite.
  3. [Inference analysis] The 78.6% latency breakdown and 21% caching ceiling would be clearer with a breakdown table by component (denoise loop, caching, etc.).

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback emphasizing the importance of validating the surrogate estimator. We address each major comment below and commit to revisions that strengthen the empirical support for the DPO claims.

read point-by-point responses

  1. Referee: [Abstract] Abstract, contribution (i): the surrogate flow-matching log-probability estimator is presented as the enabling mechanism for DPO on continuous-action models, yet the manuscript supplies no quantification of approximation error (e.g., mean absolute deviation, rank correlation, or held-out trajectory comparison) against exact ODE-integrated log-probabilities, nor an ablation replacing the surrogate with exact integration on a tractable subset. This directly affects the validity of the DPO gradients and the reported DoRA gains in contribution (ii).

    Authors: We agree that the manuscript currently lacks direct quantification of the surrogate's approximation error relative to exact ODE integration. The surrogate was introduced to avoid the computational cost of ODE solving while enabling DPO, but without explicit error metrics the fidelity remains unverified. In the revised manuscript we will add a dedicated validation subsection reporting mean absolute deviation, Spearman rank correlation, and a held-out trajectory comparison on a computationally tractable subset of trajectories. We will also include an ablation that replaces the surrogate with exact integration on that subset and reports the resulting DPO performance difference. revision: yes

  2. Referee: [Abstract / DPO experiments] Abstract / § on DPO experiments: the claim that the surrogate enables valid DPO rests on the unverified assumption that it preserves ordering and magnitude sufficiently for the preference loss; without explicit validation metrics or sensitivity analysis, the +10.4 pp mean improvement cannot be confidently attributed to the cross-paradigm method rather than benchmark-specific tuning or implementation details.

    Authors: We acknowledge that the current text does not provide explicit validation that the surrogate preserves the relative log-probability ratios required by the DPO loss, which limits confident attribution of the observed gains. The reported +10.4 pp mean improvement was obtained using the surrogate-enabled DPO pipeline; however, without the requested metrics it is difficult to isolate the contribution of the cross-paradigm approach. In revision we will add both the approximation-error metrics described above and a sensitivity analysis (varying the surrogate's internal hyperparameters) to demonstrate that the preference ordering is sufficiently preserved for effective optimization. revision: yes

Circularity Check

0 steps flagged

No circularity; purely empirical contributions with open benchmarks and code

full rationale

The paper reports three empirical contributions: a surrogate log-probability estimator for DPO on flow-matching VLAs, DoRA vs LoRA comparisons on LIBERO (with reported success rates), and inference latency breakdowns. No derivation chain, equations, or predictions are presented that reduce to fitted parameters or self-citations by construction. The surrogate is introduced as an enabling technique whose accuracy is implicitly tested via downstream DPO performance on public tasks; no self-referential definitions or load-bearing self-citations appear. Results are externally falsifiable via the linked open code and LIBERO suites.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claims rest on standard machine-learning assumptions about optimization and probability estimation plus the validity of the introduced surrogate; no free parameters or invented physical entities are described.

axioms (1)
  • domain assumption The surrogate estimator accurately approximates the log-probability required by the DPO objective for flow-matching models without full ODE integration.
    Invoked in contribution (i) to enable DPO on continuous backbones.
invented entities (1)
  • surrogate flow-matching log-probability estimator no independent evidence
    purpose: Enable DPO on continuous-action VLAs without probability-flow ODE integration
    New method introduced in the paper; no independent external validation cited in abstract.

pith-pipeline@v0.9.1-grok · 5876 in / 1415 out tokens · 53550 ms · 2026-06-30T17:45:28.189157+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 13 canonical work pages · 8 internal anchors

  1. [1]

    A general theoretical paradigm to understand learning from human preferences

    Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. A general theoretical paradigm to understand learning from human preferences.arXiv preprint arXiv:2310.12036,

    work page arXiv

  2. [2]

    $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164,

    work page internal anchor Pith review Pith/arXiv arXiv

  3. [3]

    RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control.arXiv preprint arXiv:2307.15818,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Pali- 3 vision language models: Smaller, faster, stronger,

    Xi Chen, Xiao Wang, Lucas Beyer, Alexander Kolesnikov, Jialin Wu, Paul V oigtlaender, et al. Pali-3 vision language models: Smaller, faster, stronger.arXiv preprint arXiv:2310.09199,

    work page arXiv

  5. [5]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    KTO: Model Alignment as Prospect Theoretic Optimization

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization.arXiv preprint arXiv:2402.01306,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    ORPO: Monolithic Preference Optimization without Reference Model

    12 Jiwoo Hong, Noah Lee, and James Thorne. Orpo: Monolithic preference optimization without reference model.arXiv preprint arXiv:2403.07691,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    Robomamba: Efficient vision-language-action model for robotic reasoning and manipulation.arXiv preprint arXiv:2406.04339, 2024a

    Jiaming Liu et al. Robomamba: Efficient vision-language-action model for robotic reasoning and manipulation.arXiv preprint arXiv:2406.04339, 2024a. Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang- Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. InICML, 2024b. Songming Liu, Lingxuan Wu, Ban...

    work page arXiv

  9. [9]

    Simpo: Simple preference optimization with a reference-free reward, 2024

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference- free reward.arXiv preprint arXiv:2405.14734,

    work page arXiv

  10. [10]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn

    Physical Intelligence and openpi 2025.09 release. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InNeurIPS,

    2025

  11. [11]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, Y Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    Robotic Control via Embodied Chain-of-Thought Reasoning

    13 Ag2Manip team. Ag2manip: Learning novel manipulation skills with agent-agnostic visual and action representations. InIROS, 2024a. ChatVLA team. Chatvla: Unified multimodal understanding and robot control with vision-language- action model.arXiv preprint, 2025a. CLIP-DoRA team. Clip-dora: Weight-decomposed low-rank adaptation for efficient vision-langua...

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation

    Targets autoregressive VLAs; we test transfer to flow-matchingπ 0.5 (§4.5). Jian Wen, Jian Zhang, et al. Tinyvla: Toward fast, data-efficient vision-language-action models for robotic manipulation.arXiv preprint arXiv:2409.12514,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    Qa-lora: Quantization-aware low-rank adaptation of large language models.arXiv preprint arXiv:2309.14717,

    Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen, Heng Chang, Hengheng Zhang, Zhengsu Chen, Xiaopeng Zhang, and Qi Tian. Qa-lora: Quantization-aware low-rank adaptation of large language models.arXiv preprint arXiv:2309.14717,

    work page arXiv