Pith. sign in

REVIEW 3 major objections 7 minor 7 cited by

Contrastive Representation Regularization for Vision-Language-Action Models

T0 review · 3 major / 7 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding a robot-state-aware contrastive loss to VLA training realigns VLM embeddings with proprioceptive state and improves manipulation, especially precise pick-and-place.

desk verdict A simple, plausible VLA regularization with consistent gains; the main soft spot is a scale-dependent weighting ablation the authors don't test. read the letter →

arxiv 2510.01711 v4 pith:766G5U5H submitted 2025-10-02 cs.RO cs.LG

Taeyoung Kim , Jimin Lee , Myungkyu Koo , Dongyoung Kim , Kyungmin Lee , Changyeon Kim , Younggyo Seo , Jinwoo Shin This is my paper
classification cs.ROcs.LG
keywords vision-language-actionmodelscontrastivelearningrepresentationregularizationproprioceptionrobotmanipulationInfoNCEflowmatchingmulti-viewaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language-action (VLA) models inherit rich visual and semantic representations from pretrained vision-language models, but those representations do not know about the robot's own body. This paper tries to close that gap with a lightweight auxiliary loss, Robot State-aware Contrastive Loss (RS-CL), which pulls VLA embeddings toward embeddings from episodes with similar proprioceptive states, using Euclidean distance between robot states as soft labels. On top of a strong baseline, RS-CL lifts success from 65.7% to 69.7% on a 24-task simulated kitchen benchmark and from 45.0% to 58.3% on real-robot pick-and-place and lid-closing tasks. The largest gains appear in pick-and-place, which the authors attribute to more precise positioning during grasping and placing. If the claim holds, proprioceptive supervision is a cheap and general way to make pretrained VLM features genuinely useful for control.

What carries the argument

The load-bearing object is the weighted InfoNCE loss in Eqs. 3-4 of the paper. Each training sample is summarized by a learned token, projected to an embedding, and the loss compares every sample to an augmented version of every other sample. The weight w_ij = exp(-||q_i - q_j||_2 / beta) is a soft label built from the Euclidean distance between proprioceptive state vectors; it tells the loss which pairs are control-relevant without needing discrete class labels. The companion mechanism is view cutoff, a representation-level augmentation that zeroes out the feature slice of a randomly chosen camera view, so contrastive pairs differ by viewpoint rather than by task meaning. Together they shap

What would settle it

Train the exact RS-CL setup but replace the state distance in Eq. 4 with random noise or permuted state vectors, keeping all other hyperparameters and the view-cutoff augmentation. If success rates on the simulated kitchen benchmark stay near the reported RS-CL level, the improvement is not caused by state-aligned supervision; if the gains disappear, the causal role of the state distance is confirmed.

Watch

Extended reading notes

Core claim

The authors' central claim is that the conditioning representations inside a VLA model can be made control-relevant by regularizing them with a contrastive loss whose pair weights come from the robot's proprioceptive state. Concretely, RS-CL appends a learnable summarization token to the VLM output, projects the summarized embedding, and optimizes a weighted InfoNCE loss: pairs of samples with close robot states are assigned larger weights and pulled together, while distant states are pushed apart. This runs alongside the standard flow-matching action-prediction loss, with a view-cutoff augmentation that masks one camera view's embedding slice to create cheap contrastive pairs. The authors r

Load-bearing premise

The load-bearing premise is that Euclidean distance between proprioceptive state vectors is a faithful proxy for control-relevant similarity: samples whose robot states are close are treated as near-duplicates regardless of visual scene or instruction, and if that fails, RS-CL pushes unrelated embeddings together.

Editorial extensions

If this is right

  • RS-CL can be dropped into an existing VLA training loop with only an added contrastive path, so the reported gains come without a second training stage or curated robotics reasoning datasets.
  • Because the loss supervises the representation rather than the action output, it transfers across backbone vision-language models; the from-scratch experiments show gains on several different backbones.
  • Performance gains concentrate in pick-and-place, so robot tasks whose bottleneck is precise positioning benefit most from representation alignment to state.
  • Improvements appear at every dataset size tested, from 30 to 300 demonstrations, making the method relevant to low-data robot learning.
  • The view-cutoff augmentation gives a cheap way to build contrastive pairs from multi-view observations, which may carry over to any multi-camera robot setup.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own limitations section notes that object poses and contact forces are excluded. A direct extension would be to feed those signals into the same soft-label weighting, turning RS-CL into a general alignment objective for any control-relevant state.
  • The Euclidean-distance assumption is the soft spot: if two states are numerically close but require very different actions because the scene or instruction differs, RS-CL will pull their embeddings together erroneously. A testable fix is to weight distances by vision or instruction similarity, or to learn the state metric.
  • Because view cutoff simulates viewpoint variation, RS-CL may also improve robustness to missing or occluded cameras; the real-robot close-lid result, where the wrist view becomes occluded mid-task, is consistent with that, though the paper does not isolate the augmentation's contribution to occlusion robustness.
  • The cosine schedule anneals the contrastive weight to zero, so representation refinement matters most early in training. A curriculum that later re-enables state alignment could yield further gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes RS-CL, an auxiliary contrastive regularization loss for Vision-Language-Action (VLA) models. The method appends a learnable summarization token to the VLM output, produces embedding pairs via a representation-level 'view cutoff' augmentation, and applies a weighted InfoNCE loss in which the pair weight is a softmax over negative Euclidean distances between the samples' proprioceptive states. The total objective adds this loss, with a cosine-decayed weight, to the standard flow-matching action prediction loss. Experiments on RoboCasa-Kitchen (30/100/300 demos), LIBERO, real-robot pick-and-place and close-lid tasks, and from-scratch VLA training across several VLM backbones report consistent improvements over GR00T N1.5 and other baselines, with ablations of the soft-label target and augmentation type.

Significance. If the reported results hold, RS-CL is a simple, lightweight, and broadly applicable plug-in for VLA fine-tuning, with evidence spanning multiple simulation benchmarks, real-robot tasks, and different VLM backbones. The paper's strengths include detailed training/evaluation settings, use of public datasets and checkpoints, task-wise result tables, and real-robot generalization experiments. However, the central ablation isolating the state-aware weighting is vulnerable to a distance-scale/hyperparameter confound, and the lack of repeated-seed statistics limits confidence in the magnitudes of the gains. These issues are addressable and do not undermine the overall empirical direction, but they need correction before the specific claim about proprioceptive-state supervision can be accepted.

major comments (3)
  1. [Eq. (4), Table 3a, App. B.3] The load-bearing component of RS-CL is the soft weight w_ij, defined via Euclidean distance on raw proprioceptive vectors that mix position, 6D rotation, and gripper state. Table 3a compares 'current state distance' against next-action distances while fixing β=1.0 and τ=0.2 for every target. If the characteristic distance scales of the different targets differ, a fixed β makes the weights either nearly uniform or nearly one-hot, so the observed ordering (state > next action ≈ no soft label) may reflect which target's scale happens to match β=1.0 rather than the semantic content. To support the claim that proprioceptive state is the right supervision signal, please normalize the features, tune or sweep β per target, and report the effective weight distribution (e.g., entropy of w_ij) to demonstrate the comparison is scale-fair.
  2. [Tables 1–3 and Fig. 5] All reported results are single training runs. Real-robot evaluations use 24 trials per task (App. C.3), so differences such as 50.0 vs 58.3 in Fig. 5a are within sampling noise. Simulation results use 1200 trials but a single seed. Please provide mean ± std over at least 3 seeds for the main tables (Tables 1, 2, 3) and the real-robot results; several ablation gaps are only 1–2 points, so without variance estimates it is difficult to judge whether the central improvements are robust.
  3. [Table 3b vs Table 3a] In Table 3b, the 'No augmentation' row (65.3) is below the no-regularization baseline (65.7, Table 3a), yet the text only says that similar representation-level cutoff operations give 'limited improvements' and does not discuss this regression. This suggests the view-cutoff augmentation is essential for RS-CL to help, rather than merely beneficial, and it complicates attributing the end-to-end gain to the state-aware weighting versus the augmentation. This should be discussed explicitly.
minor comments (7)
  1. [Abstract] Grammar: 'VLA models have shown its capabilities' should be 'their capabilities.'
  2. [Section 2.2] Typo: 'an representation-level augmentation' should be 'a representation-level augmentation.'
  3. [Algorithm 1] Line 2 is redundant because line 3 overwrites h. Please clarify whether the action decoder's conditioning input is the output from the sequence with or without the appended summarization token, and align the text with Eq. (2).
  4. [Table 4 (App. A.3)] The row for layer 24 appears to duplicate the numbers of the layer-18 row while changing only the average. Please verify the entries.
  5. [Section 3 baselines] The text cites 'RoboBrain (Team et al., 2025)' but Fig. 7 and Table 10 refer to 'RoboBrain2.0.' Keep the naming consistent.
  6. [App. B.3] The DTW distance for 'next action sequence' is said to use an additional temperature γ=10.0, but it is not stated how the DTW distance is normalized before exponentiation with β. This is relevant to the scale-fairness concern above.
  7. [Reproducibility statement] The reproducibility statement mentions implementation details but does not state whether code will be released. A clear code-release plan would strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

Central claim is an empirical improvement over external baselines; no load-bearing step reduces to its own inputs by construction.

full rationale

RS-CL is an auxiliary loss defined in Eq. 3-4 and combined with the flow-matching action objective in Eq. 5. The paper's headline claims are success-rate gains on RoboCasa-Kitchen, LIBERO, and real-robot tasks, compared with external baselines (GR00T N1.5, π0, π0-FAST); these are held-out task outcomes, not quantities that appear in the loss. The soft weights in Eq. 4 are part of the proposed method, not a derived prediction of the benchmarks. The Table 3a ablation compares different supervision targets by downstream task success; the fixed β=1.0 across targets is a scale/calibration concern (a correctness risk, not circularity), but the comparison is an empirical one rather than a construction. The CKNNA result in Fig. 8 confirms that optimizing L_RS-CL increases embedding-state alignment; this is a sanity check of the training objective, not evidence for the performance claim. The only self-citation (Seo et al., 2023, in Related Work) is peripheral and not load-bearing: the view-cutoff augmentation is validated by the paper's own ablation in Table 3b. No uniqueness theorem or prior-work ansatz is invoked to force the design, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method's core assumption is that proprioceptive-state distances can supervise representation geometry; this is introduced ad hoc but tested indirectly via ablations. No new physical or conceptual entities are posited; the summarization token and view cutoff are learned/operational components, not independently checkable entities.

free parameters (5)
  • λ (RS-CL weight) = 1.0, cosine-decayed to 0
    Set by hand to emphasize representation refinement early in training; central to balancing L_FM and L_RS-CL.
  • similarity temperature τ = 0.2
    Controls sharpness of InfoNCE similarity in Eq. 3; chosen without reported sensitivity analysis.
  • soft-weight temperature β = 1.0
    Controls mapping from state distance to weight in Eq. 4; chosen by hand.
  • projection head dims = 2048 hidden, 128 output
    Architecture choice for gψ; not central but affects the contrastive signal.
  • hidden layer for Qwen2.5-VL-7B = layer 18
    Selected via LIBERO ablation (Table 4); affects all from-scratch 7B results.
assumptions (4)
  • domain assumption Pre-trained VLM representations can be reshaped by a lightweight adapter without unfreezing the backbone (except in the SigLIP2 unfrozen variant)
    The method's parameter efficiency rests on a frozen-backbone assumption, adopted following GEAR (2025).
  • ad hoc to paper Euclidean distance in proprioceptive state space is a meaningful soft-label similarity for action-relevant representation learning
    Eq. 4 defines wij from exp(-||qi-qj||/β); no independent validation that state distance corresponds to task similarity.
  • standard math InfoNCE with soft weights and a cosine-similarity head shapes representation space as intended
    Standard contrastive framework from Oord et al. 2018 and Khosla et al. 2020; not derived in the paper.
  • ad hoc to paper View cutoff preserves state-relevant semantics after masking one view's embedding slice
    The augmentation assumes the remaining views suffice; ablation shows an improvement, but the mechanism is not independently established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Contrastive Representation Regularization for Vision-Language-Action Models." pith.science (2026). https://pith.science/paper/766G5U5H

@misc{pith2026251001711,
  author       = {Pith},
  title        = {Pith review of: Contrastive Representation Regularization for Vision-Language-Action Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/766G5U5H}},
  note         = {Machine review of arXiv:2510.01711}
}
read the original abstract

Vision-Language-Action (VLA) models have shown strong capabilities in robot manipulation by leveraging rich representations from pre-trained Vision-Language Models (VLMs). However, their representations arguably remain suboptimal, lacking sensitivity to robotic signals such as control actions and proprioceptive information. To address the issue, we introduce Robot State-aware Contrastive Loss (RS-CL), a simple and effective representation regularization for VLA models, designed to bridge the gap between VLM representations and robotic signals. In particular, RS-CL aligns the representations more closely with the robot's proprioceptive states by using relative distances between the states as soft supervision. Complementing the original action prediction objective, RS-CL enhances control-relevant representation learning, while being lightweight and fully compatible with standard VLA training pipelines. Our empirical results demonstrate that RS-CL substantially improves the performance of state-of-the-art VLA models; it pushes the prior art to 69.7% achieving the state-of-the-art performance on the RoboCasa-Kitchen benchmark, and boosts success rates from 45.0% to 58.3% on challenging real-robot manipulation tasks.

Figures

Figures reproduced from arXiv: 2510.01711 by the authors.

Figure 1
Figure 1. Overview. We extend the standard VLA training framework with a contrastive path. Embeddings from the pre-trained VLM are augmented by the view cutoff operation applied on a randomly selected observation view, and optimized with our Robot State-aware Contrastive Loss to attract samples with similar proprioceptive states, complementing the action prediction loss. In contrast, we aim to directly refine VLM representati… view at source ↗
Figure 2
Figure 2. Training VLM representations for action prediction. (a) We visualize VLM embeddings of robot episodes performing the same task “Open the microwave / cabinet door” across different scenes in RoboCasa-Kitchen. (b) Pre-trained VLM representations are dominated by the visual appearance (e.g., distractor objects). (c) RS-CL guides embeddings to align with the robot’s proprio￾ceptive states, yielding representations that … view at source ↗
Figure 3
Figure 3. Representation-level aug￾mentation for contrastive pairs. View cutoff is an simple augmentation that randomly masks out the embedding slice of one observation view from the VLM representation. This misalignment motivates our central hypothesis: ex￾plicitly aligning VLM representations with their physical state will improve action prediction. Based on this hypoth￾esis, we introduce Robot State-aware Contrastive Loss … view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Example of tasks used in our experiments. We study RS-CL on multitask simulation benchmarks of (a) RoboCasa-Kitchen (Nasiriany et al., 2024) and (b) LIBERO (Liu et al., 2023a). In addition, we consider (c) real-robot manipulation tasks considering pick-and-place, and a…
Figure 5
Figure 5. Figure 5: Real-robot task success rate (%). Results on (a) in-domain tasks (4 pick-and-place and 1 close-lid task), and (b) generalization tasks (visual, physical generalization, and language grounding). For the in-domain close-lid and language grounding tasks, we report both pa…
Figure 6
Figure 6. Figure 6: Qualitative results on real-robot manipulation task. Under partial-view occlusion at the wrist view, the baseline model (left) fails to align the lid with the pot, resulting in inaccurate placement. In contrast, RS-CL (right) achieves precise alignment and successful c…
Figure 7
Figure 7. Figure 7: From-scratch experiments. Success rates (%) on RoboCasa-Kitchen for VLA models trained from various VLM backbones. Vanilla indicates Qwen2.5-VL. Results show the effects of RS-CL on top of backbones further trained with robotics data, based on (a) Qwen2.5-VL-3B, (b) 7B…
Figure 8
Figure 8. Figure 8: Alignment to proprioceptive states. We measure the alignment of condition representations inside trained VLA models, to the robot’s propriocep￾tive states using CKNNA (Huh et al., 2024). RS-CL successfully improves the representation alignment to robot states of VLA mo…
Figure 9
Figure 9. Figure 9: Real-world in-domain tasks. Box to Bowl : “Pick up the green cube on the brown box and place it in the golden bowl." Basket to Bowl : “Pick up the blue sponge in the white basket and place it in the black bowl.” Box to Plate : (+ Tabletop Background changed) “Pick up t…
Figure 10
Figure 10. Figure 10: Real-world visual generalization tasks. Box to Bowl : “Pick up the yellow banana on the brown box and place it in the gold bowl.” Box to Bowl : “Pick up purple grapes on the brown box and place it in the gold bowl.” Box to Bowl : “Pick up the red strawberry on the bro…
Figure 11
Figure 11. Figure 11: Real-world physical generalization tasks. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Real-world language grounding tasks. D FURTHER ANALYSIS D.1 CONTRASTIVE REPRESENTATION REGULARIZATION [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Contrastive Action-Image Pre-training for Visuomotor Control

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    CAIP learns action-aligned visual representations via contrastive pre-training on human hand keypoints from egocentric video, outperforming DINOv2, SigLIP, MVP, and R3M with >30% gains on real dexterous manipulation tasks.

  2. FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuning

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    FiberTune is a new fine-tuning objective that preserves action-fiber visual residuals in VLA policies, yielding performance gains on simulation and physical robot tasks.

  3. Mitigating State Aliasing in Vision-Language-Action Models via Inverse Dynamics Learning

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Inverse dynamics prediction is added as an auxiliary task to reduce state aliasing in VLA models by directly supervising the vision encoder on action-relevant visual distinctions using only standard observation-action pairs.

  4. Semantic Anchoring for Robotic Action Representations

    cs.RO 2026-07 conditional novelty 5.0 of 10

    Anchoring a VLA's mid-layer action features to a frozen semantic encoder during fine-tuning, with shared/private decomposition, improves in-distribution and out-of-distribution robot manipulation success without chang...

  5. GeoProp: Grounding Robot State in Vision for Generalist Manipulation

    cs.RO 2026-07 conditional novelty 5.0 of 10

    Projecting robot end-effector state onto image feature maps and sampling co-located visual tokens improves manipulation policy success by 4-10% across 67 tasks.

  6. QuoVLA: Quotient Space for Vision-Language-Action Models

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    QuoVLA introduces a quotient-space framework that compresses VLM latents into action-sufficient representations via quantization and dual-branch design for better VLA generalization.

  7. Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Models

    cs.CV 2026-05 unverdicted novelty 3.0 of 10

    The paper quantifies the geometric gap in current VLAs via linear probing and compares three architectures for injecting geometry from GFMs while analyzing impacts of data, cameras, and reconstruction quality.

Reference graph

Works this paper leans on

27 extracted references · 17 linked inside Pith · cited by 7 Pith papers

  1. [1]

    Cosmos-Reason1: From physical common sense to embodied reasoning.arXiv preprint arXiv:2503.15558,

    Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, et al. Cosmos-Reason1: From physical common sense to embodied reasoning.arXiv preprint arXiv:2503.15558,

    arXiv

  2. [3]

    GR00T-N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734,

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GR00T-N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734,

    arXiv

  3. [4]

    π0.5: A vision-language-action model with open-world generalization

    Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π0.5: A vision-language-action model with open-world generalization. InConference on Robot Learning, 2025a. Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karo...

    arXiv

  4. [7]

    Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645,

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645,

    arXiv

  5. [8]

    Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650,

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650,

    arXiv

  6. [9]

    Libero: Benchmarking knowledge transfer for lifelong robot learning

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. InAdvances in Neural Information Processing Systems, 2023a. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. InAdvances in Neural Information Processing Systems, 2023b. ...

    arXiv

  7. [11]

    Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748,

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748,

    arXiv

  8. [12]

    Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747,

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747,

    arXiv

Show all 27 references
  1. [16]

    Robobrain 2.0 technical report.arXiv preprint arXiv:2507.02029,

    BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Minglan Lin, Zhiyu Li, Zhou Cao, Pengwei Wang, Enshen Zhou, Yi Han, et al. Robobrain 2.0 technical report.arXiv preprint arXiv:2507.02029,

    arXiv

  2. [17]

    Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features.arXiv preprint arXiv:2502.14786,

    Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdul- mohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, an...

    arXiv

  3. [18]

    Instructvla: Vision-language-action instruction tuning from understanding to manipulation.arXiv preprint arXiv:2507.17520,

    Shuai Yang, Hao Li, Yilun Chen, Bin Wang, Yang Tian, Tai Wang, Hanqing Wang, Feng Zhao, Yiyi Liao, and Jiangmiao Pang. Instructvla: Vision-language-action instruction tuning from understanding to manipulation.arXiv preprint arXiv:2507.17520,

  4. [19]

    Chatvla: Unified multimodal understanding and robot control with vision-language-action model.arXiv preprint arXiv:2502.14420,

    Zhongyi Zhou, Yichen Zhu, Minjie Zhu, Junjie Wen, Ning Liu, Zhiyuan Xu, Weibin Meng, Ran Cheng, Yaxin Peng, Chaomin Shen, et al. Chatvla: Unified multimodal understanding and robot control with vision-language-action model.arXiv preprint arXiv:2502.14420,

    arXiv

  5. [20]

    Under review

    13 Preprint. Under review. A HYPERPARAMETERS ANDIMPLEMENTATIONDETAILS A.1 HYPERPARAMETERS For the weighting coefficient for LRS-CL, λ, we initialize to 1.0 and decayed to 0 using a cosine schedule by maximum training steps, such that representation refinement is emphasized in ...

    2025

  6. [22]

    We follow the original training and inference recipe of GEAR (2025), including the prior distribution p(s) =Beta( a−s a ; 1.5,1), a= 0.999for sampling the flow-matching timestep s in equation

    2025

  7. [23]

    We omit the use of future tokens (Zheng et al., 2025), as they are beyond the scope of this work

    All models are trained with thenew_embodimenttag. We omit the use of future tokens (Zheng et al., 2025), as they are beyond the scope of this work. For RoboCasa-Kitchen, we train for 60K gradient steps with a global batch size of 64, using AdamW with a learning rate of 1e-4 un...

    2025

  8. [25]

    We randomly sample 10 trajectories per task in RoboCasa-Kitchen, totaling 240 trajectories

    is a nearest-neighbor variant of kernel align- ment (Kornblith et al., 2019). We randomly sample 10 trajectories per task in RoboCasa-Kitchen, totaling 240 trajectories. Each trajectory is processed with a window size of 16, yielding 4415 transitions. We extract the embeddings...

    2019

  9. [26]

    This result indicates the effectiveness of our proposed training framework, together with the augmen- tation strategyview cutoff

    98.299.497.2 87.8 95.7 + CL (Ours)97.4 99.0 97.2 87.4 95.3 + RS-CL (Ours) 98.498.698.2 90.4 96.4 On RoboCasa-Kitchen, a contrastive representation regularization, without other supervision from low-level robotic signals (i.e., InfoNCE) improves the performance of GR00T N1.5 (C...

    2025

  10. [27]

    trained with, and without RS-CL, by different number of demonstrations. Task GR00T N1.5 (LFM) GR00T N1.5 (L FM+λLRS-CL) 30 demos 100 demos 300 demos 30 demos 100 demos 300 demos RoboCasa Kitchen (24 tasks, PnP = Pick-and-Place) Close Double Door 44.0 86.0 80.0 54.0 78.0 86.0 C...

    2025

  11. [64]

    At inference, we use an action horizonH= 16and execute all actions without re-planning

    We set the learning rate to 2.5e-5 with cosine decay to 2.5e-6 and 1K warmup steps. At inference, we use an action horizonH= 16and execute all actions without re-planning. For RoboCasa-Kitchen, we evaluate all models with 1200 trials. For LIBERO, we evaluate 50 trials for each...

    2024

  12. [2018]

    A simple but tough- to-beat data augmentation approach for natural language understanding and generation.arXiv preprint arXiv:2009.13818,

    Dinghan Shen, Mingzhi Zheng, Yelong Shen, Yanru Qu, and Weizhu Chen. A simple but tough- to-beat data augmentation approach for natural language understanding and generation.arXiv preprint arXiv:2009.13818,

    2009 arXiv

  13. [2020]

    Smolvla: A vision- language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844,

    Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zoui- tine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, et al. Smolvla: A vision- language-action model for affordable and efficient robotics.arXiv preprint arXiv:2...

    arXiv

  14. [2021]

    Contrastive language, action, and state pre-training for robot learning.arXiv preprint arXiv:2304.10782,

    Krishan Rana, Andrew Melnik, and Niko Sünderhauf. Contrastive language, action, and state pre-training for robot learning.arXiv preprint arXiv:2304.10782,

    arXiv

  15. [2022]

    Visual embodied brain: Let multimodal large language models see, think, and control in spaces.arXiv preprint arXiv:2506.00123,

    Gen Luo, Ganlin Yang, Ziyang Gong, Guanzhou Chen, Haonan Duan, Erfei Cui, Ronglei Tong, Zhi Hou, Tianyi Zhang, Zhe Chen, et al. Visual embodied brain: Let multimodal large language models see, think, and control in spaces.arXiv preprint arXiv:2506.00123,

    arXiv

  16. [2023]

    Knowledge insulating vision-language- action models: Train fast, run fast, generalize better.arXiv preprint arXiv:2505.23705,

    Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, et al. Knowledge insulating vision-language- action models: Train fast, run fast, generalize better.arXiv preprint arXiv:2505.23705,

    arXiv

  17. [2024]

    Under review

    10 Preprint. Under review. Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U Tan, Navonil Majumder, Soujanya Poria, et al. Nora: A small open-sourced generalist vision language action model for embodied tasks.arXiv preprint arXiv:2504.19854,

    arXiv

  18. [2025]

    Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923,

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923,

    arXiv

  19. [2048]

    Layer Spatial Object Goal Long Avg

    Table 4:Hidden representation layer ablations on Qwen2.5-VL-7B backbone.We report success rates (%) on the LIBERO benchmark, varying the hidden layer index used as the conditioning representation for VLA models trained from scratch. Layer Spatial Object Goal Long Avg. 12 (with...

    2023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.