Pith. sign in

REVIEW 5 major objections 9 minor 46 references

In-Context VLA: Endowing Vision-Language-Action Models with Language via In-Context Post-Training and Agentic Tool Use

T0 review · 5 major / 9 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vision-language-action robots should consume grounded language rather than generate it: injecting tool-measured spatial facts as context, with supervision only on actions, beats chain-of-thought on success, speed, and data efficiency.

desk verdict Strong recipe and honest experiments, but the 'consume over generate' claim is confounded by test-time information asymmetry. read the letter →

arxiv 2608.05738 v1 pith:QZ75OLBB submitted 2026-08-06 cs.RO

Jiarui Yang , Wen Huang , Jiale Zhang , Maowei Hu , Hang Guo This is my paper · ORCID
classification cs.RO
keywords vision-language-actionmodelsin-contextpost-trainingagentictoolusechain-of-thoughtgroundedlanguagerobotmanipulationbehaviorcloning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language-action (VLA) models for robot manipulation are trained by behavior cloning: imitate expert actions from an image and a fixed instruction. The field's natural fix is chain-of-thought (CoT), letting the model write a textual rationale before acting, but this paper argues CoT is the wrong tool for low-level control, for three reasons: the rationale is generated from the same image the action head already sees (so it adds no information and can mislead), language tokens dominate the gradient so the policy learns to narrate rather than act, and autoregressively sampling hundreds of tokens per decision breaks closed-loop timing. The paper's alternative, VLA-Talker, gives the model the ability to consume grounded language: an external tool loop (open-vocabulary detector, monocular depth, and a vision-language fallback) measures where the gripper and target objects are, renders that evidence into diverse paraphrased sentences, and injects it into the context while the loss touches only action tokens. Under matched evidence, this configuration tops generative CoT on LIBERO (97.4% vs 81.5%), runs about 4.6 times faster per decision, needs fewer demonstrations, and transfers to eight real-world tabletop tasks. If the claim is right, the design principle for VLA language competence shifts from producing rationales to reading externally grounded evidence.

What carries the argument

The mechanism that carries the argument is in-context post-training with an action-only supervision mask. The training sequence is `[instruction, image, <spatial>evidence</spatial>, action chunk]`, and the loss is computed only on action tokens (plus a single separator), so the model never learns to write the evidence and only learns to attend to it, eliminating the objective interference and generation latency of CoT in one design choice. The evidence itself comes from an agentic tool loop that answers a single question, where are the gripper and the task-relevant objects in image space and which is farther, emitting a structured tuple of gripper pixel and depth (from analytic projection via known camera intrinsics and extrinsics), per-object centroids and normalized depths (from an open-vocabulary detector and depth estimator, with a VLM fallback), and gripper-object relations. A rendering engine then paraphrases each tuple along six axes (reference modality, referential frame, lexicon, depth verbalization, verbosity, evidence-conditioned content) so the same geometric fact appears under many surface forms, teaching the policy to interpret rather than memorize spatial language. A final trajectory-level GRPO stage with sparse success rewards aligns when the policy invokes tools with actual task outcome, shrinking average tool calls per episode from about 3.4 to 1.8.

What would settle it

Corrupt the injected evidence deliberately on a held-out scene set, shifting every object centroid by about 30 pixels or making the VLM fallback mislabel target categories, and measure success; the paper's account predicts success drops sharply and monotonically with evidence error because the policy acts on what it reads, whereas flat success near behavior-cloning level would show the gain comes from something other than consuming grounded evidence. A finer variant the paper's ablations make available: with evidence fixed, sweep the supervision weight on the injected text tokens from zero (the paper's setting) up to full language-modeling loss, and check whether accuracy declines smoothly toward the Gen-CoT level as predicted.

Watch

Extended reading notes

Core claim

The paper's central assertion is that "what a VLA needs is not the ability to generate language, but the ability to consume grounded language." VLA-Talker instantiates this by decoupling evidence acquisition from evidence use: an agentic tool-use cascade projects the gripper into image space analytically, localizes task-relevant objects with an open-vocabulary detector (falling back to a VLM locator when the detector is uncertain), and samples relative depth; the resulting evidence tuple is rendered into diverse, paraphrased spatial descriptions wrapped in `<spatial>` tags; and the policy is post-trained with the language-model loss masked out, so it learns only to condition on the injected evidence, never to produce it. The paper reports that this design improves the LIBERO average success rate from 81.5% for the matched generative-CoT variant to 97.4%, reaches 59.5% on RoboCasa-GR1 and 72.4% on SimplerEnv, and achieves 58.1% single-task success on a real AgiBot G1 humanoid. The injection-vs-generation comparison holds all evidence identical, so the only differences are whether evidence is generated or injected and what is supervised.

Load-bearing premise

The load-bearing premise is that the tool loop's measurements are accurate enough at every keyframe for the policy to act on them; the detector, depth estimator, and VLM fallback can return wrong or misleading evidence, and the paper's own limitation notes that in a sufficiently out-of-distribution scene the policy would trust that misleading evidence and act on it.

Editorial extensions

If this is right

  • Under matched evidence, supervising only action tokens on injected context beats both generating-and-supervising text (81.5% vs 97.4% LIBERO average) and injecting-but-still-supervising text (89.7%), so the operative design choice is the supervision mask, not the evidence alone.
  • Language competence for a VLA becomes measurable as robustness to phrasing: with roughly 24 diverse paraphrase realizations per evidence tuple, success under paraphrased instructions stays at 94.6% versus 77.2% for a single template.
  • Data efficiency improves: 25 demonstrations per task with VLA-Talker outperform behavior cloning trained on 50, and the gap widens as data shrinks.
  • The tool cascade's fallback routing is what keeps injected evidence reliable: at a simulated 60% detector drop rate the cascade retains 93.8% success where detector-only falls to 70.6%.
  • Trajectory-level GRPO adds +1.8% over the in-context cold-start alone and teaches economical tool use, but applied to the raw backbone it degrades to 87.8%, so RL is a complement to, not a replacement for, the grounded post-training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's error decomposition implies a next bottleneck: once grounding is solved, control precision near contact becomes 69.4% of residual failures, a direct argument for pairing evidence injection with a contact-rich refinement controller, which the paper itself names as a promising direction.
  • A testable extension the paper leaves implicit: if consuming grounded evidence is the true source of the gain, the same action-only in-context recipe should transfer across VLA backbones and action heads (including diffusion or flow-based heads) without re-engineering the rendering pipeline, since the mechanism lives in the supervision mask.
  • Because the policy trusts injected context by design, a confidence-aware tool loop that abstains or flags low-certainty evidence could push the approach into the out-of-distribution scenes the paper identifies as its main limitation.
  • The keyframe gating schedule is hand-designed, and the RL stage already learns to call tools less often on its own, so learning when to request fresh evidence end-to-end is a natural next step the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 9 minor

Summary. The paper argues that a vision-language-action model needs the ability to consume grounded language rather than to generate rationales. VLA-Talker injects tool-derived spatial evidence (open-vocabulary detection, monocular depth, analytical gripper projection, VLM fallback) as read-only <spatial> context at keyframes, supervises only action tokens during an in-context post-training stage, renders evidence in diverse paraphrases through a round-trip-filtered data engine, and then applies GRPO with sparse success rewards. The paper reports success rates on LIBERO, RoboCasa-GR1, SimplerEnv, and eight real-robot AgiBot tasks, comparing against a matched-evidence generative CoT baseline (Gen-CoT) and published VLA baselines, and claims that free-form CoT degrades low-level control through a grounding gap, objective interference, and inference latency.

Significance. If established cleanly, the central claim would be a valuable contribution: it decouples evidence acquisition from evidence consumption, makes the falsifiable prediction that action-only supervision on injected evidence beats rationale generation under equal evidence, and demonstrates a large inference-cost advantage (12.8 Hz vs 2.8 Hz, Table 10). The paper has genuine strengths: Table 3(b) versus (c) is a confound-free ablation of the supervision mask under identical test-time injection; the round-trip consistency filter with explicit pixel/depth tolerances (Appendix A) is a machine-checkable safeguard; Appendix E provides three-seed statistics with a Welch test; Appendices G and H give hyperparameter sensitivity and synthetic perception-noise stress tests; and Appendix N states limitations honestly, including the premise that tool outputs must be trustworthy, which the cascade robustness studies (Table 14, Fig. 13) partially mitigate.

major comments (5)
  1. [Tables 1 and 3; Fig. 4] Table 1 reports Gen-CoT at 96.2% LIBERO average, while Table 3(a) and Fig. 4 report the same 'matched-evidence generate-and-supervise-text' scheme at 81.5%, and Fig. 1's motivational bars (CoT ≈ 0.88) match neither value. Table 6 adds a third number for the same method at the full data budget (87.6% at 50 demonstrations), placing Gen-CoT below BC's 90.4%, whereas Table 1 places Gen-CoT (96.2%) well above BC (90.4%, Table 4). If these are different configurations (e.g., single-stage SFT versus the full two-stage recipe, or different per-task data budgets), the text must say so explicitly; as printed, the central claim's strength swings from a 1.2-point margin (Table 1) to a 15.9-point margin (Table 3), and no sentence in the paper allows a reader to reconcile the values. Please unify the definition of the matched-evidence baseline, state its configuration in every table where it appears, and re-derive the affected claims.
  2. [Tables 1, 2, 3, 5; Table 12] The headline comparisons confound test-time evidence access with the generate-versus-inject contrast. Gen-CoT must produce the evidence tuples from the RGB image at inference (the 'grounding gap' defined in the method section), whereas VLA-Talker receives the same tuples from the tool loop on keyframes at inference ('Keyframe gating'). The comparison therefore varies two factors at once: whether the policy is handed near-ground-truth spatial evidence, and whether language is generated or injected. Table 3(a) versus (c) cannot attribute the 15.9-point gap to the consume-versus-generate principle, and Table 12's 'w/o tool loop (self-guess)' result (84.3%, below plain BC at 90.4%) shows the gains track the external evidence itself. A control that receives the identical tool-loop output at inference while still generating (or supervising) the rationale is needed to support the stated attribution; only Table 3(b) versus (c), which shares test-time injection and varies only the supervision mask, is free of this confound.
  3. [Method: Keyframe gating vs. RL stage] The method section specifies tool invocation in two incompatible ways. 'Keyframe gating' states that evidence is injected only on a fixed initial/gripper-change/periodic schedule, with 'the same schedule ... applied' at inference, so the tool loop runs independently of any policy decision. The RL section, however, defines rollouts containing policy-emitted tool calls (Eq. (4): C_k), rewards tool-call syntax through I_format in Eq. (5), asserts that 'the policy is free to discover when to call tools rather than being told,' and Fig. 10(b) reports that GRPO reduces average tool calls per episode from ~3.4 to ~1.8. If the keyframe schedule is fixed, the policy cannot choose to skip or add tool calls and the Fig. 10(b) result is unexplained; if the policy can emit tool requests, the interaction between those requests and the gating schedule must be specified, including how the 12.8 Hz inference cost (Table 10) accounts for tool-loop overhead. Please clarify the mechanism and make the claims consistent.
  4. [Why Generative CoT Hurts Low-Level Control; Eq. (2)] The abstract claims the paper shows 'both empirically and analytically' that free-form CoT degrades low-level control, but the analytical component is the token-count argument around Eq. (2): because language tokens outnumber action tokens, 'gradient mass is dominated by sounding reasonable.' Token-count imbalance alone does not establish gradient domination, since per-token cross-entropy terms are typically normalized, action-token losses can be large early in training, and either term can be masked or reweighted (as the paper itself does in Table 3). Table 3(b) versus (c) empirically supports the value of action-only supervision, so the paper should either provide measured per-stage gradient norms or withdraw the 'analytically' claim.
  5. [Appendix G; Table 1] Appendix G states that all hyperparameters were selected by a coarse grid search on the LIBERO validation split and then frozen across benchmarks, and Fig. 12 selects the GRPO defaults (β = 0.01, M = 16) on LIBERO as well; the headline LIBERO average of 97.4% is therefore tuned on the same suite on which it is presented as a new SOTA (the margin over VLA-Thinker in Table 1 is 0.4 points). Please either report LIBERO with a configuration chosen without access to LIBERO, or clearly separate development-split from held-out results. The main tables (1, 2, 5) also carry no seed variance; the three-seed analysis in Appendix E covers only BC, Gen-CoT, and VLA-Talker, not the published baselines in those tables.
minor comments (9)
  1. [Fig. 1] The introductory bar chart gives no benchmark or configuration for its values (CoT ≈ 0.88, ICL ≈ 0.97), and neither number matches Tables 1 or 3; please label the source or remove the chart.
  2. [Fig. 4] Setting (b) is annotated as 0.9x latency (73 ms) and setting (c) as 1.0x (78 ms), implying that supervising the injected text yields a faster policy than action-only supervision; please explain this or correct the annotation.
  3. [Implementation paragraph] The paragraph contains a dangling cross-reference, 'the two-stage recipe of Section :', with the section number missing.
  4. [Fig. 13] The x-axis label renders as 'Injected pixel-localization noise ( , px)' with the σ symbol missing.
  5. [Table 13] The caption contains an editorial annotation ('[highlighting applied to family- and overall-average rows only]') that should be removed from the final caption.
  6. [Table 8] The real-robot table reports 20 trials per subtask with no variance or repeated-initialization information, and several margins (e.g., Pen single-task, 35 vs 15) are within plausible binomial sampling noise at this sample size; please add confidence intervals or more trials, and note that the single-task +CoT average (41.9) coincides exactly with the Baseline average (41.9).
  7. [Introduction] The phrase 'agentic tool-use interface, in which the policy queries ...' is not reflected in the described training and inference procedure, where the tool loop runs on a fixed schedule; please align the wording with the mechanism clarified in response to the major comments.
  8. [Table 16] The table reports 1,640 training episodes for LIBERO, which implies roughly 41 demonstrations per task under the standard 40-task split, a number that does not match the 50-demonstration budget used in Table 6; please state the exact per-task demonstration budget used for the main results.
  9. [Abstract] The abstract's claim of 'nearly 10%' improvement over standard behavior-cloning baselines is not tied to a specific table: on LIBERO the gain over the BC backbone is 7.0 points (Table 4), and no BC baseline is reported for RoboCasa-GR1 or SimplerEnv; please make the claim quantitative and traceable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the consume-vs-generate contrast is a genuine experimental manipulation, not a derivation that reduces to its inputs.

full rationale

The paper's central claim—that VLA language competence should be the consumption of externally grounded evidence rather than its generation—is supported by a chain of evidence that does not reduce to its own inputs. Grounded evidence is produced by an external tool loop (open-vocabulary detection, monocular depth, analytic gripper projection, and a VLM fallback) and injected as read-only context; the policy is trained with Equation (3) under an action-only supervision mask, so no token of the evidence context ever appears in the loss and the model never learns to produce it. The key controlled comparison (Table 3 and Figure 4) varies only whether the identical evidence tuple is generated-and-supervised or injected-and-action-supervised, which is a real experimental manipulation rather than a definitional equivalence. Table 12 honestly shows that removing the tool loop drops performance below plain BC, confirming that the gain is attributable to the externally acquired evidence rather than to a hidden fitted term or to a self-referential training objective. No parameter is fitted to the reported success metrics, no uniqueness theorem is imported from the authors' own prior work, and no known result is merely renamed. The Gen-CoT baseline can be criticized as confounding 'generation' with 'lack of test-time tool access,' since at inference Gen-CoT must infer object locations from pixels while VLA-Talker receives them as context; that is a legitimate external-validity concern about the comparison, however, not a circularity, because the paper's own ablation (Table 3) and the error decomposition (Table 15) isolate the generate-versus-consume mechanism. The limitations section candidly acknowledges dependence on off-the-shelf perception tools and real-world calibration requirements, further supporting that no step is self-referential. No circular step meets the required standard of exhibiting an equation, fitted constant, or citation that makes the derivation equivalent to its inputs.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No free parameters beyond standard training hyperparameters and hand-set gating choices; no invented physical entities. The 'evidence tuple' and 'tool loop' are software constructs, not postulated entities. The main assumptions are the reliability of external tools and the ability of a pretrained backbone to consume injected text.

free parameters (6)
  • GRPO KL anchor weight beta = 0.01
    Tuned via coarse grid search on LIBERO validation; controls how far the policy can drift from the in-context checkpoint.
  • GRPO rollout group size M = 16
    Tuned via coarse grid search on LIBERO validation; larger groups reduce variance but increase RL compute.
  • In-context SFT and RL learning rates = 1e-5 and 2e-6
    Chosen by coarse grid search on LIBERO validation and kept fixed across all benchmarks.
  • Paraphrase pool size n = 24 realizations per tuple
    Selected from a diversity ablation on LIBERO as the smallest pool within 1 point of saturated paraphrase robustness.
  • Keyframe gating schedule = initial frame, gripper-change frames, periodic checks
    Hand-designed heuristic; the paper notes in Limitation 3 that learning when to request evidence is left to future work.
  • Round-trip filter tolerances = 5 px offset, 0.02 normalized depth
    Hand-set thresholds for accepting paraphrase candidates; used to keep rendered captions faithful to evidence.
assumptions (4)
  • domain assumption A pretrained VLA backbone can be fine-tuned to condition on injected <spatial> language context.
    The method assumes the backbone (OpenVLA-OFT or JoyAI-RA-0.1) can learn to use text evidence even though it never generates it; no theorem guarantees this.
  • domain assumption External perception tools (GroundingDino, DepthAnything, Qwen2.5-VL) return sufficiently accurate and timely evidence at keyframes.
    The central gain comes from injecting tool-derived evidence; if the tools fail or hallucinate, the policy trusts misleading context (acknowledged in Limitation 1).
  • ad hoc to paper Token-count imbalance in Eq. (2) implies that language supervision dominates gradient mass and pushes the policy toward narration.
    The paper states this as an analytical conclusion without a formal derivation; it motivates the design but is an assumption about loss dynamics.
  • domain assumption Varying surface form while fixing geometric meaning forces the policy to learn semantics rather than surface strings.
    The diverse-rendering data engine relies on this transfer claim; it is supported only by paraphrase-robustness ablations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of In-Context VLA: Endowing Vision-Language-Action Models with Language via In-Context Post-Training and Agentic Tool Use." pith.science (2026). https://pith.science/paper/QZ75OLBB

@misc{pith2026260805738,
  author       = {Pith},
  title        = {Pith review of: In-Context VLA: Endowing Vision-Language-Action Models with Language via In-Context Post-Training and Agentic Tool Use},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QZ75OLBB}},
  note         = {Machine review of arXiv:2608.05738}
}
read the original abstract

Vision-Language-Action (VLA) models have become the dominant recipe for generalist manipulation, yet they are almost universally trained by behavior cloning: a policy imitates expert action chunks conditioned on a static image and a fixed instruction. A natural remedy is to inject explicit reasoning through textual chain-of-thought (CoT). We show, both empirically and analytically, that free-form textual CoT degrades low-level control: the reasoning it produces is ungrounded, its latency breaks closed-loop timing, and, crucially, the reasoning and action tokens are optimized against conflicting objectives so that the policy learns to narrate rather than to act. We argue that what a VLA needs is not the ability to generate language, but the ability to consume grounded language. To this end we introduce \textbf{\ourmethod{}}, a framework that endows a VLA with language competence through (i) in-context post-training, in which perceptual evidence is injected as structured context and the model is supervised only on actions, and (ii) an agentic tool-use interface, in which the policy queries open-vocabulary detectors, monocular depth, and a vision--language model to actively acquire task-relevant information. Rather than emitting a single templated caption, our data engine produces diverse, paraphrased, and evidence-conditioned spatial descriptions, so that the policy learns to interpret language it has never seen verbatim. Across the RoboCasa-GR1, SimplerEnv, and LIBERO simulation benchmarks, together with 8 real-world robot manipulation tasks, our method consistently achieves SOTA results in both performance and efficiency when compared with CoT-based approaches under matched configurations.

Figures

Figures reproduced from arXiv: 2608.05738 by the authors.

Figure 1
Figure 1. Comparison between free-form CoT generation [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the VLA-Talker framework. An agentic tool-use module (depth estimation, open-vocabulary object [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Representative inference-time rollout showing in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Per-decision latency (bars, left axis) vs. LIBERO [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Keyframe gating on LIBERO. Adding gripper [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Real-robot deployment on the AgiBot G1 humanoid [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Representative successful executions of VLA-Talker on the AgiBot G1 humanoid, sampled from three different desktop [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: A representative failure rollout of VLA-Talker on the [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: RL training dynamics. (a) Task-success reward rises steadily from [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: LIBERO success rate by suite, mean ± std over 3 seeds. Error bars shrink from BC to Gen-CoT to VLA-Talker, indicating that grounded, externally acquired evidence also reduces run-to-run variance, not just the mean. of newly added layers, and evaluation initial conditi…
Figure 12
Figure 12. Figure 12: Hyperparameter sensitivity of the GRPO stage. (a) LIBERO average success vs. KL-anchor weight [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: LIBERO average success under synthetic pixel [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 9 canonical work pages

  1. [1]

    IEEE Transactions on Neural Networks and Learning Systems , year=

    A survey on vision--language--action models for embodied ai , author=. IEEE Transactions on Neural Networks and Learning Systems , year=

    work page

  2. [2]

    arXiv preprint arXiv:2604.20100 , year=

    Joyai-ra 0.1: A foundation model for robotic autonomy , author=. arXiv preprint arXiv:2604.20100 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2406.09246 , year=

    Openvla: An open-source vision-language-action model , author=. arXiv preprint arXiv:2406.09246 , year=

    arXiv

  4. [4]

    Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers) , pages=

    Truthfulqa: Measuring how models mimic human falsehoods , author=. Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers) , pages=

  5. [5]

    Advances in Neural Information Processing Systems , volume=

    Libero: Benchmarking knowledge transfer for lifelong robot learning , author=. Advances in Neural Information Processing Systems , volume=

  6. [6]

    arXiv preprint arXiv:2204.01691 , year=

    Do as i can, not as i say: Grounding language in robotic affordances , author=. arXiv preprint arXiv:2204.01691 , year=

    arXiv

  7. [7]

    2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization=

    2025

  8. [8]

    arXiv preprint arXiv:2505.20289 , year=

    Visualtoolagent (vista): A reinforcement learning framework for visual tool selection , author=. arXiv preprint arXiv:2505.20289 , year=

    arXiv

Show all 46 references
  1. [9]

    NeurIPS 2025 Workshop on Space in Vision, Language, and Embodied AI , year=

    RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Lifelong Learning in Physical Embodied Systems , author=. NeurIPS 2025 Workshop on Space in Vision, Language, and Embodied AI , year=

    2025

  2. [10]

    9th Annual Conference on Robot Learning , year=

    Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation , author=. 9th Annual Conference on Robot Learning , year=

  3. [11]

    arXiv preprint arXiv:2508.19236 , year=

    Memoryvla: Perceptual-cognitive memory in vision-language-action models for robotic manipulation , author=. arXiv preprint arXiv:2508.19236 , year=

    arXiv

  4. [12]

    European Conference on Computer Vision , pages=

    LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents , author=. European Conference on Computer Vision , pages=

  5. [13]

    arXiv preprint arXiv:2303.03378 , year=

    Palm-e: An embodied multimodal language model , author=. arXiv preprint arXiv:2303.03378 , year=

    arXiv

  6. [14]

    arXiv preprint arXiv:2307.05973 , year=

    Voxposer: Composable 3d value maps for robotic manipulation with language models , author=. arXiv preprint arXiv:2307.05973 , year=

    arXiv

  7. [15]

    arXiv preprint arXiv:2207.05608 , year=

    Inner monologue: Embodied reasoning through planning with language models , author=. arXiv preprint arXiv:2207.05608 , year=

    arXiv

  8. [16]

    arXiv preprint arXiv:2512.04069 , year=

    SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL , author=. arXiv preprint arXiv:2512.04069 , year=

    arXiv

  9. [17]

    arXiv preprint arXiv:2605.13119 , year=

    Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models , author=. arXiv preprint arXiv:2605.13119 , year=

    arXiv

  10. [18]

    arXiv preprint arXiv:2602.01166 , year=

    Latent Reasoning VLA: Latent Thinking and Prediction for Vision-Language-Action Models , author=. arXiv preprint arXiv:2602.01166 , year=

    arXiv

  11. [19]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    TCoT: Trajectory chain-of-thoughts for robotic manipulation with failure recovery in vision-language-action model , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  12. [20]

    arXiv preprint arXiv:2601.11404 , year=

    ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models , author=. arXiv preprint arXiv:2601.11404 , year=

  13. [21]

    arXiv preprint arXiv:2603.22280 , year=

    Dualcot-vla: Visual-linguistic chain of thought via parallel reasoning for vision-language-action models , author=. arXiv preprint arXiv:2603.22280 , year=

  14. [22]

    arXiv preprint arXiv:2603.14523 , year=

    VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoning , author=. arXiv preprint arXiv:2603.14523 , year=

  15. [23]

    Intelligence, Physical and Black, Kevin and Brown, Noah and Darpinian, James and Dhabalia, Karan and Driess, Danny and Esmail, Adnan and Equi, Michael and Finn, Chelsea and Fusai, Niccolo and others , journal=. _

  16. [24]

    arXiv preprint arXiv:2410.24164 , year=

    _0 : A Vision-Language-Action Flow Model for General Robot Control , author=. arXiv preprint arXiv:2410.24164 , year=

    arXiv

  17. [25]

    arXiv preprint arXiv:2510.13778 , year=

    Internvla-m1: A spatially guided vision-language-action framework for generalist robot policy , author=. arXiv preprint arXiv:2510.13778 , year=

    arXiv

  18. [26]

    Forty-third International Conference on Machine Learning , year=

    Langforce: Bayesian decomposition of vision language action models via latent action queries , author=. Forty-third International Conference on Machine Learning , year=

  19. [27]

    arXiv preprint arXiv:2602.10098 , year=

    Vla-jepa: Enhancing vision-language-action model with latent world model , author=. arXiv preprint arXiv:2602.10098 , year=

  20. [28]

    Advances in neural information processing systems , volume=

    Videovla: Video generators can be generalizable robot manipulators , author=. Advances in neural information processing systems , volume=

  21. [29]

    arXiv preprint arXiv:2411.19650 , year=

    Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation , author=. arXiv preprint arXiv:2411.19650 , year=

    arXiv

  22. [30]

    arXiv preprint arXiv:2602.11236 , year=

    Abot-m0: Vla foundation model for robotic manipulation with action manifold learning , author=. arXiv preprint arXiv:2602.11236 , year=

    arXiv

  23. [31]

    Advances in Neural Information Processing Systems , volume=

    Thinkact: Vision-language-action reasoning via reinforced visual latent planning , author=. Advances in Neural Information Processing Systems , volume=

  24. [32]

    arXiv preprint arXiv:2512.16793 , year=

    Physbrain: Human egocentric data as a bridge from vision language models to physical intelligence , author=. arXiv preprint arXiv:2512.16793 , year=

  25. [33]

    arXiv preprint arXiv:2601.14133 , year=

    TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers , author=. arXiv preprint arXiv:2601.14133 , year=

  26. [34]

    arXiv preprint arXiv:2509.06951 , year=

    F1: A vision-language-action model bridging understanding and generation to actions , author=. arXiv preprint arXiv:2509.06951 , year=

    arXiv

  27. [35]

    arXiv preprint arXiv:2501.15830 , year=

    Spatialvla: Exploring spatial representations for visual-language-action model , author=. arXiv preprint arXiv:2501.15830 , year=

    arXiv

  28. [36]

    The International Journal of Robotics Research , volume=

    Diffusion policy: Visuomotor policy learning via action diffusion , author=. The International Journal of Robotics Research , volume=. 2025 , publisher=

    2025

  29. [37]

    arXiv preprint arXiv:2503.06669 , year=

    Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems , author=. arXiv preprint arXiv:2503.06669 , year=

    arXiv

  30. [38]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Depth anything: Unleashing the power of large-scale unlabeled data , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  31. [39]

    European conference on computer vision , pages=

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection , author=. European conference on computer vision , pages=. 2024 , organization=

    2024

  32. [40]

    arXiv preprint arXiv:2405.05941 , year=

    Evaluating real-world robot manipulation policies in simulation , author=. arXiv preprint arXiv:2405.05941 , year=

    arXiv

  33. [41]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  34. [42]

    arXiv preprint arXiv:2503.14734 , year=

    Gr00t n1: An open foundation model for generalist humanoid robots , author=. arXiv preprint arXiv:2503.14734 , year=

    arXiv

  35. [43]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Dip: Unsupervised dense in-context post-training of visual representations , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  36. [44]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

  37. [45]

    Advances in neural information processing systems , volume=

    Large language models are zero-shot reasoners , author=. Advances in neural information processing systems , volume=

  38. [46]

    arXiv preprint arXiv:2602.01067 , year=

    A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulation , author=. arXiv preprint arXiv:2602.01067 , year=

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.