Pith. sign in

REVIEW 5 major objections 4 minor 3 cited by

Mechanistic interpretability for steering vision-language-action models

T0 review · 5 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Clamping a few interpretable neurons steers a vision-language-action robot at inference time.

desk verdict First VLA steering via FFN value vectors shows real effects, but headline claims run ahead of the evidence; worth a serious referee. read the letter →

arxiv 2509.00328 v1 pith:XPATLXYN submitted 2025-08-30 cs.RO cs.LG

Bear Häon , Kaylene Stocking , Ian Chuang , Claire Tomlin This is my paper · ORCID
classification cs.ROcs.LG
keywords mechanisticinterpretabilityvision-language-actionmodelsactivationsteeringfeedforwardvaluevectorstransformerrobotmanipulationzero-shotbehaviorcontrolfoundationforrobotics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Robot policies that combine language, vision, and action are normally treated as opaque action generators. This paper tries to establish that inside their transformer layers lie readable semantic concepts—'fast', 'slow', 'up', 'high'—inherited from language pretraining, and that these concepts are causally connected to the actions the robot takes. Its method reads each feedforward neuron's value vector through the model's token vocabulary, groups neurons with similar top tokens into concept clusters, and then forces those neurons' activations to a constant at inference time. In simulation the fast cluster produces larger end-effector displacement than the slow cluster; on a real robot arm, low and slow clusters lower the trajectory and reduce movement speed without any retraining, reward signal, or environment interaction. If this holds, operators of deployed VLA robots gain a lightweight, interpretable control knob for shaping behavior.

What carries the argument

Value-vector activation steering: a transformer feedforward output is a weighted sum of fixed row vectors (value vectors), each weighted by an input-dependent activation. Projected through the language-modeling head, each value vector acts as a probability distribution over tokens, so its top tokens supply a semantic label. The steering operator sets the activations of a chosen cluster S to a constant α, giving a residual shift that propagates through the transformer. Clusters are formed either manually from top-token lists or by kNN over softmax-weighted average token embeddings, then matched to a target concept by cosine similarity.

What would settle it

Take the kNN-selected 'slow' cluster from the Slow/Fast Transport experiment and measure its effect on end-effector speed in 50 held-out task contexts with varied prompts and object positions. If the distribution of speed changes from this cluster is indistinguishable from the distribution produced by an equally sized random cluster, or if its sign flips between contexts, the claim that token-projected value vectors are behaviorally aligned directions fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the semantic structure retained from vision-language pretraining is causally active in action selection: when a neuron cluster reads, in token space, as 'slow' or 'up', raising that cluster's activation moves the robot's end effector in exactly that way. The mechanism is a simple override—set a subset S of feedforward activations to a constant α, leaving the rest intact—which creates a residual shift that propagates to the final action-token distribution. In simulation, fast clusters produced 27.73% larger end-effector displacement than slow clusters, and full-layer 'up' clusters produced the largest upward motion. On hardware, 'low' and 'slow' cluster inter

Load-bearing premise

The load-bearing premise is that neurons whose top output tokens sound alike—'slow', 'safe'—are behaviorally alike in the robot's current context, so forcing them to a constant shifts the action in that semantic direction rather than in some unrelated way.

Editorial extensions

If this is right

  • Operators can adjust a live policy's behavior by changing a small set of neuron indices and one scalar, enabling immediate speed or height modulation without collecting new data.
  • Because most feedforward neurons retain semantic content after VLA training, interpretability tools developed for language models can be carried directly into embodied policies.
  • Steering with semantically selected neurons is more effective than editing the prompt or perturbing random neurons, indicating that concept-aligned internal directions form a distinct control channel.
  • Layer-depth results localize motion-related concepts to the full network, with late layers nearly as effective, guiding where future interventions should be applied.
  • Fine-tuning mainly reallocates action-token probabilities rather than destroying semantic structure, so interpretable directions from pretraining can be expected to survive task adaptation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to select clusters by measured behavioral effect instead of token similarity; if the two selections disagree, the paper's 'semantic alignment' is only a proxy for behavior.
  • The method should transfer to other VLA families with discrete action tokens; models with continuous action heads may require a different way of grounding value vectors in token space.
  • Because the same cluster can drift in meaning across contexts, steering could be paired with closed-loop calibration that picks the activation strength α online from observed behavior rather than fixing it in advance.
  • The paper's asymmetry—low and slow steer while high and fast do not—suggests the baseline policy already sits at one end of the behavior range; a testable extension is to estimate the default operating point before choosing the steering direction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper introduces a mechanistic interpretability framework for vision-language-action (VLA) models. It projects feedforward value vectors into the token embedding space, identifies semantically interpretable neuron clusters (e.g., fast/slow, up/down) via token projections, and steers behavior by overriding the activations of selected neurons with a constant scalar. The authors report that VLA training preserves semantic structure from VLM pretraining, that action tokens appear in all layers, that fine-tuning mostly affects action-token distributions, and that interventions modulate motion magnitude in LIBERO (OpenVLA) and height/speed on a UR5 (π0-FAST). They claim this is the first such framework and that steering works zero-shot without fine-tuning, reward signals, or environment interaction.

Significance. If the claims hold, the paper opens a promising new direction: using mechanistic interpretability to build a control interface for embodied foundation models. Strengths include the use of two open-source VLAs, deterministic seeds in simulation, a concrete intervention operator, and a candid limitations section. However, the load-bearing claims currently outrun the evidence: the 'less than 25% rewired' statistic is unmeasured, the simulation results omit task success rates, the hardware speed result is open-loop rather than closed-loop, and the token-projection cluster assumption is challenged by the authors' own high/fast null results and the slow/safe confound. The central idea is promising but needs substantial strengthening before the advertised 'out-of-the-box' claim is supported.

major comments (5)
  1. [§1 bullet 1 and §3.2] The claim that 'less than 25% of FFN neurons are rewired for action prediction' is never quantified. Figure 3a/3b show that action tokens dominate the most significant z-score shifts between base and fine-tuned models, but no measurement of the fraction of rewired neurons is provided. Please define 'rewired' and report the actual fraction with confidence intervals, or remove the claim.
  2. [§4.1] Simulation steering experiments report only end-effector displacement and paired t-tests; task success rates are not reported for any steering configuration. Since the interventions alter behavior, the absence of success rates leaves open the possibility that 'fast' clusters simply produce larger, task-failing motions. Report per-condition success rates (or another task-completion metric) alongside displacement.
  3. [§4.2, Task Evaluation] The Slow/Fast Transport hardware result is not a closed-loop physical demonstration. The text states that 'we run inference for all steering variants and baselines simultaneously, but only execute the baseline predicted action with no intervention.' The box plots in Figure 7b are therefore based on predicted action displacements, not on executed robot motion. This should be clearly labeled as an open-loop action-prediction analysis, or the steered actions should be executed and measured.
  4. [Appendix C.3, Table 2, and §6] Cluster selection is based on top-token keyword frequency, and the clusters are not validated as behaviorally aligned. The 'slow' cluster mixes slow/calm/safe tokens and the 'fast' cluster mixes fast/risk tokens; the high intervention and fast intervention did not differ from baseline (Figure 7). The paper's own Limitations section concedes that clusters can conflate distinct behaviors and that meaning drifts across contexts. This directly weakens the central assumption that token-projection similarity implies behavioral alignment. Please add a behavioral validation step or per-cluster ablations, and temper the semantic labeling accordingly.
  5. [Abstract and §4.2] The abstract claims the method works 'without fine-tuning, reward signals, or environment interaction,' but the hardware experiments require LoRA fine-tuning on task-specific data (75–120 collected episodes) and the simulation experiments require sweeping α and cluster size. The method itself may not require fine-tuning, but the paper does not establish that intervention selection is fully 'out-of-the-box'; in fact, the low/slow interventions were found to work only after evaluating their high/fast counterparts. Please qualify the claim to distinguish the intervention operator from the model preparation and hyperparameter selection.
minor comments (4)
  1. [Throughout] Inconsistent capitalization: 'O PEN VLA', 'OPEN VLA', and 'OpenVLA' are used interchangeably in the main text, abstract, and figure captions. Please standardize.
  2. [Figure 5b] The caption says 'up-cluster activations injected at early, late, and full model depths' but 'early' and 'late' are not defined in the main text. Please define the depth split in Section 4.1.
  3. [Appendix B.3] The text states the full model contains 352,255 value vectors, then says the first half is 176,128 vectors. Since 352,255 is odd, the 'first half' should be 176,127 or 176,128 with an explanation of the split.
  4. [Title page] There are several typographical issues: 'Bear H¨aon' contains a nonstandard diacritic, and some PDF artifacts appear in the extracted text (e.g., 'OPEN VLA' spacing). Please proofread the camera-ready version.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: behavioral predictions are measured against external rollouts, not derived from the token-projection clusters.

full rationale

The central derivation is not circular. Value vectors are interpreted by projecting onto the token embedding basis (Eq. 2, Section 3), but the steering claims are then tested against independent physical outcomes: LIBERO end-effector displacements and UR5 height/displacement measurements. Eq. 3 defines the intervention operator without assuming the behavioral effect it is used to establish. The slow/fast and low/high clusters are selected by keyword frequency in Appendix C.3, after which the effect on speed/height is measured; so the behavioral prediction is externally falsifiable. The paper itself concedes the key gap: 'Our clustering method is based on token-level semantic similarity rather than direct behavioral outcomes' (Section 6), which is a validity risk, not a reduction to inputs. Two adjacent concerns are non-circular correctness risks: (i) the abstract's 'without ... environment interaction' claim is strained by the parameter sweep over cluster sizes and activation strengths in Section 4.1, and (ii) the slow cluster mixes 'slow' with 'safe' tokens (Table 2c), so the speed attribution is confounded. Neither concern makes the result equivalent to its input by construction. No load-bearing self-citation appears; prior coauthor work (Ref. [13]) is only in Related Work, and the interpretability method follows external work [8].

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's approach rests on established interpretability assumptions from the LLM literature rather than introducing new mathematical entities. The key additional premise, that token-space semantics of value vectors predict behavioral control, is an empirical assumption the paper only partially validates.

free parameters (4)
  • Activation coefficient α = Swept over {2,4,6,8,10,20} in simulation; fixed at 10 in hardware
    The scalar used to override activations in Eq. (3). The sweep tunes it to the desired effect; there is no principled derivation.
  • Cluster size |S| = 10 or 20 in simulation; 6 in hardware
    Number of value vectors overridden per intervention, chosen by the experimenter.
  • Manual cluster selection = Six vectors per concept for hardware, based on top-token frequency
    The specific value vectors in the steering set are hand-picked from the top tokens; this is a selection that directly determines the outcome.
  • kNN neighborhood size k = 10, 20, 40 in simulation
    Used for clustering value vectors in the temporal localization experiments; chosen without a stated criterion.
assumptions (4)
  • domain assumption The linear representation hypothesis: concepts are represented as directions in activation space.
    Adopted from [15]; underpins the interpretation of value vectors in Section 3.
  • domain assumption FFN value vectors, projected to token space, reveal the semantic meaning of a neuron.
    Adopted from [8]; used to assign concepts like 'fast' and 'up' to clusters in Sections 3-4.
  • domain assumption Overriding activations to a constant α shifts the residual stream in a way that changes model output predictably.
    Central to the steering method in Eq. (3)-(4); assumes the linear superposition of value vectors is behaviorally meaningful.
  • domain assumption Token-space semantic similarity of value vectors translates to behavioral similarity when those vectors are amplified.
    Used to select clusters for steering (Section 4, Appendix C.3); the paper's own Limitations note that this can fail (clusters can conflate behaviors, meanings drift).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mechanistic interpretability for steering vision-language-action models." pith.science (2026). https://pith.science/paper/XPATLXYN

@misc{pith2026250900328,
  author       = {Pith},
  title        = {Pith review of: Mechanistic interpretability for steering vision-language-action models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XPATLXYN}},
  note         = {Machine review of arXiv:2509.00328}
}
read the original abstract

Vision-Language-Action (VLA) models are a promising path to realizing generalist embodied agents that can quickly adapt to new tasks, modalities, and environments. However, methods for interpreting and steering VLAs fall far short of classical robotics pipelines, which are grounded in explicit models of kinematics, dynamics, and control. This lack of mechanistic insight is a central challenge for deploying learned policies in real-world robotics, where robustness and explainability are critical. Motivated by advances in mechanistic interpretability for large language models, we introduce the first framework for interpreting and steering VLAs via their internal representations, enabling direct intervention in model behavior at inference time. We project feedforward activations within transformer layers onto the token embedding basis, identifying sparse semantic directions - such as speed and direction - that are causally linked to action selection. Leveraging these findings, we introduce a general-purpose activation steering method that modulates behavior in real time, without fine-tuning, reward signals, or environment interaction. We evaluate this method on two recent open-source VLAs, Pi0 and OpenVLA, and demonstrate zero-shot behavioral control in simulation (LIBERO) and on a physical robot (UR5). This work demonstrates that interpretable components of embodied VLAs can be systematically harnessed for control - establishing a new paradigm for transparent and steerable foundation models in robotics.

Figures

Figures reproduced from arXiv: 2509.00328 by the authors.

Figure 1
Figure 1. We present a framework for steering Vision-Language-Action (VLA) models. We extract FFN vectors, project them to the VLA token space, cluster them by semantic alignment, and inject activations at inference time to modulate behavior. Our experiments demonstrate interpretable zero-shot control in both simulation (OPENVLA in LIBERO) and on a physical robot (π0 on a UR5). Abstract: Vision-Language-Action (VLA) models ar… view at source ↗
Figure 2
Figure 2. VLA training incorporates action tokens into FFN value vectors but retains semantic [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Task fine-tuning mainly affects action tokens in FFN value vectors. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Sample tasks from LIBERO-Long. Six representative long-horizon tasks - involving sequential manipulation goals such as object placement, containment, and appliance interaction. (a) Steering motion magnitude interventions. The effect of fast and slow cluster interventio…
Figure 5
Figure 5. Figure 5: Simulation results: Steering OPENVLA with value vector interventions. istically for reproducibility, enabling precise comparisons between baseline and intervention tra￾jectories. All simulation experiments use the 7B-parameter OPENVLA model, with the fine-tuned LIBERO-…
Figure 6
Figure 6. Figure 6: Physical robot experiments: Steering π0 on a UR5. 4.2 Robot Experiments We also evaluate our VLA steering method using the 3B-parameter π0-FAST VLA on a real robot. Specifically, we test the ability to steer binary control opposites - slow vs. fast and low vs. high - i…
Figure 7
Figure 7. Figure 7: Physical robot experiments: Binary control opposites. placement between each step along the trajectory across 10 rollouts for each variant and plot results in Figure 7b. More detailed analysis of individual trajectories for both tasks are in the Appendix. Robot Results…
Figure 8
Figure 8. Figure 8: Common DROID instruction tokens become more common in the value vectors of a model fine-tuned on the DROID dataset. The majority of the top-200 instruction tokens are more common in the value vectors of the π0-FAST-DROID-finetune checkpoint, corresponding to a positive…
Figure 9
Figure 9. Figure 9: Top-5 token projections for the most semantically aligned [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Robot Setup: Our hardware experiments use a UR5 robot arm equipped with a Robotiq 2F-140 gripper. The setup includes two cameras: a static scene camera overlooking the workspace and a wrist-mounted camera facing the gripper. (a) Scene Camera (b) Wrist Camera [PITH_FU…
Figure 11
Figure 11. Figure 11: Example of raw images that are input into [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Low/High Transport: End-effector height from 10 trajectories with low intervention (a) vs. high intervention (b). We plot the segment of the trajectory from when the robot picks up the penguin to when it releases it into the basket. Low interventions have a lower maxi…
Figure 13
Figure 13. Figure 13: Slow/Fast Transport: End-effector displacement (a) and cumulative end-effector dis￾placement (b) over a single trajectory with slow intervention vs. high intervention. While it may be hard to compare displacement at each action step, the cumulative displacement shows …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Present but Not Remembered: Auditing How Frozen VLAs Encode, Deploy, and Steer Visual History

    cs.CV 2026-07 conditional novelty 7.0 of 10

    VLAs linearly encode past frames as near-redundant copies of the present and deploy them only as architecture-conditional fallback or standing use, so steerability tracks deployment regime not encoding.

  2. What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Erasing objects from front-camera images shows Alpamayo 1's trajectories depend most on large vehicles, pedestrians, and traffic lights, but attributions are seed-unstable and some effects reach the output without tou...

  3. Interpreting Video Representations with Spatio-Temporal Sparse Autoencoders

    cs.CV 2026-04 conditional novelty 6.0 of 10

    Spatio-temporal contrastive SAEs recover temporal coherence lost by hard TopK, improve action probes by +3.9% and retrieval by up to 2.8× R@1, and expose a monosemanticity metric artifact.

Reference graph

Works this paper leans on

30 extracted references · 16 canonical work pages · cited by 3 Pith papers

  1. [1]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Haus- man, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky. π0: A vision-language-action flow model for general robot control, 2024. URL https://arxiv. ...

    arXiv 2024

  2. [2]

    Pertsch, K

    K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. FAST: Efficient action tokenization for vision-language-action models, 2025. URL https://arxiv.org/abs/2501.09747

    arXiv 2025

  3. [3]

    G. R. Team, S. Abeyruwan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, T. Armstrong, A. Balakr- ishna, R. Baruch, M. Bauza, M. Blokzijl, S. Bohez, K. Bousmalis, A. Brohan, T. Buschmann, A. Byravan, S. Cabi, K. Caluwaerts, F. Casarini, O. Chang, J. E. Chen, X. Chen, H.-T. L. Chi- ang, K. Choromanski, D. D’Ambrosio, S. Dasari, T. Davchev, C. Devin, N. D. Palo, ...

    arXiv 2025

  4. [4]

    Olsson, N

    C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y . Bai, A. Chen, T. Conerly, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, S. John- ston, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, and C. Olah. In-context learning and induction heads. Transformer ...

    work page 2022

  5. [5]

    Bricken, A

    T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y . Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, B. McLean, J. E. Burke, T. Hume, S. Carter, T. Henighan, and C. Olah. Towards monosemanticity: Decomposing language mod- els with diction...

    work page 2023

  6. [6]

    Cunningham, A

    H. Cunningham, A. Ewart, L. Riggs, R. Huben, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models, 2023. URL https://arxiv.org/abs/ 2309.08600

    arXiv 2023

  7. [7]

    Marks, J

    S. Marks, J. Treutlein, T. Bricken, J. Lindsey, J. Marcus, S. Mishra-Sharma, D. Ziegler, E. Ameisen, J. Batson, T. Belonax, S. R. Bowman, S. Carter, B. Chen, H. Cunningham, C. Denison, F. Dietz, S. Golechha, A. Khan, J. Kirchner, J. Leike, A. Meek, K. Nishimura- Gasparian, E. Ong, C. Olah, A. Pearce, F. Roger, J. Salle, A. Shih, M. Tong, D. Thomas, K. Riv...

    arXiv 2025

  8. [8]

    M. Geva, A. Caciularu, K. R. Wang, and Y . Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space, 2022. URL https://arxiv. org/abs/2203.14680

    arXiv 2022

Show all 30 references
  1. [9]

    C. Olah, A. Mordvintsev, and L. Schubert. Feature visualization. Distill, 2017. doi:10.23915/ distill.00007. https://distill.pub/2017/feature-visualization

    2017

  2. [10]

    Lindsey, W

    J. Lindsey, W. Gurnee, E. Ameisen, B. Chen, A. Pearce, N. L. Turner, C. Citro, D. Abra- hams, S. Carter, B. Hosmer, J. Marcus, M. Sklar, A. Templeton, T. Bricken, C. McDougall, H. Cunningham, T. Henighan, A. Jermyn, A. Jones, A. Persic, Z. Qi, T. B. Thompson, S. Zim- merman, K...

    2025

  3. [11]

    K. Meng, D. Bau, A. Andonian, and Y . Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359–17372, 2022

    2022

  4. [12]

    T.-H. Wang, W. Xiao, T. Seyde, R. Hasani, and D. Rus. Measuring interpretability of neural policies of robots with disentangled representation. In Conference on Robot Learning, pages 602–641. PMLR, 2023

    2023

  5. [13]

    Pohland and C

    S. Pohland and C. Tomlin. Understanding the dependence of perception model competency on regions in an image. In World Conference on Explainable Artificial Intelligence , pages 130–154. Springer, 2024

    2024

  6. [14]

    Glanois, P

    C. Glanois, P. Weng, M. Zimmer, D. Li, T. Yang, J. Hao, and W. Liu. A survey on interpretable reinforcement learning. Machine Learning, 113(8):5847–5890, 2024

    2024

  7. [15]

    K. Park, Y . J. Choe, and V . Veitch. The linear representation hypothesis and the geometry of large language models. In International Conference on Machine Learning , pages 39643– 39666. PMLR, 2024

    2024

  8. [16]

    M. Geva, R. Schuster, J. Berant, and O. Levy. Transformer feed-forward layers are key-value memories, 2021. URL https://arxiv.org/abs/2012.14913

    2021 arXiv

  9. [17]

    Collaboration, A

    E. Collaboration, A. O’Neill, A. Rehman, A. Gupta, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, A. Tung, A. Bewley, A. Herzog, A. Ir- pan, A. Khazatsky, A. Rai, A. Gupta, A. Wang, A. Kolobov, A. Singh, A. Garg, A. Kembhavi, A. Xie, A...

    2024 arXiv

  10. [18]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, P. Florence, C. Fu, M. G. Arenas, K. Gopalakrishnan, K. Han, K. Hausman, A. Herzog, J. Hsu, B. Ichter, A. Irpan, N. Joshi, R. Julian, D. Kalashnikov, Y . Kuang, I. L...

    2023 arXiv

  11. [19]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, et al. Openvla: An open-source vision-language-action model. In 8th Annual Conference on Robot Learning, 2024

    2024

  12. [20]

    Beyer, A

    L. Beyer, A. Steiner, A. S. Pinto, A. Kolesnikov, X. Wang, D. Salz, M. Neumann, I. Al- abdulmohsin, M. Tschannen, E. Bugliarello, T. Unterthiner, D. Keysers, S. Koppula, F. Liu, A. Grycner, A. Gritsenko, N. Houlsby, M. Kumar, K. Rong, J. Eisenschlos, R. Kabra, M. Bauer, M. Boˇ...

    2024 arXiv

  13. [21]

    Touvron, L

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V . Goswami, N. Goyal, A. Hartshorn, S. Ho...

    2023 arXiv

  14. [22]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. In Robotics: Science and Systems, 2024

    2024

  15. [23]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023. URLhttps://arxiv.org/abs/2306. 03310

    2023

  16. [24]

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen. Lora: Low- rank adaptation of large language models, 2021. URL https://arxiv.org/abs/2106. 09685

    2021

  17. [25]

    non-semantic

    N. Shazeer. GLU variants improve transformer, 2020. URL https://arxiv.org/abs/ 2002.05202. 13 A Steering Intervention Details Let x ∈ Rn be the residual input to a transformer FFN. As shown in Equation (2), the FFN output is a sum of fixed value vectors w(i) θ ∈ Rn weighted by...

    2020 arXiv

  18. [26]

    To assign meaning to each value vector, we project it into the model’s output token space using the language modeling head, as described in Section 3

    Semantic Embedding Construction. To assign meaning to each value vector, we project it into the model’s output token space using the language modeling head, as described in Section 3. For each value vector w(i) θ , we compute token logits and identify the top- 5 tokens it most...

  19. [27]

    The full OPEN VLA model contains 352,255 value vectors across all trans- former layers

    Depth Partitioning. The full OPEN VLA model contains 352,255 value vectors across all trans- former layers. To analyze temporal structure, we partition them: the first half (176,128 vectors) represents early-layer vectors, while the second half represents late-layer vectors. 1...

  20. [28]

    We apply cosine-based k-nearest neighbor (kNN) clustering to these semantic embeddings using the cuML GPU-accelerated library

    kNN Clustering. We apply cosine-based k-nearest neighbor (kNN) clustering to these semantic embeddings using the cuML GPU-accelerated library. Each value vector is assigned a cluster based on its k nearest neighbors (k ∈ {10, 20, 40}). For each cluster, we compute a semantic c...

  21. [29]

    Concept-Aligned Cluster Selection. To find clusters aligned with a target concept (e.g., “up”), we tokenize the concept word/phrase using the model’s tokenizer, embed it using the language modeling head, and compare it to all cluster centroids using cosine similarity. The clus...

  22. [30]

    place penguin 17 in basket

    Temporal Partitioning for Layer-Specific Interventions.To isolate temporal effects, the above clustering pipeline was run independently on the early-layer and late-layer vector subsets. This ensured that the selected clusters were sourced exclusively from the intended depth re...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.